Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Orthogonal matrix wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Euclidean vector wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Matrix multiplication wikipedia , lookup
Polyphonic Music Represented in Mathematical Vectors Nikoleta Hrušková, Slovak University of Technology, Faculty of Informatics and Information Technologies, Ilkovičova 3, 842 16 Bratislava, Slovakia [email protected] Abstract. Google enables searching and retrieving text, video, pictures, books and maps. Why not to look for music by melody typing and retrieve similar music file with ranking information? This paper gives proposals for representation of monophonic and polyphonic music in form of mathematical vectors or matrices allowing retrieving, comparing and evaluating music files, whatever scale they are composed. Query may be monophonic melody edited in notes or played on flash piano. Keywords: music, information retrieval, polyphony, content-based music retrieval, melodic, rhythmic, harmonic vectors, matrix representation of music. 1 Introduction Music is an intellectual property. To print it or to record it is protected by copyright law. Web offers massiveness of music works. To find requested music file happens on metadata (author, title, artist etc.). There are web pages providing finding and identifying a tune according to melody. Multimedia offer several modes of information consumption: listening, reading, watching, seeing, smelling etc. In multimedia applications (for tourism, culture, medical care, advertising) may be the query itself a multimedia excerpt. What computer-based music representation is the most suitable one, that enables effective content-based information retrieval? This is a question for new research area Music Information Retrieval. 2 Music Notation Note is a single musical tone of definite pitch and duration. Conventional musical notation is written on a five-line staff divided into bars. Notes indicate the duration and pitch of a sound; they can be arranged in order of pitch to form scales. Clefs fix the pitch of the notes; accidentals indicate brief changes in pitch. Rests specify the duration of a silence [1]. Fig. 1. Musical notation (score, symbolic format, musical sheet). Musical sheet is a set of graphical signs. They give information to performers about the notes, their pitch, duration, rhythm, dynamic, signs that inform about repetition, tempo etc. Figure 2 is an example of sheet music with monophonic (one voice) melody for one instrument, three measures (pickup measure plus two measures), time signature 4/4, and sequence of notes in key signature C-major. Every note in a melody has its pitch and duration. These are two main dimensions of a note [2]. Fig. 2. Extract of soprano melody: song N.359 "Ó, Mária primluvnica naša“[4]. 3 Music Notation in Mathematical Vectors 3.1 Monophonic Music In [2] we proposed mathematical representation for music in form of vectors. We transformed melody into pitch intervals and assigned them with numbers according to the length of interval from 0 to 10. Each type of pitch interval has the length and one of two orientations (+ up / - down). Pitch intervals create melodic vector. For duration of a note stands a number. The smallest duration interval is set on 1/2, as the shortest note in the example (Figure 2) covers exactly half of a beat. Duration intervals create rhythmic vector. Music motif from Figure 2 is according to our proposed transformation represented by two separated vectors, one for pitch-interval contour and one for rhythmic contour. M 3,4,0,0,4,3,5,0,1 R 1,1,2,2,1,1,1,1,2,2 Fig. 3. Pitch interval between two consecutive notes and duration of the note. Melody transformed into set of intervals instead of exact pitch values enables searching duplicate melodies whatever scale they are composed. It allows differentiate and monitor major and minor intervals since they evoke different human emotions. Use of melodic and rhythmic vectors for music classification and evaluation is presented in [2]. How to find, compare and evaluate electronically two music files using melodic vector is demonstrated in application Melody Comparator presented in [3]. 3.1 Polyphonic Music In polyphonic music several notes may be sounded simultaneously at any one point in time. Figure 3 gives an example of short polyphonic music (up to four voices can sound at one point in time) in symbolic format. Music is read from left to right, thus it is two dimensional representation in a time. Two staves notation is typical for piano or organ. Notes in vertical alignment are read simultaneously and played together. Fig. 3. Extract of a song N.359 "Ó, Mária primluvnica naša“ [4]. Database for our research are songs from United Catholic Songbook [4]. It includes more than 500 songs. They are mostly composed for piano or organ or special songs for choir. They are written mostly for 4 voices: soprano, alto, tenor, bass. Figure 4 depicts selection of particular voices – soprano, alto, tenor and bass from piano notation in Figure 3 in four separated staves – each for one voice. Fig. 4. Extract of a song N.359 "Ó, Mária primluvnica naša“ [4] in separated voices. New term harmonic vector is introduced. Vector describes chord at one point in time. In chord sound all notes simultaneously at one point in time. Harmony in a song is built with the pitches of these notes. Intervals between particular voices create harmonic vector. The smallest pitch interval unit is semitone. Components of the harmonic vector are following pitch-intervals: soprano-alto, alto-tenor, tenor-bass. h i (wi1, wi 2,...win 1) Let h i ( wi1, wi 2,...win 1) be a harmonic vector corresponding to one of the columns in harmonic matrix. i represents number of chords within a bar and n represents number of voices within a chord respectively n-1 represents number of intervals within a chord. For example, figure 5 depicts chords in the first measure of the song N.359 [4]0, which are the base for constructing harmonic vectors. Harmonic vector for the first chord of the first bar is: h1 (0,8,4) . A similar vector q (wq1, wq 2,...wqj) will also represent a query. Harmonic vector has floating number of components. It depends on number of voices in a chord. Fig. 5. Harmonic vectors in the 1.measure of a song N.359 [4]. Let Hk be a harmonic matrix formed from harmonic vectors per bar with number k. It represents harmony at each point in time within a bar. The number of rows in matrix is given by number of voices (n) in a song respectively number of intervals between voices (n-1). Number of columns in matrix is given by max. number of notes in a voice within a bar. Hk (h1 h 2 h 3 ... h i) Harmonic matrix H1 represents harmony in the first bar of a song from Figure 5 at each point in time. It gives chord’s representation of the song, but does not give any information about melody in any voice. 0 0 0 4 0 0 H 1 8 7 5 5 12 17 4 4 3 3 7 7 Melody is followed by voices: soprano, alto, tenor and bass. Let s k ( sk1, sk 2,...ski ) be a melodic vector for soprano voice and consists of pitch intervals and interval orientation (+/-) of soprano melody. k represents the number of a bar and i represents number of ‘voice notes’ within a bar. Melodic vector for alto/tenor/bass voice consists of pitch intervals and interval orientation (+/-) of alto/tenor/bass melody. The definition of melodic vectors for all voices is as follows: s k (sk1, sk 2,...ski ) ak (ak1, ak 2,...aki ) t k (tk1, tk 2,...tki ) b k (bk1, bk 2,...bki ) Let Mk be melodic matrix formed from melodic vectors of soprano, alto, tenor and bass (SATB) voices of the bar with number k. sk ak Mk tk bk Melodic matrix formed from melodic vectors of soprano, alto, tenor and bass (SATB) voices in bar 1. 0 0 M1 0 0 5 0 0 0 0 4 0 2 0 2 0 0 0 2 0 1 0 3 0 0 0 0 4 3 Setting of a number of chords within a bar: 1. Number of chords within a bar is set by max number of notes in a voice. Voices with less notes must be filled with notes of the same pitch. Since max number of notes in a voice can vary from bar to bar, harmonic matrix can have different number of columns from bar to bar. With such representation of chord’s number we lose information about the beat of chord’s change within a bar. To get this information, next matrix about rhythm is needed. This method was used in example of music motif mentioned above. Max number of notes in bar 1 of an example in Figure 4 is in soprano voice. There are 6 notes. Other voices are filled with notes of the same pitch to get the same number of notes within a bar. Number of notes in bar 1 is set to 6 for each voice. Therefore harmonic matrix H1 consists of 6 chords (columns). Next harmonic matrix H2 will have just 2 chords, since max number of notes in a voice is 2. Next harmonic matrices are constructed similarly. 2. Number of chords within a bar is set by time signature of the song and the shortest note duration in a song. Number of chords becomes fixed and constant for all bars in a song. Harmonic matrices of particular bars will have same number of columns within a song. Voices with longer notes than the shortest note of a song must be filled with notes of the same pitch. Many chords will be repeated, but the information about the beat of chord’s change within a bar is saved. Matrix with rhythm information is not needed. In our example (Figure 4) is the time signature 4/4 and the shortest note of the song is semiquaver (duration of half a beat). This means that one bar can be filled max with 8 notes, each with duration of half a beat (4 beats / 0,5 = 8 semiquavers). Number of notes within a bar is set to 8 and is fixed for the whole song. Other voices are filled with notes of the same pitch to get the fixed number of notes within a bar. Therefore every harmonic matrix Hk in a song consists of 8 chords (columns). Our representation of polyphonic music in matrix form ignores rests in melodic and harmonic matrices. We consider rests should be taken into account in rhythmic matrix. Relations between matrices are saved in order numbers of matrices, since our database consists of one-scale songs, there are not changes in scale within a song. Melodic and rhythmic vectors follow music horizontally. Harmonic vector follows music in vertical line. By building melodic and harmonic matrixes we can receive a good mathematical representation of a song in both lines – horizontal and vertical, and it brings exact information about melody in each voice and harmony in each chord. Combination of these two matrixes eventually with matrix for rhythm will be good base for searching similarities in musical melody, rhythm or harmony. Acknowledgement: This work was partially supported by projects VG1/0508/09 and KEGA 345-032STU-4. References 1. Contributors: Illustrated Oxford Dictionary, Dorling Kindersley Limited and Oxford University Press, ISBN 0-7513-110-3, pp. 558, (1998) 2. Habudová-Hrušková, N., Representation and Pattern Matching Techniques for Music Data. In: Student Research Conference 2011, Vol. 1 : 7th Student Research Conference in Informatics and Information Technologies Bratislava, 2011 Proceedings. - Bratislava : STU v Bratislave FIIT, 2011. - ISBN 978-80-227-3484-4. - S. 125-131, (2011) 3. Hrušková, N., Hvolka, J.: Representing, comparing and evaluating of music files, Proceedings of the International Conference on E-learning and the Knowledge Society, ASE Publishing House, ISBN 978-606-505-459-2. pp. 213-218, (2011) 4. Schneider-Trnavský, M.: Jednotný katolícky spevník (United Catholic Songbook), Spolok sv. Vojtecha, ISBN 80-7162-173-0, (1996)