Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
BASS TRACK SELECTION IN MIDI FILES AND MULTIMODAL IMPLICATIONS TO MELODY Octavio Vicente & Jose M. Iñesta gPRAI Pattern Recognition and Artificial Intelligence Group Computer Music Laboratory PROJECT Description and Retrieval of Music and Sound Information Descripción y Recuperación de Información Musical y Sonora Spain introduction CONTEXT DOMAIN: • Multimedia content management • Content-based music information retrieval DATA: • Symbolic music container files (digital scores) • Multi-track MIDI files • Organized by instruments or parts in tracks • Some tracks have particular useful information introduction WHAT KIND OF USEFUL INFORMATIONS? • MELODY TRACK – – – – Melody is what we use to remember of a song Music repository indexing (music thumbnails) Fingerprinting Music similarity and retrieval • BASS TRACK – Bass is an important feature in music structure – Harmonic analysis – Rhythm analysis introduction THE PROBLEM: • Bass track selection in multi-track MIDI files using our background in melody track selection (D. Rizo et al. “A Pattern Recognition Approach for Melody Track Selection in MIDI Files”. ISMIR 2006) introduction WHAT CAN WE A PRIORI EXPECT: • MELODY TRACK – The concept of melody is somehow elusive: • Something singable • Something catchy in a song • A monophonic part easy to remember • BASS TRACK – Seems easier at first: • Low pitches involved • Monophonic, melodic, etc. • No a priori assumtions about instrumentation track description • Statistical features are extracted from every track • A feature vector represents each track • Most descriptors include normalized versions Table 1: Normalized and non normalized features used for track content description. About the track Category Track information Pitch About its content ry and classifiers im- xtracted. Those red for building up a his dictionary, a set Pitch intervals Note durations Normalized Avg Polyphony Duration Occupation Occupation Rate Number of Notes Highest Lowest Mean Standard Deviation Largest Smallest Mean Standard Deviation Longest Shortest Mean Standard Deviation Non normalized Avg Polyphony Occupation Rate Highest Lowest Mean Standard Deviation Largest Smallest Mean Standard Deviation Longest Shortest Mean Standard Deviation It is non sense to use non normalized versions of using max and min value for all the tracks in the file track description • Some descriptors prove to be useful: Average polyphony No Yes Lowest pitch No Yes Normalized no. notes No Yes Is it a bass track? The combination of these hints will permit us to assign each track a probability of being a bass track. track description • The tool for giving the probability is a Random Forest Classifier (RFC) • due to their ability for making their own feature selection – Using K trees, each Tj gives its decision dj on t – then – where (“purity”) ratio between the number of samples of the winning class for the decision leaf (Breiman, L. (2001). “Random forests”. Machine learning, 45(1): 5–32 ) experimental setup • Three data sets used (200 files each): – CL200: classical music – JZ200: jazz – KR200: pop-rock (karaoke) • Number of bass tracks per file: The system should say NO-TRACK The system should select it The system should select any of them • Number of bass and non-bass tracks in the MIDI datasets: experimental setup Tag compilation and selection Track labels Bass labels MIDI files Bass tags dictionary Bass tags Bass tracks Genre tags Classical Jazz Pop-rock RFCs Classifier Classical Classifier Jazz Classifier Pop-rock Experiment 1 Bass versus non-bass classification: given a particular track, is it a bass one? ? Experiment 1 Bass versus non-bass classification: given a particular track, is it a bass one? Experiment 2 Bass track selection: given a file, which track contains the bass part? None 1 2 3 4 … N ? Notation: 0 For solving the no bass track situation: Experiment 2 Bass track selection: given a file, which track contains the bass part? In addition to accuracy, other evaluations are computed: • FP : the classifier selects a non-bass track • TP : the selected track contains the correct bass line • FN : no track selected but the MIDI file indeed contains at least one bass track Experiment 2 Bass track selection: given a file, which track contains the bass part? Experiment 2’ Bass track selection across styles: style specificities of the bass part The test style files were not used for training Piano left hand issue (82.8%) Experiment 3 A question of multimodal nature arises: Can we use the bass track information for improving melody track detection? • Melody tracks classification is based on the corresponding estimated from the provided by the random forest using the melody tagged data, using Constraint: A first naïve approach could be 1st estimate and remove that track for selection. • PRO: it simplifies the problem (less tracks) • CON: no new information is provided Experiment 3 The proposed approach: instead of looking at , let’s consider the probabilities of being a melody conditioned also by the knowledge of how a bass looks like and the different-track constraint If we also assume that bass and melody tracks are not mutually conditioned, we reach to and Experiment 3 Results: Multimodal bass track selection: Multimodal melody track selection: Conclusions • Global statistical features and RFC have proven to be useful for other kind of tracks other than melody. • In fact, it works better (+24.4 %) for bass than for melody (seems to be easier). • Bass track characterization depends on the music genre. • Using bass information improved significantly the melody track selection. • The improvement was lower when melody was used to select bass tracks. and future works • Generalization studies are needed – conditioned by the long and tedious work of tagging and checking the ground truth in hundreds of MIDIs • Natural extension to other tracks: – instrument-based: piano, for example, but any; – role-based: solos, intros, etc. – Study *multi*modal interactions among them BASS TRACK SELECTION IN MIDI FILES AND MULTIMODAL IMPLICATIONS TO MELODY Octavio Vicente & Jose M. Iñesta gPRAI Pattern Recognition and Artificial Intelligence Group Computer Music Laboratory PROJECT Description and Retrieval of Music and Sound Information Descripción y Recuperación de Información Musical y Sonora Spain