Identifying Successful Melodic Similarity Algorithms for use in Music


Published on

Margaret Cahill, Donncha Ó Maidín
Centre for Computational Musicology and Computer Music, Department of Computer Science and Information Systems, University of Limerick, Ireland.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Identifying Successful Melodic Similarity Algorithms for use in Music

  1. 1. Identifying Successful Melodic Similarity Algorithms for use in Music Retrieval Margaret Cahill Donncha Ó Maidín University of Limerick, Ireland. [email_address]
  2. 2. Introduction <ul><li>MIR - Music Information Retrieval </li></ul><ul><li>Field of research concerned with searching digitized collections of music </li></ul><ul><li>Various formats: </li></ul><ul><ul><li>raw and compressed audio </li></ul></ul><ul><ul><li>scores </li></ul></ul><ul><ul><li>MIDI </li></ul></ul>
  3. 3. Musical Scores <ul><li>This research is concerned with music in score format </li></ul><ul><li>Collections of scores online e.g. </li></ul><ul><ul><li>KernScores </li></ul></ul><ul><ul><li>MuseData </li></ul></ul><ul><li>Music notation editors and scanning software allow saving/exporting in score format </li></ul>
  4. 4. Melodic Similarity <ul><li>The ability to detect similar melodies allows: </li></ul><ul><ul><li>searching for query melodies </li></ul></ul><ul><ul><li>analysis of compositional style </li></ul></ul><ul><ul><li>analysis of musical form/structure </li></ul></ul><ul><ul><li>identification of recurring themes and motifs </li></ul></ul>
  5. 5. Current Approaches <ul><li>Many current algorithms are based on string-matching techniques from the IR world </li></ul><ul><ul><li>e.g. edit-distance algorithm </li></ul></ul><ul><ul><li>calculates the cost of turning one string into another using delete, replace, and insert actions </li></ul></ul><ul><li>Operations suited to text are not always suited to melody </li></ul>
  6. 6. Features of a score <ul><li>A wide range of features are available from a melodic score: </li></ul><ul><li>Which features </li></ul><ul><li>are important? </li></ul><ul><li>How should they </li></ul><ul><li>be combined? </li></ul><ul><li>Which algorithms </li></ul><ul><li>are suitable? </li></ul>Table 1: Some of the musical features available from a score implicit pattern of strong and weak stresses – metrical stress beaming ties tempo directions note duration phrasing accidentals ornamentation note pitch dynamics instrumentation articulation key signature rest time signature barlines clef
  7. 7. Our Approach <ul><li>Use music perception research as a guide to choosing suitable features </li></ul><ul><li>Use a test-bed of melodies for which we already know the degree of similarity </li></ul><ul><li>Identify potential algorithms </li></ul><ul><li>Test use of features, method of combining features, values of internal weights and parameters </li></ul><ul><li>Tweak algorithms to improve performance </li></ul>
  8. 8. The Listening Experiment <ul><li>Variations on Twinkle, Twinkle, Little Star for recorder by Duschenes. </li></ul><ul><li>Theme and Variations provides a natural range of similar melodies. </li></ul><ul><li>Provides a context for the listener – all melodies are related to the Theme. </li></ul><ul><li>The Theme is 8 bars long. </li></ul><ul><li>9 variations. </li></ul><ul><li>Good examples of some musical issues melodic similarity algorithms face: </li></ul><ul><ul><li>different time signatures </li></ul></ul><ul><ul><li>different keys </li></ul></ul><ul><ul><li>augmentation of theme (1 bar stretched to 2 bars) </li></ul></ul><ul><ul><li>triplets </li></ul></ul><ul><ul><li>single notes replaced by repeated rhythmic repetitions </li></ul></ul><ul><ul><li>ornate elaborations of the theme using both stepwise movements and arpeggic leaps </li></ul></ul>
  9. 9. Theme Figure 1: The first 2 bars of the Theme and Variations
  10. 10. Listening Experiment <ul><li>The Theme and each variation easily segments into 4-bar phrases. </li></ul><ul><li>Two 4-bar segments were extracted to form Part A and Part B of the experiment. </li></ul><ul><li>Pairs of melodies </li></ul><ul><li>played </li></ul><ul><li>Similarity was rated </li></ul><ul><li>on a 7-point scale. </li></ul><ul><li>Questionnaire and </li></ul><ul><li>comment sheet </li></ul><ul><li>Time = 24-30 minutes. </li></ul><ul><li>34 subjects participated. </li></ul>Table 2: Structure of the listening experiment
  11. 11. Consistency of Results <ul><li>Spearman’s correlation coefficient (correlation on the ranks) used to measure the consistency of subjects. </li></ul><ul><ul><li>used the similarity rating given by subjects for the sequential and random (repeat) playing of melodies. </li></ul></ul><ul><li>Only retained data from highly-correlated subjects: </li></ul><ul><ul><li>>.789, significant at the .01 level </li></ul></ul><ul><ul><li>Part A: 26 subjects. </li></ul></ul><ul><ul><li>Part B: 16 subjects. </li></ul></ul><ul><li>Ratings for random (repeat) playing discarded following the consistency check. </li></ul>
  12. 12. Consistency of Results <ul><li>Inter-subject correlation used as </li></ul><ul><li>a rough measure of the overall </li></ul><ul><li>reliability of the data. </li></ul><ul><ul><li>calculated from the correlation </li></ul></ul><ul><ul><li>matrices of each section </li></ul></ul><ul><ul><li>how each subject correlated </li></ul></ul><ul><ul><li>with every other subject </li></ul></ul><ul><li>Median ratings used for </li></ul><ul><li>comparison with algorithmic </li></ul><ul><li>measures of similarity. </li></ul>Table 3: Inter-subject correlation Table 4: Results of listening experiment – median ratings.
  13. 13. The Algorithms where: k = the time windows of the score p 1 , p 2 = pitch values of the first and second melodies in a window w k = the weight associated with that time window totaldur = the duration processed Figure 2: Algorithm - Ó Maidín 1998
  14. 14. The Algorithms <ul><li>processes the score in time windows </li></ul><ul><ul><li>the duration of the shortest full note at that particular point in the score. </li></ul></ul><ul><li>sum of the absolute pitch differences in each window multiplied by a corresponding weight for that window </li></ul><ul><li>weights can be duration, metrical accents,or combinations of these and other weighting factors </li></ul><ul><li>currently exploring variations of this algorithm using different combinations of musical features </li></ul><ul><li>transposition identified using the median value of the pitch differences with the associated weight value (similar to using frequency in statistics) </li></ul><ul><li>edit distance algorithm to be explored next </li></ul>
  15. 15. Comparing the Algorithmic and Human Measures of Similarity <ul><li>Two main approaches taken </li></ul><ul><li>Correlation </li></ul><ul><ul><li>gives an overall measure of performance across all variations rather than individual variations </li></ul></ul><ul><li>Normalize data, then apply metric </li></ul><ul><ul><li>z-scores or min-max normalization </li></ul></ul><ul><ul><li>can compare the algorithm performance for </li></ul></ul><ul><ul><li>individual variations </li></ul></ul><ul><ul><li>metric used to measure performance across </li></ul></ul><ul><ul><li>all variations </li></ul></ul>Figure 3: Possible metrics for use with normalized data
  16. 16. Current Work <ul><li>Have identified the use of duration and metrical stress weights for melodies by comparing algorithmic and human measure of similarity </li></ul><ul><li>Investigating the use of duration and metrical stress weights for melodies by comparing the algorithmic and human measures of similarity </li></ul><ul><li>Analyzing results through correlation and other metrics to determine which features are most successful for use in melodic similarity algorithms </li></ul>