Using the Shape of Music to Compute the similarity between Symbolic Musical Pieces

451 views
359 views

Published on

We describe a new approach for computing the similarity between symbolic musical pieces, based on the differences in shape between the interpolating curves defining the pieces. We outline several requirements for a symbolic musical similarity system, and then show how our model addresses them. Results obtained by an early prototype, tested with the MIREX 2005 Symbolic Music Similarity collections, show that this approach performs well. This new model opens a new and promising line for further research on the melodic similarity problem.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
451
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using the Shape of Music to Compute the similarity between Symbolic Musical Pieces

  1. 1. Using the Shape of Music toCompute the Similarity betweenSymbolic Musical PiecesJulián Urbano, Juan Lloréns,Jorge Morato and Sonia Sánchez-Cuadradohttp://julian-urbano.infoTwitter: @julian_urbano CMMR 2010 · Málaga, Spain · June 24th
  2. 2. 2Outline• Introduction• Melodic Similarity Requirements• General Solutions to the Requirements• A Model Based on Interpolation• Implementation and Experimental Results• Conclusions and Future Work
  3. 3. 3Symbolic Melodic Similarity• Given a musical piece (i.e. query), retrieve others deemed melodically similar to it (i.e. results)• Traditional approaches? [Typke et al., 2005a] ▫ Geometry [Ukkonen et al., 2003][Typke et al., 2004] ▫ n-grams [Uitdenbogerd et al., 1999][Doraisamy et al., 2003] ▫ Alignment [Hanna et al., 2007]• What do we do? ▫ Use local an alignment algorithm ▫ whose symbols are n-grams ▫ according to a geometric substitution function
  4. 4. 4General Requirements• Any Music Information Retrieval system should meet several requirements [Selfridge-Field, 1998][Byrd et al., 2002][Mongeau et al., 1990]• Particularly focused on non-experts• We just put together and thoroughly describe traditional and well-known requirements mostly related with transposition invariance ▫ Vertical requirements (i.e. pitch) ▫ Horizontal requirements (i.e. time)
  5. 5. 5Vertical Requirements• Query [simplified riff from Layla by Dereck and the Dominos]• Octave Equivalence
  6. 6. 6Vertical Requirements (II)• Query• Degree Equality
  7. 7. 7Vertical Requirements (III)• Query• Note Equality
  8. 8. 8Vertical Requirements (IV)• Query• Pitch Variation
  9. 9. 9Vertical Requirements (V)• Query• Harmonic Similarity
  10. 10. 10Vertical Requirements (and VI)• Voice Separation
  11. 11. 11Horizontal Requirements• Query [simplified beginning from op.81 no.10 by S. Heller]• Time Signature Equivalence
  12. 12. 12Horizontal Requirements (II)• Query• Tempo Equivalence
  13. 13. 13Horizontal Requirements (III)• Query• Duration Equality
  14. 14. 14Horizontal Requirements (and IV)• Query• Duration Variation
  15. 15. 15General Vertical Solutions• Octave Equivalence ▫ Disregard octave number but consider relative changes (G5 to C6 is not the same as G5 to C5).• Degree Equality ▫ Use the degrees within the tonality• Note Equality ▫ Use actual pitch values• Some approaches use both, but key signature is not always available in SMF [Hanna et al., 2007]• The accepted solution is to consider relative pitch differences between successive notes
  16. 16. 16General Horizontal Solutions• Time Signature Equivalence ▫ Just ignore it• Tempo Equivalence ▫ Use actual note durations• Duration Equality ▫ Use score durations• Again, this information is not mandatory in SMF, and users with different expertise would prefer different approaches• It is usual to just ignore time altogether, or use the duration ratio between successive notes
  17. 17. 17A Model based on Interpolation• Consider the time-pitch plane• Arrange the notes as points in the plane, according to their pitch and duration• With different voices, get new pitch-dimensions sharing the same time dimension• Define the curve Ci(t) as the one interpolating the notes of the i-th voice (pitch-dimension)
  18. 18. 18A Model based on Interpolation (II)
  19. 19. 19A Model based on Interpolation (and III)• The similarity of two pieces is thought of as their similarity in shape• Most requirements are directly met ▫ Neither pitch nor time invariants change the shape of the curve ▫ Pitch and Duration Variations can be measured analytically
  20. 20. 20Measure of Similarity• Consider the curves as polynomials ▫ C(t)=antn+an-1tn-1+…+a1t+a0• The first derivative measures how much the shape is changing at any time• The shape dissimilarity between two curves (songs) can be measured as the area between their first derivatives
  21. 21. 21It is Metric• Non-negativity ▫ diff(C, D) ≥ 0• Identity of indiscernibles ▫ diff(C, D) = 0  C = D• Symmetry ▫ diff(C, D) = diff(D, C)• Triangle inequality ▫ diff(C, E) ≤ diff(C, D) + diff(D, E)• So we could use vantage objects [Bozkaya et al., 1999]
  22. 22. 22Interpolation with Splines• Easier to handle than Lagrange’s polynomials• They avoid the Runge’s phenomenon [de Boor, 2001]
  23. 23. 23Interpolation with Splines (II)• Defined as piece-wise functions• Very handy to measure the Pitch and Duration Variations ▫ Span durations can be normalized from 0 to 1
  24. 24. 24Interpolation with Splines (and III)• Defined as parametric functions ▫ One function per dimension• Pitch and Time can be compared separately• Voices can be isolated easily ▫ Using partial derivatives• More weight can be given to pitch than to time
  25. 25. 25First Implementation• Dynamic programming has been widely used with textual representations of music ▫ Levenshtein distance ▫ Needleman-Wunsch global alignment ▫ Smith-Waterman local alignment [Smith et al., 1981]  Shown to be the most effective [Hanna et al., 2007, 2008]• The symbols in the sequences are defined as n-grams of successive notes, according to the spans defined by the curve• The substitution score between two n-grams is the area between their curves’ derivatives
  26. 26. 26First Implementation (and II)• We used degree 3 Uniform B-Splines [de Boor, 2003] ▫ Results in spans of 4 notes (n-gram length)  Noted be effective [Doraisamy et al., 2003]• Pitch relative to the first note’s ▫ 74, 81, 72, 76 ▫ 7, -2, 2 (actually 0, 7, -2, 2)• Duration relative to the first note’s ▫ 240, 480, 240, 720 ▫ 2, 1, 3 (actually 1, 2, 1, 3 or 1/7, 2/7, 1/7, 3/7)
  27. 27. 27Results• Tested with MIREX 2005 test collections ▫ Training and evaluation collections ▫ 11 queries per collection ▫ About 550 songs per collection ▫ Partially ordered lists with relevants [Typke et al., 2005b] ▫ Effectiveness measured with ADR [Typke et al., 2006]
  28. 28. 28Results (II)• Two alternatives tested ▫ Kpitch=1 and Ktime=0 ▫ Kpitch=0.75 and Ktime=0.25  Chosen by others [Doraisamy et al., 2003][Hanna et al., 2007] Tuning Collection Avg. Min. Max. Kpitch=1 , Ktime=0 Training 0.639 0.271 0.864 Kpitch=0.75 , Ktime=0.25 Training 0.643 0.312 0.864 Kpitch=1 , Ktime=0 Evaluation 0.709 0.314 0.911 Kpitch=0.75 , Ktime=0.25 Evaluation 0.710 0.314 0.911• We found the improvement of considering time completely incidental
  29. 29. 29Results (and III)• Compared with the official MIREX 2005 results ▫ We would have ranked first ▫ Best ADR scores for 5 of the 11 queries Query Splines GAM O US TWV L(P3) L(DP) FM 190.011.224-1.1.1 0.803 0.820 0.717 0.824 0.538 0.455 0.547 0.443 400.065.784-1.1.1 0.879 0.846 0.619 0.624 0.861 0.614 0.839 0.679 450.024.802-1.1.1 0.722 0.450 0.554 0.340 0.554 0.340 0.340 0.340 600.053.475-1.1.1 0.911 0.883 0.911 0.911 0.725 0.661 0.650 0.567 600.053.481-1.1.1 0.630 0.293 0.629 0.486 0.293 0.357 0.293 0.519 600.054.278-1.1.1 0.810 0.674 0.785 0.864 0.731 0.660 0.527 0.418 600.192.742-1.1.1 0.703 0.808 0.808 0.703 0.808 0.642 0.642 0.808 700.010.059-1.1.2 0.521 0.521 0.521 0.521 0.521 0.667 0.521 0.521 700.010.591-1.4.2 0.314 0.665 0.314 0.314 0.314 0.474 0.314 0.375 702.001.406-1.1.1 0.689 0.566 0.874 0.675 0.387 0.722 0.606 0.469 703.001.021-1.1.1 0.826 0.730 0.412 0.799 0.548 0.549 0.692 0.561 Average 0.710 0.660 0.650 0.642 0.571* 0.558*** 0.543 ** 0.518*** bold for best per query, italics for best per system * for significant difference at the 0.10 level, ** at the 0.05 level and *** at the 0.01 level
  30. 30. 30Conclusions• We presented a new geometric model to compute the similarity of symbolic pieces ▫ Opens a very promising line for further research• It has a very intuitive interpretation, but not so intuitive implementation• A very early prototype has shown to perform quite well with the MIREX 2005 test collections ▫ Would have ranked first ▫ Though not significantly better than the top 3• The modeling of time is once again shown not to improve the overall effectiveness
  31. 31. 31So Now What?• We presented a very early work• We are currently improving it ▫ As of today we reach avg. ADR scores of over 0.82• Other considerations ▫ Local alignment? Domain-dependant tuning? ▫ Uniform B-Splines? Cardinal? Hermite? ▫ n-grams of length 4? Split at inflection points? ▫ Area between derivatives? Between the curves? ▫ Shape as a nominal variable (concave, convex)? ▫ Harmony: all possible paths? Polyphony?• We will see... ▫ Submitting 3 or 4 versions to MIREX 2010
  32. 32. 32And That’s It! Picture by 姒儿喵喵

×