Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels <ul><ul><li>Mohamed Sordo, Cyril Laur...
Outline <ul><li>Introduction </li></ul><ul><li>Motivation and goals </li></ul><ul><li>Evaluation </li></ul><ul><li>Demonst...
introduction <ul><li>Massive Increase in volume of online music </li></ul><ul><li>Long Tail Economics </li></ul><ul><ul><l...
introduction::  last year's work... <ul><li>Searchsounds  (ISMIR, 2006) </li></ul><ul><ul><li>Exploits MP3-blogs info </li...
motivation::  annotating music collections <ul><li>Tag sparsity </li></ul><ul><ul><li>Postal Service - “Such Great Heights...
goals <ul><li>Ease the process of annotating large music collections </li></ul><ul><ul><li>No learning from audio to tags,...
our proposal Annotated
our proposal <ul><li>“wisdom of (song) crowds” </li></ul>FOR FREE!!! Annotated
our proposal <ul><li>evaluation...? “wisdom of crowds” & feedback </li></ul>FOR FREE!!! Annotated
our proposal <ul><ul><li>Relevance feedback from the users </li></ul></ul><ul><ul><ul><li>Spencer Tunick's “wisdom of crow...
our proposal <ul><li>new song, no tags </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Quirky noise pop Cute pl...
our proposal <ul><li>propose tags (“wisdom of song crowds”) </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Qui...
our proposal <ul><li>users validate/add tags </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Quirky noise pop C...
experiments <ul><li>Ground Truth </li></ul><ul><ul><li>5481 annotated songs from Magnatune (John Buckman) </li></ul></ul><...
experiments::  results <ul><li>Some basic comments... </li></ul><ul><ul><li>Increasing number of similar songs (10, 20, 30...
demo <ul><li>Searchsounds, with users </li></ul><ul><ul><li>~250,000 full songs </li></ul></ul><ul><ul><li>~48% songs anno...
 
 
 
 
 
 
 
conclusions <ul><li>Extent annotations in a music collection </li></ul><ul><ul><li>General experiment </li></ul></ul><ul><...
questions? <ul><li>Preguntes? </li></ul><ul><li>¿Preguntas? </li></ul><ul><li>Questions ? </li></ul><ul><li>Fragen ? </li>...
Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels <ul><ul><li>Mohamed Sordo, Cyril Laur...
experiments::  metrics <ul><li>SongId usana31605 </li></ul><ul><ul><li>Manually annotated (ground truth) </li></ul></ul><u...
experiments::  results <ul><li>How does CB performs...? </li></ul>
experiments::  results <ul><li>With album, artist filtering... </li></ul>
experiments::  results <ul><li>Recall the recall </li></ul>
Upcoming SlideShare
Loading in...5
×

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Tags

2,466

Published on

In this paper we present a way to annotate music collections by exploiting audio similarity. In this sense, similarity is used to propose labels (tags) to yet unlabeled songs, based on the content–based distance between them. The main goal of our work is to ease the process of annotating huge music collections, by using content-based similarity distances as a way to propagate labels among songs.

We present two different experiments. The first one propagates labels that are related with the style of the piece, whereas the second experiment deals with mood labels. On the one hand, our approach shows that using a music collection annotated at 40% with styles, and using content– based, the collection can be automatically annotated up to 78% (that is, 40% already annotated and the rest, 38%, only using propagation), with a recall greater than 0.4. On the other hand, for a smaller music collection annotated at 30% with moods, the collection can be automatically annotated up to 65% (e.g. 30% plus 35% using propagation).

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,466
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Tags

  1. 1. Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels <ul><ul><li>Mohamed Sordo, Cyril Laurier and Òscar Celma </li></ul></ul><ul><ul><li>(Music Technology Group, UPF) </li></ul></ul>
  2. 2. Outline <ul><li>Introduction </li></ul><ul><li>Motivation and goals </li></ul><ul><li>Evaluation </li></ul><ul><li>Demonstration </li></ul><ul><li>Conclusions </li></ul>
  3. 3. introduction <ul><li>Massive Increase in volume of online music </li></ul><ul><li>Long Tail Economics </li></ul><ul><ul><li>Make everything available </li></ul></ul><ul><ul><li>Help me find it </li></ul></ul>
  4. 4. introduction:: last year's work... <ul><li>Searchsounds (ISMIR, 2006) </li></ul><ul><ul><li>Exploits MP3-blogs info </li></ul></ul><ul><ul><li>Natural Lang. Processing </li></ul></ul><ul><ul><li>Audio similarity (“more like this”) </li></ul></ul>
  5. 5. motivation:: annotating music collections <ul><li>Tag sparsity </li></ul><ul><ul><li>Postal Service - “Such Great Heights” </li></ul></ul><ul><ul><ul><li>2.4 million num. plays </li></ul></ul></ul><ul><ul><ul><li>100s of tags </li></ul></ul></ul><ul><ul><li>Mike Shupps's - “All Over Town” </li></ul></ul><ul><ul><ul><li>3 num. plays </li></ul></ul></ul><ul><ul><ul><li>0 tags </li></ul></ul></ul><ul><li>Cold start problem for new songs </li></ul>
  6. 6. goals <ul><li>Ease the process of annotating large music collections </li></ul><ul><ul><li>No learning from audio to tags, but... </li></ul></ul><ul><ul><li>Content-based similarity to propose tags among songs </li></ul></ul><ul><ul><ul><li>Timbre, rhythm, tonality, etc. </li></ul></ul></ul>
  7. 7. our proposal Annotated
  8. 8. our proposal <ul><li>“wisdom of (song) crowds” </li></ul>FOR FREE!!! Annotated
  9. 9. our proposal <ul><li>evaluation...? “wisdom of crowds” & feedback </li></ul>FOR FREE!!! Annotated
  10. 10. our proposal <ul><ul><li>Relevance feedback from the users </li></ul></ul><ul><ul><ul><li>Spencer Tunick's “wisdom of crowds” </li></ul></ul></ul>http://www.flickr.com/photos/guacamoleproject/487376758/
  11. 11. our proposal <ul><li>new song, no tags </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Quirky noise pop Cute playful Drums 80s Fast Weird Sweet Quirky Cute rock pop 90s Fun metal rock Edgy guitar thrash 90s Fierce Weird concert Loud ??? ??? ??? ??? ??? ??? thrash loud metal death thrash rock Edgy gothic heavy metal 90s Weird concert Loud
  12. 12. our proposal <ul><li>propose tags (“wisdom of song crowds”) </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Quirky noise pop Cute playful Drums 80s Fast Weird Sweet Quirky Cute rock pop 90s Fun metal rock Edgy guitar thrash 90s Fierce Weird concert Loud thrash loud metal death thrash rock Edgy gothic heavy metal 90s Weird concert Loud thrash rock 90s Loud metal Weird
  13. 13. our proposal <ul><li>users validate/add tags </li></ul>rock Indie Cute guitar Drums 90s Fast Weird twee Quirky noise pop Cute playful Drums 80s Fast Weird Sweet Quirky Cute rock pop 90s Fun metal rock Edgy guitar thrash 90s Fierce Weird concert Loud thrash loud metal death thrash rock Edgy gothic heavy metal 90s Weird concert Loud thrash rock 90s Loud metal Weird
  14. 14. experiments <ul><li>Ground Truth </li></ul><ul><ul><li>5481 annotated songs from Magnatune (John Buckman) </li></ul></ul><ul><ul><li>29 Different labels (rock, instrumental, classical, relaxing, etc.) </li></ul></ul><ul><ul><li>Avg. 3 tags per song </li></ul></ul><ul><ul><li>Only CB similarity, no user feedback </li></ul></ul><ul><li>2 experiments </li></ul><ul><ul><li>All tags </li></ul></ul><ul><ul><li>Only mood tags (smaller collection) </li></ul></ul>
  15. 15. experiments:: results <ul><li>Some basic comments... </li></ul><ul><ul><li>Increasing number of similar songs (10, 20, 30,...) </li></ul></ul><ul><ul><ul><li>Bad recall, precision </li></ul></ul></ul><ul><ul><ul><li>Best results: ~10 similars </li></ul></ul></ul><ul><ul><li>Using album, artist filtering </li></ul></ul><ul><ul><ul><li>Need to increase the number of similar songs </li></ul></ul></ul><ul><ul><ul><li>Best results: ~20 similars </li></ul></ul></ul><ul><li>Metrics </li></ul><ul><ul><li>Precision / Recall (F-measure, alpha = 2) </li></ul></ul><ul><ul><li>Spearman Rho </li></ul></ul><ul><ul><li>(Details on the paper!) </li></ul></ul>
  16. 16. demo <ul><li>Searchsounds, with users </li></ul><ul><ul><li>~250,000 full songs </li></ul></ul><ul><ul><li>~48% songs annotated, avg. 2 tags per song </li></ul></ul><ul><ul><ul><li>from CDBaby, Magnatune, etc. </li></ul></ul></ul>
  17. 24. conclusions <ul><li>Extent annotations in a music collection </li></ul><ul><ul><li>General experiment </li></ul></ul><ul><ul><ul><li>10 similar, recall >0.4 </li></ul></ul></ul><ul><ul><ul><li>40%+38% = 78% </li></ul></ul></ul><ul><ul><li>Mood experiment </li></ul></ul><ul><ul><ul><li>P@1 = 0.5 </li></ul></ul></ul><ul><ul><ul><li>30%+35% = 65% </li></ul></ul></ul><ul><li>Avoid cold start problem (new song) </li></ul><ul><li>Limitations </li></ul><ul><ul><li>General experiment: Low precision, due to (valid) proposed tags, that were not in the GT </li></ul></ul><ul><ul><li>Mood experiment: “Mysterious” label </li></ul></ul>
  18. 25. questions? <ul><li>Preguntes? </li></ul><ul><li>¿Preguntas? </li></ul><ul><li>Questions ? </li></ul><ul><li>Fragen ? </li></ul><ul><li>أسئلة ؟ </li></ul>
  19. 26. Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels <ul><ul><li>Mohamed Sordo, Cyril Laurier and Òscar Celma </li></ul></ul><ul><ul><li>(Music Technology Group, UPF) </li></ul></ul>
  20. 27. experiments:: metrics <ul><li>SongId usana31605 </li></ul><ul><ul><li>Manually annotated (ground truth) </li></ul></ul><ul><ul><ul><li>Classical, Instrumental, Baroque, Piano </li></ul></ul></ul><ul><ul><li>Proposed labels (and frequency) </li></ul></ul><ul><ul><ul><li>Instrumental (0.55), Classical (0.40), Baroque (0.26), Harpsichord (0.24) </li></ul></ul></ul><ul><ul><li>Precision = TP / (TP + FN) </li></ul></ul><ul><ul><ul><li>3 / 4 (added Harpsichord) </li></ul></ul></ul><ul><ul><li>Recall = TP / (TP + FP) </li></ul></ul><ul><ul><ul><li>3 / 4 (Piano is missing) </li></ul></ul></ul><ul><ul><li>F-measure (alpha = 2 , more weight to Recall) </li></ul></ul><ul><ul><li>Spearman-Rho </li></ul></ul>
  21. 28. experiments:: results <ul><li>How does CB performs...? </li></ul>
  22. 29. experiments:: results <ul><li>With album, artist filtering... </li></ul>
  23. 30. experiments:: results <ul><li>Recall the recall </li></ul>
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×