Music Recommendation and Discovery

28,298 views

Published on

Tutorial on Music Recommendation by Oscar Celma (Gracenote) and Paul Lamere (The Echo Nest).

The world of music is changing rapidly. We are now just a few clicks away from being able to listen to nearly any song that has ever been recorded. This easy access to a nearly endless supply of music is changing how we explore, discover, share and experience music.
As the world of online music grows, music recommendation and discovery tools become an increasingly important way for music listeners to engage with music. Commercial recommenders such as Last.fm, iTunes Genius and Pandora have enjoyed commercial and critical success. But how well do these systems really work? How good are the recommendations? How far into the "long tail" do these recommenders reach?
In this tutorial we look at the current state-of-the-art in music recommendation and discovery. We examine current commercial and research systems, focusing on the advantages and the disadvantages of the various recommendation strategies. We look at some of the challenges in building music recommenders and we explore some of the novel techniques that are being used to improve future music recommendation and discovery systems.

Òscar Celma is the Chief Innovation Officer at Barcelona Music and Audio Technologies (BMAT). In 2008, Òscar obtained his Ph.D. in Computer Science and Digital Communication, in the Pompeu Fabra University (Barcelona, Spain). Òscar has a book published by Springer, titled "Music Recommendation and Discovery: The Long Tail, Long Fail and Long Play in the Music Digital Age" (2010). He holds 2 patents (US2003009344 and JP2003323188, 2002) from his work on the Vocaloid system, a singing voice-synthesizer bought by Yamaha in 2004.
Follow on Twitter: @ocelma

Paul Lamere is the Director of Developer Platform for The Echo Nest, a music intelligence company located in Boston. Paul is interested in using technology to help people explore for new and interesting music. He is active in both the music information retrieval and the recommender systems research communities. Paul authors a popular blog on music technology at MusicMachinery.com.
Follow on Twitter: @plamere

Published in: Technology, Business
1 Comment
61 Likes
Statistics
Notes
  • For the Opacity problem (slide 24) - the one piece that's wrong with you calling this one out is that the Johnny Cash track there is his cover of NIN's Hurt - a stunningly well done cover that even Trent Reznor considers superior to his original.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
28,298
On SlideShare
0
From Embeds
0
Number of Embeds
6,557
Actions
Shares
0
Downloads
586
Comments
1
Likes
61
Embeds 0
No embeds

No notes for slide

Music Recommendation and Discovery

  1. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation and Discovery Remastered Tutorial @recsys, 2011
  2. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @plamere @ocelma
  3. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  4. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  5. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation is important How many songs fit in my pocket? 10 Songs 1,000 Songs 10,000,000 Songs 1979 2001 2011
  6. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaWhats so special about music?● Huge item space● Very low cost per item● Many item types● Low consumption time● Very high per-item reuse● Highly passionate users● Highly contextual usage● Consumed in sequences● Large personal collections● Doesnt require our full attention● Highly Social
  7. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music recommendation is broken ... If you like Britney Spears you might like......Report on Pre-War IntelligenceLets look at some of theissues ....
  8. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  9. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  10. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance
  11. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – cold startnew or unpopular items If you like Gregorian Chants you might like Green Day
  12. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCold Start – New User - Enrollment
  13. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma New User – Implicit taste dataThe Audioscrobbler
  14. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Metadata Mismatches
  15. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Metadata Mismatches Why?
  16. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance - The grey sheep problem
  17. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Cultural Mismatches
  18. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  19. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty and Serendipity
  20. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaPopularity Bias - The Harry Potter Effect
  21. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma...also known as the Coldplay effect
  22. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty / Serendipity – the enemyHigh stakes competitions focused on relevance canreduce novelty and serendipity
  23. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  24. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaThe Opacity Problem“If you like NiN you might like Johnny Cash” Why???
  25. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation?If you like Norah Jones ...You might like Ravi Shankar
  26. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation?If you like Norah Jones ...You might like her father, Ravi Shankar
  27. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz??????? Photo cc by Mithrandir3
  28. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz
  29. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Hacking the recommender
  30. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  31. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaHelp! I’m stuck in the headThe limited reach of music recommendation 48% of recommendations Popularity 0% of recommendations 52% of recommendations 83 Artists 6,659 Artists 239,798 ArtistsStudy by Dr. Oscar Celma - MTG UPF Sales Rank
  32. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Discovery ChallengePersonal discovery a challenge too Listener Study Listeners 5,000 Average Songs 3,500 Per User Percent of songs never 65% listened to
  33. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context
  34. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation is not just shopping● It is not just for shopping, but... ● Discovery ● Exploration ● Play ● Organization ● Playlisting ● Recommendation for groups ● Devices● Doesnt have to look like a spreadsheet!
  35. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Context: Tools for explorationIshkurs Guide to Electronic Dance Music http://techno.org/electronic-music-guide/
  36. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  37. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  38. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  39. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Input data source ● Own data, Customer, Labels, UGC, ...● Protocol ● Ingestion format – TSV, XML, DDEX, XLS!, … ● Method – FTP, API, ... ● Frequency – Offline processing: Daily / weekly? – Data freshness! ● Documentation
  40. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Post-processing ● Data cleaning: Duplicates, normalization ● Allow customer to use its own Ids!● Add links to external sources ● Rosetta Stone
  41. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Considerations ● Allow customer to use its own IDs when using the rec. system. ● How long does it take to process the whole collection? ● Incremental updates
  42. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  43. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  44. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based● Hybrid (combination)
  45. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based “X similar to (or influenced by) Y” Editorial metadata (Genre, Decades, Location, …) Music Genome● Collaborative filtering● Social-based● Content-based● Hybrid (combination)
  46. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen to X also listen to Y”● Social-based● Content-based● Hybrid (combination)
  47. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen to X also listen to Y”● Social-based● Content-based● Hybrid (combination)
  48. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based● Content-based● Hybrid (combination)
  49. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1]● Content-based● Hybrid (combination)
  50. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1] Probability distribution:● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02● Hybrid (combination)
  51. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1] Probability distribution:● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02● Hybrid (combination) Binary: 100100000100000011000001
  52. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering Matrix Factorization. E.g: SVD, NMF, ...● Social-based● Content-based● Hybrid (combination)
  53. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● WebMIR [Schedl, 2008]Content Reviews Lyrics Blogs Social Tags Bios Playlists ● Content-based ● Hybrid (combination)
  54. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based “X and Y sound similar”● Hybrid
  55. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] – Cosine: mean/var (feature vectors) – Ad-hoc
  56. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] http://xkcd.com/26/ – Cosine: mean/var (feature vectors) – Ad-hoc
  57. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based● Hybrid Weighted (linear combination) – E.g CF * 0.2 + CT * 0.4 + CB * 0.4 Cascade – E.g 1st apply CF, then reorder by CT or CB Switching ...
  58. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  59. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  60. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Search● Metadata search Bruce*● Using filters: “Popular Irish bands from the 80s” popularity:[8.0 TO 10.0] AND iso_country:IE AND decade:1980● Audio search (and similarity) ● Query by example
  61. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity
  62. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity ? ? ?Using Last.fm-360K dataset
  63. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma SimilarityUsing Last.fm-360K dataset
  64. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity (include feedback)Using Last.fm-360K dataset
  65. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Beyond similarity
  66. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● “If Paul likes Radiohead he might also like X” vs.● “If Oscar likes Radiohead he might also like Y”
  67. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● “If Paul likes Radiohead he might also like X” vs.● “If Oscar likes Radiohead he might also like Y” SIMILARITY != RECOMMENDATION
  68. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● To whom are we recommending? Phoenix-2 (UK, 2006)
  69. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma lamere @ last.fm
  70. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma mini-lameres @ last.fm● Clustering (k-means) lamere top-50 artists
  71. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma mini-lameres @ last.fm● Clustering (k-means) lamere top-50 artists
  72. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead
  73. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - RadioheadVs. Radiohead similar artists...
  74. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - RadioheadVs. Radiohead similar artists...
  75. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - RadioheadVs. Radiohead similar artists...
  76. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - RadioheadVs. Radiohead similar artists...
  77. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Personalization (Itemization?)● ...but also which Radiohead era?
  78. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Analytics● Big data processing ● capture, storage, search, share, analysis and visualization● (local) Trend detection● Tastemakers● ...
  79. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  80. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaReal-world Music Recommendation
  81. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaReal-world Music Recommendation
  82. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaContextual Web Crawl
  83. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaAudio Processing
  84. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaHybrid Recommendation
  85. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma● 100 million registered users● 37 million active monthly users● More than 900,000 songs in catalog● More than 90,000 artists in catalog● More than 11 billion thumbs● More than 1.9 billion stations● 95% of the collection was played in July 2011
  86. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCuration and Analysis
  87. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaWeighting vectors
  88. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaFor unknown artists
  89. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaFor popular artists
  90. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  91. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  92. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCountry: UKRecord Labels: Acid Jazz, Sony BMG, ColumbiaGenres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rockYears active: 1992 - presentAssociated acts: Brand New Heavies, Guru, Julian Perretta
  93. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCountry: UKRecord Labels: Acid Jazz, Sony BMG, ColumbiaGenres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rockYears active: 1992 - presentAssociated acts: Brand New Heavies, Guru, Julian PerrettaMood: upbeat, energeticRhythm: 120bpm, no rubato, high percusivenessKey: DmTags: acid jazz funk danceSounds like: Sereia (Tiefschwarz Radio Edit) by Mundo Azul
  94. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma “I want some upbeat songs from unknown US bands, similar to Radiohead“http://ella.bmat.ws/collections/bmat/artists/radiohead/similar/tracks ?filter=mood:happy +speed:fast +iso_country:US +popularity:[0.0+TO+4.0]
  95. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  96. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation"The key utility measure is user happiness. It seems reasonable to assume that relevance of the results is the most important factor: blindingly fast, useless answers do not make a user happy." – "Introduction to Information Retrieval" (Manning, Raghavan, and Schutze, 2008)
  97. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE
  98. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE?
  99. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE
  100. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE (in music)
  101. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation● Limitations of current metrics (RMSE, P/R, ROC, Spearman Rho, Kendall Tau, etc.) ● skewness – performed on test data that users chose to rate ● do not take into account – usefulness – novelty / serendipity – topology of the (item or user) similarity graph – ...
  102. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma EvaluationIf no RMSE then...?● Predictive Accuracy vs. Perceived Quality● Does the recommendation help the user? (user satisfaction) ● Familiarity vs. Novelty● Does the recommendation help the system? ● $$$ ● Catalog exposure
  103. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG?
  104. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG? ? Mean Reciprocal Rank + User feedback
  105. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma? ?
  106. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  107. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  108. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  109. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  110. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  111. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WTF?
  112. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance Emitt Rhodes
  113. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & RelevanceWHY as important as WHAT
  114. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & RelevanceWHY as important as WHAT
  115. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & RelevanceWHY as important as WHAT
  116. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & RelevanceWHY as important as WHAT
  117. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma WTF
  118. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Other evaluation techniques rdHow can I evaluate a 3 party recommender:objective measures: coverage, reachsubject measures: Focus on precision Measure irrelevant results: The WTF test
  119. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaThe WTF Test Why the Freakomendation?
  120. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation● Research Datasets ● Million Song Dataset (CB, Social, Lyrics, Tags and more) http://labrosa.ee.columbia.edu/millionsong/ ● Last.fm (CF) http://ocelma.net/MusicRecommendationDataset/ – Last.fm 360K users <user, artist, total plays> – Last.fm 1K users <user, timestamp, artist, song>
  121. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  122. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  123. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring● Do not monitor (or test) only the Algorithm, but the WHOLE recommender system: KPIs● Catalog ● % matches against full catalog? ● Ingestion time? ● Availability?● Data & Algorithms ● Time computing (e.g. Matrix factorization)? ● Matrix size (e.g. ~10M x ~1M) in memory? – 10M vectors with 300 floats per vector → ~11Gb ● Time computing vector similarity O(n)?
  124. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma MonitoringUSAGE● Search assert_equal(ID(search(The The)), ID(The The))● Similarity assert(similarity(U2, REM) > 0.8) assert(similarity(AC/DC, Rebecca Black) < 0.3)● Recommendation 0) create_profile(@ocelma) 1) assert(similarity(@ocelma, U2) >= 0.8) 2) dislike(@ocelma, track(U2,Lemon)) 3) assert(similarity(@ocelma, album(U2,Zooropa)) < 0.8)
  125. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring● (web) API ● Measure query response – Jmeter, Apache Benchmark ● Process real logs – Fake (repeated) queries → fast because using cache?
  126. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring● (web) API ● Measure query response – Jmeter, Apache Benchmark ● Process real logs – Fake (repeated) queries → fast because using cache?
  127. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO
  128. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Conclusions● Music Recsys is multidisciplinary ● search and filtering, musicology, data mining, machine learning, personalization, social networks, text processing, complex networks, user interaction, information visualization, and signal processing (among others!)● Music Recsys is important ● These technologies will be integral in helping the next generation of music listeners find that next favorite song ● Strong industry impact● Music Recsys is special
  129. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research● How well do music recommenders work? ● lack of standardized data sets and objective evaluation methods● How to recognize and incorporate context into recommendations? ● listener’s context (exercising, exploring, working, driving, relaxing, and so on)
  130. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research● How to make recommendations for all music? ● consider all music including new, unknown, and unpopular content.● What effect will automatic music recommenders have on the collective music taste?
  131. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation and Discovery Remastered Tutorial @recsys, 2011

×