Music Recommendation and Discovery

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation and Discovery Remastered Tutorial @recsys, 2011

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @plamere @ocelma

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaINTRO CHORUS VERSE BRIDGE OUTRO

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation is important How many songs fit in my pocket? 10 Songs 1,000 Songs 10,000,000 Songs 1979 2001 2011

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaWhats so special about music?● Huge item space● Very low cost per item● Many item types● Low consumption time● Very high per-item reuse● Highly passionate users● Highly contextual usage● Consumed in sequences● Large personal collections● Doesnt require our full attention● Highly Social

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music recommendation is broken ... If you like Britney Spears you might like......Report on Pre-War IntelligenceLets look at some of theissues ....

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation?● Relevance● Novelty / Serendipity● Transparency / Trust● Reach● Context

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – cold startnew or unpopular items If you like Gregorian Chants you might like Green Day

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCold Start – New User - Enrollment

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma New User – Implicit taste dataThe Audioscrobbler

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Metadata Mismatches

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Metadata Mismatches Why?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance - The grey sheep problem

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaRelevance – Cultural Mismatches

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty and Serendipity

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaPopularity Bias - The Harry Potter Effect

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma...also known as the Coldplay effect

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty / Serendipity – the enemyHigh stakes competitions focused on relevance canreduce novelty and serendipity

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaThe Opacity Problem“If you like NiN you might like Johnny Cash” Why???

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation?If you like Norah Jones ...You might like Ravi Shankar

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation?If you like Norah Jones ...You might like her father, Ravi Shankar

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz??????? Photo cc by Mithrandir3

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Hacking the recommender

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaHelp! I’m stuck in the headThe limited reach of music recommendation 48% of recommendations Popularity 0% of recommendations 52% of recommendations 83 Artists 6,659 Artists 239,798 ArtistsStudy by Dr. Oscar Celma - MTG UPF Sales Rank

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Discovery ChallengePersonal discovery a challenge too Listener Study Listeners 5,000 Average Songs 3,500 Per User Percent of songs never 65% listened to

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation is not just shopping● It is not just for shopping, but... ● Discovery ● Exploration ● Play ● Organization ● Playlisting ● Recommendation for groups ● Devices● Doesnt have to look like a spreadsheet!

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Context: Tools for explorationIshkurs Guide to Electronic Dance Music http://techno.org/electronic-music-guide/

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Input data source ● Own data, Customer, Labels, UGC, ...● Protocol ● Ingestion format – TSV, XML, DDEX, XLS!, … ● Method – FTP, API, ... ● Frequency – Offline processing: Daily / weekly? – Data freshness! ● Documentation

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Post-processing ● Data cleaning: Duplicates, normalization ● Allow customer to use its own Ids!● Add links to external sources ● Rosetta Stone

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process● Considerations ● Allow customer to use its own IDs when using the rec. system. ● How long does it take to process the whole collection? ● Incremental updates

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based “X similar to (or influenced by) Y” Editorial metadata (Genre, Decades, Location, …) Music Genome● Collaborative filtering● Social-based● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen to X also listen to Y”● Social-based● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1]● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1] Probability distribution:● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering “people who listen toRawalso listen to Y” X plays:● Social-based Normalize to [5..1] Probability distribution:● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02● Hybrid (combination) Binary: 100100000100000011000001

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering Matrix Factorization. E.g: SVD, NMF, ...● Social-based● Content-based● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● WebMIR [Schedl, 2008]Content Reviews Lyrics Blogs Social Tags Bios Playlists ● Content-based ● Hybrid (combination)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based “X and Y sound similar”● Hybrid

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] – Cosine: mean/var (feature vectors) – Ad-hoc

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] http://xkcd.com/26/ – Cosine: mean/var (feature vectors) – Ad-hoc

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmamusic recommendation approaches● Expert-based● Collaborative filtering● Social-based● Content-based● Hybrid Weighted (linear combination) – E.g CF * 0.2 + CT * 0.4 + CB * 0.4 Cascade – E.g 1st apply CF, then reorder by CT or CB Switching ...

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Search● Metadata search Bruce*● Using filters: “Popular Irish bands from the 80s” popularity:[8.0 TO 10.0] AND iso_country:IE AND decade:1980● Audio search (and similarity) ● Query by example

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity ? ? ?Using Last.fm-360K dataset

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma SimilarityUsing Last.fm-360K dataset

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity (include feedback)Using Last.fm-360K dataset

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Beyond similarity

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● “If Paul likes Radiohead he might also like X” vs.● “If Oscar likes Radiohead he might also like Y”

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● “If Paul likes Radiohead he might also like X” vs.● “If Oscar likes Radiohead he might also like Y” SIMILARITY != RECOMMENDATION

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation● To whom are we recommending? Phoenix-2 (UK, 2006)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma lamere @ last.fm

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma mini-lameres @ last.fm● Clustering (k-means) lamere top-50 artists

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - RadioheadVs. Radiohead similar artists...

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Personalization (Itemization?)● ...but also which Radiohead era?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Analytics● Big data processing ● capture, storage, search, share, analysis and visualization● (local) Trend detection● Tastemakers● ...

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaReal-world Music Recommendation

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaContextual Web Crawl

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaAudio Processing

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaHybrid Recommendation

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma● 100 million registered users● 37 million active monthly users● More than 900,000 songs in catalog● More than 90,000 artists in catalog● More than 11 billion thumbs● More than 1.9 billion stations● 95% of the collection was played in July 2011

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCuration and Analysis

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaWeighting vectors

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaFor unknown artists

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaFor popular artists

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCountry: UKRecord Labels: Acid Jazz, Sony BMG, ColumbiaGenres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rockYears active: 1992 - presentAssociated acts: Brand New Heavies, Guru, Julian Perretta

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaCountry: UKRecord Labels: Acid Jazz, Sony BMG, ColumbiaGenres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rockYears active: 1992 - presentAssociated acts: Brand New Heavies, Guru, Julian PerrettaMood: upbeat, energeticRhythm: 120bpm, no rubato, high percusivenessKey: DmTags: acid jazz funk danceSounds like: Sereia (Tiefschwarz Radio Edit) by Mundo Azul

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma “I want some upbeat songs from unknown US bands, similar to Radiohead“http://ella.bmat.ws/collections/bmat/artists/radiohead/similar/tracks ?filter=mood:happy +speed:fast +iso_country:US +popularity:[0.0+TO+4.0]

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation"The key utility measure is user happiness. It seems reasonable to assume that relevance of the results is the most important factor: blindingly fast, useless answers do not make a user happy." – "Introduction to Information Retrieval" (Manning, Raghavan, and Schutze, 2008)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE (in music)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation● Limitations of current metrics (RMSE, P/R, ROC, Spearman Rho, Kendall Tau, etc.) ● skewness – performed on test data that users chose to rate ● do not take into account – usefulness – novelty / serendipity – topology of the (item or user) similarity graph – ...

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma EvaluationIf no RMSE then...?● Predictive Accuracy vs. Perceived Quality● Does the recommendation help the user? (user satisfaction) ● Familiarity vs. Novelty● Does the recommendation help the system? ● $$$ ● Catalog exposure

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG? ? Mean Reciprocal Rank + User feedback

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma? ?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WTF?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance Emitt Rhodes

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & RelevanceWHY as important as WHAT

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma WTF

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Other evaluation techniques rdHow can I evaluate a 3 party recommender:objective measures: coverage, reachsubject measures: Focus on precision Measure irrelevant results: The WTF test

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaThe WTF Test Why the Freakomendation?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation● Research Datasets ● Million Song Dataset (CB, Social, Lyrics, Tags and more) http://labrosa.ee.columbia.edu/millionsong/ ● Last.fm (CF) http://ocelma.net/MusicRecommendationDataset/ – Last.fm 360K users <user, artist, total plays> – Last.fm 1K users <user, timestamp, artist, song>

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring● Do not monitor (or test) only the Algorithm, but the WHOLE recommender system: KPIs● Catalog ● % matches against full catalog? ● Ingestion time? ● Availability?● Data & Algorithms ● Time computing (e.g. Matrix factorization)? ● Matrix size (e.g. ~10M x ~1M) in memory? – 10M vectors with 300 floats per vector → ~11Gb ● Time computing vector similarity O(n)?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma MonitoringUSAGE● Search assert_equal(ID(search(The The)), ID(The The))● Similarity assert(similarity(U2, REM) > 0.8) assert(similarity(AC/DC, Rebecca Black) < 0.3)● Recommendation 0) create_profile(@ocelma) 1) assert(similarity(@ocelma, U2) >= 0.8) 2) dislike(@ocelma, track(U2,Lemon)) 3) assert(similarity(@ocelma, album(U2,Zooropa)) < 0.8)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring● (web) API ● Measure query response – Jmeter, Apache Benchmark ● Process real logs – Fake (repeated) queries → fast because using cache?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Conclusions● Music Recsys is multidisciplinary ● search and filtering, musicology, data mining, machine learning, personalization, social networks, text processing, complex networks, user interaction, information visualization, and signal processing (among others!)● Music Recsys is important ● These technologies will be integral in helping the next generation of music listeners find that next favorite song ● Strong industry impact● Music Recsys is special

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research● How well do music recommenders work? ● lack of standardized data sets and objective evaluation methods● How to recognize and incorporate context into recommendations? ● listener’s context (exercising, exploring, working, driving, relaxing, and so on)

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research● How to make recommendations for all music? ● consider all music including new, unknown, and unpopular content.● What effect will automatic music recommenders have on the collective music taste?

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celmaMusic Recommendation and Discovery Remastered Tutorial @recsys, 2011

http://blog.echonest.com	1963
http://musicmachinery.com	1466
http://www.analyticsteam.com	85
http://mediamus.blogspot.com	72
http://nbcg2.blogspot.com	37
http://mediamus.blogspot.fr	26
http://www.scoop.it	25
http://a0.twimg.com	22
http://musictecaris.blogspot.com	13
http://www.musictecaris.blogspot.com	6
http://webcache.googleusercontent.com	5
http://www.bonvote.com	5
http://translate.googleusercontent.com	3
http://paper.li	2
http://fisicaiquimicade3resob2011-12.wikispaces.com	2
http://strumbox.com	2
http://www.netvibes.com	1
http://tweetedtimes.com	1
http://feeds2.feedburner.com	1
http://us-w1.rockmelt.com	1
http://musictecaris.blogspot.com.es	1

Music Recommendation and Discovery

by Paul Lamere on Oct 24, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

21 Embeds 3,739

Statistics

Music Recommendation and Discovery — Presentation Transcript