Music Recommendation and Discovery

recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

Music Recommendation and Discovery
Remastered

Tutorial

@recsys, 2011


@plamere @ocelma


INTRO CHORUS VERSE BRIDGE OUTRO


Music Recommendation is important

How many songs fit in my pocket?

10 Songs 1,000 Songs 10,000,000 Songs
1979 2001 2011


What's so special about music?
● Huge item space
● Very low cost per item
● Many item types
● Low consumption time
● Very high per-item reuse
● Highly passionate users
● Highly contextual usage
● Consumed in sequences
● Large personal collections
● Doesn't require our full attention
● Highly Social


Music recommendation is broken ...
If you like Britney Spears
you might like...

...Report on Pre-War Intelligence

Let's look at some of the
issues ....


What makes a good music recommendation?
● Relevance
● Novelty / Serendipity
● Transparency / Trust
● Reach
● Context


Relevance


Relevance – cold start
new or unpopular items
If you like Gregorian Chants you might like Green Day


Cold Start – New User - Enrollment


New User – Implicit taste data
The Audioscrobbler


Relevance – Metadata Mismatches


Relevance – Metadata Mismatches

Why?


Relevance - The grey sheep problem


Relevance – Cultural Mismatches


What makes a good music recommendation?

● Relevance
● Novelty / Serendipity
● Transparency / Trust
● Reach
● Context


Novelty and Serendipity


Popularity Bias - The Harry Potter Effect


...also known as the Coldplay effect


Novelty / Serendipity – the enemy

High stakes competitions focused on relevance can
reduce novelty and serendipity


The Opacity Problem
“If you like NiN you might like Johnny Cash”

Why???


Is this a good recommendation?

If you like Norah Jones ...

You might like Ravi Shankar


Is this a good recommendation?

If you like Norah Jones ...

You might like her father, Ravi Shankar


Brutal Death Metal Quiz

???????

Photo cc by Mithrandir3


Brutal Death Metal Quiz


Hacking the recommender


Help! I’m stuck in the head
The limited reach of music recommendation

48% of recommendations
Popularity

0% of
recommendations

52% of recommendations

83 Artists 6,659 Artists 239,798 Artists

Study by Dr. Oscar Celma - MTG UPF
Sales Rank


Music Discovery Challenge
Personal discovery a challenge too

Listener Study
Listeners 5,000

Average Songs
3,500
Per User

Percent of
songs never 65%
listened to


Music Recommendation is not just
shopping
● It is not just for shopping, but...
● Discovery
● Exploration
● Play
● Organization
● Playlisting
● Recommendation for groups
● Devices
● Doesn't have to look like a spreadsheet!


Context: Tools for exploration
Ishkur's Guide to Electronic Dance Music

http://techno.org/electronic-music-guide/


Ingestion process
● Input data source
● Own data, Customer, Labels, UGC, ...
● Protocol
● Ingestion format
– TSV, XML, DDEX, XLS!, …
● Method
– FTP, API, ...
● Frequency
– Offline processing: Daily / weekly?
– Data freshness!
● Documentation


Ingestion process
● Post-processing
● Data cleaning: Duplicates, normalization
● Allow customer to use its own Ids!
● Add links to external sources
● Rosetta Stone


Ingestion process
● Considerations
● Allow customer to use its own IDs when using the
rec. system.
● How long does it take to process the whole
collection?
● Incremental updates


music recommendation approaches
● Expert-based
● Collaborative filtering
● Social-based
● Content-based
● Hybrid (combination)


● Expert-based
“X similar to (or influenced by) Y”
Editorial metadata (Genre, Decades, Location, …)
Music Genome
● Social-based
● Content-based


● Expert-based
“people who listen to X also listen to Y”
● Social-based
● Content-based


● Expert-based
“people who listen toRawalso listen to Y”
X plays:
● Social-based
● Content-based


● Expert-based
X plays:
● Social-based Normalize to [5..1]

● Content-based


● Expert-based
X plays:
Probability distribution:
● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02


● Expert-based
X plays:
Probability distribution:
● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02
● Hybrid (combination) Binary:
100100000100000011000001


● Expert-based
Matrix Factorization. E.g: SVD, NMF, ...
● Social-based
● Content-based


● Expert-based
● Social-based
●
WebMIR [Schedl, 2008]

Content Reviews Lyrics Blogs Social Tags Bios Playlists

● Content-based


● Expert-based
● Social-based
● Content-based
“X and Y sound similar”
● Hybrid


● Expert-based
● Social-based
● Content-based
Audio features
– Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon,
2005], Harmony [Gomez, 2006], ...
Similarity
– KL-divergence: GMM [Aucouturier, 2002]
– EMD [Logan, 2001]
– Euclidean: PCA [Cano, 2005]
– Cosine: mean/var (feature vectors)
– Ad-hoc


● Expert-based
● Social-based
● Content-based
Audio features
– Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon,
2005], Harmony [Gomez, 2006], ...
Similarity
– KL-divergence: GMM [Aucouturier, 2002]
– EMD [Logan, 2001]
– Euclidean: PCA [Cano, 2005]
http://xkcd.com/26/
– Cosine: mean/var (feature vectors)
– Ad-hoc


● Expert-based
● Social-based
● Content-based
● Hybrid
Weighted (linear combination)
– E.g CF * 0.2 + CT * 0.4 + CB * 0.4
Cascade
– E.g 1st apply CF, then reorder by CT or CB
Switching
...


Search
● Metadata search
Bruce*

● Using filters: “Popular Irish bands from the 80s”
popularity:[8.0 TO 10.0] AND
iso_country:IE AND decade:1980

● Audio search (and similarity)
● Query by example


Similarity


Similarity

? ? ?

Using Last.fm-360K dataset


Similarity



Similarity' (include feedback)



Beyond similarity


Recommendation
● “If Paul likes Radiohead he might also like X”
vs.
● “If Oscar likes Radiohead he might also like Y”


Recommendation
● “If Paul likes Radiohead he might also like X”
vs.
● “If Oscar likes Radiohead he might also like Y”

SIMILARITY != RECOMMENDATION


Recommendation
● To whom are we recommending? Phoenix-2 (UK, 2006)


lamere @ last.fm


mini-lamere's @ last.fm
●
Clustering (k-means) lamere top-50 artists


@lamere - Radiohead


@lamere - Radiohead

Vs. Radiohead similar artists...


Personalization (Itemization?)
● ...but also which Radiohead era?


Analytics
● Big data processing
● capture, storage, search, share, analysis and
visualization
● (local) Trend detection
● Tastemakers
● ...


Real-world Music Recommendation


Contextual Web Crawl


Audio Processing


Hybrid Recommendation


● 100 million registered users
● 37 million active monthly users

● More than 900,000 songs in catalog

● More than 90,000 artists in catalog

● More than 11 billion thumbs

● More than 1.9 billion stations

● 95% of the collection was played in July 2011


Curation and Analysis


Weighting vectors


For unknown artists


For popular artists


Country: UK
Record Labels: Acid Jazz, Sony BMG, Columbia
Genres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock
Years active: 1992 - present
Associated acts: Brand New Heavies, Guru, Julian Perretta


Country: UK
Record Labels: Acid Jazz, Sony BMG, Columbia
Genres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock
Years active: 1992 - present
Associated acts: Brand New Heavies, Guru, Julian Perretta

Mood: upbeat, energetic
Rhythm: 120bpm, no rubato, high percusiveness
Key: Dm
Tags: acid jazz funk dance
Sounds like: Sereia (Tiefschwarz Radio Edit) by Mundo Azul


“I want some upbeat songs from unknown US bands,
similar to Radiohead“

http://ella.bmat.ws/collections/bmat/artists/radiohead/similar/tracks
?filter=mood:happy
+speed:fast
+iso_country:US
+popularity:[0.0+TO+4.0]


Evaluation
"The key utility measure is user happiness. It seems
reasonable to assume that relevance of the results is
the most important factor: blindingly fast, useless
answers do not make a user happy."

– "Introduction to Information Retrieval"
(Manning, Raghavan, and Schutze, 2008)


RMSE


RMSE?


NO RMSE


NO RMSE
(in music)


Evaluation
● Limitations of current metrics (RMSE, P/R, ROC,
Spearman Rho, Kendall Tau, etc.)
● skewness
– performed on test data that users chose to rate
● do not take into account
– usefulness
– novelty / serendipity
– topology of the (item or user) similarity graph
– ...


Evaluation
If no RMSE then...?
● Predictive Accuracy vs. Perceived Quality
● Does the recommendation help the user? (user
satisfaction)
● Familiarity vs. Novelty
● Does the recommendation help the system?
● $$$
● Catalog exposure


NEXT SONG?


NEXT SONG?

?

Mean Reciprocal Rank
+
User feedback


? ?


Novelty & Relevance


Novelty & Relevance

WTF?


Novelty & Relevance

Emitt Rhodes


Novelty & Relevance

WHY as important as WHAT


WTF


Other evaluation techniques
rd
How can I evaluate a 3 party recommender:

objective measures:
coverage, reach

subject measures:
Focus on precision
Measure irrelevant results: The WTF test


The WTF Test

Why the Freakomendation?


Evaluation
● Research Datasets
● Million Song Dataset (CB, Social, Lyrics, Tags and
more)
http://labrosa.ee.columbia.edu/millionsong/

● Last.fm (CF)
http://ocelma.net/MusicRecommendationDataset/
– Last.fm 360K users <user, artist, total plays>
– Last.fm 1K users <user, timestamp, artist, song>


Monitoring
● Do not monitor (or test) only the Algorithm, but the
WHOLE recommender system: KPIs
● Catalog
● % matches against full catalog?
● Ingestion time?
● Availability?
● Data & Algorithms
● Time computing (e.g. Matrix factorization)?
● Matrix size (e.g. ~10M x ~1M) in memory?
– 10M vectors with 300 floats per vector → ~11Gb
●
Time computing vector similarity O(n)?


Monitoring
USAGE
● Search
assert_equal(ID(search('The The')), ID('The The'))

● Similarity
assert(similarity(U2, REM) > 0.8)
assert(similarity(AC/DC, Rebecca Black) < 0.3)

● Recommendation
0) create_profile(@ocelma)
1) assert(similarity(@ocelma, U2) >= 0.8)
2) dislike(@ocelma, track(U2,Lemon))
3) assert(similarity(@ocelma, album(U2,Zooropa)) < 0.8)


Monitoring
● (web) API
● Measure query response
– Jmeter, Apache Benchmark
● Process real logs
– Fake (repeated) queries → fast because using cache?


Conclusions
● Music Recsys is multidisciplinary
● search and filtering, musicology, data mining,
machine learning, personalization, social networks,
text processing, complex networks, user interaction,
information visualization, and signal processing
(among others!)
● Music Recsys is important
● These technologies will be integral in helping the next
generation of music listeners find that next favorite
song
● Strong industry impact
● Music Recsys is special


Further research
● How well do music recommenders work?
● lack of standardized data sets and objective
evaluation methods

● How to recognize and incorporate context into
recommendations?
● listener’s context (exercising, exploring, working,
driving, relaxing, and so on)


Further research
● How to make recommendations for all music?
● consider all music including new, unknown, and
unpopular content.

● What effect will automatic music recommenders
have on the collective music taste?

Music Recommendation and Discovery

Music Recommendation and Discovery

Recommended

More Related Content

Viewers also liked

Viewers also liked(20)

More from Paul Lamere

More from Paul Lamere(10)

Recently uploaded

Recently uploaded(20)

Music Recommendation and Discovery