SlideShare a Scribd company logo
1 of 30
Download to read offline
MMCF: Multimodal Collaborative Filtering
for Automatic Playlist Continuation
Team ‘hello world!’ (2nd place), main track
Hojin Yang*, Yoonki Jeong, Minjin Choi, and Jongwuk Lee
Sungkyunkwan University, Republic of Korea
2
Motivation & Challenges
Automatic Playlist Continuation
➢Dataset: Million Playlist Dataset (MPD)
3
Playlist title
Tracks
in the
playlist
Metadata of tracks
(artist, album)
Challenge Set
4
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
Few tracks in the
first part
Many tracks in the
first part
Many tracks in the
random position
Challenge Set
5
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
No tracks in the playlist
How to deal with an edge case?
Challenge Set
6
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
Scarce information
How to treat playlists with scarce
information?
Challenge Set
7
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
Various types of Input
How to deal with various types of
input?
8
Proposed Model
Overview of the Proposed Model
➢An ensemble method with two components.
◆ Autoencoder for tracks and metadata for tracks.
◆ CharCNN for playlist titles.
9
Overview of the Proposed Model
➢An ensemble method with two components
◆ Autoencoder for tracks and metadata for tracks
◆ CharCNN for playlist titles
10
Autoencoder
11
encoder decoder
Compressed representation
of the input image
28X28X3 pixels
➢Learn a latent representation by reconstructing the
input itself
28X28X3 pixels
Collaborative Autoencoder
12
1
0
1
1
0
1Hey Jude
Rehab
Yesterday
Dancing Queen
Mamma Mia
Viva la Vida
encoder decoder
➢Learn a latent representation of a given playlist
consisting of a set of tracks
0.9
0.01
0.78
0.9
0. 6
0.8
✔
Top-1
Recommendation
Output
Collaborative Autoencoder
13
1
0
1
1
0
1
1
0
1
0
0
0
0.9
0.01
0.78
0.9
0. 6
0.8Hey Jude
Rehab
Yesterday
Dancing Queen
Mamma Mia
Viva la Vida
dropout
encoder decoder
1
0
1
1
0
1
➢Training with dropout
◆ Some positive input values are corrupted (set to zero).
Collaborative Autoencoder
14
➢Training with dropout
◆ Some positive input values are corrupted (set to zero).
How to utilize the metadata such
as artists and albums?
1
0
1
1
0
1
1
0
1
0
0
0
0.9
0.01
0.78
0.9
0. 6
0.8Hey Jude
Rehab
Yesterday
Dancing Queen
Mamma Mia
Viva la Vida
dropout
encoder decoder
1
0
1
1
0
1
Utilizing Metadata
15
1
0
1
1
0
1
0
1
1
0
0.9
0.01
0.78
0.9
0. 6
0.8
0.2
0.98
0.9
0.6
Hey Jude
Rehab
Yesterday
Dancing Queen
Mamma Mia
Viva la Vida
encoder decoder
➢Simply concatenate an artist vector corresponding
to the track vector.
➢Randomly choose either the playlist or its artists as
input.
16
Training Strategy: Hide-and-Seek
Training Strategy: Hide-and-Seek
➢Randomly choose either the playlist or its artists as
input.
17
Training Strategy: Hide-and-Seek
➢Randomly choose either the playlist or its artists as
input.
18
➢About 0.5~9.0% improvement over the baseline
19
sequential shuffled
Input 1 5 10 25 100 25 100
Track 0.111 0.153 0.183 0.215 0.159 0.306 0.301
Track + Artist 0.121 0.156 0.184 0.216 0.172 0.317 0.303
Gain(%) +9.00 +1.96 +0.55 +0.47 +8.18 +3.60 +0.66
* R-precision
Training Strategy: Hide-and-Seek
➢How to design a dropout strategy?
20
1
0
1
1
0
1
0
1
1
0
1
0
1
0
0
0
0
0
0
0
0.9
0.23
0.78
0.1
0.1
0.8
0.2
0.98
0.9
0.1
Hey Jude
Rehab
Yesterday
Dancing Queen
Mamma Mia
Viva la Vida
dropout
encoder decoder
1
0
1
1
0
1
0
1
1
0
Training Strategy: Dropout Scheme
Hide
Training Strategy: Dropout Scheme
➢Dropout ratios vary in the input size.
21
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
Dropout a lot
Dropout a little
➢Dropout can be sequential or random.
22
1 2 3 4 5 6 7 8 9 10
# of tracks 0 1 5 10 5 10 25 100 25 100
Title
available
Yes Yes Yes Yes No No Yes Yes Yes Yes
Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled
# of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
Dropout
the back part
Dropout
randomly
Training Strategy: Dropout Scheme
CharCNN for Playlist Titles
➢An ensemble method with two components
◆ Autoencoder for tracks and metadata for tracks
◆ CharCNN for playlist titles
23
Character-level CNN for NLP
➢Effective for capturing spatial locality of a
sequence of texts
24
I like this
song
very
much
0.1 0.3 0.2 0.6
0.2 0.6 -1.2 -0.2
-2.1 0.2 0.1 0.4
-2.1 0.9 -3.1 1.4
0.1 0.3 -0.2 0.1
0.4 0.1 0.7 0.1
I
like
this
song
very
Filter (3 by k )
2.2
2.3
-1.3
0.9
max
pooling
Conv layer
2.3
Feature
much
k-dimension embedding
convolutional
Character-level CNN for NLP
➢Effective for capturing spatial locality of a
sequence of texts
25
I like this
song
very
much
0.1 0.3 0.2 0.6
0.2 0.6 -1.2 -0.2
-2.1 0.2 0.1 0.4
-2.1 0.9 -3.1 1.4
0.1 0.3 -0.2 0.1
0.4 0.1 0.7 0.1
I
like
this
song
very
Filters (3 by k )
convolutional
2.2
2.3
-1.3
0.9
max
pooling
2.3
Feature
much
k-dimension embedding
Conv layer
2.2
2.3
-1.3
0.9
Conv layers
2.2
2.3
-1.3
0.9
1.2
2.4
-1.1
0.4
max
pooling
2.3
1.2
2.4
Feature vector
convolutional
CharCNN for Playlist Titles
➢The playlist title is represented by a short text, implying
the abstract description of a playlist.
➢Leverage character-level embedding instead of
word-level embedding.
26
Conv layers
Feature vector
Combining Two Models
➢Simplest method: 𝑤𝒊𝒕𝒆𝒎 = 0.5 and 𝑤𝒕𝒊𝒕𝒍𝒆 = 0.5
27
Combining Two Models
➢The accuracy of the AE relies on the number of
tracks in a playlist.
◆ Dynamic: Set weights according to the number of items.
28
Items
Playlist Title
Chill songs
0.7
0.4
0.9
0.1
0.2
0.1
0.2
0.3
0.7
0.1
0.6
0.4
0.7
0.2
0.2
AE
CNN
𝑤_𝑖𝑡𝑒𝑚 = 5
𝑤_𝑡𝑖𝑡𝑙𝑒 = 1
Combining Two Models
➢The weight of two models is combined
dynamically depending on the size of input.
➢The playlist title is primarily useful for playlists with
very few tracks.
29
sequential shuffled
Title leverage 0 1 5 10 25 100 25 100
Item only - 0.121 0.156 0.184 0.216 0.172 0.317 0.303
Title only 0.078 - - - - - - -
Constant(0.5) 0.078 0.131 0.158 0.181 0.211 0.161 0.310 0.292
Dynamic 0.078 0.131 0.158 0.184 0.216 0.172 0.317 0.303
Gain(%) - - - +1.6 +2.3 +6.8 +2.2 +3.7
* R-precision
Q&A
30
Thank you!
Hojin Yang
- undergraduate student, SKKU
- code: github.com/hojinYang/spotify_recSys_challenge_2018
- email: hojin.yang7@gmail.com

More Related Content

What's hot

Interactive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and SpotifyInteractive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and Spotify
Chris Johnson
 

What's hot (20)

Music recommendations @ MLConf 2014
Music recommendations @ MLConf 2014Music recommendations @ MLConf 2014
Music recommendations @ MLConf 2014
 
Music Personalization : Real time Platforms.
Music Personalization : Real time Platforms.Music Personalization : Real time Platforms.
Music Personalization : Real time Platforms.
 
Personalizing the listening experience
Personalizing the listening experiencePersonalizing the listening experience
Personalizing the listening experience
 
Music Recommendations at Scale with Spark
Music Recommendations at Scale with SparkMusic Recommendations at Scale with Spark
Music Recommendations at Scale with Spark
 
Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018 Artwork Personalization at Netflix Fernando Amat RecSys2018
Artwork Personalization at Netflix Fernando Amat RecSys2018
 
Collaborative Filtering at Spotify
Collaborative Filtering at SpotifyCollaborative Filtering at Spotify
Collaborative Filtering at Spotify
 
Counterfactual Learning for Recommendation
Counterfactual Learning for RecommendationCounterfactual Learning for Recommendation
Counterfactual Learning for Recommendation
 
Scala Data Pipelines @ Spotify
Scala Data Pipelines @ SpotifyScala Data Pipelines @ Spotify
Scala Data Pipelines @ Spotify
 
A Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at NetflixA Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at Netflix
 
Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018
 
Big data and machine learning @ Spotify
Big data and machine learning @ SpotifyBig data and machine learning @ Spotify
Big data and machine learning @ Spotify
 
Deep learning for audio-based music recommendation
Deep learning for audio-based music recommendationDeep learning for audio-based music recommendation
Deep learning for audio-based music recommendation
 
Learning a Personalized Homepage
Learning a Personalized HomepageLearning a Personalized Homepage
Learning a Personalized Homepage
 
Homepage Personalization at Spotify
Homepage Personalization at SpotifyHomepage Personalization at Spotify
Homepage Personalization at Spotify
 
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
 
ML+Hadoop at NYC Predictive Analytics
ML+Hadoop at NYC Predictive AnalyticsML+Hadoop at NYC Predictive Analytics
ML+Hadoop at NYC Predictive Analytics
 
Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)
 
Spotify Machine Learning Solution for Music Discovery
Spotify Machine Learning Solution for Music DiscoverySpotify Machine Learning Solution for Music Discovery
Spotify Machine Learning Solution for Music Discovery
 
추천시스템 이제는 돈이 되어야 한다.
추천시스템 이제는 돈이 되어야 한다.추천시스템 이제는 돈이 되어야 한다.
추천시스템 이제는 돈이 되어야 한다.
 
Interactive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and SpotifyInteractive Recommender Systems with Netflix and Spotify
Interactive Recommender Systems with Netflix and Spotify
 

Similar to MMCF: Multimodal Collaborative Filtering for Automatic Playlist Conitnuation

Cepicky os-mapping-frameworks
Cepicky os-mapping-frameworksCepicky os-mapping-frameworks
Cepicky os-mapping-frameworks
Jachym Cepicky
 

Similar to MMCF: Multimodal Collaborative Filtering for Automatic Playlist Conitnuation (16)

Emotion and Theme Recognition of Music Using Convolutional Neural Networks
Emotion and Theme Recognition of Music Using Convolutional Neural NetworksEmotion and Theme Recognition of Music Using Convolutional Neural Networks
Emotion and Theme Recognition of Music Using Convolutional Neural Networks
 
Chord recognition mac lab presentation
Chord recognition mac lab presentationChord recognition mac lab presentation
Chord recognition mac lab presentation
 
CNN (v2).pptx
CNN (v2).pptxCNN (v2).pptx
CNN (v2).pptx
 
Audio chord recognition using deep neural networks
Audio chord recognition using deep neural networksAudio chord recognition using deep neural networks
Audio chord recognition using deep neural networks
 
Deep Learning with Audio Signals: Prepare, Process, Design, Expect
Deep Learning with Audio Signals: Prepare, Process, Design, ExpectDeep Learning with Audio Signals: Prepare, Process, Design, Expect
Deep Learning with Audio Signals: Prepare, Process, Design, Expect
 
jpg image processing nagham salim_as.ppt
jpg image processing nagham salim_as.pptjpg image processing nagham salim_as.ppt
jpg image processing nagham salim_as.ppt
 
Automatic Set List Identification and Song Segmentation of Full-Length Concer...
Automatic Set List Identification and Song Segmentation of Full-Length Concer...Automatic Set List Identification and Song Segmentation of Full-Length Concer...
Automatic Set List Identification and Song Segmentation of Full-Length Concer...
 
Practical Deep Learning Using Tensor Flow - Sandeep Kath
Practical Deep Learning Using Tensor Flow - Sandeep KathPractical Deep Learning Using Tensor Flow - Sandeep Kath
Practical Deep Learning Using Tensor Flow - Sandeep Kath
 
Recent Advances in Natural Language Processing
Recent Advances in Natural Language ProcessingRecent Advances in Natural Language Processing
Recent Advances in Natural Language Processing
 
Cepicky os-mapping-frameworks
Cepicky os-mapping-frameworksCepicky os-mapping-frameworks
Cepicky os-mapping-frameworks
 
Splice site recognition among different organisms
Splice site recognition among different organismsSplice site recognition among different organisms
Splice site recognition among different organisms
 
Genome assembly: then and now — v1.2
Genome assembly: then and now — v1.2Genome assembly: then and now — v1.2
Genome assembly: then and now — v1.2
 
Row Pattern Matching 12c MATCH_RECOGNIZE OOW14
Row Pattern Matching 12c MATCH_RECOGNIZE OOW14Row Pattern Matching 12c MATCH_RECOGNIZE OOW14
Row Pattern Matching 12c MATCH_RECOGNIZE OOW14
 
Chpater 6
Chpater 6Chpater 6
Chpater 6
 
Hsjs.pdf
Hsjs.pdfHsjs.pdf
Hsjs.pdf
 
Dsoop (co 221) 1
Dsoop (co 221) 1Dsoop (co 221) 1
Dsoop (co 221) 1
 

Recently uploaded

原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
great91
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
yulianti213969
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
acoha1
 
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
aqpto5bt
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
yulianti213969
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
acoha1
 
Go paperless and transform your procurement process with the Hive Collaborati...
Go paperless and transform your procurement process with the Hive Collaborati...Go paperless and transform your procurement process with the Hive Collaborati...
Go paperless and transform your procurement process with the Hive Collaborati...
LitoGarin1
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
Amil baba
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
dq9vz1isj
 

Recently uploaded (20)

原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
 
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书)圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
一比一原版(ucla文凭证书)加州大学洛杉矶分校毕业证学历认证官方成绩单
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
 
Go paperless and transform your procurement process with the Hive Collaborati...
Go paperless and transform your procurement process with the Hive Collaborati...Go paperless and transform your procurement process with the Hive Collaborati...
Go paperless and transform your procurement process with the Hive Collaborati...
 
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
 
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024
 
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam DunksNOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic information
 

MMCF: Multimodal Collaborative Filtering for Automatic Playlist Conitnuation

  • 1. MMCF: Multimodal Collaborative Filtering for Automatic Playlist Continuation Team ‘hello world!’ (2nd place), main track Hojin Yang*, Yoonki Jeong, Minjin Choi, and Jongwuk Lee Sungkyunkwan University, Republic of Korea
  • 3. Automatic Playlist Continuation ➢Dataset: Million Playlist Dataset (MPD) 3 Playlist title Tracks in the playlist Metadata of tracks (artist, album)
  • 4. Challenge Set 4 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Few tracks in the first part Many tracks in the first part Many tracks in the random position
  • 5. Challenge Set 5 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 No tracks in the playlist How to deal with an edge case?
  • 6. Challenge Set 6 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Scarce information How to treat playlists with scarce information?
  • 7. Challenge Set 7 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Various types of Input How to deal with various types of input?
  • 9. Overview of the Proposed Model ➢An ensemble method with two components. ◆ Autoencoder for tracks and metadata for tracks. ◆ CharCNN for playlist titles. 9
  • 10. Overview of the Proposed Model ➢An ensemble method with two components ◆ Autoencoder for tracks and metadata for tracks ◆ CharCNN for playlist titles 10
  • 11. Autoencoder 11 encoder decoder Compressed representation of the input image 28X28X3 pixels ➢Learn a latent representation by reconstructing the input itself 28X28X3 pixels
  • 12. Collaborative Autoencoder 12 1 0 1 1 0 1Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder ➢Learn a latent representation of a given playlist consisting of a set of tracks 0.9 0.01 0.78 0.9 0. 6 0.8 ✔ Top-1 Recommendation Output
  • 13. Collaborative Autoencoder 13 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1 ➢Training with dropout ◆ Some positive input values are corrupted (set to zero).
  • 14. Collaborative Autoencoder 14 ➢Training with dropout ◆ Some positive input values are corrupted (set to zero). How to utilize the metadata such as artists and albums? 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1
  • 15. Utilizing Metadata 15 1 0 1 1 0 1 0 1 1 0 0.9 0.01 0.78 0.9 0. 6 0.8 0.2 0.98 0.9 0.6 Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder ➢Simply concatenate an artist vector corresponding to the track vector.
  • 16. ➢Randomly choose either the playlist or its artists as input. 16 Training Strategy: Hide-and-Seek
  • 17. Training Strategy: Hide-and-Seek ➢Randomly choose either the playlist or its artists as input. 17
  • 18. Training Strategy: Hide-and-Seek ➢Randomly choose either the playlist or its artists as input. 18
  • 19. ➢About 0.5~9.0% improvement over the baseline 19 sequential shuffled Input 1 5 10 25 100 25 100 Track 0.111 0.153 0.183 0.215 0.159 0.306 0.301 Track + Artist 0.121 0.156 0.184 0.216 0.172 0.317 0.303 Gain(%) +9.00 +1.96 +0.55 +0.47 +8.18 +3.60 +0.66 * R-precision Training Strategy: Hide-and-Seek
  • 20. ➢How to design a dropout strategy? 20 1 0 1 1 0 1 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0.9 0.23 0.78 0.1 0.1 0.8 0.2 0.98 0.9 0.1 Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1 0 1 1 0 Training Strategy: Dropout Scheme Hide
  • 21. Training Strategy: Dropout Scheme ➢Dropout ratios vary in the input size. 21 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Dropout a lot Dropout a little
  • 22. ➢Dropout can be sequential or random. 22 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Dropout the back part Dropout randomly Training Strategy: Dropout Scheme
  • 23. CharCNN for Playlist Titles ➢An ensemble method with two components ◆ Autoencoder for tracks and metadata for tracks ◆ CharCNN for playlist titles 23
  • 24. Character-level CNN for NLP ➢Effective for capturing spatial locality of a sequence of texts 24 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filter (3 by k ) 2.2 2.3 -1.3 0.9 max pooling Conv layer 2.3 Feature much k-dimension embedding convolutional
  • 25. Character-level CNN for NLP ➢Effective for capturing spatial locality of a sequence of texts 25 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filters (3 by k ) convolutional 2.2 2.3 -1.3 0.9 max pooling 2.3 Feature much k-dimension embedding Conv layer 2.2 2.3 -1.3 0.9 Conv layers 2.2 2.3 -1.3 0.9 1.2 2.4 -1.1 0.4 max pooling 2.3 1.2 2.4 Feature vector convolutional
  • 26. CharCNN for Playlist Titles ➢The playlist title is represented by a short text, implying the abstract description of a playlist. ➢Leverage character-level embedding instead of word-level embedding. 26 Conv layers Feature vector
  • 27. Combining Two Models ➢Simplest method: 𝑤𝒊𝒕𝒆𝒎 = 0.5 and 𝑤𝒕𝒊𝒕𝒍𝒆 = 0.5 27
  • 28. Combining Two Models ➢The accuracy of the AE relies on the number of tracks in a playlist. ◆ Dynamic: Set weights according to the number of items. 28 Items Playlist Title Chill songs 0.7 0.4 0.9 0.1 0.2 0.1 0.2 0.3 0.7 0.1 0.6 0.4 0.7 0.2 0.2 AE CNN 𝑤_𝑖𝑡𝑒𝑚 = 5 𝑤_𝑡𝑖𝑡𝑙𝑒 = 1
  • 29. Combining Two Models ➢The weight of two models is combined dynamically depending on the size of input. ➢The playlist title is primarily useful for playlists with very few tracks. 29 sequential shuffled Title leverage 0 1 5 10 25 100 25 100 Item only - 0.121 0.156 0.184 0.216 0.172 0.317 0.303 Title only 0.078 - - - - - - - Constant(0.5) 0.078 0.131 0.158 0.181 0.211 0.161 0.310 0.292 Dynamic 0.078 0.131 0.158 0.184 0.216 0.172 0.317 0.303 Gain(%) - - - +1.6 +2.3 +6.8 +2.2 +3.7 * R-precision
  • 30. Q&A 30 Thank you! Hojin Yang - undergraduate student, SKKU - code: github.com/hojinYang/spotify_recSys_challenge_2018 - email: hojin.yang7@gmail.com