Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MMCF: Multimodal Collaborative Filtering for Automatic Playlist Conitnuation

231 views

Published on

The slides used for presentation in the 'ecSys challenge workshop 2018'. The challenge is co-organized by Spotify. Our team('hello world!') won the 2nd place.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

MMCF: Multimodal Collaborative Filtering for Automatic Playlist Conitnuation

  1. 1. MMCF: Multimodal Collaborative Filtering for Automatic Playlist Continuation Team ‘hello world!’ (2nd place), main track Hojin Yang*, Yoonki Jeong, Minjin Choi, and Jongwuk Lee Sungkyunkwan University, Republic of Korea
  2. 2. 2 Motivation & Challenges
  3. 3. Automatic Playlist Continuation ➢Dataset: Million Playlist Dataset (MPD) 3 Playlist title Tracks in the playlist Metadata of tracks (artist, album)
  4. 4. Challenge Set 4 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Few tracks in the first part Many tracks in the first part Many tracks in the random position
  5. 5. Challenge Set 5 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 No tracks in the playlist How to deal with an edge case?
  6. 6. Challenge Set 6 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Scarce information How to treat playlists with scarce information?
  7. 7. Challenge Set 7 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Various types of Input How to deal with various types of input?
  8. 8. 8 Proposed Model
  9. 9. Overview of the Proposed Model ➢An ensemble method with two components. ◆ Autoencoder for tracks and metadata for tracks. ◆ CharCNN for playlist titles. 9
  10. 10. Overview of the Proposed Model ➢An ensemble method with two components ◆ Autoencoder for tracks and metadata for tracks ◆ CharCNN for playlist titles 10
  11. 11. Autoencoder 11 encoder decoder Compressed representation of the input image 28X28X3 pixels ➢Learn a latent representation by reconstructing the input itself 28X28X3 pixels
  12. 12. Collaborative Autoencoder 12 1 0 1 1 0 1Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder ➢Learn a latent representation of a given playlist consisting of a set of tracks 0.9 0.01 0.78 0.9 0. 6 0.8 ✔ Top-1 Recommendation Output
  13. 13. Collaborative Autoencoder 13 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1 ➢Training with dropout ◆ Some positive input values are corrupted (set to zero).
  14. 14. Collaborative Autoencoder 14 ➢Training with dropout ◆ Some positive input values are corrupted (set to zero). How to utilize the metadata such as artists and albums? 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1
  15. 15. Utilizing Metadata 15 1 0 1 1 0 1 0 1 1 0 0.9 0.01 0.78 0.9 0. 6 0.8 0.2 0.98 0.9 0.6 Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder ➢Simply concatenate an artist vector corresponding to the track vector.
  16. 16. ➢Randomly choose either the playlist or its artists as input. 16 Training Strategy: Hide-and-Seek
  17. 17. Training Strategy: Hide-and-Seek ➢Randomly choose either the playlist or its artists as input. 17
  18. 18. Training Strategy: Hide-and-Seek ➢Randomly choose either the playlist or its artists as input. 18
  19. 19. ➢About 0.5~9.0% improvement over the baseline 19 sequential shuffled Input 1 5 10 25 100 25 100 Track 0.111 0.153 0.183 0.215 0.159 0.306 0.301 Track + Artist 0.121 0.156 0.184 0.216 0.172 0.317 0.303 Gain(%) +9.00 +1.96 +0.55 +0.47 +8.18 +3.60 +0.66 * R-precision Training Strategy: Hide-and-Seek
  20. 20. ➢How to design a dropout strategy? 20 1 0 1 1 0 1 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0.9 0.23 0.78 0.1 0.1 0.8 0.2 0.98 0.9 0.1 Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida dropout encoder decoder 1 0 1 1 0 1 0 1 1 0 Training Strategy: Dropout Scheme Hide
  21. 21. Training Strategy: Dropout Scheme ➢Dropout ratios vary in the input size. 21 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Dropout a lot Dropout a little
  22. 22. ➢Dropout can be sequential or random. 22 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Dropout the back part Dropout randomly Training Strategy: Dropout Scheme
  23. 23. CharCNN for Playlist Titles ➢An ensemble method with two components ◆ Autoencoder for tracks and metadata for tracks ◆ CharCNN for playlist titles 23
  24. 24. Character-level CNN for NLP ➢Effective for capturing spatial locality of a sequence of texts 24 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filter (3 by k ) 2.2 2.3 -1.3 0.9 max pooling Conv layer 2.3 Feature much k-dimension embedding convolutional
  25. 25. Character-level CNN for NLP ➢Effective for capturing spatial locality of a sequence of texts 25 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filters (3 by k ) convolutional 2.2 2.3 -1.3 0.9 max pooling 2.3 Feature much k-dimension embedding Conv layer 2.2 2.3 -1.3 0.9 Conv layers 2.2 2.3 -1.3 0.9 1.2 2.4 -1.1 0.4 max pooling 2.3 1.2 2.4 Feature vector convolutional
  26. 26. CharCNN for Playlist Titles ➢The playlist title is represented by a short text, implying the abstract description of a playlist. ➢Leverage character-level embedding instead of word-level embedding. 26 Conv layers Feature vector
  27. 27. Combining Two Models ➢Simplest method: 𝑤𝒊𝒕𝒆𝒎 = 0.5 and 𝑤𝒕𝒊𝒕𝒍𝒆 = 0.5 27
  28. 28. Combining Two Models ➢The accuracy of the AE relies on the number of tracks in a playlist. ◆ Dynamic: Set weights according to the number of items. 28 Items Playlist Title Chill songs 0.7 0.4 0.9 0.1 0.2 0.1 0.2 0.3 0.7 0.1 0.6 0.4 0.7 0.2 0.2 AE CNN 𝑤_𝑖𝑡𝑒𝑚 = 5 𝑤_𝑡𝑖𝑡𝑙𝑒 = 1
  29. 29. Combining Two Models ➢The weight of two models is combined dynamically depending on the size of input. ➢The playlist title is primarily useful for playlists with very few tracks. 29 sequential shuffled Title leverage 0 1 5 10 25 100 25 100 Item only - 0.121 0.156 0.184 0.216 0.172 0.317 0.303 Title only 0.078 - - - - - - - Constant(0.5) 0.078 0.131 0.158 0.181 0.211 0.161 0.310 0.292 Dynamic 0.078 0.131 0.158 0.184 0.216 0.172 0.317 0.303 Gain(%) - - - +1.6 +2.3 +6.8 +2.2 +3.7 * R-precision
  30. 30. Q&A 30 Thank you! Hojin Yang - undergraduate student, SKKU - code: github.com/hojinYang/spotify_recSys_challenge_2018 - email: hojin.yang7@gmail.com

×