In this presentation, I give an overview of the machine learning algorithms behind Spotify’s extraordinarily popular Discover Weekly playlist. I provide a brief introduction to what the playlist is, explain how music recommendation engines have evolved over time, then break down the three main algorithm types powering Spotify’s recommendations: (1) collaborative filtering, (2) Natural Language Processing (NLP), and (3) Raw audio analysis. Video of the presentation can be found here: https://www.youtube.com/watch?v=PUtYNjInopA

1. Spotify Discover Weekly:
Sophia Ciocca
The machine learning behind your
music recommendations

2. What You’ll Learn:
• What is Spotify Discover Weekly?
• How does it work?
–General data flow
–Three types of Recommendation models ***

4. It’s pretty popular...

5. Music Curation is Nothing New
-- manual curation
-- manually tag attributes
-- audio analysis,
text analysis on metadata
-- collaborative filtering

6. So how does Spotify generate
recommendations?

7. Music Curation is Nothing New
-- manual curation
-- manually tag attributes
-- audio analysis,
text analysis on metadata
-- collaborative filtering
1
2
3

8. Image credit: Chris Johnson, Spotify

9. Image credit: Chris Johnson, Spotify

10. Image credit: Chris Johnson, Spotify

11. Image credit: Chris Johnson, Spotify

12. 3 Types of
Recommendation Models
1. Collaborative filtering models
(on your and other users’ behavior.)
2. Natural Language Processing (NLP) models
(on text -- e.g. music blogs/internet,
descriptions/song names)
3. Audio models
(on raw audio tracks)

13. Image credit: Chris Johnson, Spotify
1: collaborative
filtering

14. Collaborative Filtering --
the “Netflix Prize”
Recommendation Model #1: Collaborative Filtering

15. ...except Spotify doesn’t have explicit
feedback like Netflix does.
...so it uses implicit feedback
stream counts
Recommendation Model #1: Collaborative Filtering

16. Image credit: Chris Johnson, Spotify
Recommendation Model #1: Collaborative Filtering

17. But how does collaborative filtering work, with 100
million users?
Recommendation Model #1: Collaborative Filtering

18. How does Collaborative Filtering work, exactly?
Recommendation Model #1: Collaborative Filtering
user vector song vector

19. Image credit: Chris Johnson, Spotify
2: NLP
models

20. Recommendation Model #2: NLP Models
Natural Language Processing (NLP)
the ability of a computer to understand human speech.

21. Image credit: Chris Johnson, Spotify
3: audio
models

22. Why analyze the audio itself, too?
1. To improve accuracy
2. To make sure new songs are
included!
Recommendation Model #3: Raw Audio Models

23. How can we analyze raw audio files?
Recommendation Model #3: Raw Audio Models

24. Convolutional neural networks
to run over the acoustics!
(the same technology behind
facial recognition)
Recommendation Model #3: Raw Audio Models

25. Convolutional Neural Networks
Image credit: Sander Dieleman
Recommendation Model #3: Raw Audio Models

26. Image credit: Chris Johnson, Spotify
1: collaborative
filtering
2: NLP
models
3: audio
models

27. Image credit: Chris Johnson, Spotify
1: collaborative
filtering
2: NLP
models
3: audio
models

28. Additional Resources:
general spotify data flow:
https://qz.com/571007/the-magic-that-makes-spotifys-discover-weekly-playlists-
so-damn-good/
https://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-spotifys-
discover-weekly
collaborative filtering:
https://www.slideshare.net/MrChrisJohnson/collaborative-filtering-with-spark
raw audio models:
http://benanne.github.io/2014/08/05/spotify-cnns.html
detailed exploration of all 3 model types:
https://notes.variogr.am/2012/12/11/how-music-recommendation-works-and-
doesnt-work/

29. Thank you!
sophiaciocca@gmail.com