Spotify: P2P music-on-demand streaming


Published on

Slides from my talk on Codebits 2010, video of the session available at

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Spotify: P2P music-on-demand streaming

  1. 1. Spotify: P2P music-on- demand streaming Ricardo Santos – Spotify @ricardovice
  2. 2. Basic requirements • Lots of music • Available everywhere, with great user interfaces • More convenient than piracy • Fast start of playback • High availability (enough nines to put CDs in the basement) • Large scale (many many users)
  3. 3. The music • Over 10 million tracks • Growing every day, around 10k per day • 96-320 kbps audio streams, most are Ogg Vorbis q5, 160kbps
  4. 4. Accessibility • People should always be able to access music • Whenever they want • Wherever they are
  5. 5. but enough with the marketing talk...
  6. 6. Latency matters • High latency can be a problem, not only in First Person Shooters • Increased latency of Google searches by 100 – 400ms decreased usage by 0.2 – 0.6% (Jake Brutlag, 2009) • Slow performance is one of the major reasons users abandon services • Users don't come back
  7. 7. Latency matters • Focus on low latency • On average, the human notion of “instantly” is 200ms • The median latency to begin play a track in Spotify is 265ms • Due to disk lookup, at times it's actually faster to start playing a track from network than from disk • The SLA is maintained by monitoring latency in the client
  8. 8. Playing a track • Check local cache • Request first piece from Spotify servers • Meanwhile, search P2P for remainder • Switch between servers & P2P as needed • Towards the end of a track, start pre- fetching the next one via P2P rather than our servers
  9. 9. When to start playing? • Trade off between stutter & latency • Look at last 15 min of transfer rates • Model as Markov chain and simulate • Coupled with some heuristics
  10. 10. • Production storage is a cache with fast drives & lots of RAM • Serves the most popular content • A cache miss will generate a request to master storage • User will experience longer latency • Production storage is available in several data centers to ensure closeness to the user (latency wise) Production storage
  11. 11. Master storage • Works as a DHT, with some redundancy • Contains all available tracks but has slower drives and access • Tracks are kept in several formats, adding up to around 290TB
  12. 12. P2P helps • Easier to scale • Less servers • Less bandwidth • Better uptime • Less costs • Fun!
  13. 13. P2P overview • Not a piracy network, all tracks are added by Spotify • Used on all desktop clients (no mobile) • Each client connected to <= 60 others • All nodes are equals (no super nodes) • A track is downloaded from several peers
  14. 14. P2P custom protocol • Ask for most urgent pieces first • If a peer is slow, re-request from new peers • When buffers run low, download from central servers • If loading from servers, estimate at what point P2P will catch up • If buffers are very low, stop uploading
  15. 15. P2P finding peers • Partial central tracker (BitTorrent-style) • Broadcast query in small neighborhood (Gnutella-style) • Two mechanisms results in higher availability • Limited broadcast for local (LAN) peer discovery (cherry on top...)
  16. 16. P2P security • The client needs to be able to play music, but we have to prevent reverse engineering from doing the same • Therefor we can't openly discuss the details (Security Trough Obscurity) but... • Closed environment • Verify integrity of downloaded files • Data transfers are encrypted • Usernames are not exposed in P2P network, all peers assigned pseudonym
  17. 17. So, what's the outcome? • At over 10 million users the responses are • 55.4% from client cache • 35.8% from the P2P network • 8.8% from the servers
  18. 18. I'd like to know more... • Get in touch with us • Checkout Gunnar Kreitz's slides and academic papers on the subject:
  19. 19. Oh, and we have cake as well! :D