Your SlideShare is downloading. ×
Spotify: P2P music-on-demand streaming
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Spotify: P2P music-on-demand streaming

4,651
views

Published on

Slides from my talk on Codebits 2010, video of the session available at http://codebits.eu/intra/s/session/125

Slides from my talk on Codebits 2010, video of the session available at http://codebits.eu/intra/s/session/125


0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,651
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
1
Comments
0
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Spotify: P2P music-on- demand streaming Ricardo Santos – Spotify @ricardovice
  • 2. Basic requirements • Lots of music • Available everywhere, with great user interfaces • More convenient than piracy • Fast start of playback • High availability (enough nines to put CDs in the basement) • Large scale (many many users)
  • 3. The music • Over 10 million tracks • Growing every day, around 10k per day • 96-320 kbps audio streams, most are Ogg Vorbis q5, 160kbps
  • 4. Accessibility • People should always be able to access music • Whenever they want • Wherever they are
  • 5. but enough with the marketing talk...
  • 6. Latency matters • High latency can be a problem, not only in First Person Shooters • Increased latency of Google searches by 100 – 400ms decreased usage by 0.2 – 0.6% (Jake Brutlag, 2009) • Slow performance is one of the major reasons users abandon services • Users don't come back
  • 7. Latency matters • Focus on low latency • On average, the human notion of “instantly” is 200ms • The median latency to begin play a track in Spotify is 265ms • Due to disk lookup, at times it's actually faster to start playing a track from network than from disk • The SLA is maintained by monitoring latency in the client
  • 8. Playing a track • Check local cache • Request first piece from Spotify servers • Meanwhile, search P2P for remainder • Switch between servers & P2P as needed • Towards the end of a track, start pre- fetching the next one via P2P rather than our servers
  • 9. When to start playing? • Trade off between stutter & latency • Look at last 15 min of transfer rates • Model as Markov chain and simulate • Coupled with some heuristics
  • 10. • Production storage is a cache with fast drives & lots of RAM • Serves the most popular content • A cache miss will generate a request to master storage • User will experience longer latency • Production storage is available in several data centers to ensure closeness to the user (latency wise) Production storage
  • 11. Master storage • Works as a DHT, with some redundancy • Contains all available tracks but has slower drives and access • Tracks are kept in several formats, adding up to around 290TB
  • 12. P2P helps • Easier to scale • Less servers • Less bandwidth • Better uptime • Less costs • Fun!
  • 13. P2P overview • Not a piracy network, all tracks are added by Spotify • Used on all desktop clients (no mobile) • Each client connected to <= 60 others • All nodes are equals (no super nodes) • A track is downloaded from several peers
  • 14. P2P custom protocol • Ask for most urgent pieces first • If a peer is slow, re-request from new peers • When buffers run low, download from central servers • If loading from servers, estimate at what point P2P will catch up • If buffers are very low, stop uploading
  • 15. P2P finding peers • Partial central tracker (BitTorrent-style) • Broadcast query in small neighborhood (Gnutella-style) • Two mechanisms results in higher availability • Limited broadcast for local (LAN) peer discovery (cherry on top...)
  • 16. P2P security • The client needs to be able to play music, but we have to prevent reverse engineering from doing the same • Therefor we can't openly discuss the details (Security Trough Obscurity) but... • Closed environment • Verify integrity of downloaded files • Data transfers are encrypted • Usernames are not exposed in P2P network, all peers assigned pseudonym
  • 17. So, what's the outcome? • At over 10 million users the responses are • 55.4% from client cache • 35.8% from the P2P network • 8.8% from the servers
  • 18. I'd like to know more... • Get in touch with us • Checkout Gunnar Kreitz's slides and academic papers on the subject: http://www.csc.kth.se/~gkreitz/spotify-p2p10/
  • 19. Oh, and we have cake as well! :D spotify.com/jobs jobs@spotify.com