Horizontal Scalability for Great Success                          Nick Barkas                            @snb             ...
Outline        Introduction          • Spotify          • Kinds of scalability        Designing scalable network applicati...
Introductiononsdag den 22 juni 2011
What is Spotify?        On-demand music streaming service        Also play your music files, or buy mp3 downloads        P...
Overview of Spotify network                              Backend services        search                          playlist ...
Scaling vertically: “bigger” machines    +Maybe no or only small code changes    +Fewer servers is easier operationally   ...
Scaling horizontally: more machines    +You can always add more machines!    +No threads (maybe)    +Possible to run in “t...
Why horizontal for Spotify?        We are too big          • Over 13 million songs          • And over 10 million users, w...
Why the GIL is kind of a good thing        Forces horizontally scalable design        Multiple cores require multiple Pyth...
Designing scalable network applicationsonsdag den 22 juni 2011
Separate services for separate features        The UNIX way: small, simple programs doing one thing well          •     Ca...
Many instances of each service        N instances/machine where N <= # cores, many machines        Need a way to spread re...
Sharding data        Each server/instance responsible for subset of data        Can be easy if you share nothing        Mu...
Brewer’s CAP theorem                          Consistency        Availability                                     Partitio...
Brewer’s CAP theorem                          Consistency        Availability                                     Partitio...
Eventual consistency        Lots of NoSQLish options work this way          • Reads of just written data not guaranteed to...
Sometimes you need consistency        Locking, atomic operations          • Creating globally unique keys, e.g. usernames ...
Tips for many instances of a service        Processor affinity        Watch out for connection limits (e.g. in RBDMS)     ...
Related Spotify tools and methodsonsdag den 22 juni 2011
Supervision        Spotify developed daemon that launches other daemons          • Usually as many instances as cores - 1 ...
Finding services and load balancing        Each service has an SRV DNS record          •     One record with same name for...
Finding services and load balancing         Example SRV record        _frobnicator._http.example.com. 3600 SRV   10    50 ...
Distributed hash tables (DHT) in DNS         Service instances correspond to a range of hash keys                         ...
DHT DNS record examples         Ring segment, per instance      tokens.8081.frob1.example.com. 3600 TXT                   ...
Further reading about DHTs        “Chord: A scalable peer-to-peer lookup service for internet applications”          • htt...
One last thing to remember        /dev/null is web scale!                                  Image: http://www.xtranormal.co...
Questions?          Or write me something: snb@spotify.com, @snb on Twitteronsdag den 22 juni 2011
Thank youonsdag den 22 juni 2011
Upcoming SlideShare
Loading in …5
×

Spotify: Horizontal Scalability for Great Success

3,809 views

Published on

Talk for EuroPython 2011 by Nick Barkas from Spotify. Discussion of some things to consider when building a scalable network service, including details about how we handle the challenges that come along with this at Spotify

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,809
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
99
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Spotify: Horizontal Scalability for Great Success

  1. 1. Horizontal Scalability for Great Success Nick Barkas @snb EuroPython - 22 June 2011onsdag den 22 juni 2011
  2. 2. Outline Introduction • Spotify • Kinds of scalability Designing scalable network applications • Distributing work • Handling shared data Related Spotify tools and methods • Supervision • Round-robin DNS and SRV records • Distributed hash tables (DHT)onsdag den 22 juni 2011
  3. 3. Introductiononsdag den 22 juni 2011
  4. 4. What is Spotify? On-demand music streaming service Also play your music files, or buy mp3 downloads Premium subscriptions with mobile and offline support, or free with ads Create playlists available on any computer or device with Spotify Connect with friends via Facebook and send songs to each other Sync downloads and your own music files with iPods Available today in Sweden, Norway, Finland, UK, Netherlands, France, and Spainonsdag den 22 juni 2011
  5. 5. Overview of Spotify network Backend services search playlist storage user browse ... access web api point Clientsonsdag den 22 juni 2011
  6. 6. Scaling vertically: “bigger” machines +Maybe no or only small code changes +Fewer servers is easier operationally -Hardware prices don’t scale linearly -Servers can only get so big -Multithreading can be hard -Single point of failure (SPOF)onsdag den 22 juni 2011
  7. 7. Scaling horizontally: more machines +You can always add more machines! +No threads (maybe) +Possible to run in “the cloud” (EC2, Rackspace) -Need some kind of load balancer -Data sharing/synchronization can be hard -Complexity: many pieces, maybe hidden SPOFs ±Fundamental to the application’s designonsdag den 22 juni 2011
  8. 8. Why horizontal for Spotify? We are too big • Over 13 million songs • And over 10 million users, who have lots of playlists CPython kind of doesn’t give us a choice anyway • Global interpreter lock (GIL) = no simultaneous threadsonsdag den 22 juni 2011
  9. 9. Why the GIL is kind of a good thing Forces horizontally scalable design Multiple cores require multiple Python processes • Basically the same when scaling to multiple machines Multi-process apps encourage share-nothing design • Sharing nothing avoids difficult, slow synchronizationonsdag den 22 juni 2011
  10. 10. Designing scalable network applicationsonsdag den 22 juni 2011
  11. 11. Separate services for separate features The UNIX way: small, simple programs doing one thing well • Can do the same with network services • Simple applications are easier to scale • Can focus on services with high usage/availability needs • Development is fast and scalable too ‣ If well-defined interfaces between servicesonsdag den 22 juni 2011
  12. 12. Many instances of each service N instances/machine where N <= # cores, many machines Need a way to spread requests amongst instances • Hardware load balancers • Round-robin DNS • Proxy servers (Varnish, Nginx, Squid, LigHTTPD, Apache...)onsdag den 22 juni 2011
  13. 13. Sharding data Each server/instance responsible for subset of data Can be easy if you share nothing Must direct client to instance that has its data Harder if you want things like replicationonsdag den 22 juni 2011
  14. 14. Brewer’s CAP theorem Consistency Availability Partition Tolerance You only get to have one or two Image: http://thecake.info/onsdag den 22 juni 2011
  15. 15. Brewer’s CAP theorem Consistency Availability Partition Tolerance You only get to have one or two. The cake is a lie. Image: http://thecake.info/onsdag den 22 juni 2011
  16. 16. Eventual consistency Lots of NoSQLish options work this way • Reads of just written data not guaranteed to be up-to-date Example: Cassandra • Combination of ideas from Dynamo and BigTable • Available (fast writes, replication) • Partition tolerant (retries later if replica node unreachable) • Also can get consistency if willing to sacrifice the other two • But rather young project, big learning curveonsdag den 22 juni 2011
  17. 17. Sometimes you need consistency Locking, atomic operations • Creating globally unique keys, e.g. usernames • Transactions, e.g. billing PostgreSQL (and other RDBMSs) are great at this • Availability via replication, hot standby masters • Store only what you absolutely must in global databasesonsdag den 22 juni 2011
  18. 18. Tips for many instances of a service Processor affinity Watch out for connection limits (e.g. in RBDMS) Lots of processes can share memcached OS page cache for read-heavy dataonsdag den 22 juni 2011
  19. 19. Related Spotify tools and methodsonsdag den 22 juni 2011
  20. 20. Supervision Spotify developed daemon that launches other daemons • Usually as many instances as cores - 1 • Restarts supervised instance if one fails • Also restarts all instances of an application on upgrade See also: systemdonsdag den 22 juni 2011
  21. 21. Finding services and load balancing Each service has an SRV DNS record • One record with same name for each service instance • Clients (AP) resolve to find servers providing that service • Lowest priority record is chosen with weighted shuffle • Clients must retry other instances in case of failuresonsdag den 22 juni 2011
  22. 22. Finding services and load balancing Example SRV record _frobnicator._http.example.com. 3600 SRV 10 50 8081 frob1.example.com. name TTL type prio weight port host Sometimes also use Varnish or Nginx for HTTP services • Can have caching tooonsdag den 22 juni 2011
  23. 23. Distributed hash tables (DHT) in DNS Service instances correspond to a range of hash keys e7 F A 14 key k = 1c c1 E B 3f key j = bd D 9e C 68 • Distributes data among service instances ‣ Instance B owns keys in (14, 3f], E owns (9e, c1] • Redundancy? Hash again, write to replica instance • Must transition data when ring changes • Good also for non-sharded data: cache localityonsdag den 22 juni 2011
  24. 24. DHT DNS record examples Ring segment, per instance tokens.8081.frob1.example.com. 3600 TXT “00112233445566778899aabbccddeeff” name: tokens.port.host. TTL type last key Configuration of DHT config._frobnicator._http.example.com. 3600 TXT “slaves=0” name: config.srv_name. TTL type no replication config._frobnicator._http.example.com. 3600 TXT “slaves=2 redundancy=host” name: config.srv_name. TTL type three replicas on separate hostsonsdag den 22 juni 2011
  25. 25. Further reading about DHTs “Chord: A scalable peer-to-peer lookup service for internet applications” • http://portal.acm.org/citation.cfm?id=964723.383071 “Dynamo: Amazon’s Highly Available Key-value Store” • http://portal.acm.org/citation.cfm?id=1294281onsdag den 22 juni 2011
  26. 26. One last thing to remember /dev/null is web scale! Image: http://www.xtranormal.com/watch/6995033/onsdag den 22 juni 2011
  27. 27. Questions? Or write me something: snb@spotify.com, @snb on Twitteronsdag den 22 juni 2011
  28. 28. Thank youonsdag den 22 juni 2011

×