Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Data Sharding   Michał Gruchała michal@gruchala.info  WebClusters 2011
TODO● Background● Theory● Practice● Summary
BackgroundMicroblogging site ● user messages (blog) ● cockpit/wallClassic architecture ● database ● web server(s) ● loadba...
BackgroundWeb servers, load balancers ● one server ● ... ● 1000 servers ● not a problemDatabase ● one database ● two datab...
BackgroundReplication ● increase read performance (raid1) ● increase data safety (raid1) ● does not increase systems capac...
BackgroundScalability ● stateless elements scale well ● stateful elements     ○ quite easy to scale        ■ if we want mo...
BackgroundSharding ;)               AB    CD        ABCD         GH               EF        EFGH        IJKL              ...
Theory
TheoryScaling ● Scale Back     ○ delete, archive unuset data ● Scale Up (vertical)     ○ more power, more disks ● Scale Ou...
TheorySharding ● split one big database into many smaller databases    ○ spread rows    ○ spread them across many servers ...
TheorySharding key ● shard by a key ● all data with that key will be on the same shard ● i.e. shard by user - all informat...
TheorySharding function ● maps keys to shards ● where to find the data ● where to store the data                    shard ...
TheorySharding function ● Dynamic    ○ Mapping in a database table  ● Fixed      ○ Moduloshard number = id % shards_count ...
TheoryAdvantages ● Linear write/read performance scalability (raid0) ● Capacity increase (raid0) ● Smaller databases are e...
TheoryChallenges ● Globally unique IDs    ○ unique across all shards       ■ auto_increment_increment, auto_increment_offs...
ChallengesRe-sharding   1,4,7    2,5,8   3,6,9     1,6      2,7    3,8      4,9      5   ● consistent hasingor   ● more sh...
ChallengesCross-shard ● queries     ○ sent to many shards     ○ collect result from one     ○ avoidable (better sharding k...
ChallengesNetwork ● more machines, more smaller streams ● full-mesh between webservers and shards ● pconnect vs. connectCo...
Practice
PracticeMicroblogging site ● see users messages ● see stream/wallClassic architecture ● database ● web server(s) ● loadbal...
Practice                                          who   whoseData                     Johns messages?     1     2id     lo...
PracticeUser ● no need for sharding                                          UserMessagesharded by user (owner field) ● sh...
Practice                 shard0                      id       owner    message   who   whose                      1       ...
PracticeBobs blog ● Bobs messages    ○ find Bobs id in User table (id = 2)    ○ find Bobs shard (2%2 = 0, shard0)    ○ fet...
Practice              shard0                   id       owner    message   who   whose                   1        2       ...
PracticeWho follows Andy ? ● find Andys id in User table (id=3) ● find Andys shard (3%2 = 1, shard1) ● hmmm
Practice              shard0                    id       owner    message   who   whose                    1        2     ...
Practice                shard0                     id       owner    message   who   whose                     1        2 ...
Summary
SummaryShard or not to shard ● many reads, little writes? - dont ● many writes and no capacity problems? - dont (use SSD) ...
SummaryIf You have to shard ● always use sharding + replication = raid10     ○ sharding reduces high availability (like ra...
Wake Up!
Upcoming SlideShare
Loading in …5
×

Data sharding

2,798 views

Published on

WebClusters'11 presenation about database sharding

Data sharding

  1. 1. Data Sharding Michał Gruchała michal@gruchala.info WebClusters 2011
  2. 2. TODO● Background● Theory● Practice● Summary
  3. 3. BackgroundMicroblogging site ● user messages (blog) ● cockpit/wallClassic architecture ● database ● web server(s) ● loadbalancer(s)
  4. 4. BackgroundWeb servers, load balancers ● one server ● ... ● 1000 servers ● not a problemDatabase ● one database ● two databases (master -> slave) ● two databases (master <-> master) ● n databases (slave(s)<-master<->master->slave(s))a lot of replication ;)
  5. 5. BackgroundReplication ● increase read performance (raid1) ● increase data safety (raid1) ● does not increase systems capacity (GBs)
  6. 6. BackgroundScalability ● stateless elements scale well ● stateful elements ○ quite easy to scale ■ if we want more reads (cache, replication) ○ hard to scale ■ if we want more writes ■ if we want more capacity
  7. 7. BackgroundSharding ;) AB CD ABCD GH EF EFGH IJKL IJ KL
  8. 8. Theory
  9. 9. TheoryScaling ● Scale Back ○ delete, archive unuset data ● Scale Up (vertical) ○ more power, more disks ● Scale Out (horizontal) ○ add machines ■ functional partitioning ■ replication ■ sharding
  10. 10. TheorySharding ● split one big database into many smaller databases ○ spread rows ○ spread them across many servers ● shared-nothing partitioning ● not a replication
  11. 11. TheorySharding key ● shard by a key ● all data with that key will be on the same shard ● i.e. shard by user - all informations connected to user are on one shard (user info, messages, friends list)user 1 -> shard 1user 2 -> shard 2user 3 -> shard 1user 4 -> shard 2 ● choosing a right key is very important!
  12. 12. TheorySharding function ● maps keys to shards ● where to find the data ● where to store the data shard number = sf(key)
  13. 13. TheorySharding function ● Dynamic ○ Mapping in a database table ● Fixed ○ Moduloshard number = id % shards_count ○ Hash + Moduloshard number = md5(email) % shards_count ○ Consistent hasing http://en.wikipedia.org/wiki/Consistent_hashing
  14. 14. TheoryAdvantages ● Linear write/read performance scalability (raid0) ● Capacity increase (raid0) ● Smaller databases are easier to manage ○ alter ○ backup/restore ○ truncate ;) ● Smaller databases are faster ○ as may fit into memory ● Cost effective ○ 80core, 20 HD, 80GB RAM vs ○ 10 x (8core, 2HD, 8GB RAM)
  15. 15. TheoryChallenges ● Globally unique IDs ○ unique across all shards ■ auto_increment_increment, auto_increment_offset ■ global IDs table ○ not unique across shards ■ IDs in dbs - not unique ■ shard_number - unique ■ global unique ID = shard_number + db ID
  16. 16. ChallengesRe-sharding 1,4,7 2,5,8 3,6,9 1,6 2,7 3,8 4,9 5 ● consistent hasingor ● more shards than machines/nodes(i.e. 100 shards on 10 machines)
  17. 17. ChallengesCross-shard ● queries ○ sent to many shards ○ collect result from one ○ avoidable (better sharding key, more sharding keys) ● joins ○ send query to many shards ○ join results in an application ○ sometimes unavoidable
  18. 18. ChallengesNetwork ● more machines, more smaller streams ● full-mesh between webservers and shards ● pconnect vs. connectComplexity ● usually sharding is done in application logic
  19. 19. Practice
  20. 20. PracticeMicroblogging site ● see users messages ● see stream/wallClassic architecture ● database ● web server(s) ● loadbalancer(s)
  21. 21. Practice who whoseData Johns messages? 1 2id login Johns follows? 3 41 John 3 22 Bob 1 33 Andy id owner message 5 24 Claire 1 2 M1 2 15 Megan 2 1 M2 1 5 3 2 M3 4 3 4 3 M4 4 1 5 2 M5
  22. 22. PracticeUser ● no need for sharding UserMessagesharded by user (owner field) ● shard_number = owner % 2Followsharded by user (who field) ● shard_number = who % 2 Message Message Follow Follow2 shards, 3 machines Follow shard0 shard1
  23. 23. Practice shard0 id owner message who whose 1 2 M1 2 1id login 3 2 M3 4 31 John 5 2 M5 4 12 Bob3 Andy shard1 who whose4 Claire 1 25 Megan id owner message 3 4 2 1 M2 3 2 4 3 M4 1 3 5 2 mapping? 1 5
  24. 24. PracticeBobs blog ● Bobs messages ○ find Bobs id in User table (id = 2) ○ find Bobs shard (2%2 = 0, shard0) ○ fetch Messages (shard0) where owner = 2 ● People Bob follows ○ find Bobs id in User table (id = 2) ○ find Bobs shard (2%2 = 0, shard0) ○ fetch whose id from Follow table (shard0) ○ fetch people info from User table
  25. 25. Practice shard0 id owner message who whose 1 2 M1 2 1id login 3 2 M3 4 31 John 5 2 M5 4 12 Bob3 Andy shard1 who whose4 Claire 1 25 Megan id owner message 3 4 2 1 M2 3 2 4 3 M4 1 3 5 2 1 5
  26. 26. PracticeWho follows Andy ? ● find Andys id in User table (id=3) ● find Andys shard (3%2 = 1, shard1) ● hmmm
  27. 27. Practice shard0 id owner message who whose 1 2 M1 2 1id login 3 2 M3 4 31 John 5 2 M5 4 12 Bob3 Andy shard1 who whose4 Claire5 Megan 1 2 id owner message 3 4 2 1 M2 3 2 Cross-shard 4 3 M4 1 3 query! 5 2 1 5
  28. 28. Practice shard0 id owner message who whose 1 2 M1 2 1id login 3 2 M3 4 31 John 5 2 M5 4 12 Bob3 Andy shard1 who whose4 Claire5 Megan 1 2 id owner message 3 4 2 1 M2 3 2 Ideas? 4 3 M4 1 3 5 2 1 5
  29. 29. Summary
  30. 30. SummaryShard or not to shard ● many reads, little writes? - dont ● many writes and no capacity problems? - dont (use SSD) ● capacity problems? - yes ● many writes and capacity problems? - yes ● scale-up is affordable? - dont shardAs You see... it depends!
  31. 31. SummaryIf You have to shard ● always use sharding + replication = raid10 ○ sharding reduces high availability (like raid0) ● more shards than You need ○ i.e. 4 machines, 100 shards ○ or dynamic allocation ● think of network capacity (full-mesh) ○ load sharding (google it ;)) ● sharding key - important! ○ cross-shard queries
  32. 32. Wake Up!

×