Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sharding with MongoDB (Eliot Horowitz)


Published on

Published in: Technology
  • Be the first to comment

Sharding with MongoDB (Eliot Horowitz)

  1. open-source, high-performance, document-oriented database
  2. Why Sharding • Vertical scaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher
  3. What is Sharding • ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)
  4. Overview • Automatic partitioning and management • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent
  5. Range Based • collection is broken into chunks by range
  6. Auto-sharding Shards mongo mongo mongo .. Config mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client
  7. Config Servers • 3 of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up
  8. Shards • Can be master, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes
  9. mongos • Sharding Router • Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network traffic
  10. Writes • Inserts : require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered
  11. Queries • By shard key: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort
  12. Operations • split: breaking a chunk into 2 • migrate: move a chunk from 1 shard to another
  13. Balancing • distributes chunks automatically • can look at: disk ops, cpu, data size
  14. Multi DataCenter • intelligent geo honing • auto failover
  15. Sharding Limitations • Unique indexes not based on shard key • current version limited to ~ 20 petabytes
  16. After Party! Download MongoDB and let us know what you think @eliothorowitz @mongodb