open-source, high-performance,
  document-oriented database
Why Sharding

• Vertical scaling is limited by physics and
  cost
• Hard to scale vertically in the cloud
• Can scale wide...
What is Sharding

• ad-hoc partitioning
• consistent hashing (Dynamo)
• range based partitioning (BigTable/PNUTS)
Overview
• Automatic partitioning and management
• range based
• Can convert from single master to sharded
  system with 0...
Range Based



• collection is broken into chunks by range
Auto-sharding
                  Shards
          mongo   mongo    mongo
                                         ..
Config ...
Config Servers

• 3 of them
• changes are made with 2 phase commit
• if any are down, meta data goes read only
• system is ...
Shards


• Can be master, master/slave or replica sets
• Replica sets gives sharding + full auto-
  failover
• Regular mon...
mongos

• Sharding Router
• Acts just like a mongod to clients
• Can have 1 or as many as you want
• Can run on appserver ...
Writes

• Inserts : require shard key, routed
• Removes: routed and/or scattered
• Updates: routed or scattered
Queries

• By shard key: routed
• sorted by shard key: routed in order
• by non shard key: scatter gather
• sorted by non ...
Operations


• split: breaking a chunk into 2
• migrate: move a chunk from 1 shard to
  another
Balancing


• distributes chunks automatically
• can look at: disk ops, cpu, data size
Multi DataCenter


• intelligent geo honing
• auto failover
Sharding Limitations


• Unique indexes not based on shard key
• current version limited to ~ 20 petabytes
After Party!

Download MongoDB
   http://www.mongodb.org

and let us know what you think
 @eliothorowitz @mongodb
Upcoming SlideShare
Loading in...5
×

Sharding with MongoDB (Eliot Horowitz)

21,955

Published on

Published in: Technology
0 Comments
40 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
21,955
On Slideshare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
441
Comments
0
Likes
40
Embeds 0
No embeds

No notes for slide


  • sharding isn’t new












  • osa thai - open bar, food
  • Sharding with MongoDB (Eliot Horowitz)

    1. 1. open-source, high-performance, document-oriented database
    2. 2. Why Sharding • Vertical scaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher
    3. 3. What is Sharding • ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)
    4. 4. Overview • Automatic partitioning and management • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent
    5. 5. Range Based • collection is broken into chunks by range
    6. 6. Auto-sharding Shards mongo mongo mongo .. Config mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client
    7. 7. Config Servers • 3 of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up
    8. 8. Shards • Can be master, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes
    9. 9. mongos • Sharding Router • Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network traffic
    10. 10. Writes • Inserts : require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered
    11. 11. Queries • By shard key: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort
    12. 12. Operations • split: breaking a chunk into 2 • migrate: move a chunk from 1 shard to another
    13. 13. Balancing • distributes chunks automatically • can look at: disk ops, cpu, data size
    14. 14. Multi DataCenter • intelligent geo honing • auto failover
    15. 15. Sharding Limitations • Unique indexes not based on shard key • current version limited to ~ 20 petabytes
    16. 16. After Party! Download MongoDB http://www.mongodb.org and let us know what you think @eliothorowitz @mongodb
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×