Your SlideShare is downloading. ×

Sharding with MongoDB (Eliot Horowitz)

2,238

Published on

Eliot's presentation at MongoUK

Eliot's presentation at MongoUK

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,238
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
42
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide


  • sharding isn’t new















  • Transcript

    • 1. Eliot Horowitz @eliothorowitz MongoUK June 18, 2010
    • 2. Why Distributed Databases • Vertical scaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher
    • 3. Methods of Distribution • ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)
    • 4. Mongo Sharding • Can distrbute databases, collections or a objects in a collection • Choose how you partition data • Balancing, migrations, management all automatic
    • 5. • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent
    • 6. Range Based • collection is broken into chunks by range
    • 7. Under the Hood Shards mongo mongo mongo .. Config mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client
    • 8. Config Servers • 3 of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up
    • 9. Shards • Can be master, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes
    • 10. mongos • Sharding Router • Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network traffic
    • 11. Writes • Inserts : require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered
    • 12. Queries • By shard key: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort
    • 13. Operations • split: breaking a chunk into 2 • migrate: move a chunk from 1 shard to another
    • 14. Balancing • distributes chunks automatically • can look at: disk ops, cpu, data size
    • 15. Setting it Up • Start servers • add shards: db.runCommand( { addshard : "10.1.1.5" } ) • turn on partitioning: db.runCommand( { enablesharding : "test" } • shard a collection: db.runCommand( { shardcollection : "test.data" , key : { num : 1 } } )
    • 16. Multi DataCenter • intelligent geo honing • auto failover
    • 17. Sharding Limitations • Unique indexes not based on shard key • current version limited to ~ 20 petabytes
    • 18. Download MongoDB http://www.mongodb.org After Party! Bar 1920 (its close) Don’t worry - world cup will be on - first drink free. and let us know what you think @eliothorowitz @mongodb

    ×