Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Sharding with MongoDB (Eliot Horowitz)

2,221
views

Published on

Eliot's presentation at MongoUK

Eliot's presentation at MongoUK

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,221
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
42
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide


  • sharding isn’t new















  • Transcript

    • 1. Eliot Horowitz @eliothorowitz MongoUK June 18, 2010
    • 2. Why Distributed Databases • Vertical scaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher
    • 3. Methods of Distribution • ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)
    • 4. Mongo Sharding • Can distrbute databases, collections or a objects in a collection • Choose how you partition data • Balancing, migrations, management all automatic
    • 5. • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent
    • 6. Range Based • collection is broken into chunks by range
    • 7. Under the Hood Shards mongo mongo mongo .. Config mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client
    • 8. Config Servers • 3 of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up
    • 9. Shards • Can be master, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes
    • 10. mongos • Sharding Router • Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network traffic
    • 11. Writes • Inserts : require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered
    • 12. Queries • By shard key: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort
    • 13. Operations • split: breaking a chunk into 2 • migrate: move a chunk from 1 shard to another
    • 14. Balancing • distributes chunks automatically • can look at: disk ops, cpu, data size
    • 15. Setting it Up • Start servers • add shards: db.runCommand( { addshard : "10.1.1.5" } ) • turn on partitioning: db.runCommand( { enablesharding : "test" } • shard a collection: db.runCommand( { shardcollection : "test.data" , key : { num : 1 } } )
    • 16. Multi DataCenter • intelligent geo honing • auto failover
    • 17. Sharding Limitations • Unique indexes not based on shard key • current version limited to ~ 20 petabytes
    • 18. Download MongoDB http://www.mongodb.org After Party! Bar 1920 (its close) Don’t worry - world cup will be on - first drink free. and let us know what you think @eliothorowitz @mongodb