Back to Basics 2017: Introduction to Sharding
•Download as PPTX, PDF•
0 likes•1,929 views
Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations by providing the capability for horizontal scaling.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Back to Basics 2017: Introduction to Sharding
Similar to Back to Basics 2017: Introduction to Sharding (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
Back to Basics 2017: Introduction to Sharding
- 2. Back to Basics 2017 : Webinar 4 Introduction to Sharding Joe Drumgoole Director of Developer Advocacy, EMEA MongoDB @jdrumgoole V1.1
- 3. 3 Summary of Part 1 to 3 • Introduction to NoSQL • Your First MongoDB Application • Introduction to Replica Sets • MongoDB Compass, MongoDB Atlas
- 4. 4 Agenda • Sharding – What is it? Why do we need it? • The architecture of a sharded cluster • Sharded cluster constraints • How a sharded cluster works in practice
- 6. Replica Set with MongoDB Application Driver Primary /data Secondary /data Secondary /data
- 7. Replica Set Bottlenecks Application Driver Primary /data Secondary /data Secondary /data RAM Limits on single server CPU Limits on single server Network Bandwidth Disk I/O
- 8. What is Sharding? Application mongos mongos mongos Driver
- 9. But There is More Application mongos mongos mongos Driver Config Server
- 10. 10 Construction • Build Cluster • Identify shard key • Sharding happens on individual collections • To shard a collection: sh.shardcollections( "MUGS.members",{ "members.member_id" : 1 } )
- 11. 11 Shard Keys • User defines shard key • Shard key defines range of data • Key space is like points on a line • Range is a segment of that line
- 12. 12 Shard Key Constraints • Shard keys are immutable • Shard keys should have high cardinality • Shard keys must be unique • Shard key must exist in every document • Limited to 512 bytes in size • Cannot be a multi-key (array)
- 14. 14 Chunk is a Section on the Range
- 16. 16 How Data is Distributed • Initially 1 chunk • Default max chunk size: 64mb • MongoDB automatically splits & migrates chunks when max reached
- 18. 18 Acquiring the Balancer Lock
- 21. 21 Clean Up
- 22. Routing Requests
- 23. 23 Routing Requests - Targeted
- 24. 24 Routing Requests – Non-Targeted
- 26. 26 Picking a Shard Key • Cardinality • Write Distribution • Query Isolation
- 27. Q&A
Editor's Notes
- Who I am, how long have I been at MongoDB.
- Data is partitioned. Mongos distribute data.
- Data is partitioned. Mongos distribute data.
- Once chunk size is reached, mongos asks mongod to split a chunk + internal function called splitVector() mongod counts number of documents on each side of split + based on avg. document size `db.stats()` Chunk split is a **logical** operation (no data has moved) Max on first chunk should be 14
- Balancer is running on mongos Once the difference in chunks between the most dense shard and the least dense shard is above the migration threshold, a balancing round starts