Deploying MongoDB can be a challenge if you don't understand how resources are used nor how to plan for the capacity of your systems. If you need to deploy, or grow, a MongoDB single instance, replica set, or tens of sharded clusters then you probably share the same challenges in trying to size that deployment. This talk will cover what resources MongoDB uses, and how to plan for their use in your deployment. Topics covered will include understanding how to model and plan capacity needs from the perspective of a new deployment, growing an existing one, and defining where the steps along scalability on your path to the top. The goal of this presentation will be to provide you with the tools needed to be successful in managing your MongoDB capacity planning tasks.
30. What is failure?
• We have failed at Capacity Planning when our
resources don’t meet our requirements
• Because our requirements can have many
dimensions, we may exceed our requirements in
one characteristic but not meet them in another
• This means that we can spend many $$$ and still
fail!
31. What about Legacy Hardware?
• Let’s hope whatever worked for this legacy
technology also works for MongoDB
• Same principles of Capacity Planning still apply
33. • Before it's too late!
• When?
Capacity Planning: When
Start Launch Version 2
34. Capacity Planning is Measurement
Measuring early gives you a comparison
point for when you need to do it again
35. Velocity of Change
• Limitations -> takes time
– Data Movement
– Allocation/Provisioning (servers/mem/disk)
• Improvement
– Limit Size of Change (if you can)
– Increase Frequency
– MEASURE its effect
– Practice
41. Starter Questions
• What is the working set?
– How does that equate to memory
– How much disk access will that require
• How efficient are the queries?
• What is the rate of data change?
• How big are the highs and lows?
42. Deployment Types
All of these use the same resources:
• Single Instance
• Multiple Instances (Replica Set)
• Cluster (Sharding)
• Data Centers
7: Why, What, When, Where, How, WhoSkipping Who is you – that's why you're here, right?Where – hopefully on the same systems you'll be running in production, How -- bonusShow evidenceCapacity planning prerequisitesWhy do we need to do this?Epic fail – what's the consequence of not doing this?Thankless job – if you get it right no one notices. But if you get it wrong...Failure to project performance drop-off as the amount of data increases
Once we launch, we don't want to have avoidable down time due to poorly selected HWAs our success grows we want to stay in front of the demand curveWe want to meet business' and users' expectationsWe want to keep our jobs and get big raises! ;)
Initialize -> ElectionPrimary + data replication from primary to secondary
Break into separate slides - Show actual examples (possibly from mms) illustrating these points
Note, change the color to match the actual things.
top three – all add to memory what happens in RAM...if you have 64GB – 20GB sorting aggregation and connectionshow that each using more decreases availability for others2.4 will add a way to estimate "working set"
top three – all add to memory what happens in RAM...if you have 64GB – 20GB sorting aggregation and connectionshow that each using more decreases availability for others2.4 will add a way to estimate "working set"
Take out “Documents (and Collections)”
PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data
PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data