How do you determine whether your MongoDB Atlas cluster is over provisioned, whether the new feature in your next application release will crush your cluster, or when to increase cluster size based upon planned usage growth?  MongoDB Atlas provides over a hundred metrics enabling visibility into the inner workings of MongoDB performance, but how do apply all this information to make capacity planning decisions? This presentation will enable you to effectively analyze your MongoDB performance to optimize your MongoDB Atlas spend and ensure smooth application operation into the future.

1. Atlas Cluster Capacity Analysis:
How much runway do I have left in
my cluster?
Jay Runkel – Master Solutions Architect
03:00PM – 03:45PM
Exhibit Hall B

2. @
#MDBlocal
Jay Runkel
Master Solutions Architect
jayrunkel
TORONTO

3. #MDBLocal
Your application is deployed in Atlas
Everything is going well
MongoDB performance is excellent…

4. #MDBLocal
How do you know whether?
This will continue…
There is plenty of runway left in their cluster.
Cluster capacity issues are just
around the corner…

5. #MDBLocal
Atlas provides this?
How do you leverage this information to
determine available capacity based upon
expectations for future growth?

6. Earning your Atlas Wings

7. #MDBLocal
Determining available runway in 3 steps
1. Verify database is optimally configured
2. Evaluate remaining capacity
• RAM
• CPU
• IOPS
3. Extrapolate to future

8. Step 1
Verifying Database is Optimally Configured

9. #MDBLocal
Step 1 – Verifying Database is Optimally Configured
How much work is being done?
• Op Counters
• Document Metrics
Are indexes properly configured?
• Query Targeting
• Scan and Order
Are there slow queries?
• Query Execution Times
• Performance Advisor

10. #MDBLocal
OpCounters
command
query
update
delete
getmore
insert

11. #MDBLocal
Document Metrics
Deleted Average rate per second of documents deleted meets your specified threshold.
Inserted Average rate per second of documents inserted meets your specified threshold.
Returned Average rate per second of documents returned meets your specified threshold.
Updated Average rate per second of documents updated meets your specified threshold.

12. #MDBLocal
Query Targeting
• Scanned/Returned
• # of index entries scanned per document
returned
• Scanned Objects/Returned
• # of documents scanned per document
returned
• Ideal ratio should be 1
• High ratios indicate lack of indexes (Collection
Scan) or partial index usage

13. #MDBLocal
Query Targeting
Ideal ratio of 1

14. #MDBLocal
Query Targeting
Less than optimal ratio of 46.8K

15. #MDBLocal
Scan and Order
Is MongoDB leveraging indexes to sort data?
Scan and Order
The average rate per second over the selected sample period of queries
that return sorted results that cannot perform the sort operation using an
index.

16. #MDBLocal
Query Execution Times

17. #MDBLocal
Performance Advisor

18. Step 2
How must extra capacity do we have?

19. #MDBLocal
Step 2 – How much extra capacity do we have?
Start with CPU and IO
• CPU Utilization
• IO Utilization
• IOPS
Next Dig Deeper
• Cache Activity
• Cache Usage

20. #MDBLocal
There are only 4 resources:
1. CPU
2. RAM
3. Disk Space
4. IOPS

21. #MDBLocal
There are only 4 resources:
1. CPU
2. RAM
3. Disk Space
4. IOPS
Atlas will automatically
scale this one for you

22. #MDBLocal
CPU

23. #MDBLocal
Atlas Provides 3 CPU Metrics
1. System CPU – All processes (may be > 100%, multiple cores)
2. Normalized – All processes, normalized 0 – 100%
3. Process – MongoDB only

24. #MDBLocal
Minimal Additional CPU Capacity
M40
• 15 GB RAM
• 4 vCPUs

25. #MDBLocal
Extra CPU Capacity
M40
• 15 GB RAM
• 4 vCPUs

26. #MDBLocal
RAM and IO

27. #MDBLocal
RAM and Disk activity are interrelated
• More RAM è less disk activity
• Less RAM à more disk activity
• Why?

28. #MDBLocal
Data and indexes are cached in RAM
File System
collections indexes
CPU
Memory
indexes
documents
{} {} {}
{} {}
More RAM à more data in cache à less read from disk

29. MongoDB allocates ~50% RAM for cache
OS MDB ? File System Cache Wired Tiger Cache
Uncompressed
Documents
Indexes
Compressed
Documents
Indexes
Disk

30. #MDBLocal
How much RAM is enough?
WT Cache (~1/2 RAM) > Working Set
Working Set = Indexes + Frequently accessed documents
Atlas displays the index size:

31. #MDBLocal
Index size gives us the lower bound for RAM
Working Set = Indexes + Frequently accessed documents
How big are the frequently accessed documents?
There isn’t a an Atlas chart for that:

32. #MDBLocal
Cache Activity can provide some visibility
Disk
FSCacheWTCache
Application

33. #MDBLocal
Working set doesn’t fit in RAM
Disk
FSCacheWTCache
Application
High cache and disk activity

34. #MDBLocal
What does this look like in Atlas?
M40
• 15 GB RAM
• 4 vCPUs
Cache Activity
Cache Usage

35. #MDBLocal
What does this look like in Atlas?
M40
• 15 GB RAM
• 4 vCPUs
Cache Activity
Cache Usage
Disk Utilization

36. #MDBLocal
Available runway is short
Queues are empty
Query execution
times aren’t horrible

37. #MDBLocal
How do we add more runway (capacity)?
1. Add RAM
• More of working set in RAM
• Shorter latency
• Less IO
2. Add more IOPS
• Read more data from disk per second
Which should you choose?

38. #MDBLocal
Add more RAM or IO?
Memory is much faster
• ~ .1 us = memory access
• ~150 us = 4K random read from SSD
• Putting more of the working set in RAM will have big impact
• Upgrade instance size
• Add shard
• Adding IOPS will provide a smaller boost

39. #MDBLocal
Example: working set fits in RAM
M10
• 1.7 GB RAM
• .5 vCPU
• Lots of queries
• Low cache activity

40. #MDBLocal
Low disk activity à working set fits in RAM
• Excellent query
performance
• Low disk
utilization

41. #MDBLocal
What about writes?
• Available IOPS must be greater than:
• IOPS required for writes
• IOPS for reads not in cache
• Cache activity will provide visibility into write IOPS as well

42. #MDBLocal
Cache usage with max write throughput
M10
• 1.7 GB RAM
• .5 vCPU
• Write intensive
workload

43. #MDBLocal
Cache usage with max write throughput
Reads
Writes
WT Cache Size:
• 256 MB
• Max Dirty (20%) : 51.2 MB

44. #MDBLocal
In this case, CPU is the limiting factor

45. Step 3
Do we have enough future capacity?

46. #MDBLocal
Step 3 – Do we have enough future capacity?
At this point:
1. MongoDB is optimally configured
2. We understand our current level of utilization
• CPU
• IO/IOPS
3. We estimated our working set
4. Validated our working set estimate by looking at cache activity

47. #MDBLocal
Step 3 – Do we have enough future capacity?
We need to understand how the workload will change in the future:
• How much more data?
• How many more queries?
• New data types and queries?

48. #MDBLocal
For writes
# Writes Today
# Write IOPS Today
# Future Writes
# Future IOPS
=
# Future IOPS = (# Future Writes * Write IOPS Today) / # Writes Today
Assumes application
doesn’t change
Make sure there is sufficient CPU capacity for additional writes

49. #MDBLocal
For Reads
Need to understand how working set will change:
1. Static working set
• More queries against same working set
2. Linear growth
• More data, but same documents/indexes/queries
3. New working set
• Changes to documents, indexes, queries

50. #MDBLocal
Static Working Set, but more queries
Disk
FSCacheWTCache
Application
W
o
r
k
i
n
g
S
e
t

51. #MDBLocal
Static Working Set, but more queries
1. Scale reads IOPS proportionally (similar to writes)
2. Verify available CPU capacity
• Scale up or out as necessary

52. #MDBLocal
Linear growth, same queries, same documents
Disk
FSCacheWTCache
Application
W
o
r
k
i
n
g
S
e
t
W
o
r
k
i
n
g
S
e
t

53. #MDBLocal
Linear growth, same data, same documents
1. Scale write IOPS proportionally
2. Scale reads IOPS proportionally
3. Scale RAM proportionally
4. Verify available CPU capacity
• Scale up or out as necessary

54. #MDBLocal
Changes to documents, queries, indexes, volume, …
Extrapolate if changes are small
If not, estimate RAM, Disk, IOPS as in:
• Sizing MongoDB Clusters
• MongoDB World 2017

55. #MDBLocal
Review: How much runway is available?

56. #MDBLocal
Review: How much runway is available?
1. Verify optimally configured
• Indexes support queries
• Query targeting, Scan and Order, Performance Profiler
2. Evaluate remaining capacity
• CPU
• IO
• Use cache activity to evaluate working set size
3. Extrapolate to future
• Extrapolate linearly, if application changes are small