1. Rick Copeland @rick446
Arborian Consulting, LLC

2.  Now a consultant, but formerly…
 Software engineer at SourceForge, early adopter of
MongoDB (version 0.8)
 Wrote the SQLAlchemy book (I love SQL when it’s
used well)
 Mainly write Python now, but have done C++, C#,
Java, Javascript, VHDL, Verilog, …

3.  You can do it with an RDBMS as long as you…
 Don’t use joins
 Don’t use transactions
 Use read-only slaves
 Use memcached
 Denormalize your data
 Use custom sharding/partitioning
 Do a lot of vertical scaling
▪ (we’re going to need a bigger box)

4. +1
Year

6.  Use documents to improve locality
 Optimize your indexes
 Be aware of your working set
 Scaling your disks
 Replication for fault-tolerance and read scaling
 Sharding for read and write scaling

7. Relational (SQL) MongoDB
Database Database Dynamic
Typing
Table Collection B-tree
(range-based)
Index Index
Row Document
Think JSON
Column Field
Primitive types +
arrays, documents

8. {
title: "Slides for Scaling with MongoDB",
author: "Rick Copeland",
date: ISODate("20012-02-29T19:30:00Z"),
text: "My slides are available on speakerdeck.com",
comments: [
{ author: "anonymous",
date: ISODate("20012-02-29T19:30:01Z"),
text: "Fristpsot!" },
{ author: "mark”,
date: ISODate("20012-02-29T19:45:23Z"),
text: "Nice slides" } ] }
Embed comment data in
blog post document

9. Seek = 5+ ms Read = really really fast

10. Post
Comment
Author

11. Post
Author
Comment
Comment
Comment
Comment
Comment

12. Find where x equals 7
1 2 3 4 5 6 7
Looked at 7 objects

13. Find where x
equals 7 4
2 6
1 3 5 7
Looked at 3 objects

14. Entire index
must fit in
RAM

15. Only small
portion in
RAM

16.  Working set =
 sizeof(frequently used data)
 + sizeof(frequently used indexes)
 Right-aligned indexes reduce working set size
 Working set should fit in available RAM for best
performance
 Page faults are the biggest cause of performance
loss in MongoDB

17. >db.foo.stats()
Data Size
{
"ns" : "test.foo",
"count" : 1338330,
"size" : 46915928, Average doc size
"avgObjSize" : 35.05557523181876,
"storageSize" : 86092032,
"numExtents" : 12,
"nindexes" : 2, Size on disk (or RAM!)
"lastExtentSize" : 20872960,
"paddingFactor" : 1,
"flags" : 0, Size of all indexes
"totalIndexSize" : 99860480,
"indexSizes" : {
"_id_" : 55877632,
"x_1" : 43982848},
"ok" : 1 Size of each index
}

18. ~200 seeks / second

19. ~200 seeks / second ~200 seeks / second ~200 seeks / second
 Faster, but less reliable

20. ~400 seeks / second ~400 seeks / second ~400 seeks / second
 Faster and more reliable ($$$ though)

21.  Old and busted  master/slave replication
 The new hotness  replica sets with automatic
failover
Read / Write Primary
Read Secondary
Read Secondary

22.  Primary handles all
writes
 Application optionally
sends reads to slaves
 Heartbeat manages
automatic failover

23.  Special collection (the oplog) records operations
idempotently
 Secondaries read from primary oplog and replay
operations locally
 Space is preallocated and fixed for the oplog

24. {
"ts" : Timestamp(1317653790000, 2),
Insert
"h" : -6022751846629753359,
"op" : "i",
"ns" : "confoo.People", Collection name
"o" : {
"_id" : ObjectId("4e89cd1e0364241932324269"),
"first" : "Rick",
"last" : "Copeland”
}
} Object to insert

25.  Use heartbeat signal to detect failure
 When primary can’t be reached, elect a new one
 Replica that’s the most up-to-date is chosen
 If there is skew, changes not on new primary are
saved to a .bson file for manual reconciliation
 Application can require data to be replicated to a
majority to ensure this doesn’t happen

26.  Priority
 Slower nodes with lower priority
 Backup or read-only nodes to never be primary
 slaveDelay
 Fat-finger protection
 Data center awareness and tagging
 Application can ensure complex replication
guarantees

27.  Reads scale nicely
 As long as the working set fits in RAM
 … and you don’t mind eventual consistency
 Sharding to the rescue!
 Automatically partitioned data sets
 Scale writes and reads
 Automatic load balancing between the shards

28. Configuration
MongoS MongoS
Config 1 Config 2 Config 3
Shard 1 Shard 2 Shard 3 Shard 4
0..10 10..20 20..30 30..40
Primary Primary Primary Primary
Secondary Secondary Secondary Secondary
Secondary Secondary Secondary Secondary

29.  Sharding is per-collection and range-based
 The highest-impact choice (and hardest to
change decision) you make is the shard key
 Random keys: good for writes, bad for reads
 Right-aligned index: bad for writes
 Small # of discrete keys: very bad
 Ideal: balance writes, make reads routable by mongos
 Optimal shard key selection is hard

30. Primary Data Center Secondary Data Center
Shard 1 Shard 1 Shard 1
Priority 1 Priority 1 Priority 0
Shard 2 Shard 2 Shard 2
Priority 1 Priority 1 Priority 0
Shard 3 Shard 3 Shard 3
RS3
Priority 1 Priority 1 Priority 0
Config 1 Config 2 Config 3

31.  Writes and reads both scale (with good choice of
shard key)
 Reads scale while remaining strongly consistent
 Partitioning ensures you get more usable RAM
 Pitfall: don’t wait too long to add capacity

32. Rick Copeland @rick446
Arborian Consulting, LLC