1. Unearthing 4 Hidden
Gems of MongoDB 2.6
Dan Pasette
VP of Core Engineering

2. MongoDB 2.6 was big.

3. > Power of 2 as Default Allocation Strategy
> Server-side Timeouts
> Query Engine Introspection
> Background Indexing on Secondaries
Here’s what you missed:
Allocation. Timeouts. Introspection. Indexing.

4. Gem 1.
Power of 2 Allocation is
now on by default.
Allocation. Timeouts. Introspection. Indexing.

5. What happened?
Before 2.6, the default record
allocation used an exact-fit strategy.
Allocation. Timeouts. Introspection. Indexing.

6. Why does the allocation
strategy matter?
Allocation. Timeouts. Introspection. Indexing.

7. Allocation. Timeouts. Introspection. Indexing.
Take this section of disk with
some records of various sizes.

8. Allocation. Timeouts. Introspection. Indexing.
Documents are resized,
and records are removed.

9. Allocation. Timeouts. Introspection. Indexing.
Space is wasted because
new records cannot fit.

10. Allocation. Timeouts. Introspection. Indexing.
Fragmentation!

11. Allocation. Timeouts. Introspection. Indexing.
When document needs to
move, references must be
updated. That costs I/O.

12. The 2.6 Fix
By default, round up the record size
to the next power of two.
Allocation. Timeouts. Introspection. Indexing.

13. Allocation. Timeouts. Introspection. Indexing.
5121024

14. Allocation. Timeouts. Introspection. Indexing.
5121024
804 412
Reduce disk frag!

15. We’ve made capacity
planning predictable.
Allocation. Timeouts. Introspection. Indexing.

16. We’ve made capacity
planning predictable.
What about operations?
Allocation. Timeouts. Introspection. Indexing.

17. Gem 2.
Server-side Timeouts.
Allocation. Timeouts. Introspection. Indexing.

18. Server
“Our collection was indexed in staging,
but we forgot to index in production.”
Allocation. Timeouts. Introspection. Indexing.

19. Server
C
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,
but we forgot to index in production.”

20. Server
C
C retry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,
but we forgot to index in production.”

21. Server
C
C
C
retry!
retry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,
but we forgot to index in production.”

22. The 2.6 Fix
Use maxTimeMS to limit how long
an op can run in the database.
Allocation. Timeouts. Introspection. Indexing.

23. We’ve stopped the op.
Allocation. Timeouts. Introspection. Indexing.

24. We’ve stopped the op.
But why was it slow?
Allocation. Timeouts. Introspection. Indexing.

25. Gem 3.
Get inside the
Query Optimizer.
Allocation. Timeouts. Introspection. Indexing.

26. What happened?
For 2.6, we rewrote the query
execution framework — and
most of the time, it just works.
Allocation. Timeouts. Introspection. Indexing.

27. Allocation. Timeouts. Introspection. Indexing.
Query Planner
Query
Parser

28. Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan
Cache
?
Query
Parser

29. Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Miss
Query
Parser

30. Allocation. Timeouts. Introspection. Indexing.
Query
Parser
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Miss
Plan Enumerator
Plan
Plan
Plan

31. Allocation. Timeouts. Introspection. Indexing.
Query
Parser
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner

32. Allocation. Timeouts. Introspection. Indexing.
Query
Parser
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache

33. Allocation. Timeouts. Introspection. Indexing.
Query
Parser
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache

34. Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
Query
Parser
The plan cache cuts out
a lot of processing.

35. But sometimes,
queries are slow.
Allocation. Timeouts. Introspection. Indexing.

36. Allocation. Timeouts. Introspection. Indexing.
Query
Parser
Query Planner Plan
Cache
?
Hit
Return
Plan Runner
!!!
The plan cache is caching
a suboptimal plan.

37. The 2.6 Fix
Add logging and introspection tools
to view and manipulate the cache,
such as getPlanCache.
Allocation. Timeouts. Introspection. Indexing.

38. Now we know what’s
wrong with the query.
Allocation. Timeouts. Introspection. Indexing.

39. Now we know what’s
wrong with the query.
Let’s fix it!
Allocation. Timeouts. Introspection. Indexing.

40. Gem 4.
Background Indexing
on Secondaries.
Allocation. Timeouts. Introspection. Indexing.

41. P
S S
Allocation. Timeouts. Introspection. Indexing.

42. P
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.

43. C
P
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.

44. What happened?
Before 2.6, background index builds
became foreground index builds
when replicated to secondaries.
Allocation. Timeouts. Introspection. Indexing.

45. The 2.6 Fix
Now, background index builds
stay in the background.
Allocation. Timeouts. Introspection. Indexing.

46. C
P
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.

47. > Power of 2 as Default Allocation Strategy
> Server-side Timeouts
> Query Engine Introspection
> Background Indexing on Secondaries
Here’s what we
covered:
Allocation. Timeouts. Introspection. Indexing.

48. Small gems, big impact.

49. Small gems, big impact.
Thank you!