Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hidden Gems in the 2.6 Release

1,620 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Hidden Gems in the 2.6 Release

  1. 1. Unearthing 4 Hidden Gems of MongoDB 2.6 Dan Pasette VP of Core Engineering
  2. 2. MongoDB 2.6 was big.
  3. 3. > Power of 2 as Default Allocation Strategy > Server-side Timeouts > Query Engine Introspection > Background Indexing on Secondaries Here’s what you missed: Allocation. Timeouts. Introspection. Indexing.
  4. 4. Gem 1. Power of 2 Allocation is now on by default. Allocation. Timeouts. Introspection. Indexing.
  5. 5. What happened? Before 2.6, the default record allocation used an exact-fit strategy. Allocation. Timeouts. Introspection. Indexing.
  6. 6. Why does the allocation strategy matter? Allocation. Timeouts. Introspection. Indexing.
  7. 7. Allocation. Timeouts. Introspection. Indexing. Take this section of disk with some records of various sizes.
  8. 8. Allocation. Timeouts. Introspection. Indexing. Documents are resized, and records are removed.
  9. 9. Allocation. Timeouts. Introspection. Indexing. Space is wasted because new records cannot fit.
  10. 10. Allocation. Timeouts. Introspection. Indexing. Fragmentation!
  11. 11. Allocation. Timeouts. Introspection. Indexing. When document needs to move, references must be updated. That costs I/O.
  12. 12. The 2.6 Fix By default, round up the record size to the next power of two. Allocation. Timeouts. Introspection. Indexing.
  13. 13. Allocation. Timeouts. Introspection. Indexing. 5121024
  14. 14. Allocation. Timeouts. Introspection. Indexing. 5121024 804 412 Reduce disk frag!
  15. 15. We’ve made capacity planning predictable. Allocation. Timeouts. Introspection. Indexing.
  16. 16. We’ve made capacity planning predictable. What about operations? Allocation. Timeouts. Introspection. Indexing.
  17. 17. Gem 2. Server-side Timeouts. Allocation. Timeouts. Introspection. Indexing.
  18. 18. Server “Our collection was indexed in staging, but we forgot to index in production.” Allocation. Timeouts. Introspection. Indexing.
  19. 19. Server C Allocation. Timeouts. Introspection. Indexing. “Our collection was indexed in staging, but we forgot to index in production.”
  20. 20. Server C C retry! Allocation. Timeouts. Introspection. Indexing. “Our collection was indexed in staging, but we forgot to index in production.”
  21. 21. Server C C C retry! retry! Allocation. Timeouts. Introspection. Indexing. “Our collection was indexed in staging, but we forgot to index in production.”
  22. 22. The 2.6 Fix Use maxTimeMS to limit how long an op can run in the database. Allocation. Timeouts. Introspection. Indexing.
  23. 23. We’ve stopped the op. Allocation. Timeouts. Introspection. Indexing.
  24. 24. We’ve stopped the op. But why was it slow? Allocation. Timeouts. Introspection. Indexing.
  25. 25. Gem 3. Get inside the Query Optimizer. Allocation. Timeouts. Introspection. Indexing.
  26. 26. What happened? For 2.6, we rewrote the query execution framework — and most of the time, it just works. Allocation. Timeouts. Introspection. Indexing.
  27. 27. Allocation. Timeouts. Introspection. Indexing. Query Planner Query Parser
  28. 28. Allocation. Timeouts. Introspection. Indexing. Query Planner Plan Cache ? Query Parser
  29. 29. Allocation. Timeouts. Introspection. Indexing. Query Planner Plan Cache ? Hit Return Plan Runner Miss Query Parser
  30. 30. Allocation. Timeouts. Introspection. Indexing. Query Parser Query Planner Plan Cache ? Hit Return Plan Runner Miss Plan Enumerator Plan Plan Plan
  31. 31. Allocation. Timeouts. Introspection. Indexing. Query Parser Query Planner Plan Cache ? Hit Return Plan Runner Miss Plan Enumerator Plan Plan Plan Multiplan Runner
  32. 32. Allocation. Timeouts. Introspection. Indexing. Query Parser Query Planner Plan Cache ? Hit Return Plan Runner Miss Plan Enumerator Plan Plan Plan Multiplan Runner Winner! Cache
  33. 33. Allocation. Timeouts. Introspection. Indexing. Query Parser Query Planner Plan Cache ? Hit Return Plan Runner Miss Plan Enumerator Plan Plan Plan Multiplan Runner Winner! Cache
  34. 34. Allocation. Timeouts. Introspection. Indexing. Query Planner Plan Cache ? Hit Return Plan Runner Query Parser The plan cache cuts out a lot of processing.
  35. 35. But sometimes, queries are slow. Allocation. Timeouts. Introspection. Indexing.
  36. 36. Allocation. Timeouts. Introspection. Indexing. Query Parser Query Planner Plan Cache ? Hit Return Plan Runner !!! The plan cache is caching a suboptimal plan.
  37. 37. The 2.6 Fix Add logging and introspection tools to view and manipulate the cache, such as getPlanCache. Allocation. Timeouts. Introspection. Indexing.
  38. 38. Now we know what’s wrong with the query. Allocation. Timeouts. Introspection. Indexing.
  39. 39. Now we know what’s wrong with the query. Let’s fix it! Allocation. Timeouts. Introspection. Indexing.
  40. 40. Gem 4. Background Indexing on Secondaries. Allocation. Timeouts. Introspection. Indexing.
  41. 41. P S S Allocation. Timeouts. Introspection. Indexing.
  42. 42. P S S > db.coll.ensureIndex({ … }, { background: true }) Allocation. Timeouts. Introspection. Indexing.
  43. 43. C P S S > db.coll.ensureIndex({ … }, { background: true }) Allocation. Timeouts. Introspection. Indexing.
  44. 44. What happened? Before 2.6, background index builds became foreground index builds when replicated to secondaries. Allocation. Timeouts. Introspection. Indexing.
  45. 45. The 2.6 Fix Now, background index builds stay in the background. Allocation. Timeouts. Introspection. Indexing.
  46. 46. C P S S > db.coll.ensureIndex({ … }, { background: true }) Allocation. Timeouts. Introspection. Indexing.
  47. 47. > Power of 2 as Default Allocation Strategy > Server-side Timeouts > Query Engine Introspection > Background Indexing on Secondaries Here’s what we covered: Allocation. Timeouts. Introspection. Indexing.
  48. 48. Small gems, big impact.
  49. 49. Small gems, big impact. Thank you!

×