Internal Anatomy of an Update

1,022 views

Published on

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,022
On SlideShare
0
From Embeds
0
Number of Embeds
582
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Internal Anatomy of an Update

  1. 1. Update Anatomy The guide to how an update works
  2. 2. Me = Scott Hernandez ● Worked on Morphia (+java), community ● Over 3 years with MongoDB ● Worked on update re-write in 2.6 ● Working on replication team now ● Lots of experience with many parts
  3. 3. User Components update({f:1}, {$set: {a: 2}}, {multi:true}) ● Query ● Update Modifiers (or replacement doc) ● Options o multi, upsert o writeConcern
  4. 4. New Code (in 2.6) ● New update and query code ● Replaced all modifier implementations ● New validation and error messages ● Using new “MutableBSON” document class ● New oplog entry generation code
  5. 5. Internal Components ● Update Command (-> request) ● Update Executor: Runner + Driver ● *Modifiers (Inc, Set, Unset, Bit, …) ● Validation, Modification, and Replication
  6. 6. Two Paths To Update Wire Protocol Op User Write Commands Internal Update Request
  7. 7. … don’t forget FindAndModify ;)
  8. 8. Update Request 1. Create o Validate args o Prepare what we can (reduce lock time) o Record system state (repl, sharding, etc) 2. Send for execution
  9. 9. What is a modifier {$set: {a: 1} }, {$min: {c: “b”} } ● Modifier: “set”, “min” ● Field Path: “a”, “c” (+ “b”) ● Value: 1, “b”
  10. 10. Implementing a modifier db/ops/modifier_interface.h: ● init ( modExpr, opts ) // validation ● prepare ( doc, fieldName, execInfo ) // work ● apply ( ) // update document ● log ( ) // oplog entry created
  11. 11. Modifier::init ● Validate form of update, and args ● Parse path ● Indicate if positional, and supported
  12. 12. Modifier::prepare ● Work with doc, but don’t change it ● Validate field, type, operation ● Substitute positional path element, if appl. ● Do work, stored to the side ● Return ExecInfo (no-op, fields affected)
  13. 13. Modifier::apply ● Apply prepared work to doc
  14. 14. Modifier::log ● Calculate transformation logic for oplog entry ● Update idempotent oplog entry o Set o Unset o Full Replacement
  15. 15. db/ops/modifier_* + tests ● add_to_set.h/cpp ● bit.h/cpp ● compare.h/cpp ● current_date.h/cpp ● inc.h/cpp ● obj_replace.h/cpp ● pop.h/cpp ● pull.h/cpp ● pull_all.h/cpp ● push.h/cpp ● rename.h/cpp ● set.h/cpp ● unset.h/cpp
  16. 16. Update Executor ● Finish prep ● If no docs + upsert, prepare insert doc ● Run driver (mods) against each doc ● Ensure, validate, log for replication ● Return UpdateResult
  17. 17. Update Flow Query + Update Cursor ● Init ● Prepare ● Apply { $inc: {a:1}, { $inc: {b:1}} ● Init ● Prepare ● Apply { $set: {c: {e:1} }} ● Log ● Log Entry Oplog No results, upsert:true Insert Entry
  18. 18. Update Flow Notes ● Yields based on timing/in-memory ● Each document update is atomic ● Periodically allows journal to flush, if needed
  19. 19. Questions? Round 1: No, then we have more slides
  20. 20. 2.6 Stuff ● New modifiers ● Improvements ● Behavioral Changes
  21. 21. New Modifiers ● Mul(tiply) existing field by constant: $mul {a:1} + {$mul:{a:10}} -> {a:10} ● Min/Max of existing field w/constant: $min/$max {a:1} + {$min:{a:100}} -> {a:1} (no-op) ● Store current date/ts in a field: $currentDate {} + {$currentDate:{ dt: true}} -> {dt: new Date()}
  22. 22. Improvements ● $push o $sort elements (works with numbers/strings/etc) o $sort and/or $slice (doesn’t require $sort for $slice) o $slice off either end (no middle yet) o $position lets you specify where in the array o $each, $sort, $slice no longer order dep. ● $bit o xor support
  23. 23. Improvements ● Error messages with context o The field 'l' must be an array but is of type NumberLong in document {_id: ObjectId('52a0a61672a5f2f771099b54')} o Cannot apply $inc to a value of non-integral type. {_id: ObjectId('52a0a61672a5f2f771099b54')} has the field 'd' of non-integral type Date
  24. 24. Improvements ● No reordering of fields (except: _id is first) ● Better building of doc during insert o update({a:1, b:1}, {$inc:{c:1}}, 0,1) -> {a:1, b:1, c:1} o now uses parsed query tree to get equalities ● Will now return new _id for all types ● Accurate tracking of no-ops, modified docs
  25. 25. Improvements ● Validation o Immutable Fields (_id, shard keys) o Storage restrictions (no $field, “.”, _id field types) o DBRef validation ● Can now query + set _id/shard-key fields o update({_id:1}, {$set:{_id:1, ...}}) // ok o update({_id:1}, {_id:2, ...}) // error ● Cleanup bad data ($unset/$rename etc)
  26. 26. Questions? No, well thanks for coming!

×