Your SlideShare is downloading. ×
  • Like

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Internal Anatomy of an Update

  • 504 views
Published

 

Published in Technology , Health & Medicine
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
504
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
10
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Update Anatomy The guide to how an update works
  • 2. Me = Scott Hernandez ● Worked on Morphia (+java), community ● Over 3 years with MongoDB ● Worked on update re-write in 2.6 ● Working on replication team now ● Lots of experience with many parts
  • 3. User Components update({f:1}, {$set: {a: 2}}, {multi:true}) ● Query ● Update Modifiers (or replacement doc) ● Options o multi, upsert o writeConcern
  • 4. New Code (in 2.6) ● New update and query code ● Replaced all modifier implementations ● New validation and error messages ● Using new “MutableBSON” document class ● New oplog entry generation code
  • 5. Internal Components ● Update Command (-> request) ● Update Executor: Runner + Driver ● *Modifiers (Inc, Set, Unset, Bit, …) ● Validation, Modification, and Replication
  • 6. Two Paths To Update Wire Protocol Op User Write Commands Internal Update Request
  • 7. … don’t forget FindAndModify ;)
  • 8. Update Request 1. Create o Validate args o Prepare what we can (reduce lock time) o Record system state (repl, sharding, etc) 2. Send for execution
  • 9. What is a modifier {$set: {a: 1} }, {$min: {c: “b”} } ● Modifier: “set”, “min” ● Field Path: “a”, “c” (+ “b”) ● Value: 1, “b”
  • 10. Implementing a modifier db/ops/modifier_interface.h: ● init ( modExpr, opts ) // validation ● prepare ( doc, fieldName, execInfo ) // work ● apply ( ) // update document ● log ( ) // oplog entry created
  • 11. Modifier::init ● Validate form of update, and args ● Parse path ● Indicate if positional, and supported
  • 12. Modifier::prepare ● Work with doc, but don’t change it ● Validate field, type, operation ● Substitute positional path element, if appl. ● Do work, stored to the side ● Return ExecInfo (no-op, fields affected)
  • 13. Modifier::apply ● Apply prepared work to doc
  • 14. Modifier::log ● Calculate transformation logic for oplog entry ● Update idempotent oplog entry o Set o Unset o Full Replacement
  • 15. db/ops/modifier_* + tests ● add_to_set.h/cpp ● bit.h/cpp ● compare.h/cpp ● current_date.h/cpp ● inc.h/cpp ● obj_replace.h/cpp ● pop.h/cpp ● pull.h/cpp ● pull_all.h/cpp ● push.h/cpp ● rename.h/cpp ● set.h/cpp ● unset.h/cpp
  • 16. Update Executor ● Finish prep ● If no docs + upsert, prepare insert doc ● Run driver (mods) against each doc ● Ensure, validate, log for replication ● Return UpdateResult
  • 17. Update Flow Query + Update Cursor ● Init ● Prepare ● Apply { $inc: {a:1}, { $inc: {b:1}} ● Init ● Prepare ● Apply { $set: {c: {e:1} }} ● Log ● Log Entry Oplog No results, upsert:true Insert Entry
  • 18. Update Flow Notes ● Yields based on timing/in-memory ● Each document update is atomic ● Periodically allows journal to flush, if needed
  • 19. Questions? Round 1: No, then we have more slides
  • 20. 2.6 Stuff ● New modifiers ● Improvements ● Behavioral Changes
  • 21. New Modifiers ● Mul(tiply) existing field by constant: $mul {a:1} + {$mul:{a:10}} -> {a:10} ● Min/Max of existing field w/constant: $min/$max {a:1} + {$min:{a:100}} -> {a:1} (no-op) ● Store current date/ts in a field: $currentDate {} + {$currentDate:{ dt: true}} -> {dt: new Date()}
  • 22. Improvements ● $push o $sort elements (works with numbers/strings/etc) o $sort and/or $slice (doesn’t require $sort for $slice) o $slice off either end (no middle yet) o $position lets you specify where in the array o $each, $sort, $slice no longer order dep. ● $bit o xor support
  • 23. Improvements ● Error messages with context o The field 'l' must be an array but is of type NumberLong in document {_id: ObjectId('52a0a61672a5f2f771099b54')} o Cannot apply $inc to a value of non-integral type. {_id: ObjectId('52a0a61672a5f2f771099b54')} has the field 'd' of non-integral type Date
  • 24. Improvements ● No reordering of fields (except: _id is first) ● Better building of doc during insert o update({a:1, b:1}, {$inc:{c:1}}, 0,1) -> {a:1, b:1, c:1} o now uses parsed query tree to get equalities ● Will now return new _id for all types ● Accurate tracking of no-ops, modified docs
  • 25. Improvements ● Validation o Immutable Fields (_id, shard keys) o Storage restrictions (no $field, “.”, _id field types) o DBRef validation ● Can now query + set _id/shard-key fields o update({_id:1}, {$set:{_id:1, ...}}) // ok o update({_id:1}, {_id:2, ...}) // error ● Cleanup bad data ($unset/$rename etc)
  • 26. Questions? No, well thanks for coming!