MongoDB.local Berlin: How and when to use multi-document transactions
•
0 likes•288 views
1) MongoDB 4.0 introduced multi-document ACID transactions that allow atomic operations across multiple collections. 2) Transactions provide snapshot isolation, all-or-nothing execution, and read your own writes behavior. They abort and roll back on write conflicts with other transactions or operations. 3) Care must be taken to avoid excessive WiredTiger cache pressure from long-running transactions holding snapshots open. Transaction size and lifetime should be limited.
Recommended
More Related Content
Similar to MongoDB.local Berlin: How and when to use multi-document transactions
Similar to MongoDB.local Berlin: How and when to use multi-document transactions (6)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
MongoDB.local Berlin: How and when to use multi-document transactions
- 1. How and When to Use Multi-Document Transactions @aly_cabral
- 4. After 7 years …
- 5. The evolution of MongoDB 3.0 3.2 Document Validation $lookup Fast Failover Simpler Scalability Aggregation ++ Encryption At Rest In-Memory Storage Engine BI Connector MongoDB Compass APM Integration Profiler Visualization Auto Index Builds Backups to File System Doc-Level Concurrency Compression Storage Engine API ≤50 replicas Auditing ++ Ops Manager Linearizable reads Intra-cluster compression Views Log Redaction Graph Processing Decimal Collations Faceted Navigation Zones ++ Aggregation ++ Auto-balancing ++ ARM, Power, zSeries BI & Spark Connectors ++ Compass ++ Hardware Monitoring Server Pool LDAP Authorization Encrypted Backups Cloud Foundry Integration 3.4 3.6 Change Streams Retryable Writes Expressive Array Updates Query Expressivity Causal Consistency Consistent Sharded Sec. Reads Compass Community Ops Manager ++ Query Advisor Schema Validation End to End Compression IP Whitelisting Default Bind to Localhost Sessions WiredTiger 1m+ Collections MongoDB BI Connector ++ Expressive $lookUp R Driver Atlas Cross Region Replication Atlas Auto Storage Scaling 4.0 Multi-Document ACID Transactions Atlas Global Clusters Atlas HIPAA Atlas LDAP Atlas Audit Atlas Encrypted Storage Engine Atlas AWS Backup Snapshots Atlas Full CRUD Agg Pipeline Type Conversions 40% Faster Shard Migrations Snapshot Reads Non-Blocking Secondary Reads SHA-2 TLS 1.1+ Compass Agg Pipeline Builder Compass Export to Code Charts Beta Free Monitoring Cloud Service Ops Manager K8s & OpenShift MongoDB Stitch GA MongoDB Mobile Beta
- 8. So now we have transactions in MongoDB… Owners ID First Name Last Name Account Type Pets ID Name Type Weight OwnerID Address ID Street City OwnerID Contact ID Phone Number OwnerID
- 9. So now we have transactions in MongoDB… {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}}
- 10. Umbrellas The fact that you have umbrellas, do you need to use it all the time ?
- 11. with client.start_session() as s: s.start_transaction(): try: owners.insert_one(doc1, session=s) owners.insert_one(doc2, session=s) except: s.abort_transaction() raise s.commit_transaction()
- 14. All or Nothing Execution TXN Application MongoDB Uncommitted Writes { Document 1 } { Document 2 } { Document 3 } Committed Writes
- 15. Snapshot Isolation 100 101 102
- 16. Snapshot Isolation 100 101 102 Snapshot 102
- 17. Snapshot Isolation 100 101 102 103 104 105 106 107 108 Snapshot 102
- 18. Snapshot Isolation 100 101 102 103 104 105 106 107 108 109 110 111 Snapshot 102 Snapshot 108
- 19. Read Your Own Writes Operation Read TXN {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} Uncommitted Writes
- 20. Read Your Own Writes Operation Read TXN {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} Uncommitted Writes street: “92nd” db.pets.updateOne({_id: “OWNER12345”, {$set: “Address.street”: “92nd”}})
- 21. Read Your Own Writes Operation Read TXN {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} Uncommitted Writes street: “92nd” db.pets.find({_id: “OWNER12345”, {“Address.street”: 1}})
- 22. Read Your Own Writes Operation Read TXN {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} Uncommitted Writes Address.street = 94th
- 23. Read Your Own Writes Operation Read TXN {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} Uncommitted Writes Address.street = 94th Address.street = 92nd db.pets.find({_id: “OWNER12345”, {“Address.street”: 1}})
- 25. TXN Write TXN Write {Doc 1} TXN write is unable to obtain a write lock! TXN is ABORTED rolled-back entirely
- 26. TXN Write Operation Non-Transactional Write {Doc 1} Operation Blocks Behind TXN Commit Infinitely retries with Backoff Logic until MaxTimeMS is reached Operation non-transactional write is unable to obtain a write lock!
- 27. When non-transactional writes encounter write conflicts Operation Blocks Behind TXN Commit Infinitely retries with Backoff Logic until MaxTimeMS is reached When transactions encounter write conflicts Transactions abort rollback entirely
- 28. TXN Read TXN Write {Doc 1} Reads do not lock a document from being modified Only writes make write conflicts
- 29. TXN Write TXN Write {_id: “OWNER12345”, name: { first: “Aly”, last: “Cabral”}, accountType: “Paid”, pets: [ { type: “Dog”, name: “Lady”}], contact: { phoneNumber: 8008675309}, Address: { city: “New York”, street: “94th”, zipecode: 10025}} street: “94th” No-op writes are not writes Only writes make write conflicts
- 31. @aly_cabral
- 32. @aly_cabral
- 33. @aly_cabral
- 34. @aly_cabral
- 35. @aly_cabral
- 37. P S S Writes are replicated once committed
- 38. Cache Pressure
- 39. Cache Pressure
- 40. Cache Pressure
- 41. Cache Pressure
- 42. Cache Pressure
- 43. … … keys K K K KVV V V txn time data next Update Structure txn time data next 1) Cache pressure builds up by write volume after a snapshot 2) Transactions use the same snapshot throughout it’s duration 3) The update structure can only be cleaned up after the snapshot is evicted WiredTiger Cache
- 44. How to Avoid WiredTiger Cache Pressure: transactionLifetimeLimitSeconds = default to 60s Commit READ-ONLY Transactions Abort Abandoned Transactions <1000 Documents Modified, <16MB Oplog Entry
- 45. DLL vs DML
- 46. DLL vs DML
- 48. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 49. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 50. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 51. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 52. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 53. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 54. DDL Operations createIndex() createCollection() dropDatabase() namespace Pending DDL Intent Lock
- 56. 1) All data modeling rules still apply 2) Transactions should not be the most common operation (sanity check for rule 1) 3) Pass in the session to all statements 4) Implement retry logic, transactions can always abort 5) Don’t unnecessarily leave snapshots open 6) To trigger write conflicts, make sure you’re doing writes 7) Plan for DDL Operations