Application development just got a lot easier! MongoDB 4.0 supports multi-document, multi-collection, multi-statement ACID transactions. We’ll walk through characteristics, syntax, optimization strategies, and all the guarantees transactions provide to simplify and speed up your application development. However, with great power comes great responsibility; you’ll also learn some details of the implementation of transactions in MongoDB and why transactions aren’t always the answer. Learn how to spot dangerous usage patterns and leverage the other features of MongoDB to meet your requirements before pushing to prod.

1. How and When to Use
Multi-Document
Transactions

2. VP Engineering, Storage
MongoDB
Dr Michael Cahill

3. {_id: “OWNER12345”,
name: { first: “Aly”,
last: “Cabral”},
accountType: “Paid”,
pets: [ { type: “Dog”,
name: “Lady”}],
contact: { phoneNumber: 8008675309},
Address: { city: “New York”,
street: “94th”,
zipecode: 10025}}
So now you have transactions…
@aly_cabral
Owners
ID First Name Last Name Account
Type
Pets
ID Name Type Weight OwnerID
Address
ID Street City OwnerID
Contact
ID Phone
Number
OwnerID

4. Transactions Use Cases
@aly_cabral
Many to Many
Relationships
Positions and Trades
Event Processing
Account Creation
Application Event
Logging
Application Auditing

5. with client.start_session() as s:
s.start_transaction():
try:
owners.insert_one(doc1, session=s)
owners.insert_one(doc2, session=s)
except:
s.abort_transaction()
raise
s.commit_transaction()
@aly_cabral

6. Transactions Characteristics

7. Multi- Collection
Database
Statement
Document
@aly_cabral

8. All or Nothing Execution
TXN
Application
MongoDB
Uncommitted Writes
{ Document 1 } { Document 2 }
{ Document 3 }
Committed Writes

9. Snapshot Isolation
@aly_cabral
100 101 102 103 104 105 106 107 108 109 110 111
Snapshot
102
Snapshot
108

10. Read Your Own Writes
@aly_cabral
Operation
Read
TXN
{_id: “OWNER12345”,
name: { first: “Aly”,
last: “Cabral”},
accountType: “Paid”,
pets: [ { type: “Dog”,
name: “Lady”}],
contact: { phoneNumber: 8008675309},
Address: { city: “New York”,
street: “94th”,
zipecode: 10025}}
Uncommitted Writes
street: “92nd”
Address.street = 94th
Address.street = 92nd

11. Write Conflicts

12. @aly_cabral
TXN
Write
TXN
Write
{Doc 1}
TXN write is unable to obtain a write lock!
TXN is ABORTED
rolledback entirely

13. @aly_cabral
TXN
Write
Operation
Non-Transactional Write
{Doc 1}
Operation Blocks Behind TXN Commit
Infinitely retries with Backoff Logic until MaxTimeMS is reached
Operation non-transactional write is unable to obtain a write lock!

14. @aly_cabral
When non-transactional writes
encounter write conflicts
Operation Blocks Behind TXN
Commit
Infinitely retries with Backoff Logic until MaxTimeMS is reached
When transactions encounter write
conflicts
Transactions abort
rollback entirely

15. @aly_cabral
TXN
Read
TXN
Write
{Doc 1}
Reads do not lock a document from being modified
Only writes make write conflicts

16. @aly_cabral
TXN
Write
TXN
Write
{_id: “OWNER12345”,
name: { first: “Aly”,
last: “Cabral”},
accountType: “Paid”,
pets: [ { type: “Dog”,
name: “Lady”}],
contact: { phoneNumber: 8008675309},
Address: { city: “New York”,
street: “94th”,
zipecode: 10025}}
street: “94th”
No-op writes are not writes
Only writes make write conflicts

17. Retrying Transactions

21. Production Considerations

22. @aly_cabral
P
S
S
Writes are replicated once committed

23. @aly_cabral
… …
keys
K K K KVV V V
txn
time
data
next
Update Structure
txn
time
data
next
1) Cache pressure builds up by write volume after a snapshot
2) Transactions use the same snapshot throughout it’s duration
3) The update structure can only be cleaned up after the snapshot is evicted
WiredTiger Cache

24. @aly_cabral
Working within Transaction limits
transactionLifetimeLimitSeconds = default to 60s
Commit READ-ONLY Transactions
Abort Abandoned Transactions
<1000 Documents Modified, <16MB Oplog Entry

25. @aly_cabral
DDL Operations
createIndex()
createCollection()
dropDatabase()
namespace
Pending DDL
Intent Lock

26. 1) All data modeling rules still apply
2) Transactions should not be the most common
operation (sanity check for rule 1)
3) Pass in the session to all statements
4) Implement retry logic, transactions can always
abort
5) Don’t leave transactions active unnecessarily
6) To trigger write conflicts, make sure you’re
doing writes
7) Plan for DDL Operations

27. Path to Transactions
1. ACID

28. “ACID transactions are a key capability for business critical
transactional systems, specifically around commerce processing.
No other database has both the power of NoSQL and cross
collection ACID transaction support.This combination will make it
easy for developers to write mission critical applications leveraging
the power of MongoDB.”
-- Dharmesh Panchmatia,
Director of E-commerce, Cisco Systems

29. Transactions: Acid

30. Transactions: aCid

31. Consistency, availability, partition
tolerance

32. Transactions: acId

33. Transactions: aciD

34. Path to Transactions
2. Why is this work hard?

35. Multi-year, all-hands company effort
• The storage layer,
• Sharding architecture,
• Introducing a global logical clock,
• New replication consensus protocol,
• Metadata management,
… and that’s just part of the list!
Not forgetting the driver teams, documentation and education.

36. Reason #1:
single-node ordering vs. group
ordering
MongoDB WiredTiger

37. Replication example
... 37 38 39
... 37 38 39
... 37 38 39
... 37 38 39
Primary Oplog
Secondary reads

38. Multi-threaded secondary oplog
application
OpLog ... 37 38 39 40 4241
Primary oplog order
Secondary apply order
40 39 41 38

39. “People assume that time is a strict progression
of cause to effect, but actually, from a non-linear,
non-subjective point of view, it is more like a big
ball of wibbily-wobbly timey-wimey … stuff.”
-- Doctor Who

40. Reason #2: increasing entanglement
Getting the right answer requires communication.

41. Path to Transactions
3. Timestamps

42. MongoDB’s timestamp
... 40 4138 39 42
ACID for a set of systems
durability based on replication

43. WiredTiger’s transaction ID
... 40 4138 39 42
ACID for a single-node
durability based on checkpoints and journaling

44. Timestamps override transaction IDs
64-bit value
fast comparisons
strictly increasing
MSB hex string API

45. Timestamp stored with each MVCC
update
FRUIT
mango
apple
banana
TXN ID
TIMESTAM
P

46. MongoDB secondary example
Set the timestamp at update or commit
OpLog
... 37 38 39 40 4241
Primary oplog order
Secondary timestamp order
... 37 38 39 40 4241

47. Overlapping reads with oplog
application
Secondary timestamp order
... 37 38 39 40 4241
Timestamp #1 Timestamp #2
3.6
4.0
Blocking
Parallel

48. Queries can be “as of” a timestamp
Reads see most recent
version committed with
an earlier timestamp

49. We need an “oldest” timestamp

50. Durability can be “as of” a timestamp

51. Replication rollback for a single
server
... 37 38 39 40 4241
Primary Oplog
... 37 38 39 117 119118
Corrected (Secondary) Oplog
Replacements

52. We need a “stable” timestamp

53. Sydney & New York

54. Thank You!