Jay Runkel, Master Solutions Architect
Read Isolation - Making your reads clean, committed,
and repeatable
jayrunkel
Read Isolation
We are all going to a Philadelphia Phillies game!
In San Francisco, on Aug 8th
We all want to sit in Section 126
One person in each row places order for each
row
Let’s all book at the same time
Ready...
Set...
Go...
There will likely be collisions
1.Many simultaneous reads and writes
2.Likely that:
§ Some ticket purchase attempts will fail (conflicts)
§ System may become sluggish
§ Users may perceive system as erratic as they try to book seats that
“disappear”
We could solve this by using a queue
A lot people would wait...
Agenda
1. Isolation
2. RDBMS Isolation Levels
3. MongoDB Isolation Levels
Operations and Transactions
Definitions for today:
• Operation – any database query (insert, update, select/find,
delete)
• Transaction – A sequence of operations contained in a start
transaction/commit/rollback block
Part 1 - Isolation
It’s the I in ACID
Atomicity
Consistency
Isolation
Durability
Databases process many concurrent operations
These operations may access the same records and conflict with one another
Safest Approach
Execute all operations/transactions sequentially
One at a time
• All other operations/transactions wait
• No conflicts
• Performance sucks
Baseball Example: Form a line
Isolation vs. Concurrency
Concurrency
Isolation
Isolation vs. Concurrency
Concurrency
Isolation
Read Uncommitted
Read Committed
Repeatable Reads
Serializable
We will define these
levels in a second
A little more background
information first
Part 2 - Isolation
Levels
(lack of) Isolation Challenges
Dirty Reads
Read data from a record that has been modified by another running transaction and not yet committed.
Phantom Reads
Only committed data is read
But, new records may be added (or deleted) by another transaction that commits after the current transaction has
started.
Non-Repeatable Reads
Only committed data is read
Reading the same record more than once within a transaction may return different results
(the record is updated by a transaction that commits between the reads)
Time Transaction 1 Transaction 2
0 Starts
1 Starts Update Seat 99
2 Read Seat 99 <irrelevant operation>
3 Commits <irrelevant operation>
4 Commits
Dirty Read
Transaction 1 returns the results transaction 2
ROLLBACK!
Time Transaction 1 Transaction 2
0 Starts
1 Select all seats in section Starts
2 Returning seats Reserve seat 28
3 Returning seats Reserve seat 29
4 Returning seats Free seat 27
5 Return seat 28 Commits
6 Commits
Phantom Reads
Some or all of the changes made by transaction
2 show up in the results of transaction 1
Time Transaction 1 Transaction 2
0 Starts
1 Read Seat 29 Starts
2 <irrelevant operation> Reserve Seat 29
3 <irrelevant operation> Commits
4 Read Seat 29
5 Commits
Non-Repeatable Read
The second read in transaction 1 sees the changes made by transaction 2
RDBMS Isolation Levels
Dirty Reads Non-Repeatable Reads Phantom Reads
Read Uncommitted yes yes yes
Read Committed no yes yes
Repeatable Reads no no yes
Serializable no no no
Most RDBMS Don’t Offer All Levels
Dirty Reads Non-Repeatable Reads Phantom Reads
Read Uncommitted yes yes yes
Read Committed no yes yes
Repeatable Reads no no yes
Serializable no no no
Oracle
Part 3 - MongoDB
Read Isolation
Read Isolation in MongoDB is determined by
Transactions (or not)
ReadConcern
WriteConcern (sometimes)
A quick detour on read and write concern
ReadConcern
Controls consistency and isolation level of reads
# of nodes (in a replica set) that have “committed” the write
§ Default is “local”
db.test.find({x:100}).readConcern(“local”)
ReadConcern local
Node 1
Node 2 Node 3
Application
Driver
{x: 101}
{x: 100}
{x : 100}{x: 100}
write
{x: 101}
Application
Driver
{x: 101}
db.test.find({x : {$gt : 100}})
.readConcern(“local”)
Node 1
Node 2 Node 3
Application
Driver
{x: 101}
{x: 101}
{x : 100}{x: 100}
write
Application
Driver
{x: 100}
ReadConcern: Majority
db.test.find({x : {$gt : 100}})
.readConcern(“majority”)
Write Concern
• Intelligent write receipt/confirmation
– Specifies the the number of nodes that must have written the
write to disk
– Default number is 1
db.test.insert({x:100},{writeConcern:{w:2}})
ReadConcerns
ReadConcern Description
Available Local value, doesn’t account for chunk migration
Local Local value, accounts for chunk migration
Majority Write has been acknowledged by a majority nodes in the replica set
Linearizable not available with transactions
majority plus confirms with secondaries that primary can commit majority
writes
Snapshot transactions only
transaction operations are guaranteed to have read from a snapshot of
majority-committed data.
Node 1
Node 2 Node 3
Application
Driver
Write Concern 1
{x: 100}
{x: 100}
{x : 100}{x: 100}
writeack
db.test.insert({x:100},
{writeConcern:{w:1}})
Node 1
Node 2 Node 3
Application
Driver
Write Concern Majority
{x: 100}
{x: 100}
{x : 100}{x: 100}
writeack
db.test.insert({x:100},
{writeConcern:{w:”majority”}})
End of Detour
Read Isolation Without Transactions
Available/Local ReadConcern → Read
Uncommitted
Read current version in memory
• Possibly before acknowledgement sent
• May be rolled back during a failover
• Dirty reads
Default isolation level for MongoDB
• Standalone, replica set, sharded clusters
Dirty Reads - Reading Rolled Back Data
Node 1
Node 2 Node 3
Application
Driver
{x: 101}
{x: 100}
{x : 100}{x: 100}
write
{x: 101}
Application
Driver
{x: 101}
Node 1
(Primary)
Dirty Reads - Reading Rolled Back Data
Node 2 Node 3
Application
Driver
{x: 101}
{x : 100}{x: 100}
Application
Driver
{x: 101}
{x: 100}
{x: 100}
Multi-Document Reads with Local/Available
No point-in-time read operations
• Writes made after the start of the read operation may be reflected in
results
• Inserts, updates, deletes
• Multiple versions of the same document may be returned
• Matching documents may be missed
Time Process 1 Process 2
0 Starts
1 Select all seats in section Starts
2 Returning seats Select seat 29
3 Returning seats Select seat 30
4 Returning seats Free seat 27
5 Returning seats Free seat 28
6 Completes Select seat 31
7 Completes
Phantom Reads (No Transactions)
Some of the changes made by transaction 2 show
up in the results of transaction 1
Majority - Without Transactions
No Dirty Reads
• Can’t read data that could be rolled back
Phantom reads are still possible
• Documents read by a query returning multiple documents can still be
affected by other write operations
Read Isolation With Transactions
Isolation With Multi-Document Transactions
“local” and “majority” ReadConcerns are upgraded to “snapshot”
isolation
This is true in MongoDB 4.0
§ May not be true in future versions
§ Code should specify the required isolation
Snapshot Isolation
With WriteConcern: majority
• Operations are guaranteed to have read from a snapshot of
majority-committed data
• No dirty or phantom reads
With WriteConcern:1
• Read from local snapshot
• Dirty reads are possible
• More performant, less durable
Snapshot Isolation – WriteConcern: 1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
startTransaction({wc:1})
update({_id: 1}, {$set : {x: 2}})
update({_id: 2}, {$set : {y: 2}})
...
{x:2}
{y:2}
ack {x:2}
{y:2}
{x:2}
{y:2}
commit
S2 S2 S2
Si = Snapshot at time i
Time Transaction 1 Transaction 2
0 s1.startTransaction(wc:1)
1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:1)
2 update({_id: 2}, {$set : {y: 2}}) find()
3 update({_id: 3}, {$set : {z: 2}}) ...
4 Commit (primary) Commit(primary)
5 Commit(majority)
6 Commit (majority)
7
Multi-Document Transaction, RC: snapshot, WC:1
The find in transaction 2 returns the state of the
documents before the start of transaction 1
Multi-Document Transaction, RC: snapshot, WC:1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
startTransaction({wc:1})
update({_id: 1}, {$set : {x: 2}})
update({_id: 2}, {$set : {y: 2}})
...
{x:2}
{y:2}
ack {x:2}
{y:2}
{x:2}
{y:2}
commit
S2 S2 S2
startTransaction({wc:1})
find()
...
Time Transaction 1 Transaction 2
0 s1.startTransaction(wc:1)
1 update({_id: 1}, {$set : {x: 2}})
2 update({_id: 2}, {$set : {y: 2}})
3 update({_id: 3}, {$set : {z: 2}})
4 Commit (primary)
5 Acknowledgement s2.startTransaction(wc:1)
6 Commit(majority) find()
7 ...
8 Commit
Multi-Document Transaction, RC: snapshot, WC:1
The find in transaction 2 returns the state of the
documents after (primary commit) of transaction 1
Multi-Document Transaction, RC: snapshot, WC:1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
startTransaction({wc:1})
update({_id: 1}, {$set : {x: 2}})
update({_id: 2}, {$set : {y: 2}})
...
{x:2}
{y:2}
ack {x:2}
{y:2}
{x:2}
{y:2}
commit
S2 S2 S2
startTransaction({wc:1})
find()
...
Snapshot Isolation – writeConcern: majority
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
startTransaction({wc:maj})
update({_id: 1}, {$set : {x: 2}})
update({_id: 2}, {$set : {y: 2}})
...
{x:2}
{y:2}
ack {x:2}
{y:2}
{x:2}
{y:2}
commit
S2 S2 S2
Time Transaction 1 Transaction 2
0 s1.startTransaction(wc:majority)
1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:majority)
2 update({_id: 2}, {$set : {y: 2}}) find()
3 update({_id: 3}, {$set : {z: 2}}) ...
4 Commit (primary) Commit(primary)
5
6 Commit(majority)
7 Acknowledgement
Multi-Document Trans., RC: snapshot, WC:maj
The find in transaction 2 returns the state of the
documents before the start of transaction 1
Time Transaction 1 Transaction 2
0 s1.startTransaction(wc:majority)
1 update({_id: 1}, {$set : {x: 2}})
2 update({_id: 2}, {$set : {y: 2}})
3 update({_id: 3}, {$set : {z: 2}})
4 Commit (primary)
5 s2.startTransaction(wc:majority)
6 Commit(majority) find()
7 Acknowledgement ...
8 Commit
Multi-Document Trans., RC: snapshot, WC:maj
The find in transaction 2 returns the state of the
documents before the start of transaction 1
Multi-Document Transaction, RC: snapshot,
WC:maj
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
{x:1}
{y:1}
S1
startTransaction({wc:maj})
update({_id: 1}, {$set : {x: 2}})
update({_id: 2}, {$set : {y: 2}})
...
{x:2}
{y:2}
ack {x:2}
{y:2}
{x:2}
{y:2}
commit
S2 S2 S2
startTransaction({wc:1})
find()
...
Review
ReadConcern Transactions WriteConcern Isolation Level
Available No ReadUncommitted
With sharding, may miss or return duplicate
documents
Local No ReadUncommitted
Majority No ReadCommitted
Linearizable No ReadCommitted
Guarantees connection to “real” primary
Snapshot Yes 1 No phantom reads
Reads are repeatable
Dirty reads are possible
Snapshot Yes Majority No dirty or phantom reads
Reads are repeatableConcurrency Isolation
????Questions
MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committed, and Repeatable

MongoDB World 2019: MongoDB Read Isolation: Making Your Reads Clean, Committed, and Repeatable

  • 1.
    Jay Runkel, MasterSolutions Architect Read Isolation - Making your reads clean, committed, and repeatable jayrunkel
  • 2.
  • 3.
    We are allgoing to a Philadelphia Phillies game!
  • 4.
  • 5.
    We all wantto sit in Section 126
  • 6.
    One person ineach row places order for each row
  • 7.
    Let’s all bookat the same time Ready... Set... Go...
  • 8.
    There will likelybe collisions 1.Many simultaneous reads and writes 2.Likely that: § Some ticket purchase attempts will fail (conflicts) § System may become sluggish § Users may perceive system as erratic as they try to book seats that “disappear”
  • 9.
    We could solvethis by using a queue A lot people would wait...
  • 10.
    Agenda 1. Isolation 2. RDBMSIsolation Levels 3. MongoDB Isolation Levels
  • 11.
    Operations and Transactions Definitionsfor today: • Operation – any database query (insert, update, select/find, delete) • Transaction – A sequence of operations contained in a start transaction/commit/rollback block
  • 12.
    Part 1 -Isolation
  • 13.
    It’s the Iin ACID Atomicity Consistency Isolation Durability
  • 14.
    Databases process manyconcurrent operations These operations may access the same records and conflict with one another
  • 15.
    Safest Approach Execute alloperations/transactions sequentially One at a time • All other operations/transactions wait • No conflicts • Performance sucks Baseball Example: Form a line
  • 16.
  • 17.
    Isolation vs. Concurrency Concurrency Isolation ReadUncommitted Read Committed Repeatable Reads Serializable We will define these levels in a second A little more background information first
  • 18.
    Part 2 -Isolation Levels
  • 19.
    (lack of) IsolationChallenges Dirty Reads Read data from a record that has been modified by another running transaction and not yet committed. Phantom Reads Only committed data is read But, new records may be added (or deleted) by another transaction that commits after the current transaction has started. Non-Repeatable Reads Only committed data is read Reading the same record more than once within a transaction may return different results (the record is updated by a transaction that commits between the reads)
  • 20.
    Time Transaction 1Transaction 2 0 Starts 1 Starts Update Seat 99 2 Read Seat 99 <irrelevant operation> 3 Commits <irrelevant operation> 4 Commits Dirty Read Transaction 1 returns the results transaction 2 ROLLBACK!
  • 21.
    Time Transaction 1Transaction 2 0 Starts 1 Select all seats in section Starts 2 Returning seats Reserve seat 28 3 Returning seats Reserve seat 29 4 Returning seats Free seat 27 5 Return seat 28 Commits 6 Commits Phantom Reads Some or all of the changes made by transaction 2 show up in the results of transaction 1
  • 22.
    Time Transaction 1Transaction 2 0 Starts 1 Read Seat 29 Starts 2 <irrelevant operation> Reserve Seat 29 3 <irrelevant operation> Commits 4 Read Seat 29 5 Commits Non-Repeatable Read The second read in transaction 1 sees the changes made by transaction 2
  • 23.
    RDBMS Isolation Levels DirtyReads Non-Repeatable Reads Phantom Reads Read Uncommitted yes yes yes Read Committed no yes yes Repeatable Reads no no yes Serializable no no no
  • 24.
    Most RDBMS Don’tOffer All Levels Dirty Reads Non-Repeatable Reads Phantom Reads Read Uncommitted yes yes yes Read Committed no yes yes Repeatable Reads no no yes Serializable no no no Oracle
  • 25.
    Part 3 -MongoDB Read Isolation
  • 26.
    Read Isolation inMongoDB is determined by Transactions (or not) ReadConcern WriteConcern (sometimes)
  • 27.
    A quick detouron read and write concern
  • 28.
    ReadConcern Controls consistency andisolation level of reads # of nodes (in a replica set) that have “committed” the write § Default is “local” db.test.find({x:100}).readConcern(“local”)
  • 29.
    ReadConcern local Node 1 Node2 Node 3 Application Driver {x: 101} {x: 100} {x : 100}{x: 100} write {x: 101} Application Driver {x: 101} db.test.find({x : {$gt : 100}}) .readConcern(“local”)
  • 30.
    Node 1 Node 2Node 3 Application Driver {x: 101} {x: 101} {x : 100}{x: 100} write Application Driver {x: 100} ReadConcern: Majority db.test.find({x : {$gt : 100}}) .readConcern(“majority”)
  • 31.
    Write Concern • Intelligentwrite receipt/confirmation – Specifies the the number of nodes that must have written the write to disk – Default number is 1 db.test.insert({x:100},{writeConcern:{w:2}})
  • 32.
    ReadConcerns ReadConcern Description Available Localvalue, doesn’t account for chunk migration Local Local value, accounts for chunk migration Majority Write has been acknowledged by a majority nodes in the replica set Linearizable not available with transactions majority plus confirms with secondaries that primary can commit majority writes Snapshot transactions only transaction operations are guaranteed to have read from a snapshot of majority-committed data.
  • 33.
    Node 1 Node 2Node 3 Application Driver Write Concern 1 {x: 100} {x: 100} {x : 100}{x: 100} writeack db.test.insert({x:100}, {writeConcern:{w:1}})
  • 34.
    Node 1 Node 2Node 3 Application Driver Write Concern Majority {x: 100} {x: 100} {x : 100}{x: 100} writeack db.test.insert({x:100}, {writeConcern:{w:”majority”}})
  • 35.
  • 36.
  • 37.
    Available/Local ReadConcern →Read Uncommitted Read current version in memory • Possibly before acknowledgement sent • May be rolled back during a failover • Dirty reads Default isolation level for MongoDB • Standalone, replica set, sharded clusters
  • 38.
    Dirty Reads -Reading Rolled Back Data Node 1 Node 2 Node 3 Application Driver {x: 101} {x: 100} {x : 100}{x: 100} write {x: 101} Application Driver {x: 101} Node 1 (Primary)
  • 39.
    Dirty Reads -Reading Rolled Back Data Node 2 Node 3 Application Driver {x: 101} {x : 100}{x: 100} Application Driver {x: 101} {x: 100} {x: 100}
  • 40.
    Multi-Document Reads withLocal/Available No point-in-time read operations • Writes made after the start of the read operation may be reflected in results • Inserts, updates, deletes • Multiple versions of the same document may be returned • Matching documents may be missed
  • 41.
    Time Process 1Process 2 0 Starts 1 Select all seats in section Starts 2 Returning seats Select seat 29 3 Returning seats Select seat 30 4 Returning seats Free seat 27 5 Returning seats Free seat 28 6 Completes Select seat 31 7 Completes Phantom Reads (No Transactions) Some of the changes made by transaction 2 show up in the results of transaction 1
  • 42.
    Majority - WithoutTransactions No Dirty Reads • Can’t read data that could be rolled back Phantom reads are still possible • Documents read by a query returning multiple documents can still be affected by other write operations
  • 43.
    Read Isolation WithTransactions
  • 44.
    Isolation With Multi-DocumentTransactions “local” and “majority” ReadConcerns are upgraded to “snapshot” isolation This is true in MongoDB 4.0 § May not be true in future versions § Code should specify the required isolation
  • 45.
    Snapshot Isolation With WriteConcern:majority • Operations are guaranteed to have read from a snapshot of majority-committed data • No dirty or phantom reads With WriteConcern:1 • Read from local snapshot • Dirty reads are possible • More performant, less durable
  • 46.
    Snapshot Isolation –WriteConcern: 1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 Si = Snapshot at time i
  • 47.
    Time Transaction 1Transaction 2 0 s1.startTransaction(wc:1) 1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:1) 2 update({_id: 2}, {$set : {y: 2}}) find() 3 update({_id: 3}, {$set : {z: 2}}) ... 4 Commit (primary) Commit(primary) 5 Commit(majority) 6 Commit (majority) 7 Multi-Document Transaction, RC: snapshot, WC:1 The find in transaction 2 returns the state of the documents before the start of transaction 1
  • 48.
    Multi-Document Transaction, RC:snapshot, WC:1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
  • 49.
    Time Transaction 1Transaction 2 0 s1.startTransaction(wc:1) 1 update({_id: 1}, {$set : {x: 2}}) 2 update({_id: 2}, {$set : {y: 2}}) 3 update({_id: 3}, {$set : {z: 2}}) 4 Commit (primary) 5 Acknowledgement s2.startTransaction(wc:1) 6 Commit(majority) find() 7 ... 8 Commit Multi-Document Transaction, RC: snapshot, WC:1 The find in transaction 2 returns the state of the documents after (primary commit) of transaction 1
  • 50.
    Multi-Document Transaction, RC:snapshot, WC:1 {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:1}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
  • 51.
    Snapshot Isolation –writeConcern: majority {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:maj}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2
  • 52.
    Time Transaction 1Transaction 2 0 s1.startTransaction(wc:majority) 1 update({_id: 1}, {$set : {x: 2}}) s2.startTransaction(wc:majority) 2 update({_id: 2}, {$set : {y: 2}}) find() 3 update({_id: 3}, {$set : {z: 2}}) ... 4 Commit (primary) Commit(primary) 5 6 Commit(majority) 7 Acknowledgement Multi-Document Trans., RC: snapshot, WC:maj The find in transaction 2 returns the state of the documents before the start of transaction 1
  • 53.
    Time Transaction 1Transaction 2 0 s1.startTransaction(wc:majority) 1 update({_id: 1}, {$set : {x: 2}}) 2 update({_id: 2}, {$set : {y: 2}}) 3 update({_id: 3}, {$set : {z: 2}}) 4 Commit (primary) 5 s2.startTransaction(wc:majority) 6 Commit(majority) find() 7 Acknowledgement ... 8 Commit Multi-Document Trans., RC: snapshot, WC:maj The find in transaction 2 returns the state of the documents before the start of transaction 1
  • 54.
    Multi-Document Transaction, RC:snapshot, WC:maj {x:1} {y:1} S1 {x:1} {y:1} S1 {x:1} {y:1} S1 startTransaction({wc:maj}) update({_id: 1}, {$set : {x: 2}}) update({_id: 2}, {$set : {y: 2}}) ... {x:2} {y:2} ack {x:2} {y:2} {x:2} {y:2} commit S2 S2 S2 startTransaction({wc:1}) find() ...
  • 55.
    Review ReadConcern Transactions WriteConcernIsolation Level Available No ReadUncommitted With sharding, may miss or return duplicate documents Local No ReadUncommitted Majority No ReadCommitted Linearizable No ReadCommitted Guarantees connection to “real” primary Snapshot Yes 1 No phantom reads Reads are repeatable Dirty reads are possible Snapshot Yes Majority No dirty or phantom reads Reads are repeatableConcurrency Isolation
  • 56.