SQL Transactions - What they are good for and how they work

2. ( ) Transactions are Brackets Starting a transaction: Explicit: START TRANSACTION Implicit: INSERT / UPDATE / DELETE SELECT CREATE Terminating a transaction: Explicit: COMMIT ROLLBACK Implementation deﬁned: Due to errors CREATE … Varies  amongst  products

3. Transactions have Characteristics Access-Mode: READ_ONLY READ_WRITE Constraint Mode: IMMEDIATE DEFERRED Transaction-Isolation-Level: READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE Diagnostic Size: Archaic

4. Transactions have Characteristics Access-Mode: READ_ONLY READ_WRITE Constraint Mode: IMMEDIATE DEFERRED Transaction-Isolation-Level: READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE Diagnostic Size: Archaic Not strictly a  transaction   characteristic

5. Setting Transaction Characteristics With explicit transaction begin: START TRANSACTION ISOLATION LEVEL SERIALIZABLE READ WRITE For next (implicitly started) transaction: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE READ WRITE The standard uses SERIALIZEABLE as default,  but most implementations default to READ COMMITTED. In practice: Use an API if possible.

6. Use Cases for Transactions

7. Use Cases for Transactions When writing

8. Use Cases for Transactions Backing out incomplete changes… … by the program with rollback … by the RDBMS in case of crash When writing

9. Use Cases for Transactions Backing out incomplete changes… … by the program with rollback … by the RDBMS in case of crash When writing (Boring)

10. Use Cases for Transactions Backing out incomplete changes… … by the program with rollback … by the RDBMS in case of crash When writing When reading (Boring)

11. Use Cases for Transactions Backing out incomplete changes… … by the program with rollback … by the RDBMS in case of crash When writing When reading Simplify concurrent programs… … by utilising the right  transaction isolation level (Boring)

12. Use Cases for Transactions Backing out incomplete changes… … by the program with rollback … by the RDBMS in case of crash When writing When reading Simplify concurrent programs… … by utilising the right  transaction isolation level (Boring) Amazing!

13. Standard Transaction Isolation Levels Phenomena covered by the Standard Dirty Read Non-Repeatable Read Phantom Read READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE STOP STOP STOP STOPSTOPSTOP

14. Standard Transaction Isolation Levels Phenomena covered by the Standard Dirty Read Non-Repeatable Read Phantom Read READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE STOP STOP STOP STOPSTOPSTOP Incomplete,  unfortunately

15. How Comes? SQL is a declarative language:  the standard doesn’t say how to implement it,  it just describes the effects it has. Unfortunately, SQL-92 was written with locking in mind,  and thus only describes the effects in the granularity  in which they appear whey implemented using locking. Although a commonly known issue, it was never changed. https://en.wikipedia.org/wiki/Snapshot_isolation

16. Transaction Isolation Using Locks Phenomena covered by the Standard Dirty Read Non-Repeatable Read Phantom Read READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE ▶ Row (short) ▶ Row (TX) ▶ Row (TX) ▶ Predicate (TX) STOP STOP STOP STOPSTOPSTOP

17. Transaction Isolation Using Locks Phenomena covered by the Standard Dirty Read Non-Repeatable Read Phantom Read READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE ▶ Row (short) ▶ Row (TX) ▶ Row (TX) ▶ Predicate (TX) STOP STOP STOP STOPSTOPSTOP Locks preventing  phenomena:  What (how long)

21. Example: Write Skew Make sure the sum of two rows is positive  (e.g., two bank accounts belonging to the same person) Code to withdraw from one account  (naive schema and not considering concurrency) UPDATE accounts  SET balance = balance - 200  WHERE account = 1 SELECT SUM(balance)  FROM accounts  WHERE account IN (1,2) >= 0 commit   <0 rollback

22. Examples: Disclaimer The following examples just demonstrate  one possible way two transactions could interact. The examples are by no means exhaustive.

23. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2

24. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1

25. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2

26. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2)

27. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) -200

28. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) -200

29. Write Skew in Lock-based READ UNCOMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) -200 -200

30. Write Skew in Lock-based READ COMMITTED

31. Write Skew in Lock-based READ COMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2

32. Write Skew in Lock-based READ COMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1

33. Write Skew in Lock-based READ COMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2

34. Write Skew in Lock-based READ COMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) Blocked ▶

35. Write Skew in Lock-based READ COMMITTED Account Balance 1 100 2 100 Transaction 1 Transaction 2 ▶ 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 ▶ 1 -100 2 -100 ◀ UPDATE accounts SET balance=balance-200 WHERE account=2 ▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2) Blocked Dead lock ▶ ◀▶ 1 -100 2 -100 ◀ SELECT sum(balance) FROM accounts WHERE account IN (1,2)▶

36. First Write, then Read: Conclusion For this code, READ COMMITTED is enough  to make one of the transactions succeed  (assuming a proper deadlock detection).

37. First Write, then Read: Conclusion For this code, READ COMMITTED is enough  to make one of the transactions succeed  (assuming a proper deadlock detection). Exercise: Does this hold true in these cases too?

38. First Write, then Read: Conclusion For this code, READ COMMITTED is enough  to make one of the transactions succeed  (assuming a proper deadlock detection). Exercise: Does this hold true in these cases too? SELECT sum(balance)  FROM accounts  WHERE account IN (1,2) (iff >= 200) UPDATE accounts  SET balance = balance - 200  WHERE account = :a

39. First Write, then Read: Conclusion For this code, READ COMMITTED is enough  to make one of the transactions succeed  (assuming a proper deadlock detection). Exercise: Does this hold true in these cases too? SELECT sum(balance)  FROM accounts  WHERE account IN (1,2) (iff >= 200) UPDATE accounts  SET balance = balance - 200  WHERE account = :a SELECT balance FROM accounts  WHERE account = :a UPDATE accounts SET balance = :b  WHERE account = :a SELECT sum(balance)  FROM accounts  WHERE account IN (1,2)

40. Write-Skew: General Case When using lock-based isolation   REPEATABLE READ  covers with write-skew too.

41. What’s Bad About Locking?

42. What’s Bad About Locking? Writers will always need to block writers anyway

43. What’s Bad About Locking? Writers will always need to block writers anyway Readers never block readers

44. What’s Bad About Locking? Writers will always need to block writers anyway Readers never block readers Writers block readers,  readers block writers  (except in READ UNCOMMITTED)

45. What’s Bad About Locking? Writers will always need to block writers anyway Readers never block readers Writers block readers,  readers block writers  (except in READ UNCOMMITTED) Great!

46. What’s Bad About Locking? Writers will always need to block writers anyway Readers never block readers Writers block readers,  readers block writers  (except in READ UNCOMMITTED) Not so good Great!

47. Multi Version Concurrency Control (MVCC)

48. Multi Version Concurrency Control (MVCC) Instead of waiting for writers,  just use the previous committed version of the data  (pretend the read happened before the write)

49. Multi Version Concurrency Control (MVCC) Instead of waiting for writers,  just use the previous committed version of the data  (pretend the read happened before the write) The reader eﬀectively works on a snapshot

50. Multi Version Concurrency Control (MVCC) Instead of waiting for writers,  just use the previous committed version of the data  (pretend the read happened before the write) The reader eﬀectively works on a snapshot This prevents all three phenomena mentioned in the standard:  dirty read  non-repeatable read  phantom read

51. Write Skew When Using a Snapshot Transaction 1 Transaction 2Account Balance 1 100 2 100

52. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100

53. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 100 2 -100 UPDATE accounts SET balance=balance-200 WHERE account=2 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100

56. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 100 2 -100 UPDATE accounts SET balance=balance-200 WHERE account=2 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100 SELECT sum(balance) FROM accounts WHERE account IN (1,2) 1 -100 2 100

57. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 100 2 -100 UPDATE accounts SET balance=balance-200 WHERE account=2 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100 SELECT sum(balance) FROM accounts WHERE account IN (1,2) 1 -100 2 100 1 100 2 -100 SELECT sum(balance) FROM accounts WHERE account IN (1,2)

58. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 100 2 -100 UPDATE accounts SET balance=balance-200 WHERE account=2 Account Balance 1 -100 2 -100 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100 SELECT sum(balance) FROM accounts WHERE account IN (1,2) 1 -100 2 100 1 100 2 -100 SELECT sum(balance) FROM accounts WHERE account IN (1,2)

59. Write Skew When Using a Snapshot Transaction 1 Transaction 2 1 100 2 -100 UPDATE accounts SET balance=balance-200 WHERE account=2 Account Balance 1 -100 2 -100 1 -100 2 100 UPDATE accounts SET balance=balance-200 WHERE account=1 Account Balance 1 100 2 100 SELECT sum(balance) FROM accounts WHERE account IN (1,2) 1 -100 2 100 1 100 2 -100 SELECT sum(balance) FROM accounts WHERE account IN (1,2)

60. Can Snapshots and Serialisability Coexist?

61. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities.

62. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all)

63. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic

64. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic READ ONLY transactions are an important special case

65. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic READ ONLY transactions are an important special case InnoDB  SQL Server  DB2

66. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic READ ONLY transactions are an important special case InnoDB  SQL Server  DB2 PostgreSQL

67. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic READ ONLY transactions are an important special case InnoDB  SQL Server  DB2 PostgreSQL Oracle?

68. Can Snapshots and Serialisability Coexist? Simple snapshots cannot prevent write skew abnormalities. But most transactions are  not vulnerable to write skew anyway.  (e.g., TPC-C is not vulnerable at all) The only two ways to prevent write skew  abnormalities at the database are  (1) putting locks when reading and  (2) prooﬁng the serialization graph is acyclic READ ONLY transactions are an important special case InnoDB  SQL Server  DB2 PostgreSQL

69. Can Snapshots and Serialisability Coexist? PostgreSQL 9.1+ is the only(?) database that  uses MVCC and detects serialization graph cycles.  ➜ Use SERIALIZABLE and you are done  In InnoDB (MySQL, MariaDB) and SQL Server, MVCC and Lock-Based SERIALIZABLE isolation can be

70. How to use Snapshots and Prevent Write Skew Transaction type READ ONLY READ/WRITE DB2 LUW REPEATABLE READ  (=SERIALIZABLE) REPEATABLE READ  (=SERIALIZALBE) InnoDB  (MySQL, MariaDB) REPEATABLE READ1 SERIALIZABLE PostgreSQL 9.1+ SERIALIZABLE SERIALIZABLE Oracle SERIALIZABLE SERIALIZABLE SQL Server 2005+ SNAPSHOT SERIALIZABLE 1: MySQL’s REPEATABLE READ also protects against phantom reads. KEY No shared locks  No write skew issues Shared locks  No write skew issues No shared locks  Write skew issues

71. SQL Server is Special Except SQL Server, all MVCC capable databases use it per default. In SQL Server, it must be enabled:  ALTER DATABASE … SET allow_snapshot_isolation on; This will make write operations keep old versions  (needs more space in permanent DB and tempdb) Then you can use SNAPSHOT isolation (e.g., in read-only transactions).  Staying in SERIALISABLE for read/write transactions prevents write skew issues.  Note: Hint remain effective: NOLOCK will still do dirty-read in SNAPSHOT isolation.

72. Keep Transactions Focused Don’t select data that is irrelevant for the transaction.  (an innocent looking function-call might query something you are not aware of) Do unrelated stuff in distinct transactions. Work fast.  (Never, ever keep a transaction open waiting for someone)  (In the code, but also not in your ad-hoc SQL session!) Tune your statements.  (That makes your transactions shorter too)

SQL Transactions - What they are good for and how they work

Markus Winand

SQL Transactions - What they are good for and how they work