There will always be locking inside your SQL Server box! In this session we go deep into how locking mechanism works, what are the main problems around locking, how we can resolve them and when isolation levels can actually be of help!
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Welcome to the nightmare of locking, blocking and isolation levels!
1. Българска SQL & BI
потребителска група
http://bgsqlgroup.com SQL & BI User Group Bulgaria on FB
2. WELCOME TO THE NIGHTMARE OF
LOCKING, BLOCKING
AND ISOLATION LEVELS!
Magi Naumova
Boris Hristov
BG SQL User Group Meeting,
4/4/2013
3. About Magi
Working with SQL Server from v6.5
MCT from 1998
SQL Server Trainer and Consultant for more than 15 years, having
more over 100 projects in Bulgaria, Finland, Germany, UK, Greece..
5 years Senior Consultant in Microsoft
Microsoft Certified Master SQL Server 2008
MVP SQL Server
SolidQ Mentor & Trainer
Now I am running SQL Master Academy training program
www.SQLMasterAcademy.com
4. About Bobi
SQL Server DBA at HP
Trainer at New Bulgarian University and FMI
MCT & MCITP DBA SQL Server 2008
brshristov@live.com
www.borishristov.com
5. Agenda…
Transactions. What are they?
Locks. What is there for us?
Troubleshooting locking problems
Transaction Isolation Levels
7. What Are Transactions?
A transaction is an Atomic unit of work
A transaction leaves data in a Consistent
state
A transaction is Isolated from other
concurrent transactions
A transaction is Durable
8. Auto Commit Transactions
Default transaction mode
Every TSQL statement is committed or rolled back on completion
Compile errors result in entire batch not being executed
Run time errors may allow part of the batch to commit
---run time error
-- compile error - partially executed
USE AdventureWorks2012;
GO
CREATE TABLE TestBatch (Cola INT PRIMARY KEY, Colb CHAR(3));
GO
INSERT INTO TestBatch VALUES (1, 'aaa');
INSERT INTO TestBatch VALUES (2, 'bbb');
INSERT INTO TestBatch VALUSE (3, 'ccc'); ---Duplicate key error.
VALUES (1, -- error, error!
GO
SELECT * FROM TestBatch; -- Returns no rows. 2.
rows 1 and
GO
9. Implicit Transactions
SQL Server is responsible for opening the transaction
We are responsible for committing or rolling it back
Can be turned on from Connections tab in Server Properties
SET IMPLICIT_TRANSACTIONS ON SET IMPLICIT_TRANSACTIONS ON
USE AdventureWorks2012 USE AdventureWorks2012
GO GO
UPDATE [Person].[Address] UPDATE [Person].[Address]
SET AddressLine1='Microsoft, Bulgaria' SET AddressLine1='Microsoft, Bulgaria'
WHERE AddressID=2 WHERE AddressID=2
COMMIT – this will write a change to the db ROLLBACK – this will not write a change to the d
10. Explicit Transactions
A transaction in which start and end of transaction is explicitly declared
BEGIN TRANSACTION
COMMIT TRANSACTION OR ROLLBACK TRANSACTION
XACT_ABORT ON/OFF – control the rollback behavior
SET XACT_ABORT ON – if run time error is generated everything is rolled back
USE AdventureWorks2012
GO
BEGIN TRANSACTION FundsTransfer
GO
EXEC HumanResources.DebitAccount '100', 'account1';
EXEC HumanResources.CreditAccount '100', 'account2';
COMMIT TRANSACTION;
12. Methods of Concurrency Control
Two main concurrency control types:
1. Pessimistic – SQL Server uses locks, causes blocks and who said
deadlocks?
2. Optimistic
2. Optimistic – SQL Server generates versions for everyone, but the
updates…
13. What Are Locks and what is locking?
Lock – internal memory structure that “tells” us what we all do with the
resources inside the system
Locking – mechanism to protect the resources and guarantee consistent data
14. Common lock types
Shared (S) Update (U)
Used for: Reading Used for: Preparing to modify
Duration: Released almost immediately Duration: End of the transaction or until
(depends on the isolation level) converted to exclusive (X)
Exclusive (X) Intent
Used for: Modifying Used for: Preventing incompatible
Duration: End of the transaction locks
Duration: End of the transaction
15. Lock Compatibility
Not all locks are compatible with other locks.
Lock Shared Update Exclusive
Shared (S)
X
Update (U)
X X
Exclusive
(X)
X X X
17. Let’s update a row. What do we need?
S
A query!
IX
USE AdventureWorks2012
GO
UPDATE [Person].[Address]
SET AddressLine1='Microsoft, Bulgaria' Header IX
WHERE AddressID=2
Row
Row
Row
Row X
Row
18. Methods to View Locking Information
Dynamic SQL Server Performance
Management Profiler or monitor or Activity
Views Extended Monitor
Events
20. Locking and blocking
Locking and blocking are often confused!
Locking
• The action of taking and potentially holding locks
• Used to implement concurrency control
Blocking is result of locking!
• One process needs to wait for another process to release locked resources
• In a multiuser environment, there is always, always blocking!
• Only a problem if it lasts too long
21. Lock granularity and escalation
SQL Server decides (during compilation) the granularity of locks to be used:
• Row
• Page
Lock granularity can be controlled via LOCK HINTS (PAGLOCK, TABLOCKX,
etc…)
Escalation always happens this way:
Row -> table lock (or partition lock if possible)
Page -> table lock (or partition lock if possible)
Lock escalation can be disabled:
• Trace flag 1211 – disables lock escalation on server level
• Trace flag 1224 – disables lock escalation on server level until 40% of the memory
used is consumed
23. Controlling Lock escalation
Switch the escalation level (per table)
SELECT lock_escalation_desc
FROM sys.tables
WHERE name = 'Person.Address'
ALTER TABLE Person.Address SET (LOCK_ESCALATION = {AUTO |
TABLE | DISABLE)
AUTO – Partition-level escalation if the table is partitioned
TABLE – Always table-level escalation
DISABLE – Do not escalate until absolutely necessary
24. Lock Duration
SET LOCK_TIMEOUT 5000
SET LOCK_TIMEOUT specifies number
of milliseconds to wait BEGIN TRAN
Default LOCK_TIMEOUT is -1 = wait
indefinitely UPDATE
Production.ProductInventory
Session level option SET Quantity = 500
WHERE ProductID = 1;
When timeout expires, error 1222 is
returned -- what happens if the update
times out?
Not a good idea to be used!
Does not resolve blocking problems! DELETE FROM
Production.ProductModel
WHERE Name LIKE 'Mountain%';
COMMIT
25. Blocking a.k.a live locking
Blocking occurs because of locked resources
First incompatible lock request waits
All other locks requests (even if compatible) WAIT
How to resolve blocking problems:
1. Keep the transactions as short as possible
2. No user interactions required in the middle of the transaction
3. Reduce row by row operations (cursors)
4. Use indexes
5. Consider a server to offload some of the workloads
5. Choose isolation level
26. What Are Deadlocks?
Who is victim?
• Cost for Rollback
• Deadlock priority – SET DEADLOCK_PRIOIRTY
Task A Resource 1
Task B
Resource 2
27. DEMO
Capturing locking information
Who is blocking who
Lock escalation – both to table and partition
Deadlock and the SET DEADLOCK_PRIORITY option
29. Pessimistic Isolation Levels
Changing Isolation level
◦ SET TRANSACTION ISOLATION LEVEL ….. on session level
eXclusive lock
Transaction 1
Read
Update
Select
Uncommitted
Transaction 2
Lowering the default Read Committed
◦ NOLOCK hint – set on the table level
◦ SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
You have Dirty Reads in this isolation level! Saying you want less blocking at the cost of inconsistent data!
If you need to use it a lot, that means, you need a reporting database (or optimistic locking)
30. Pessimistic Isolation Levels
Increasing the default level to Repeatable Read
Repeatable
Transaction 1 S(hared) lock
Read
select Transaction 2
Update
S lock is held until the end of the transaction in order to read the same data
Every transaction that has to update rows has to wait until T1 completes
Inserts (phantoms) performed by another transaction can still occur in the table
31. Pessimistic Isolation Levels
Increasing to Serializable
Transaction 1 S(hared) lock
Serializable
select
Transaction 2
Insert
Repeatable Read + Holding the locks for a zone of rows
If there is no index (or not used) the whole tale will be locked
Highest level of isolation, lowest level of concurrency
Be careful with using Frameworks (EF, LYNC)
33. Optimistic Concurrency
Based on Row versioning
◦ When updating a row, the previous version is stored in the version store
◦ The new row contains a pointer to the old row in the version store
Transaction 1
V1 V2
Select
Transaction 2
Readers do not block writers and writers do not block readers.
BUT writers can and will block writers, this can cause conflicts.
Adds 14bytes to every row
Needs Version Store i.e. TempDB space
34. Implementation – RCSI and SI
RCSI
◦ Statement-Level versioning, i.e. any query sees the last version of data as of the beginning of the statement
◦ Requires ALTER ATABASE SET READ_COMMITTED_SNAPSHOT ON
Transaction 1
V1 V2
Select Select in RCSI
Transaction 2 Select in SI
Snapshot Isolation Level
◦ Session Level versioning, i.e. the most recent committed version of the rows as of the beginning of the transaction
◦ Requires ALTER DATABASE SET ALLOW_SNAPSHOT_ISOLATION ON
◦ Requires SET TRANSACTION ISOLATION LEVEL SNAPSHOT
◦ If used then better together with RCSI! Else, you have drawbacks of versioning only!
36. Managing Version store
The Version Store
◦ SQL Server starts generating versions in tempdb as soon as a database is enabled for one of the
snapshot-based isolation levels
◦ Maintains link lists of rows, the end of the list is the oldest version of that particular row
Managing Version Store
◦ SQL Server maintains a clean-up thread
◦ If file cannot grow, a snapshot query fails (tempdb growth errors)
◦ Monitoring counters
◦ Free Space in tempdb
◦ Version Store Size
◦ Version Generation Rate and Version Cleanup Rate
◦ Longest Transaction Running Time
◦ Snapshot Transactions
38. Summary
If you experience blocking consider implementing Optimistic Concurrency
Consider RCSI only, it is way cheeper
RCSI + SI only if needed higher isolation level per transaction (RR in pesimistic)
If you have to use often no blocking hints the consider either reporting DB or optimistic
concurrency
Be careful with READPAST, NOLOCK, ….LOCK TIMEOUT and any other work arounds
If you keep default concurrency, then follow the recommendations for minimizing blocking and
lock escalations
Be careful with hidden hints/isolation levels changes using EF, LYNC, etc
39. Resources
MCM Readiness videos on locking lecture and demo
MCM Readiness video on Snapshot Isolation Level
http://blogs.msdn.com/b/bartd/archive/tags/sql+lockin
g/
http://www.sqlskills.com/blogs/paul/category/locking/
Lock hints -
http://www.techrepublic.com/article/control-sql-server-
locking-with-hints/5181472
Когато започнах се надявах да отида на безплатен ивент на подобна тема за това, за да мога да я разбера по – добре, а сега имам щастието самия аз да ви дам моите знания. Наистина е леко по – специална за мен и има огромна стойност Та...
Atomic – Error handling and how we use itConsistent – Foreign keys, check constraints, triggers, validation rules.Isolated – locking, isolation levelsDurable – transaction log – rolling back and forward
Every Transact-SQL statement is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back.
Mention try - catch
You use locking, because you do not like the fact that someone can change the data you are reading at the momentOptimistic – versionsКонтрол на достъпа
How much memory is used – locks on server level (decommissioned)Up to 60% of the buffer pool – out of lock memory errorsIn startup (with 0 option) it allocates 2500 locks. 2%LOCK BLOCK Lock Owner block64 bytes – 32 bit system 32 bytes – 32 bit 128 bytes – 64 bit system 64 bytes – 64 bit
Обясни какво се случва – как се минава от SUXИнтент лок – нужни са, защото SQL Server може да хваща локове на много различни обекти. Ако някой процес иска да локне таблицата, все пак трябва да има начин да му се каже, че някой ъпдейтва стойности в нея, нали?
Обясни какво се случва – как се минава от SUX+ Shared <- Exclusive <- SharedSIX - convert lock. Example – shared lock on a row now has to become X lock. However, for the exclusive to occur, we ed IX in the upper levels – that is the moment when we convert our S and IS locks to X and SIX locks
Всяка една транзакция преминава от тук. Обяснение защо.
Why do we need Shared locks – to explain it to the audienceWhy do we need intend locks and what is their goal? Why are they helping us?
Този слайд трябва да е последен в секцията introduction to locksDynamic Management ViewsSystem stored proceduresSQL ProfilerPerformance Monitor countersBlocked Process ReportDeadlock events and trace flag 1222SQL Server 2008 addsPerformance Data CollectionExtended Events
Not exactly – it always decides during compilation ROW or PAGE level – tow options only (when standard DML operations, not altering object)After that it could escalate to partition/table level
Memory takes memory 60…. InternalsEscalating in case of 40% of the memory used for locks Or 5000 locks on one objectIf escalation fails because of incompatible lock – sql server waits for another 1250 locks Always to table(partitioning) – auto to partition from 2008What is the solution?
Should be VERY VERY Careful when choosing to control it manually!! Could easy cause more problems, and it is rarely used as a lock escalation solution nor as a resolution of blocking problems! Could be set in specific cases only
The READPAST query hintERROR HANDLING for error 1222!
Blocking what is that, when does it happen, reasons behind (holding locks because of long running user trans (examples), long running trans because of resource usage problems (IO waits..), lock escalations because of row by row operations (examples), how to identify/ how to isolate reasons behindКонтекст: by default – pesimistic, read committed
Threads are killedLOCK_MONITOR checks every 5 seconds for deadlock. Cycle and conversion deadlocksError 1205 should be handled Some operations cannot be chosen for deadlock victims (rollbacks)After the deadlock occurs, the engine still continues to work and check every so often (100 ms) for other deadlocks
Shortly:isolation level RU, Isolation level RRReadpast - difference between READPAST, READUNCOMMITTED,and setting LOCK_TIMEOUT to 0. All these techniques let you avoid blocking problems,but the behavior is slightly different in each caseSerializable with Table lockDiagnose and recognize:Adam Machanic'ssp_WhoIsActive @get_additional_info = 1 will give you the isolation level, and I've used that for quick diagnostics to find out who's doing dirty reads. Just good for a first pass though.+ Aaron’s script: http://www.mssqltips.com/sqlservertip/2475/sql-server-queries-with-hints/