Welcome to the nightmare of locking, blocking and isolation levels!


Published on

There will always be locking inside your SQL Server box! In this session we go deep into how locking mechanism works, what are the main problems around locking, how we can resolve them and when isolation levels can actually be of help!

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Когато започнах се надявах да отида на безплатен ивент на подобна тема за това, за да мога да я разбера по – добре, а сега имам щастието самия аз да ви дам моите знания. Наистина е леко по – специална за мен и има огромна стойност Та...
  • Atomic – Error handling and how we use itConsistent – Foreign keys, check constraints, triggers, validation rules.Isolated – locking, isolation levelsDurable – transaction log – rolling back and forward
  • Every Transact-SQL statement is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back.
  • Mention try - catch
  • You use locking, because you do not like the fact that someone can change the data you are reading at the momentOptimistic – versionsКонтрол на достъпа
  • How much memory is used – locks on server level (decommissioned)Up to 60% of the buffer pool – out of lock memory errorsIn startup (with 0 option) it allocates 2500 locks. 2%LOCK BLOCK Lock Owner block64 bytes – 32 bit system 32 bytes – 32 bit 128 bytes – 64 bit system 64 bytes – 64 bit
  • Обясни какво се случва – как се минава от SUXИнтент лок – нужни са, защото SQL Server може да хваща локове на много различни обекти. Ако някой процес иска да локне таблицата, все пак трябва да има начин да му се каже, че някой ъпдейтва стойности в нея, нали?
  • Обясни какво се случва – как се минава от SUX+ Shared <- Exclusive <- SharedSIX - convert lock. Example – shared lock on a row now has to become X lock. However, for the exclusive to occur, we ed IX in the upper levels – that is the moment when we convert our S and IS locks to X and SIX locks
  • Всяка една транзакция преминава от тук. Обяснение защо.
  • Why do we need Shared locks – to explain it to the audienceWhy do we need intend locks and what is their goal? Why are they helping us?
  • Този слайд трябва да е последен в секцията introduction to locksDynamic Management ViewsSystem stored proceduresSQL ProfilerPerformance Monitor countersBlocked Process ReportDeadlock events and trace flag 1222SQL Server 2008 addsPerformance Data CollectionExtended Events
  • Not exactly – it always decides during compilation ROW or PAGE level – tow options only (when standard DML operations, not altering object)After that it could escalate to partition/table level
  • Memory takes memory 60…. InternalsEscalating in case of 40% of the memory used for locks Or 5000 locks on one objectIf escalation fails because of incompatible lock – sql server waits for another 1250 locks Always to table(partitioning) – auto to partition from 2008What is the solution?
  • Should be VERY VERY Careful when choosing to control it manually!! Could easy cause more problems, and it is rarely used as a lock escalation solution nor as a resolution of blocking problems! Could be set in specific cases only
  • The READPAST query hintERROR HANDLING for error 1222!
  • Blocking what is that, when does it happen, reasons behind (holding locks because of long running user trans (examples), long running trans because of resource usage problems (IO waits..), lock escalations because of row by row operations (examples), how to identify/ how to isolate reasons behindКонтекст: by default – pesimistic, read committed
  • Threads are killedLOCK_MONITOR checks every 5 seconds for deadlock. Cycle and conversion deadlocksError 1205 should be handled Some operations cannot be chosen for deadlock victims (rollbacks)After the deadlock occurs, the engine still continues to work and check every so often (100 ms) for other deadlocks
  • Shortly:isolation level RU, Isolation level RRReadpast - difference between READPAST, READUNCOMMITTED,and setting LOCK_TIMEOUT to 0. All these techniques let you avoid blocking problems,but the behavior is slightly different in each caseSerializable with Table lockDiagnose and recognize:Adam Machanic'ssp_WhoIsActive @get_additional_info = 1 will give you the isolation level, and I've used that for quick diagnostics to find out who's doing dirty reads.  Just good for a first pass though.+ Aaron’s script: http://www.mssqltips.com/sqlservertip/2475/sql-server-queries-with-hints/
  • RCSI, snapshot isolation, update conflicts
  • Snapshot transaction metadatasys.dm_tran_ version_store; sys.dm_tran_transactions_snapshot; sys.dm_tran_active_snapshot_database_transactions
  • Magi to add more if neededRecap of what we discussed in the previous slide?
  • Welcome to the nightmare of locking, blocking and isolation levels!

    1. 1. Българска SQL & BI потребителска групаhttp://bgsqlgroup.com SQL & BI User Group Bulgaria on FB
    2. 2. WELCOME TO THE NIGHTMARE OF LOCKING, BLOCKING AND ISOLATION LEVELS! Magi Naumova Boris Hristov BG SQL User Group Meeting, 4/4/2013
    3. 3. About Magi Working with SQL Server from v6.5 MCT from 1998 SQL Server Trainer and Consultant for more than 15 years, having more over 100 projects in Bulgaria, Finland, Germany, UK, Greece.. 5 years Senior Consultant in Microsoft Microsoft Certified Master SQL Server 2008 MVP SQL Server SolidQ Mentor & Trainer Now I am running SQL Master Academy training programwww.SQLMasterAcademy.com
    4. 4. About Bobi SQL Server DBA at HP Trainer at New Bulgarian University and FMI MCT & MCITP DBA SQL Server 2008 brshristov@live.com www.borishristov.com
    5. 5. Agenda…Transactions. What are they?Locks. What is there for us?Troubleshooting locking problemsTransaction Isolation Levels
    6. 6. Transactions. What are they?
    7. 7. What Are Transactions?A transaction is an Atomic unit of workA transaction leaves data in a ConsistentstateA transaction is Isolated from otherconcurrent transactionsA transaction is Durable
    8. 8. Auto Commit TransactionsDefault transaction modeEvery TSQL statement is committed or rolled back on completionCompile errors result in entire batch not being executedRun time errors may allow part of the batch to commit ---run time error -- compile error - partially executed USE AdventureWorks2012; GO CREATE TABLE TestBatch (Cola INT PRIMARY KEY, Colb CHAR(3)); GO INSERT INTO TestBatch VALUES (1, aaa); INSERT INTO TestBatch VALUES (2, bbb); INSERT INTO TestBatch VALUSE (3, ccc); ---Duplicate key error. VALUES (1, -- error, error! GO SELECT * FROM TestBatch; -- Returns no rows. 2. rows 1 and GO
    9. 9. Implicit TransactionsSQL Server is responsible for opening the transactionWe are responsible for committing or rolling it backCan be turned on from Connections tab in Server PropertiesSET IMPLICIT_TRANSACTIONS ON SET IMPLICIT_TRANSACTIONS ONUSE AdventureWorks2012 USE AdventureWorks2012GO GOUPDATE [Person].[Address] UPDATE [Person].[Address]SET AddressLine1=Microsoft, Bulgaria SET AddressLine1=Microsoft, BulgariaWHERE AddressID=2 WHERE AddressID=2COMMIT – this will write a change to the db ROLLBACK – this will not write a change to the d
    10. 10. Explicit TransactionsA transaction in which start and end of transaction is explicitly declaredBEGIN TRANSACTIONCOMMIT TRANSACTION OR ROLLBACK TRANSACTIONXACT_ABORT ON/OFF – control the rollback behavior SET XACT_ABORT ON – if run time error is generated everything is rolled back USE AdventureWorks2012 GO BEGIN TRANSACTION FundsTransfer GO EXEC HumanResources.DebitAccount 100, account1; EXEC HumanResources.CreditAccount 100, account2; COMMIT TRANSACTION;
    11. 11. Locks. What is there for us?
    12. 12. Methods of Concurrency ControlTwo main concurrency control types:1. Pessimistic – SQL Server uses locks, causes blocks and who said deadlocks?2. Optimistic2. Optimistic – SQL Server generates versions for everyone, but the updates…
    13. 13. What Are Locks and what is locking?Lock – internal memory structure that “tells” us what we all do with theresources inside the systemLocking – mechanism to protect the resources and guarantee consistent data
    14. 14. Common lock types Shared (S) Update (U) Used for: Reading Used for: Preparing to modifyDuration: Released almost immediately Duration: End of the transaction or until (depends on the isolation level) converted to exclusive (X) Exclusive (X) Intent Used for: Modifying Used for: Preventing incompatible Duration: End of the transaction locks Duration: End of the transaction
    15. 15. Lock CompatibilityNot all locks are compatible with other locks. Lock Shared Update Exclusive Shared (S)   X Update (U)  X X Exclusive (X) X X X
    16. 16. Lock Hierarchy Database Table Page Row
    17. 17. Let’s update a row. What do we need? S A query!  IX USE AdventureWorks2012 GO UPDATE [Person].[Address] SET AddressLine1=Microsoft, Bulgaria Header IX WHERE AddressID=2 Row Row Row Row X Row
    18. 18. Methods to View Locking Information Dynamic SQL Server Performance Management Profiler or monitor or Activity Views Extended Monitor Events
    19. 19. Troubleshooting locking problems
    20. 20. Locking and blocking Locking and blocking are often confused!Locking • The action of taking and potentially holding locks • Used to implement concurrency controlBlocking is result of locking! • One process needs to wait for another process to release locked resources • In a multiuser environment, there is always, always blocking! • Only a problem if it lasts too long
    21. 21. Lock granularity and escalationSQL Server decides (during compilation) the granularity of locks to be used: • Row • PageLock granularity can be controlled via LOCK HINTS (PAGLOCK, TABLOCKX,etc…)Escalation always happens this way:Row -> table lock (or partition lock if possible)Page -> table lock (or partition lock if possible)Lock escalation can be disabled: • Trace flag 1211 – disables lock escalation on server level • Trace flag 1224 – disables lock escalation on server level until 40% of the memory used is consumed
    22. 22. Lock escalationS SIX XIXXX >= 5000XX
    23. 23. Controlling Lock escalationSwitch the escalation level (per table) SELECT lock_escalation_desc FROM sys.tables WHERE name = Person.Address ALTER TABLE Person.Address SET (LOCK_ESCALATION = {AUTO | TABLE | DISABLE)AUTO – Partition-level escalation if the table is partitionedTABLE – Always table-level escalationDISABLE – Do not escalate until absolutely necessary
    24. 24. Lock Duration SET LOCK_TIMEOUT 5000SET LOCK_TIMEOUT specifies numberof milliseconds to wait BEGIN TRANDefault LOCK_TIMEOUT is -1 = waitindefinitely UPDATE Production.ProductInventorySession level option SET Quantity = 500 WHERE ProductID = 1;When timeout expires, error 1222 isreturned -- what happens if the update times out?Not a good idea to be used!Does not resolve blocking problems! DELETE FROM Production.ProductModel WHERE Name LIKE Mountain%; COMMIT
    25. 25. Blocking a.k.a live lockingBlocking occurs because of locked resourcesFirst incompatible lock request waitsAll other locks requests (even if compatible) WAITHow to resolve blocking problems:1. Keep the transactions as short as possible2. No user interactions required in the middle of the transaction3. Reduce row by row operations (cursors)4. Use indexes5. Consider a server to offload some of the workloads5. Choose isolation level
    26. 26. What Are Deadlocks?Who is victim?• Cost for Rollback• Deadlock priority – SET DEADLOCK_PRIOIRTY Task A Resource 1 Task B Resource 2
    27. 27. DEMO Capturing locking information Who is blocking who Lock escalation – both to table and partitionDeadlock and the SET DEADLOCK_PRIORITY option
    28. 28. Transaction isolation levelsTransaction Isolation Levels control when locks are taken andhow long they are held
    29. 29. Pessimistic Isolation Levels Changing Isolation level ◦ SET TRANSACTION ISOLATION LEVEL ….. on session level eXclusive lockTransaction 1 Read Update Select Uncommitted Transaction 2 Lowering the default Read Committed ◦ NOLOCK hint – set on the table level ◦ SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED You have Dirty Reads in this isolation level! Saying you want less blocking at the cost of inconsistent data! If you need to use it a lot, that means, you need a reporting database (or optimistic locking)
    30. 30. Pessimistic Isolation LevelsIncreasing the default level to Repeatable Read Repeatable Transaction 1 S(hared) lock Read select Transaction 2 Update S lock is held until the end of the transaction in order to read the same data Every transaction that has to update rows has to wait until T1 completes Inserts (phantoms) performed by another transaction can still occur in the table
    31. 31. Pessimistic Isolation LevelsIncreasing to Serializable Transaction 1 S(hared) lock Serializable select Transaction 2 Insert Repeatable Read + Holding the locks for a zone of rows If there is no index (or not used) the whole tale will be locked Highest level of isolation, lowest level of concurrency Be careful with using Frameworks (EF, LYNC)
    32. 32. DEMOPessimistic Isolation Levels
    33. 33. Optimistic ConcurrencyBased on Row versioning ◦ When updating a row, the previous version is stored in the version store ◦ The new row contains a pointer to the old row in the version store Transaction 1 V1 V2 Select Transaction 2Readers do not block writers and writers do not block readers.BUT writers can and will block writers, this can cause conflicts.Adds 14bytes to every rowNeeds Version Store i.e. TempDB space
    34. 34. Implementation – RCSI and SIRCSI ◦ Statement-Level versioning, i.e. any query sees the last version of data as of the beginning of the statement ◦ Requires ALTER ATABASE SET READ_COMMITTED_SNAPSHOT ON Transaction 1 V1 V2 Select Select in RCSI Transaction 2 Select in SISnapshot Isolation Level ◦ Session Level versioning, i.e. the most recent committed version of the rows as of the beginning of the transaction ◦ Requires ALTER DATABASE SET ALLOW_SNAPSHOT_ISOLATION ON ◦ Requires SET TRANSACTION ISOLATION LEVEL SNAPSHOT ◦ If used then better together with RCSI! Else, you have drawbacks of versioning only!
    35. 35. DEMOOptimistic Concurrency Isolation Levels
    36. 36. Managing Version storeThe Version Store ◦ SQL Server starts generating versions in tempdb as soon as a database is enabled for one of the snapshot-based isolation levels ◦ Maintains link lists of rows, the end of the list is the oldest version of that particular rowManaging Version Store ◦ SQL Server maintains a clean-up thread ◦ If file cannot grow, a snapshot query fails (tempdb growth errors) ◦ Monitoring counters ◦ Free Space in tempdb ◦ Version Store Size ◦ Version Generation Rate and Version Cleanup Rate ◦ Longest Transaction Running Time ◦ Snapshot Transactions
    37. 37. DEMOVersioning details and managing version store
    38. 38. SummaryIf you experience blocking consider implementing Optimistic ConcurrencyConsider RCSI only, it is way cheeperRCSI + SI only if needed higher isolation level per transaction (RR in pesimistic)If you have to use often no blocking hints the consider either reporting DB or optimisticconcurrencyBe careful with READPAST, NOLOCK, ….LOCK TIMEOUT and any other work aroundsIf you keep default concurrency, then follow the recommendations for minimizing blocking andlock escalationsBe careful with hidden hints/isolation levels changes using EF, LYNC, etc
    39. 39. ResourcesMCM Readiness videos on locking lecture and demoMCM Readiness video on Snapshot Isolation Levelhttp://blogs.msdn.com/b/bartd/archive/tags/sql+locking/http://www.sqlskills.com/blogs/paul/category/locking/Lock hints -http://www.techrepublic.com/article/control-sql-server-locking-with-hints/5181472
    40. 40. Thank you.• Magi.Naumova@SQLMasterAcademy.com ?• www.maginaumova.com• brshristov@live.com• www.borishristov.com