Your SlideShare is downloading. ×
0
אין משתמשים – אין בעיות<br />Meir Dudai<br />SQL Server MVP<br />Valinor<br />
Content slide<br />
When do I need to start worrying?<br />100’s of queries/second<br />1000’s of DMLs/second<br />VLDB’s<br />
When do I need to start worrying?<br />Application was designed and developed poorly…<br />
When do I need to start worrying?<br />You have just migrated from other platform<br />
What can I do?<br />Throw in more hardware – not very useful<br />
What can I really do?<br />Improve application design<br />Use database enhancements and features<br />Change system archi...
Who Am I?<br />Meir Dudai<br /><ul><li>MVP
SQL Server consultant
Editor of SQLServer.co.il</li></li></ul><li>How to increase concurrency without changing your application<br />How to adju...
Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural desig...
Common Concurrency Issues with Applications<br />Wrong transaction isolation level<br />Incorrect locking granularity<br /...
Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural desig...
GreenRoad Case Study:RCSI<br />GreenRoad is an Israeli startup improving driving behavior for fleets and consumers<br />Wo...
Tran2 (Select)<br />Tran1 (Update)<br />Reader Writer Blocking<br />Data Page<br />S-Lock<br />Blocked<br />Row-1<br />Row...
Read Committed SnapshotStatement-Level ‘Snapshot Isolation’<br />New “flavor” of read committed<br />Turn ON/OFF on a data...
Demo<br />
Lock Escalation<br />HOBT<br />T1: IX<br />T1: X<br />Page<br />Page<br />Page<br />T1: X<br />T1: X<br />T1: X<br />T1: X...
Lock Escalation<br />Converting finer-grain locks to coarse grain locks.<br />Row to Table<br />Page to Table.<br />Benefi...
Partitioned Tables and Indexes<br />SQL Server 2005 introduced partitioning, which some customers use to scale a query wor...
Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions<br />Only way to solve th...
Lock Escalation: The Solution<br />SQL Server 2008 allows lock escalation to the partition level, allowing concurrent acce...
Demo<br />
Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural desig...
Silent Blocker: Page Latches<br />What is a Pagelatch?<br />Access patterns leading to latch contention<br />Clustering in...
Silent Blocker: Page Latches<br />What can you do?<br />Cluster on some other column<br />Partition the table on some othe...
Demo<br />
Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural desig...
Codename Lyngby Case Study: Power of bulk inserts<br />Security/audit system<br />50GB of data inserted daily<br />Data is...
Benefits of Bulk Inserts<br />Improved concurrency<br />Allow application-database decoupling<br />In Lyngby case – 50GB a...
Bulk Inserts & RCSI: SQLCAT Benchmark<br />What happens when you query a table while bulk inserts are running?<br />
Bulk Inserts & RCSI: SQLCAT Benchmark<br />And with RCSI?<br />Bulk loading does not affect the size of the version store ...
StreamInsight:Complex Event Processing<br />SQL Server 2008 R2 feature<br />Processing and querying of event data streams<...
Isn’t This Just a Database Application?<br />Event<br />request<br />output stream<br />input stream<br />response<br />
Caching<br />Your database can get some rest...<br />
Microsoft AppFabric<br />
Other solutions<br />Memcache<br />Requires application change<br />3rd party<br />Usually transparent<br />
Tapuz Case Study: Offloading Traffic<br />Tapuz is the leading user content based website in Israel<br />1,200 forums<br /...
Database Scale Out<br />Queries scaled out (often geographically) similar to reporting cases<br />Databases replicate reci...
Demo<br />
Denali HADR<br />V.Next, code named “Denali” offers a brand new HADR feature<br />High-availability and disaster recovery ...
Denali HADR<br />Coming up next!<br />
Q&A<br />
Related sessions<br />Monday<br />14:00-15:15 <br />New High Availability Solution in SQL Server “Denali”<br />Justin Eric...
Open your mind to uncommon features and technologies in SQL Server and try them<br />Using these features, amazing concurr...
משובים ופייסבוק<br />מירב- השלמה<br />
Upcoming SlideShare
Loading in...5
×

Meir Dudai - Concurrency

784

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
784
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • דמו של פייסבוק
  • מדריך הטרמפיסט לטקאד
  • מישהו קיבל Giveaways?
  • No escalation from Row to PageEach lock structure is 100 bytesNumber of locks acquired on a rowset &gt; 5000 - Periodically after that
  • Self-explanatory. See Books Online for more details about partitioning. Also see http://msdn2.microsoft.com/en-us/library/ms345146.aspx
  • Query 1 is updating partition 1. It hits the escalation threshold and escalates to a TABLE X lock. Query 2 tries to update partition 3 and can’t because of the table X lock, even though nothing in partition 3 is being altered by query 1.
  • Instead of escalating to the table level and blocking all other queries on the table, you can now choose to have escalation to the partition level, which allows concurrent queries on different partitions.
  • Sunil Agarwal – Principal Program Manager
  • אחד האהובים עלי
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Database applications work in much slower times. For complex event processing applications, the latencies involved are much smaller.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Transcript of "Meir Dudai - Concurrency"

    1. 1. אין משתמשים – אין בעיות<br />Meir Dudai<br />SQL Server MVP<br />Valinor<br />
    2. 2. Content slide<br />
    3. 3. When do I need to start worrying?<br />100’s of queries/second<br />1000’s of DMLs/second<br />VLDB’s<br />
    4. 4. When do I need to start worrying?<br />Application was designed and developed poorly…<br />
    5. 5. When do I need to start worrying?<br />You have just migrated from other platform<br />
    6. 6. What can I do?<br />Throw in more hardware – not very useful<br />
    7. 7. What can I really do?<br />Improve application design<br />Use database enhancements and features<br />Change system architecture<br />
    8. 8. Who Am I?<br />Meir Dudai<br /><ul><li>MVP
    9. 9. SQL Server consultant
    10. 10. Editor of SQLServer.co.il</li></li></ul><li>How to increase concurrency without changing your application<br />How to adjust your architecture to achieve better concurrency<br />
    11. 11. Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural design<br />Bulk Load<br />StreamInsight<br />Caching<br />Offloading Traffic<br />Denali HADR<br />
    12. 12. Common Concurrency Issues with Applications<br />Wrong transaction isolation level<br />Incorrect locking granularity<br />Missing Indexes<br />Accessing large dataset leading to lock escalation<br />Long running transactions<br />User interactions in active transaction<br />Accessing objects in reverse order across transactions<br />
    13. 13. Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural design<br />Bulk Load<br />StreamInsight<br />Caching<br />Offloading Traffic<br />Denali HADR<br />
    14. 14. GreenRoad Case Study:RCSI<br />GreenRoad is an Israeli startup improving driving behavior for fleets and consumers<br />Works as SAAS<br />More than 400 inserts/sec<br />Users running queries and reports frequently<br />Method used to increase concurrency:<br />RCSI: Read Committed Snapshot Isolation<br />
    15. 15. Tran2 (Select)<br />Tran1 (Update)<br />Reader Writer Blocking<br />Data Page<br />S-Lock<br />Blocked<br />Row-1<br />Row-1<br />X-Lock<br />
    16. 16. Read Committed SnapshotStatement-Level ‘Snapshot Isolation’<br />New “flavor” of read committed<br />Turn ON/OFF on a database<br />Readers see committed values as of beginning of statement<br />Writers do not block Readers<br />Readers do not block Writers<br />Writers do block writers<br />Can greatly reduce locking / deadlocking without changing applications<br />
    17. 17. Demo<br />
    18. 18. Lock Escalation<br />HOBT<br />T1: IX<br />T1: X<br />Page<br />Page<br />Page<br />T1: X<br />T1: X<br />T1: X<br />T1: X<br />T1: X<br />Row<br />Row<br />Row<br />T1: X<br />T1: X<br />T1: X<br />Lock Escalation<br />T1: IX<br />
    19. 19. Lock Escalation<br />Converting finer-grain locks to coarse grain locks.<br />Row to Table<br />Page to Table.<br />Benefits<br />Reduced locking overhead<br />Reduces Memory requirement<br />Triggered when<br />Number of locks acquired on a rowset > 5000<br />Memory pressure<br />
    20. 20. Partitioned Tables and Indexes<br />SQL Server 2005 introduced partitioning, which some customers use to scale a query workload<br />Another common use is to streamline maintenance and enable fast range inserts and removals from tables<br />Partitioned<br />Table<br />FG1<br />FG2<br />FG3<br />Partition 2<br />Partition 3<br />Partition 1<br />
    21. 21. Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions<br />Only way to solve this in SQL Server 2005 is to disable lock escalation<br />Lock Escalation: The Problem<br />Query 1<br />Query 2<br />Partitioned<br />Table<br />IX<br />X<br />ESCALATE<br />update<br />update<br />Partition 2<br />Partition 1<br />Partition 3<br />FG1<br />FG2<br />FG3<br />
    22. 22. Lock Escalation: The Solution<br />SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions<br />Escalation to partition level does not block queries on other partitions<br />Query 1<br />Query 2<br />Partitioned<br />Table<br />IX<br />ESCALATE<br />update<br />update<br />X<br />Partition 2<br />Partition 1<br />Partition 3<br />FG1<br />FG2<br />FG3<br />
    23. 23. Demo<br />
    24. 24. Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural design<br />Bulk Load<br />StreamInsight<br />Caching<br />Offloading Traffic<br />Denali HADR<br />
    25. 25. Silent Blocker: Page Latches<br />What is a Pagelatch?<br />Access patterns leading to latch contention<br />Clustering index has a increasing primary key with high rate of inserts <br />Relatively small tables with a lot of read/write/delete<br />Indexes with low cardinality in the key with high rate of inserts<br />Allocation Structures (e.g. SGAM)<br />Observed with 16+ concurrent threads<br />Page Header<br />R1<br />Free Offset<br />R2<br />R3<br />
    26. 26. Silent Blocker: Page Latches<br />What can you do?<br />Cluster on some other column<br />Partition the table on some other attribute (computed column (hash))<br />Add another major key column<br />Change the application<br />
    27. 27. Demo<br />
    28. 28. Some basics<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Page Latches<br />Architectural design<br />Bulk Load<br />StreamInsight<br />Caching<br />Offloading Traffic<br />Denali HADR<br />
    29. 29. Codename Lyngby Case Study: Power of bulk inserts<br />Security/audit system<br />50GB of data inserted daily<br />Data is kept for one year and should be available for queries and reports at any time<br />Method used to increase concurrency:<br />Bulk inserts<br />
    30. 30. Benefits of Bulk Inserts<br />Improved concurrency<br />Allow application-database decoupling<br />In Lyngby case – 50GB a day!<br />http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge<br />
    31. 31. Bulk Inserts & RCSI: SQLCAT Benchmark<br />What happens when you query a table while bulk inserts are running?<br />
    32. 32. Bulk Inserts & RCSI: SQLCAT Benchmark<br />And with RCSI?<br />Bulk loading does not affect the size of the version store in tempdb under RCSI<br />Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table<br />Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency<br />More details about this benchmark:<br />http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx<br />
    33. 33. StreamInsight:Complex Event Processing<br />SQL Server 2008 R2 feature<br />Processing and querying of event data streams<br />Data queried while “in flight”<br />May involve multiple concurrent event sources<br />Works with high data rates<br />Aims for near-zero latency<br />
    34. 34. Isn’t This Just a Database Application?<br />Event<br />request<br />output stream<br />input stream<br />response<br />
    35. 35. Caching<br />Your database can get some rest...<br />
    36. 36. Microsoft AppFabric<br />
    37. 37. Other solutions<br />Memcache<br />Requires application change<br />3rd party<br />Usually transparent<br />
    38. 38. Tapuz Case Study: Offloading Traffic<br />Tapuz is the leading user content based website in Israel<br />1,200 forums<br />74,000 blogs<br />500,000 visitors each month<br />More than 4000 queries/sec<br />Method used to increase concurrency:<br />Offloading using Transactional Replication<br />
    39. 39. Database Scale Out<br />Queries scaled out (often geographically) similar to reporting cases<br />Databases replicate reciprocally and are writable<br />Redundancy provides fault tolerance and lowers maintenance downtime<br />Online upgrades possible<br />
    40. 40. Demo<br />
    41. 41. Denali HADR<br />V.Next, code named “Denali” offers a brand new HADR feature<br />High-availability and disaster recovery feature, using up to 4 replicas<br />Replicas can be synchronous or asynchronous <br />CTP supports only single <br />asynchronous replica<br />
    42. 42. Denali HADR<br />Coming up next!<br />
    43. 43. Q&A<br />
    44. 44. Related sessions<br />Monday<br />14:00-15:15 <br />New High Availability Solution in SQL Server “Denali”<br />Justin Erickson & Oren Bouni<br />Tuesday<br />09:30-10:45 <br />Mission Critical: Improving High<br /> Availability and Reliability<br />Assaf Fraenkel <br />12:30-13:30 <br />Designing High Performance I/O System with SQL Server<br />Dubi Lebel<br />11:15-12:15 <br />Troubleshoot Like a Pro with the Microsoft SQL Server 2008 Tools <br />Guy Glantser<br />
    45. 45. Open your mind to uncommon features and technologies in SQL Server and try them<br />Using these features, amazing concurrency can be achieved by any DBA using SQL Server!<br />
    46. 46. משובים ופייסבוק<br />מירב- השלמה<br />
    47. 47. Recommended Courses: SQL 2008<br />Maintaining a Microsoft SQL Server 2008 Database <br />Implementing a Microsoft SQL Server 2008 Database <br />ועכשיו מבצע מיוחד!<br />הרשם לאחד מקורסי SQL המוצעים כאן <br />קנה בחינת הסמכה משלימה:<br />70-432 TS: Microsoft SQL Server 2008, Implementation and Maintenance<br />70-433 TS: Microsoft SQL Server 2008, Database Development<br />וקבל מנוי שנתי ל- TechNet ואפשרות לבחינה חוזרת ללא עלות <br />לפרטים נוספים והרשמה, פנה למכללות המוסמכות <br />
    48. 48. משובים ופייסבוק<br />מירב- השלמה<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×