Your SlideShare is downloading. ×
0
Extreme Performance for DBAs<br />מאיר דודאי| SQL Server MVP| ואלינור<br />
All transaction isolation level<br />Differences between Serializeable and Repateable Read<br />RangeX-S locking vsRangeX-...
All transaction isolation level<br />Differences between Serializeable and Repateable Read<br />RangeX-S locking vsRangeX-...
לפני קצת יותר מ-10 שנים...<br />
לפני קצת יותר מ-10 שנים...<br />
לפני קצת יותר מ-10 שנים...<br />
מטרות<br />היכרות עם תהליך P&T<br />כלים חדשים ולא מוכרים לניהול Performance<br />הבנת יישום נכון של Compression<br />היכר...
מי אני?<br />מאיר דודאי:<br />SQL Server MVP<br />יועץ בתחום בסיסי נתונים<br />ניסיון של למעלה מ-10 שנים ב-SQL Server<br /...
ואלינור<br />החברה המובילה בישראל במתן שירותים מקצועיים בתחום בסיסי נתונים: SQL Server, Oracle, MySQL<br />צוות יועצים מוב...
My Assumptions About You<br />You may be a part-time or full-time DBA<br />You are familiar with SQL Server in previous ve...
ארגז הכלים החדש שלכם<br />
Agenda<br /><ul><li>Utility Control Point
Detecting bottlenecks
New Performance Monitor in Windows 2008/R2
Performance Data Collector</li></li></ul><li>Agenda<br /><ul><li>Utility Control Point
Detecting bottlenecks
New Performance Monitor in Windows 2008/R2
Performance Data Collector</li></li></ul><li>Database evolution<br />Underutilized hardware<br />Hardware computing capaci...
Having fun with deployment<br />
DAC Concepts<br />Data-tier Application Component<br /><ul><li>.dacpac= unit of deployment (data-tier application + develo...
Data-tier Application Component
Improves collaboration between developer and DBA
Moves developers from a procedural model to a declarative model
V1 is targeted at small apps</li></ul>Schema<br />LOGICAL<br />Tables, Views, Procs, UDFs<br />PHYSICAL<br />Users, Logins...
Key Concept: Utility Control Point <br /><ul><li>Provides unified view of an organization’s SQL Server usage
Shows health of
SQL Server instances
Data-tier applications
Database files, filegroups and volumes
Provides resource monitoring
CPU utilization
Storage space</li></ul>SQL02<br />SQL01<br />SQL03<br />SQL05<br />SQL04<br />DBA<br />UCP<br />Managed Instances<br />Man...
DEVELOP<br />DEPLOY<br />MANAGE<br />SQL Server Management Studio<br />FinApp<br />Dev DB<br />1<br />Deploy /<br />Upgrad...
Utility Control Point<br />
UCP<br />
UCP<br />
UCP Reports<br />
UCP Reports<br />http://bit.ly/j8xMwH<br />
UCP Reports<br />
What’s missing?	<br />Supports only SQL Server 2008 R2 (or later)<br />Better management<br />More counters can be useful…...
Fake DAC<br />
What’s next?	<br />Denali’s Contained Databases…<br />
Agenda<br /><ul><li>Utility Control Point
Detecting bottlenecks
New Performance Monitor in Windows 2008/R2
Performance Data Collector</li></li></ul><li>Performance Tools<br />DTA <br />SSMS<br />Built-in Reports<br />Activity Mon...
SSMS Activity Monitor<br />
Performance Reports<br />
Using Profiler to Detect Blocks<br />
Clear Trace & Trace Analyzer<br />
Agenda<br /><ul><li>Utility Control Point
Detecting bottlenecks
New Performance Monitor in Windows 2008/R2
Performance Data Collector</li></li></ul><li>Windows Performance Tools<br />Performance Monitor<br />Reliability Monitor<b...
Performance & Resource Monitor<br />
New performance counters<br />Database Mirroring <br />Compression stats<br />Transactions volume<br />Mirroring performan...
Custom counters<br />
Agenda<br /><ul><li>Utility Control Point
Detecting bottlenecks
New Performance Monitor in Windows 2008/R2
Performance Data Collector</li></li></ul><li>Monitor with Insight<br /><ul><li>Performance data collection
OS perf indicators
Performance data
Extensible collection
Central repository
Consolidate information
Open interface
Management reports
Canned reports
Extensible </li></li></ul><li>Open Your Mind!<br />
Agenda<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Architectural design<br />Bulk Load<br />S...
GreenRoad Case Study:RCSI<br />GreenRoad is an Israeli startup improving driving behavior for fleets and consumers<br />Wo...
Tran2 (Select)<br />Tran1 (Update)<br />Reader Writer Blocking<br />Data Page<br />S-Lock<br />Blocked<br />Row-1<br />Row...
Read Committed Snapshot<br />New “flavor” of read committed<br />Turn ON/OFF on a database<br />Readers see committed valu...
Demo<br />
Lock Escalation<br />HOBT<br />T1: IX<br />T1: X<br />Page<br />Page<br />Page<br />T1: X<br />T1: X<br />T1: X<br />T1: X...
Lock Escalation<br />Converting finer-grain locks to coarse grain locks.<br />Row to Table<br />Page to Table.<br />Benefi...
Partitioned Tables and Indexes<br />SQL Server 2005 introduced partitioning, which some customers use to scale a query wor...
Lock Escalation: The Problem<br />Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL pa...
Lock Escalation: The Solution<br />SQL Server 2008 allows lock escalation to the partition level, allowing concurrent acce...
Demo<br />
Filtered Indexes<br />An index with a WHERE clause to specify a criteria<br />Essentially index only a subset of a the tab...
Using Filtered Indexes<br />Advantages of Filtered Indexes<br />Improve query performance, partly by enhancing execution p...
Using Filtered Indexes<br />Scenarios for Filtered Indexes<br />Sparse columns, where most data is null<br />Columns with ...
Upcoming SlideShare
Loading in...5
×

1 extreme performance - part i

849

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
849
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • No escalation from Row to PageEach lock structure is 100 bytesNumber of locks acquired on a rowset &gt; 5000 - Periodically after that
  • Self-explanatory. See Books Online for more details about partitioning. Also see http://msdn2.microsoft.com/en-us/library/ms345146.aspx
  • Query 1 is updating partition 1. It hits the escalation threshold and escalates to a TABLE X lock. Query 2 tries to update partition 3 and can’t because of the table X lock, even though nothing in partition 3 is being altered by query 1.
  • Instead of escalating to the table level and blocking all other queries on the table, you can now choose to have escalation to the partition level, which allows concurrent queries on different partitions.
  • אחד האהובים עלי
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Database applications work in much slower times. For complex event processing applications, the latencies involved are much smaller.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Transcript of "1 extreme performance - part i"

    1. 1. Extreme Performance for DBAs<br />מאיר דודאי| SQL Server MVP| ואלינור<br />
    2. 2. All transaction isolation level<br />Differences between Serializeable and Repateable Read<br />RangeX-S locking vsRangeX-U<br />Deep dive into bookmark lookup <br />Page internals<br />Indexes on Sparse columns<br />
    3. 3. All transaction isolation level<br />Differences between Serializeable and Repateable Read<br />RangeX-S locking vsRangeX-U<br />Deep dive into bookmark lookup <br />Page internals<br />Indexes on Sparse columns<br />
    4. 4. לפני קצת יותר מ-10 שנים...<br />
    5. 5. לפני קצת יותר מ-10 שנים...<br />
    6. 6. לפני קצת יותר מ-10 שנים...<br />
    7. 7. מטרות<br />היכרות עם תהליך P&T<br />כלים חדשים ולא מוכרים לניהול Performance<br />הבנת יישום נכון של Compression<br />היכרות עם יכולות Scale out בגירסאות הנוכחיות וב-Denali<br />
    8. 8. מי אני?<br />מאיר דודאי:<br />SQL Server MVP<br />יועץ בתחום בסיסי נתונים<br />ניסיון של למעלה מ-10 שנים ב-SQL Server<br />עורך האתר:<br />
    9. 9. ואלינור<br />החברה המובילה בישראל במתן שירותים מקצועיים בתחום בסיסי נתונים: SQL Server, Oracle, MySQL<br />צוות יועצים מוביל המתמחה בכלל הרבדים שלבסיסי הנתונים<br />שותפים של Microsoft ו- Oracle<br />נציגי מוצרים מובילים בתחום ה-DB<br />מעל 300 לקוחות בכל המגזרים <br />
    10. 10. My Assumptions About You<br />You may be a part-time or full-time DBA<br />You are familiar with SQL Server in previous versions<br />You would like to learn about new features in SQL Server 2008&R2&2011 that can help you to manage and improve performance<br />You are interested in good lunch, two days free from work, and some giveaways from advertisers<br />
    11. 11.
    12. 12.
    13. 13.
    14. 14. ארגז הכלים החדש שלכם<br />
    15. 15. Agenda<br /><ul><li>Utility Control Point
    16. 16. Detecting bottlenecks
    17. 17. New Performance Monitor in Windows 2008/R2
    18. 18. Performance Data Collector</li></li></ul><li>Agenda<br /><ul><li>Utility Control Point
    19. 19. Detecting bottlenecks
    20. 20. New Performance Monitor in Windows 2008/R2
    21. 21. Performance Data Collector</li></li></ul><li>Database evolution<br />Underutilized hardware<br />Hardware computing capacity<br />Overburdened Administrators<br />Number of database apps<br />Number of DBAs<br />1990<br />2000<br />2010<br />
    22. 22. Having fun with deployment<br />
    23. 23. DAC Concepts<br />Data-tier Application Component<br /><ul><li>.dacpac= unit of deployment (data-tier application + developer intent)
    24. 24. Data-tier Application Component
    25. 25. Improves collaboration between developer and DBA
    26. 26. Moves developers from a procedural model to a declarative model
    27. 27. V1 is targeted at small apps</li></ul>Schema<br />LOGICAL<br />Tables, Views, Procs, UDFs<br />PHYSICAL<br />Users, Logins, Indexes<br />DAC Deployment Profile<br />Deployment Requirements, Management Policies<br />
    28. 28. Key Concept: Utility Control Point <br /><ul><li>Provides unified view of an organization’s SQL Server usage
    29. 29. Shows health of
    30. 30. SQL Server instances
    31. 31. Data-tier applications
    32. 32. Database files, filegroups and volumes
    33. 33. Provides resource monitoring
    34. 34. CPU utilization
    35. 35. Storage space</li></ul>SQL02<br />SQL01<br />SQL03<br />SQL05<br />SQL04<br />DBA<br />UCP<br />Managed Instances<br />Management Studio<br />
    36. 36. DEVELOP<br />DEPLOY<br />MANAGE<br />SQL Server Management Studio<br />FinApp<br />Dev DB<br />1<br />Deploy /<br />Upgrade DAC<br />Reverse Engineer DAC<br />2<br />SQL Server Management Studio<br />9<br />4<br />8<br />Visual Studio 2010<br />Manage, Register, Uninstall, Extract, Upgrade DAC<br />Control<br />Point<br />Create policies<br />DBA<br />DBA<br />7<br />3<br />Deploy /<br />Upgrade DAC<br />HR<br />.dacpac<br />Hand-off to DBA<br />Compile<br />+ Build<br />FinApp<br />Prod DB<br />6<br />Developer<br />SALES<br />5<br />Managed Instances<br />
    37. 37. Utility Control Point<br />
    38. 38. UCP<br />
    39. 39. UCP<br />
    40. 40. UCP Reports<br />
    41. 41. UCP Reports<br />http://bit.ly/j8xMwH<br />
    42. 42. UCP Reports<br />
    43. 43. What’s missing? <br />Supports only SQL Server 2008 R2 (or later)<br />Better management<br />More counters can be useful…<br />Monitors only DAC<br />Or is it?<br />
    44. 44. Fake DAC<br />
    45. 45. What’s next? <br />Denali’s Contained Databases…<br />
    46. 46. Agenda<br /><ul><li>Utility Control Point
    47. 47. Detecting bottlenecks
    48. 48. New Performance Monitor in Windows 2008/R2
    49. 49. Performance Data Collector</li></li></ul><li>Performance Tools<br />DTA <br />SSMS<br />Built-in Reports<br />Activity Monitor (or what’s left of it)<br />Profiler assistants:<br />ClearTrace<br />Trace Analyzer<br />UCP<br />BPA<br />Extended Events<br />
    50. 50. SSMS Activity Monitor<br />
    51. 51. Performance Reports<br />
    52. 52. Using Profiler to Detect Blocks<br />
    53. 53. Clear Trace & Trace Analyzer<br />
    54. 54. Agenda<br /><ul><li>Utility Control Point
    55. 55. Detecting bottlenecks
    56. 56. New Performance Monitor in Windows 2008/R2
    57. 57. Performance Data Collector</li></li></ul><li>Windows Performance Tools<br />Performance Monitor<br />Reliability Monitor<br />Resource Monitor<br />PAL (Performance Analysis of Logs)<br />
    58. 58. Performance & Resource Monitor<br />
    59. 59. New performance counters<br />Database Mirroring <br />Compression stats<br />Transactions volume<br />Mirroring performance<br />Custom counters<br />
    60. 60. Custom counters<br />
    61. 61. Agenda<br /><ul><li>Utility Control Point
    62. 62. Detecting bottlenecks
    63. 63. New Performance Monitor in Windows 2008/R2
    64. 64. Performance Data Collector</li></li></ul><li>Monitor with Insight<br /><ul><li>Performance data collection
    65. 65. OS perf indicators
    66. 66. Performance data
    67. 67. Extensible collection
    68. 68. Central repository
    69. 69. Consolidate information
    70. 70. Open interface
    71. 71. Management reports
    72. 72. Canned reports
    73. 73. Extensible </li></li></ul><li>Open Your Mind!<br />
    74. 74. Agenda<br />Magic settings<br />RCSI <br />Partition-level lock escalation<br />Architectural design<br />Bulk Load<br />StreamInsight<br />Caching<br />Offloading Traffic<br />Denali HADR<br />
    75. 75. GreenRoad Case Study:RCSI<br />GreenRoad is an Israeli startup improving driving behavior for fleets and consumers<br />Works as SAAS<br />More than 400 inserts/sec<br />Users running queries and reports frequently<br />Method used to increase concurrency:<br />RCSI: Read Committed Snapshot Isolation<br />
    76. 76. Tran2 (Select)<br />Tran1 (Update)<br />Reader Writer Blocking<br />Data Page<br />S-Lock<br />Blocked<br />Row-1<br />Row-1<br />X-Lock<br />
    77. 77. Read Committed Snapshot<br />New “flavor” of read committed<br />Turn ON/OFF on a database<br />Readers see committed values as of beginning of statement<br />Writers do not block Readers<br />Readers do not block Writers<br />Writers do block writers<br />Can greatly reduce locking / deadlocking without changing applications<br />
    78. 78. Demo<br />
    79. 79. Lock Escalation<br />HOBT<br />T1: IX<br />T1: X<br />Page<br />Page<br />Page<br />T1: X<br />T1: X<br />T1: X<br />T1: X<br />T1: X<br />Row<br />Row<br />Row<br />T1: X<br />T1: X<br />T1: X<br />Lock Escalation<br />T1: IX<br />
    80. 80. Lock Escalation<br />Converting finer-grain locks to coarse grain locks.<br />Row to Table<br />Page to Table.<br />Benefits<br />Reduced locking overhead<br />Reduces Memory requirement<br />Triggered when<br />Number of locks acquired on a rowset > 5000<br />Memory pressure<br />
    81. 81. Partitioned Tables and Indexes<br />SQL Server 2005 introduced partitioning, which some customers use to scale a query workload<br />Another common use is to streamline maintenance and enable fast range inserts and removals from tables<br />Partitioned<br />Table<br />FG1<br />FG2<br />FG3<br />Partition 2<br />Partition 3<br />Partition 1<br />
    82. 82. Lock Escalation: The Problem<br />Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions<br />Only way to solve this in SQL Server 2005 is to disable lock escalation<br />Query 1<br />Query 2<br />Partitioned<br />Table<br />IX<br />X<br />ESCALATE<br />update<br />update<br />Partition 2<br />Partition 1<br />Partition 3<br />FG1<br />FG2<br />FG3<br />
    83. 83. Lock Escalation: The Solution<br />SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions<br />Escalation to partition level does not block queries on other partitions<br />Query 1<br />Query 2<br />Partitioned<br />Table<br />IX<br />ESCALATE<br />update<br />update<br />X<br />Partition 2<br />Partition 1<br />Partition 3<br />FG1<br />FG2<br />FG3<br />
    84. 84. Demo<br />
    85. 85. Filtered Indexes<br />An index with a WHERE clause to specify a criteria<br />Essentially index only a subset of a the table<br />Query optimizer most likely to use when WHERE clause matches that of the filtered index<br />
    86. 86. Using Filtered Indexes<br />Advantages of Filtered Indexes<br />Improve query performance, partly by enhancing execution plan quality<br />Smaller index maintenance costs<br />Less disk storage<br />
    87. 87. Using Filtered Indexes<br />Scenarios for Filtered Indexes<br />Sparse columns, where most data is null<br />Columns with categories of values<br />Columns with distinct ranges of values<br />
    88. 88. Filtered indexes<br />
    89. 89. New server option in SQL Server 2008<br />Only a stub is cached on first execution<br />Full plan cached after second execution<br />SP_CONFIGURE 'show advanced options',1RECONFIGUREGO<br />SP_CONFIGURE 'optimize for ad hoc workloads',1RECONFIGUREGO<br />Optimize for ad hoc workloads<br />
    90. 90. “I would not use it on some DWH systems”<br />“Why is this not on by default?”<br />“I can’t think of a reason why opt for adhoc workloads would ever really be a problem”<br />“I definitely recommend it!”<br />“Best movie of 2011! Breathtaking!”<br />“But paintball guns are also not very pleasant!”<br />How good is it?<br />
    91. 91. Codename Lyngby Case Study: Power of bulk inserts<br />Security/audit system<br />50GB of data inserted daily<br />Data is kept for one year and should be available for queries and reports at any time<br />Method used to increase concurrency:<br />Bulk inserts<br />
    92. 92. Benefits of Bulk Inserts<br />Improved concurrency<br />Allow application-database decoupling<br />In Lyngby case – 50GB a day!<br />http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge<br />
    93. 93. Bulk Inserts & RCSI: SQLCAT Benchmark<br />What happens when you query a table while bulk inserts are running?<br />
    94. 94. Bulk Inserts & RCSI: SQLCAT Benchmark<br />And with RCSI?<br />Bulk loading does not affect the size of the version store in tempdb under RCSI<br />Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table<br />Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency<br />More details about this benchmark:<br />http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx<br />
    95. 95. StreamInsight:Complex Event Processing<br />SQL Server 2008 R2 feature<br />Processing and querying of event data streams<br />Data queried while “in flight”<br />May involve multiple concurrent event sources<br />Works with high data rates<br />Aims for near-zero latency<br />
    96. 96. Isn’t This Just a Database Application?<br />Event<br />request<br />output stream<br />input stream<br />response<br />
    97. 97. Caching<br />Your database can get some rest...<br />
    98. 98. Microsoft AppFabric<br />
    99. 99. Other solutions<br />Memcache<br />Requires application change<br />3rd party<br />Usually transparent<br />
    100. 100. Tapuz Case Study: Offloading Traffic<br />Tapuz is the leading user content based website in Israel<br />1,200 forums<br />74,000 blogs<br />500,000 visitors each month<br />More than 4000 queries/sec<br />Method used to increase concurrency:<br />Offloading using Transactional Replication<br />
    101. 101. Compression<br />
    102. 102. As data volume grows…<br /><ul><li>Large databases =
    103. 103. Storage Cost
    104. 104. Workload Performance
    105. 105. Manageability Cost
    106. 106. Backup/Recovery</li></li></ul><li>Solution – Compress Data Files?<br />No benefit in-memory<br />Need to compress larger chunks<br />Does not scale for random access<br />
    107. 107. Solution – SQL Server 2008 / R2<br />Store data efficiently in the row/page<br />(+) More data can fit in memory<br />(+) Better Performance for I/O bound workload<br />(-) Performance degradation for CPU bound workload<br />
    108. 108. Data Compression<br />SQL Server 2008 <br />ROW and PAGE compression<br />Backup Compression<br />SQL Server 2008 R2<br />Unicode compression<br />
    109. 109. Enabling Compression Examples<br />Latest partition uncompressed<br />Uncompressed<br />PAGE Compressed<br />ROW Compressed<br />Jan-Mar<br />Apr-June<br />July-Sept<br />Oct-Dec <br />
    110. 110. Should it be so complex?<br />In real life – usually compress the entire large tables using page compression…<br />
    111. 111. Summary – Compression<br />Can reduce size of database significantly<br />Lower total cost of ownership (TCO)<br />Easy to enable/disable<br />No application changes<br />Performance gains!<br />
    112. 112. Control Your Resources<br />
    113. 113. Resource Hogs<br />What are some of the biggest resource hogs a DBA runs into?<br />Bulk Load Processes<br />Data Archiving<br />Poorly Coded Queries (Runaways)<br />Reporting…<br />
    114. 114. Reporting in Production<br />Is this a best practice???<br />
    115. 115. Reporting In Production<br />NO!<br />
    116. 116. Reporting In Production<br />NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO<br />
    117. 117. Reporting In Production<br /> OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!<br />
    118. 118. Resource Governor<br />Similar to a governor you would find on a car<br />It limits the “speed” that a SQL Server session can run at<br />
    119. 119. Resource Governor – Workloads<br />Ability to differentiate workloads based on connection property<br />e.g. app_name, login, user, db name<br />Per-request limits<br />Max memory %<br />Max CPU time<br />Grant timeout<br />Max Requests<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×