Practical SQL query monitoring and optimization


Published on

Practical SQL query monitoring and optimization
Today the project owners demand results as soon as possible and most often - for yesterday. Time to market is crucial and it is practical to deliver bit-by-bit, get feedback and grow with the number of your customers. But as the project grows, the team does too and not all have the same expertise. As well rarely in the beginning the requirements clear enough to allow performance-wise SQL interaction. In most cases there does not exist an ORM that can solve this task for you and you will need to have hard T-SQL writer in the team. If you already know this story or are going this way then in this practical session we will share how to monitor, measure and optimize your SQL code and DB layer interaction.

Published in: Data & Analytics
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Practical SQL query monitoring and optimization

  1. 1. Practical SQL Query Monitoring and Optimization Identify Data Layer Bottlenecks
  2. 2. About me Project Manager @ 10 years professional experience Microsoft Certified Specialist Business Interests ASP.NET, AJAX, jQuery SOA, Integration GIS, Mapping SQL optimization 2 |
  3. 3. A Well Known Story Monitor and Measure Application Analyze Tier Interaction Getting Performance Hints Identify Problematic Queries Optimization Query Performance Measure the Effect Maintain 3 |
  4. 4. Warning: A Bottleneck Visual Studio Performance Wizard CPU and Memory Sampling Concurrency Tier Interaction SQL Server Profiler Create trace Analyze trace results Replay trace Log analyzer Most frequent and timely queries Identify long-running locks (and deadlocks) Identify I/O activity SQL Solutions Trace Analyzer / Deadlock Detector
  5. 5. Row vs. Chunk at a Time Row at a time (RAT) processing SELECT * FROM Activities WHERE ParentId = 1 SELECT * FROM Activities WHERE ParentId = 2 SELECT * FROM Activities WHERE ParentId = 16 Chunk at a time (CAT) processing SELECT * FROM Activities WHERE ParentId IN (1,2,16) Likely to happen when: Components with own DataSource extraction Complex code by different developers
  6. 6. DEMO 1 Row vs. Chunk at a Time Analyzing Trace Logs
  7. 7. Getting Help Analyze Execution Plan Database Tuning Advisor Suggests indexes and statistics Opt.1: Improvement 20% Opt.2: Improvement 20% Dynamic Management Views Missing Indexes (sys.dm_db_missing_index_groups) Slowest Queries (sys.dm_exec_query_stats)
  8. 8. DEMO 2 Tuning Advisor Dynamic Management Views
  9. 9. Analyze Query Plan Tools SQL Server Management Studio SQL Sentry: free @ Query Plan Colour-scaled cost information Cost filter by CPU, I/O or both Better plan layout Top Operations Tuning Tips
  10. 10. Query Plan Basics Table Scan (Heap) Clustered index missing Scans data pages (all data) Index Scan Scans index pages (less data) Relies on order of pages Index Seek Highly selective queries (10-15% of rows) Indexed Views Require: Schema binding Require: Deterministic functions
  11. 11. DEMO 3 Query Plan Explorer OR vs. UNION
  12. 12. Indexes Index types Clustered / Non-clustered Filtered index(SQL 2008) Covering index Objective Optimal performance Reduce slow scans Difference in large data sets
  13. 13. Are Indexes Magical ? Choosing Indexes Measure query impact Measure index size Maintenance Fragmentation = poor disk I/O Reorganize vs. Rebuild Automate: Nightly Job Anti-patterns Create indexes blindly Execute non-indexable queries SELECT * FROM Employees WHERE CAST(PNumber AS INT) 1 SELECT * FROM Customers WHERE ContactName LIKE ‘%a%’ SELECT * FROM Employees WHERE FName=‘a’ OR LName=‘b’ Reject indexes because of their overhead
  14. 14. SQL Server Statistics Objective WHERE clause order Statistics on data distribution Query Optimizer creates efficient plan Up-to-date check Automatically created for index keys Estimated vs. Actual Rows Maintenance DB Default: AUTOCREATE and AUTOUPDATE EXEC sp_updatestats Automate: Nightly Job
  15. 15. User Defined Functions Procedures vs. Functions Performance vs. Reusability Complex conditions Used through the whole application i.e. IsCaseClosed Chunk at a Time Table valued parameters (SQL 2008 and on) Pass result from one function to next
  16. 16. Optimize Locks Data Integrity Data commit waits NOLOCK Prevent Deadlocks Returns uncommitted data Incorrect results Less/More rows could be returned Update one entity, retrieve another Index page splits under heavy load(GUID)
  17. 17. DEMO 4 The NOLOCK Problem
  18. 18. Common Table Expressions (CTE) Alias of a query (SQL 2005) Advantages Readability and reusability Implement recursive query Materializes when used Uses indexes (Table v@r not) Disadvantages Can be used only once Must be used immediately
  19. 19. Other UNION vs. UNION ALL Data Types Do you really need BIGINT? Requires storage Requires time to read Requires memory BIGINT INT Delta (MB) Delta (%) Primary Key 1267 977 290 23% By_PropID_CountryID 813 741 72 9% By_String 303 230 73 24%
  20. 20. SQL Resource Governor Shared DB Server Prioritize queries Reporting vs. Dashboard Memory, CPU Connections, Parallelism RG Configuration Pools Workload Groups Custom classification function Available in SQL Server Enterprise
  21. 21. Test Under Load Life’s complicated Concurrent queries Multiple user Cached Buffers Tools SQL Server Profiler Replay SQL Server 2012 Distributed Replay Snap In @ SQL Load Generator
  22. 22. Summary Steps Capture/Analysis/Optimization Measure the improvement Maintain Remember Reoccurring improvement Ideally you should analyze all SQL Changes can affect SQL performance in data size result set size
  23. 23. DEMO 5 Trace Replay SQL Load Generator Resource Governor
  24. 24. Other Approaches Caching Application (Static) Session Context WCF Session Entity Framework Caching Paging Lazy loading Parallel binding Historical / Live data tables
  25. 25. SQL Saturday #152 Sponsors