Statistics and the Query Optimizer

521 views
373 views

Published on

Statistics are the driving force behind all the decisions made by the query optimizer. This session explores how statistics works in order to deliver a better understanding of why you get the performance you do in your systems. Automatica and manual creation as well as automatic and manual maintenance are covered in detail.

Published in: Data & Analytics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
521
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Statistics and the Query Optimizer

  1. 1. Grant Fritchey | www.ScaryDBA.com www.ScaryDBA.com Statistics and the Query Optimization Grant Fritchey Product Evangelist Red Gate Software
  2. 2. Grant Fritchey | www.ScaryDBA.com Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries
  3. 3. Grant Fritchey | www.ScaryDBA.com grant@scarydba.com www.scarydba.com @gfritchey www.linkedin.com/in/s carydba Grant Fritchey Product Evangelist, Red GateSoftware
  4. 4. Grant Fritchey | www.ScaryDBA.com STATISTICS IN ACTION 4
  5. 5. Grant Fritchey | www.ScaryDBA.com CREATE PROC dbo.spAddressByCity @City NVARCHAR(30) AS SELECT a.AddressID, a.AddressLine1, a.AddressLine2, a.City, sp.[Name] AS StateProvinceName, a.PostalCode FROM Person.Address AS a JOIN Person.StateProvince AS sp ON a.StateProvinceID = sp.StateProvinceID WHERE a.City = @City;
  6. 6. Grant Fritchey | www.ScaryDBA.com 6
  7. 7. Grant Fritchey | www.ScaryDBA.com 7
  8. 8. Grant Fritchey | www.ScaryDBA.com 8
  9. 9. Grant Fritchey | www.ScaryDBA.com WHAT ARE STATISTICS 9
  10. 10. Grant Fritchey | www.ScaryDBA.com 10 DBCC SHOW_STATISTICS('Person.Address',_WA_Sys_00000004_164452B1);
  11. 11. Grant Fritchey | www.ScaryDBA.com Key Statistic Terms  Rows Sampled  Steps  Density  Range_hi_key  Range_rows  Eq_rows  Avg_range_rows  Cardinality  Cardinality  Cardinality 11
  12. 12. Grant Fritchey | www.ScaryDBA.com Cardinality  Selectivity » Returned values /TotalValues  Histogram  Density » 1/distinct values  Compound Columns » Selectivity * Selectivity » Density * Density  +Trace Flags which we’ll talk about 12
  13. 13. Grant Fritchey | www.ScaryDBA.com Estimates vs. Cardinality  Comparing variables  Using != and NOT  Predicates comparing columns within a table  Functions w/o constant value  Joins using arithmetic or string operations  No statistics!  Skewed distribution (eh) 13
  14. 14. Grant Fritchey | www.ScaryDBA.com Statistics are used to…  Determine cardinality which is used to… » Determine number of rows processed which is used to… — Determine cost of the operation in the plan which is used to… – Pick the operations used in the plan which is used to » Return your data in an efficient manner which is used for whatever the heck the business wants 14
  15. 15. Grant Fritchey | www.ScaryDBA.com CAPTURING BEHAVIOR 15
  16. 16. Grant Fritchey | www.ScaryDBA.com Capture Mechanisms  Static » DBCC SHOW_STATISTICS » Execution Plans (sort of)  Dynamic » Trace Events » Extended Events 16
  17. 17. Grant Fritchey | www.ScaryDBA.com Auto_stats 17
  18. 18. Grant Fritchey | www.ScaryDBA.com Missing_column_statistics 18
  19. 19. Grant Fritchey | www.ScaryDBA.com Query_optimizer_estimate _cardinality 19
  20. 20. Grant Fritchey | www.ScaryDBA.com STATS ARE CREATED 20
  21. 21. Grant Fritchey | www.ScaryDBA.com 21 SELECT s.name, s.auto_created, s.user_created, s.filter_definition, sc.column_id, c.name AS ColumnName FROM sys.stats AS s JOIN sys.stats_columns AS sc ON sc.stats_id = s.stats_id AND sc.object_id = s.object_id JOIN sys.columns AS c ON c.column_id = sc.column_id AND c.object_id = s.object_id WHERE s.object_id = OBJECT_ID('dbo.Address2')
  22. 22. Grant Fritchey | www.ScaryDBA.com 22
  23. 23. Grant Fritchey | www.ScaryDBA.com 23
  24. 24. Grant Fritchey | www.ScaryDBA.com 24
  25. 25. Grant Fritchey | www.ScaryDBA.com What? 25 _WA_Sys_00000001_0A1E72EE
  26. 26. Grant Fritchey | www.ScaryDBA.com What? 26 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID
  27. 27. Grant Fritchey | www.ScaryDBA.com What? 27 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number
  28. 28. Grant Fritchey | www.ScaryDBA.com What? 28 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System
  29. 29. Grant Fritchey | www.ScaryDBA.com What? 29 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System The US State ofWashington… yes I’m serious
  30. 30. Grant Fritchey | www.ScaryDBA.com YOU CAN CREATE STATS 30
  31. 31. Grant Fritchey | www.ScaryDBA.com 31 CREATE STATISTICS MyStats ON Person.Person (Suffix) WITH FULLSCAN;
  32. 32. Grant Fritchey | www.ScaryDBA.com 32 CREATE STATISTICS MyJrStats ON Person.Person (Suffix) WHERE Suffix = 'Jr.' WITH FULLSCAN;
  33. 33. Grant Fritchey | www.ScaryDBA.com 33 CREATE STATISTICS MyComboStats ON Person.Person (Title,Suffix) WHERE Suffix = 'PhD' WITH FULLSCAN;
  34. 34. Grant Fritchey | www.ScaryDBA.com …Or Drop Them 34 DROP STATISTICS Person.Person.MyStats; DROP STATISTICS Person.Person.MyJrStats; DROP STATISTICS Person.Person.MyComboStats;
  35. 35. Grant Fritchey | www.ScaryDBA.com STATS ARE MAINTAINED 35
  36. 36. Grant Fritchey | www.ScaryDBA.com Dungeons and Dragons  Add 1 Row when 0  Add > 500 Rows when < 500  Add 20% + 500 Rows when > 500 36
  37. 37. Grant Fritchey | www.ScaryDBA.com 37 CREATE INDEX AddressCity ON dbo.Address2 (City); DBCC SHOW_STATISTICS(Address2,AddressCity)
  38. 38. Grant Fritchey | www.ScaryDBA.com 38 SELECT * FROM dbo.Address2 AS a WHERE City = 'Springfield';
  39. 39. Grant Fritchey | www.ScaryDBA.com 39
  40. 40. Grant Fritchey | www.ScaryDBA.com Advanced D&D  Trace Flag 2371 » SQL Sever 2008 R2 Sp1 and above » After 25,000 rows — Percentage changed based on number of rows — Reducing as the number grows  Trace Flag 4137 » Minimum selectivity instead of multiplication on AND predicates  Trace Flag 9471 (SQL Server 2014) » Minimum selectivity on AND and OR predicates  Trace Flag 9472 (SQL Server 2014) » Independence (pre-2014 behavior) 40
  41. 41. Grant Fritchey | www.ScaryDBA.com YOU CAN SHOULD MAINTAIN STATS MANUALLY 41
  42. 42. Grant Fritchey | www.ScaryDBA.com Automatic Maintenance  AUTO_CREATE_STATISTICS  AUTO_UPDATE_STATISTICS » Uses rules in previous section  AUTO_UPDATE_STATISTICS_ASYNC » Test, test, test 42
  43. 43. Grant Fritchey | www.ScaryDBA.com sp_updatestats 43 ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … if ((@ind_rowmodctr <> 0) or ((@is_ver_current is not null) and (@is_ver_current = 0))) …
  44. 44. Grant Fritchey | www.ScaryDBA.com sp_updatestats 44 ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … if ((@ind_rowmodctr <> 0) or ((@is_ver_current is not null) and (@is_ver_current = 0))) …
  45. 45. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 45 UPDATE STATISTICS Person.Address;
  46. 46. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 46 UPDATE STATISTICS Person.Address WITH FULLSCAN;
  47. 47. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 47 UPDATE STATISTICS dbo.Address2 AddressCity;
  48. 48. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 48 UPDATE STATISTICS dbo.Address2 AddressCity WITH FULLSCAN,NORECOMPUTE;
  49. 49. Grant Fritchey | www.ScaryDBA.com STATISTICS AND OPTIMIZER AT WORK 49
  50. 50. Grant Fritchey | www.ScaryDBA.com Statistics Matter 50
  51. 51. Grant Fritchey | www.ScaryDBA.com Compatibility Levels 51 ALTER DATABASE AdventureWorks2012 SET COMPATIBILITY_LEVEL = 120;
  52. 52. Grant Fritchey | www.ScaryDBA.com Optimizer Switches 52 OPTION (QUERYTRACEON 2312); DBCC TRACEON(4199); DBCC FREEPROCCACHE();
  53. 53. Grant Fritchey | www.ScaryDBA.com Recommendations  Compatibility setting  Automatic creation  Automatic update  Update asynchronous where necessary  Use appropriate sample rate 53
  54. 54. Grant Fritchey | www.ScaryDBA.com Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries
  55. 55. Grant Fritchey | www.ScaryDBA.com Resources  Scarydba.com/resources  Understanding SQL Server Cardinality Estimations  Fixing Cardinality Estimation Errors  Statistics Used by the Optimizer  First look at the query_optimizer_estimate_cardinality XE Event  Changes to automatic update statistics inSQL Server – traceflag 2371  Cardinality Estimation for Multiple Predicates 55
  56. 56. Grant Fritchey | www.ScaryDBA.com Questions? 56

×