Upcoming SlideShare
×

# Statistics and the Query Optimizer

521 views
373 views

Published on

Statistics are the driving force behind all the decisions made by the query optimizer. This session explores how statistics works in order to deliver a better understanding of why you get the performance you do in your systems. Automatica and manual creation as well as automatic and manual maintenance are covered in detail.

Published in: Data & Analytics
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
521
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
17
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Statistics and the Query Optimizer

1. 1. Grant Fritchey | www.ScaryDBA.com www.ScaryDBA.com Statistics and the Query Optimization Grant Fritchey Product Evangelist Red Gate Software
2. 2. Grant Fritchey | www.ScaryDBA.com Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries
3. 3. Grant Fritchey | www.ScaryDBA.com grant@scarydba.com www.scarydba.com @gfritchey www.linkedin.com/in/s carydba Grant Fritchey Product Evangelist, Red GateSoftware
4. 4. Grant Fritchey | www.ScaryDBA.com STATISTICS IN ACTION 4
5. 5. Grant Fritchey | www.ScaryDBA.com CREATE PROC dbo.spAddressByCity @City NVARCHAR(30) AS SELECT a.AddressID, a.AddressLine1, a.AddressLine2, a.City, sp.[Name] AS StateProvinceName, a.PostalCode FROM Person.Address AS a JOIN Person.StateProvince AS sp ON a.StateProvinceID = sp.StateProvinceID WHERE a.City = @City;
6. 6. Grant Fritchey | www.ScaryDBA.com 6
7. 7. Grant Fritchey | www.ScaryDBA.com 7
8. 8. Grant Fritchey | www.ScaryDBA.com 8
9. 9. Grant Fritchey | www.ScaryDBA.com WHAT ARE STATISTICS 9
10. 10. Grant Fritchey | www.ScaryDBA.com 10 DBCC SHOW_STATISTICS('Person.Address',_WA_Sys_00000004_164452B1);
11. 11. Grant Fritchey | www.ScaryDBA.com Key Statistic Terms  Rows Sampled  Steps  Density  Range_hi_key  Range_rows  Eq_rows  Avg_range_rows  Cardinality  Cardinality  Cardinality 11
12. 12. Grant Fritchey | www.ScaryDBA.com Cardinality  Selectivity » Returned values /TotalValues  Histogram  Density » 1/distinct values  Compound Columns » Selectivity * Selectivity » Density * Density  +Trace Flags which we’ll talk about 12
13. 13. Grant Fritchey | www.ScaryDBA.com Estimates vs. Cardinality  Comparing variables  Using != and NOT  Predicates comparing columns within a table  Functions w/o constant value  Joins using arithmetic or string operations  No statistics!  Skewed distribution (eh) 13
14. 14. Grant Fritchey | www.ScaryDBA.com Statistics are used to…  Determine cardinality which is used to… » Determine number of rows processed which is used to… — Determine cost of the operation in the plan which is used to… – Pick the operations used in the plan which is used to » Return your data in an efficient manner which is used for whatever the heck the business wants 14
15. 15. Grant Fritchey | www.ScaryDBA.com CAPTURING BEHAVIOR 15
16. 16. Grant Fritchey | www.ScaryDBA.com Capture Mechanisms  Static » DBCC SHOW_STATISTICS » Execution Plans (sort of)  Dynamic » Trace Events » Extended Events 16
17. 17. Grant Fritchey | www.ScaryDBA.com Auto_stats 17
18. 18. Grant Fritchey | www.ScaryDBA.com Missing_column_statistics 18
19. 19. Grant Fritchey | www.ScaryDBA.com Query_optimizer_estimate _cardinality 19
20. 20. Grant Fritchey | www.ScaryDBA.com STATS ARE CREATED 20
21. 21. Grant Fritchey | www.ScaryDBA.com 21 SELECT s.name, s.auto_created, s.user_created, s.filter_definition, sc.column_id, c.name AS ColumnName FROM sys.stats AS s JOIN sys.stats_columns AS sc ON sc.stats_id = s.stats_id AND sc.object_id = s.object_id JOIN sys.columns AS c ON c.column_id = sc.column_id AND c.object_id = s.object_id WHERE s.object_id = OBJECT_ID('dbo.Address2')
22. 22. Grant Fritchey | www.ScaryDBA.com 22
23. 23. Grant Fritchey | www.ScaryDBA.com 23
24. 24. Grant Fritchey | www.ScaryDBA.com 24
25. 25. Grant Fritchey | www.ScaryDBA.com What? 25 _WA_Sys_00000001_0A1E72EE
26. 26. Grant Fritchey | www.ScaryDBA.com What? 26 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID
27. 27. Grant Fritchey | www.ScaryDBA.com What? 27 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number
28. 28. Grant Fritchey | www.ScaryDBA.com What? 28 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System
29. 29. Grant Fritchey | www.ScaryDBA.com What? 29 _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System The US State ofWashington… yes I’m serious
30. 30. Grant Fritchey | www.ScaryDBA.com YOU CAN CREATE STATS 30
31. 31. Grant Fritchey | www.ScaryDBA.com 31 CREATE STATISTICS MyStats ON Person.Person (Suffix) WITH FULLSCAN;
32. 32. Grant Fritchey | www.ScaryDBA.com 32 CREATE STATISTICS MyJrStats ON Person.Person (Suffix) WHERE Suffix = 'Jr.' WITH FULLSCAN;
33. 33. Grant Fritchey | www.ScaryDBA.com 33 CREATE STATISTICS MyComboStats ON Person.Person (Title,Suffix) WHERE Suffix = 'PhD' WITH FULLSCAN;
34. 34. Grant Fritchey | www.ScaryDBA.com …Or Drop Them 34 DROP STATISTICS Person.Person.MyStats; DROP STATISTICS Person.Person.MyJrStats; DROP STATISTICS Person.Person.MyComboStats;
35. 35. Grant Fritchey | www.ScaryDBA.com STATS ARE MAINTAINED 35
36. 36. Grant Fritchey | www.ScaryDBA.com Dungeons and Dragons  Add 1 Row when 0  Add > 500 Rows when < 500  Add 20% + 500 Rows when > 500 36
38. 38. Grant Fritchey | www.ScaryDBA.com 38 SELECT * FROM dbo.Address2 AS a WHERE City = 'Springfield';
39. 39. Grant Fritchey | www.ScaryDBA.com 39
40. 40. Grant Fritchey | www.ScaryDBA.com Advanced D&D  Trace Flag 2371 » SQL Sever 2008 R2 Sp1 and above » After 25,000 rows — Percentage changed based on number of rows — Reducing as the number grows  Trace Flag 4137 » Minimum selectivity instead of multiplication on AND predicates  Trace Flag 9471 (SQL Server 2014) » Minimum selectivity on AND and OR predicates  Trace Flag 9472 (SQL Server 2014) » Independence (pre-2014 behavior) 40
41. 41. Grant Fritchey | www.ScaryDBA.com YOU CAN SHOULD MAINTAIN STATS MANUALLY 41
42. 42. Grant Fritchey | www.ScaryDBA.com Automatic Maintenance  AUTO_CREATE_STATISTICS  AUTO_UPDATE_STATISTICS » Uses rules in previous section  AUTO_UPDATE_STATISTICS_ASYNC » Test, test, test 42
43. 43. Grant Fritchey | www.ScaryDBA.com sp_updatestats 43 ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … if ((@ind_rowmodctr <> 0) or ((@is_ver_current is not null) and (@is_ver_current = 0))) …
44. 44. Grant Fritchey | www.ScaryDBA.com sp_updatestats 44 ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … if ((@ind_rowmodctr <> 0) or ((@is_ver_current is not null) and (@is_ver_current = 0))) …
45. 45. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 45 UPDATE STATISTICS Person.Address;
46. 46. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 46 UPDATE STATISTICS Person.Address WITH FULLSCAN;
47. 47. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 47 UPDATE STATISTICS dbo.Address2 AddressCity;
48. 48. Grant Fritchey | www.ScaryDBA.com UPDATE STATISTICS 48 UPDATE STATISTICS dbo.Address2 AddressCity WITH FULLSCAN,NORECOMPUTE;
49. 49. Grant Fritchey | www.ScaryDBA.com STATISTICS AND OPTIMIZER AT WORK 49
50. 50. Grant Fritchey | www.ScaryDBA.com Statistics Matter 50
51. 51. Grant Fritchey | www.ScaryDBA.com Compatibility Levels 51 ALTER DATABASE AdventureWorks2012 SET COMPATIBILITY_LEVEL = 120;
52. 52. Grant Fritchey | www.ScaryDBA.com Optimizer Switches 52 OPTION (QUERYTRACEON 2312); DBCC TRACEON(4199); DBCC FREEPROCCACHE();
53. 53. Grant Fritchey | www.ScaryDBA.com Recommendations  Compatibility setting  Automatic creation  Automatic update  Update asynchronous where necessary  Use appropriate sample rate 53
54. 54. Grant Fritchey | www.ScaryDBA.com Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries
55. 55. Grant Fritchey | www.ScaryDBA.com Resources  Scarydba.com/resources  Understanding SQL Server Cardinality Estimations  Fixing Cardinality Estimation Errors  Statistics Used by the Optimizer  First look at the query_optimizer_estimate_cardinality XE Event  Changes to automatic update statistics inSQL Server – traceflag 2371  Cardinality Estimation for Multiple Predicates 55
56. 56. Grant Fritchey | www.ScaryDBA.com Questions? 56