Statistics and
the Query Optimizer
Grant Fritchey
Product Evangelist
Red Gate Software
www.ScaryDBA.com
Grant Fritchey | w...
Goals
 Learn how SQL Server creates, stores and maintains
statistics
 Understand how the optimizer consumes statistics
t...
grant@scarydba.com www.scarydba.com

@gfritchey

Grant Fritchey
Product Evangelist, Red Gate Software

Grant Fritchey | ww...
STATISTICS IN ACTION

Grant Fritchey | www.ScaryDBA.com

4
CREATE PROC dbo.spAddressByCity @City NVARCHAR(30)
AS
SELECT a.AddressID,
a.AddressLine1,
a.AddressLine2,
a.City,
sp.[Name...
Grant Fritchey | www.ScaryDBA.com

6
Grant Fritchey | www.ScaryDBA.com

7
Grant Fritchey | www.ScaryDBA.com

8
WHAT ARE STATISTICS

Grant Fritchey | www.ScaryDBA.com

9
DBCC
SHOW_STATISTICS('Person.Address',_WA_Sys_00000004_164452B1);

Grant Fritchey | www.ScaryDBA.com

10
Key Statistic Terms
 Rows Sampled
 Steps
 Density
 Range_hi_key
 Range_rows
 Eq_rows
 Avg_range_rows
 Cardinality
...
Cardinality
 Selectivity
» Returned values / Total Values
 Histogram
 Density
» 1/distinct values
 Compound Columns
» ...
Estimates vs. Cardinality
 Comparing variables
 Using != and NOT
 Predicates comparing columns within a table
 Functio...
Statistics are used to…
 Determine cardinality which is used to…
» Determine number of rows processed which is
used to…
—...
CAPTURING BEHAVIOR

Grant Fritchey | www.ScaryDBA.com

15
Capture Mechanisms
 Static
» DBCC SHOW_STATISTICS
» Execution Plans (sort of)
 Dynamic
» Trace Events
» Extended Events
...
Auto_stats

Grant Fritchey | www.ScaryDBA.com

17
Missing_column_statistics

Grant Fritchey | www.ScaryDBA.com

18
Query_optimizer_estimate
_cardinality

Grant Fritchey | www.ScaryDBA.com

19
STATS ARE CREATED

Grant Fritchey | www.ScaryDBA.com

20
SELECT

s.name,
s.auto_created,
s.user_created,
s.filter_definition,
sc.column_id,
c.name AS ColumnName
FROM
sys.stats AS ...
Grant Fritchey | www.ScaryDBA.com

22
Grant Fritchey | www.ScaryDBA.com

23
Grant Fritchey | www.ScaryDBA.com

24
What?
_WA_Sys_00000001_0A1E72EE

Grant Fritchey | www.ScaryDBA.com

25
What?
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID

Grant Fritchey | www.ScaryDBA.com

26
What?
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID

Table Column Number

Grant Fritchey | www.ScaryDBA.com

27
What?
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID

Table Column Number
System

Grant Fritchey | www.ScaryDBA.com

28
What?
The US State of Washington… yes I’m serious
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID

Table Column Number
Sys...
YOU CAN CREATE STATS

Grant Fritchey | www.ScaryDBA.com

30
CREATE STATISTICS MyStats
ON Person.Person (Suffix)
WITH FULLSCAN;

Grant Fritchey | www.ScaryDBA.com

31
CREATE STATISTICS MyJrStats
ON Person.Person (Suffix)
WHERE Suffix = 'Jr.'
WITH FULLSCAN;

Grant Fritchey | www.ScaryDBA.c...
CREATE STATISTICS MyComboStats
ON Person.Person (Title,Suffix)
WHERE Suffix = 'PhD'
WITH FULLSCAN;

Grant Fritchey | www.S...
…Or Drop Them

DROP STATISTICS Person.Person.MyStats;
DROP STATISTICS Person.Person.MyJrStats;
DROP STATISTICS Person.Pers...
STATS ARE MAINTAINED

Grant Fritchey | www.ScaryDBA.com

35
Dungeons and Dragons
 Add 1 Row when 0
 Add > 500 Rows when < 500
 Add 20% + 500 Rows when > 500

Grant Fritchey | www....
CREATE INDEX AddressCity
ON dbo.Address2 (City);

DBCC SHOW_STATISTICS(Address2,AddressCity)

Grant Fritchey | www.ScaryDB...
SELECT * FROM dbo.Address2 AS a
WHERE City = 'Springfield';

Grant Fritchey | www.ScaryDBA.com

38
Grant Fritchey | www.ScaryDBA.com

39
Advanced D&D
 Trace Flag 2371
» SQL Sever 2008 R2 Sp1 and above
» After 25,000 rows
— Percentage changed based on number ...
YOU CAN SHOULD
MAINTAIN STATS
MANUALLY
Grant Fritchey | www.ScaryDBA.com

41
Automatic Maintenance
 AUTO_CREATE_STATISTICS
 AUTO_UPDATE_STATISTICS
» Uses rules in previous section
 AUTO_UPDATE_STA...
sp_updatestats
ALTER procedure [sys].[sp_updatestats]
@resample char(8)='NO'
As
…
if ((@ind_rowmodctr <> 0) or ((@is_ver_c...
sp_updatestats
ALTER procedure [sys].[sp_updatestats]
@resample char(8)='NO'
As
…

@ind_rowmodctr <> 0

if ((

)

or ((@is...
UPDATE STATISTICS

UPDATE STATISTICS Person.Address;

Grant Fritchey | www.ScaryDBA.com

45
UPDATE STATISTICS

UPDATE STATISTICS Person.Address
WITH FULLSCAN;

Grant Fritchey | www.ScaryDBA.com

46
UPDATE STATISTICS

UPDATE STATISTICS dbo.Address2
AddressCity;

Grant Fritchey | www.ScaryDBA.com

47
UPDATE STATISTICS

UPDATE STATISTICS
dbo.Address2 AddressCity
WITH FULLSCAN,NORECOMPUTE;

Grant Fritchey | www.ScaryDBA.co...
STATISTICS AND
OPTIMIZER AT WORK
Grant Fritchey | www.ScaryDBA.com

49
Statistics Matter

Grant Fritchey | www.ScaryDBA.com

50
Compatibility Levels

ALTER DATABASE
AdventureWorks2012 SET
COMPATIBILITY_LEVEL = 120;

Grant Fritchey | www.ScaryDBA.com
...
Optimizer Switches
OPTION

(QUERYTRACEON 2312);

DBCC TRACEON(4199);
DBCC FREEPROCCACHE();

Grant Fritchey | www.ScaryDBA....
Recommendations
 Compatibility setting
 Automatic creation
 Automatic update
 Update asynchronous where necessary
 Us...
Goals
 Learn how SQL Server creates, stores and maintains
statistics
 Understand how the optimizer consumes statistics
t...
Resources
 Understanding SQL Server Cardinality Estimations
 Fixing Cardinality Estimation Errors
 Statistics Used by t...
Questions?

Grant Fritchey | www.ScaryDBA.com

56
Upcoming SlideShare
Loading in...5
×

Statistics And the Query Optimizer

487

Published on

There are any number of tricks and traps around getting the query optimizer to provide you with an optimal execution plan that gets you your data quickly and efficiently. But, at the end of the day, the principal driving factor of the optimizer, and therefore of your queries, are the statistics that define your data. This session teaches you how those statistics are put together and maintained by SQL Server. Different types of maintenance results in different levels of accuracy within statistics so we detail what the structures and information looks like after this maintenance. Your understanding of how the optimizer works with statistics will better enable you to understand why you’re getting the performance and types of execution plans that you are getting. Understanding enables you to write better t-sql statements and deal with performance problems such as bad parameter sniffing.

Published in: Technology, Sports
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
487
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • I never believed those stories about queries that I read in the forums until one night…
  • Add traceflag to this slide
  • @resample = determines if previous sample rate is used.
  • @resample = determines if previous sample rate is used.
  • When stats update, everything gets marked for recompile.
  • When stats update, everything gets marked for recompile.
  • NORECOMPUTE disables stats updates in the future, so this builds them and then turns them off.
  • Statistics And the Query Optimizer

    1. 1. Statistics and the Query Optimizer Grant Fritchey Product Evangelist Red Gate Software www.ScaryDBA.com Grant Fritchey | www.ScaryDBA.com
    2. 2. Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries Grant Fritchey | www.ScaryDBA.com
    3. 3. grant@scarydba.com www.scarydba.com @gfritchey Grant Fritchey Product Evangelist, Red Gate Software Grant Fritchey | www.ScaryDBA.com www.linkedin.com/in/s carydba
    4. 4. STATISTICS IN ACTION Grant Fritchey | www.ScaryDBA.com 4
    5. 5. CREATE PROC dbo.spAddressByCity @City NVARCHAR(30) AS SELECT a.AddressID, a.AddressLine1, a.AddressLine2, a.City, sp.[Name] AS StateProvinceName, a.PostalCode FROM Person.Address AS a JOIN Person.StateProvince AS sp ON a.StateProvinceID = sp.StateProvinceID WHERE a.City = @City; Grant Fritchey | www.ScaryDBA.com
    6. 6. Grant Fritchey | www.ScaryDBA.com 6
    7. 7. Grant Fritchey | www.ScaryDBA.com 7
    8. 8. Grant Fritchey | www.ScaryDBA.com 8
    9. 9. WHAT ARE STATISTICS Grant Fritchey | www.ScaryDBA.com 9
    10. 10. DBCC SHOW_STATISTICS('Person.Address',_WA_Sys_00000004_164452B1); Grant Fritchey | www.ScaryDBA.com 10
    11. 11. Key Statistic Terms  Rows Sampled  Steps  Density  Range_hi_key  Range_rows  Eq_rows  Avg_range_rows  Cardinality  Cardinality  Cardinality Grant Fritchey | www.ScaryDBA.com 11
    12. 12. Cardinality  Selectivity » Returned values / Total Values  Histogram  Density » 1/distinct values  Compound Columns » Selectivity * Selectivity » Density * Density  + Trace Flags which we’ll talk about Grant Fritchey | www.ScaryDBA.com 12
    13. 13. Estimates vs. Cardinality  Comparing variables  Using != and NOT  Predicates comparing columns within a table  Functions w/o constant value  Joins using arithmetic or string operations  No statistics!  Skewed distribution (eh) Grant Fritchey | www.ScaryDBA.com 13
    14. 14. Statistics are used to…  Determine cardinality which is used to… » Determine number of rows processed which is used to… — Determine cost of the operation in the plan which is used to… – Pick the operations used in the plan which is used to » Return your data in an efficient manner which is used for whatever the heck the business wants Grant Fritchey | www.ScaryDBA.com 14
    15. 15. CAPTURING BEHAVIOR Grant Fritchey | www.ScaryDBA.com 15
    16. 16. Capture Mechanisms  Static » DBCC SHOW_STATISTICS » Execution Plans (sort of)  Dynamic » Trace Events » Extended Events Grant Fritchey | www.ScaryDBA.com 16
    17. 17. Auto_stats Grant Fritchey | www.ScaryDBA.com 17
    18. 18. Missing_column_statistics Grant Fritchey | www.ScaryDBA.com 18
    19. 19. Query_optimizer_estimate _cardinality Grant Fritchey | www.ScaryDBA.com 19
    20. 20. STATS ARE CREATED Grant Fritchey | www.ScaryDBA.com 20
    21. 21. SELECT s.name, s.auto_created, s.user_created, s.filter_definition, sc.column_id, c.name AS ColumnName FROM sys.stats AS s JOIN sys.stats_columns AS sc ON sc.stats_id = s.stats_id AND sc.object_id = s.object_id JOIN sys.columns AS c ON c.column_id = sc.column_id AND c.object_id = s.object_id WHERE s.object_id = OBJECT_ID('dbo.Address2') Grant Fritchey | www.ScaryDBA.com 21
    22. 22. Grant Fritchey | www.ScaryDBA.com 22
    23. 23. Grant Fritchey | www.ScaryDBA.com 23
    24. 24. Grant Fritchey | www.ScaryDBA.com 24
    25. 25. What? _WA_Sys_00000001_0A1E72EE Grant Fritchey | www.ScaryDBA.com 25
    26. 26. What? _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Grant Fritchey | www.ScaryDBA.com 26
    27. 27. What? _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number Grant Fritchey | www.ScaryDBA.com 27
    28. 28. What? _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System Grant Fritchey | www.ScaryDBA.com 28
    29. 29. What? The US State of Washington… yes I’m serious _WA_Sys_00000001_0A1E72EE Hexidecimal Object ID Table Column Number System Grant Fritchey | www.ScaryDBA.com 29
    30. 30. YOU CAN CREATE STATS Grant Fritchey | www.ScaryDBA.com 30
    31. 31. CREATE STATISTICS MyStats ON Person.Person (Suffix) WITH FULLSCAN; Grant Fritchey | www.ScaryDBA.com 31
    32. 32. CREATE STATISTICS MyJrStats ON Person.Person (Suffix) WHERE Suffix = 'Jr.' WITH FULLSCAN; Grant Fritchey | www.ScaryDBA.com 32
    33. 33. CREATE STATISTICS MyComboStats ON Person.Person (Title,Suffix) WHERE Suffix = 'PhD' WITH FULLSCAN; Grant Fritchey | www.ScaryDBA.com 33
    34. 34. …Or Drop Them DROP STATISTICS Person.Person.MyStats; DROP STATISTICS Person.Person.MyJrStats; DROP STATISTICS Person.Person.MyComboStats; Grant Fritchey | www.ScaryDBA.com 34
    35. 35. STATS ARE MAINTAINED Grant Fritchey | www.ScaryDBA.com 35
    36. 36. Dungeons and Dragons  Add 1 Row when 0  Add > 500 Rows when < 500  Add 20% + 500 Rows when > 500 Grant Fritchey | www.ScaryDBA.com 36
    37. 37. CREATE INDEX AddressCity ON dbo.Address2 (City); DBCC SHOW_STATISTICS(Address2,AddressCity) Grant Fritchey | www.ScaryDBA.com 37
    38. 38. SELECT * FROM dbo.Address2 AS a WHERE City = 'Springfield'; Grant Fritchey | www.ScaryDBA.com 38
    39. 39. Grant Fritchey | www.ScaryDBA.com 39
    40. 40. Advanced D&D  Trace Flag 2371 » SQL Sever 2008 R2 Sp1 and above » After 25,000 rows — Percentage changed based on number of rows — Reducing as the number grows  Trace Flag 4137 » Minimum selectivity instead of multiplication on AND predicates  Trace Flag 9471 (SQL Server 2014) » Minimum selectivity on AND and OR predicates  Trace Flag 9472 (SQL Server 2014) » Independence (pre-2014 behavior) Grant Fritchey | www.ScaryDBA.com 40
    41. 41. YOU CAN SHOULD MAINTAIN STATS MANUALLY Grant Fritchey | www.ScaryDBA.com 41
    42. 42. Automatic Maintenance  AUTO_CREATE_STATISTICS  AUTO_UPDATE_STATISTICS » Uses rules in previous section  AUTO_UPDATE_STATISTICS_ASYNC » Test, test, test Grant Fritchey | www.ScaryDBA.com 42
    43. 43. sp_updatestats ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … if ((@ind_rowmodctr <> 0) or ((@is_ver_current is not null) and (@is_ver_current = 0))) … Grant Fritchey | www.ScaryDBA.com 43
    44. 44. sp_updatestats ALTER procedure [sys].[sp_updatestats] @resample char(8)='NO' As … @ind_rowmodctr <> 0 if (( ) or ((@is_ver_current is not null) and (@is_ver_current = 0))) … Grant Fritchey | www.ScaryDBA.com 44
    45. 45. UPDATE STATISTICS UPDATE STATISTICS Person.Address; Grant Fritchey | www.ScaryDBA.com 45
    46. 46. UPDATE STATISTICS UPDATE STATISTICS Person.Address WITH FULLSCAN; Grant Fritchey | www.ScaryDBA.com 46
    47. 47. UPDATE STATISTICS UPDATE STATISTICS dbo.Address2 AddressCity; Grant Fritchey | www.ScaryDBA.com 47
    48. 48. UPDATE STATISTICS UPDATE STATISTICS dbo.Address2 AddressCity WITH FULLSCAN,NORECOMPUTE; Grant Fritchey | www.ScaryDBA.com 48
    49. 49. STATISTICS AND OPTIMIZER AT WORK Grant Fritchey | www.ScaryDBA.com 49
    50. 50. Statistics Matter Grant Fritchey | www.ScaryDBA.com 50
    51. 51. Compatibility Levels ALTER DATABASE AdventureWorks2012 SET COMPATIBILITY_LEVEL = 120; Grant Fritchey | www.ScaryDBA.com 51
    52. 52. Optimizer Switches OPTION (QUERYTRACEON 2312); DBCC TRACEON(4199); DBCC FREEPROCCACHE(); Grant Fritchey | www.ScaryDBA.com 52
    53. 53. Recommendations  Compatibility setting  Automatic creation  Automatic update  Update asynchronous where necessary  Use appropriate sample rate Grant Fritchey | www.ScaryDBA.com 53
    54. 54. Goals  Learn how SQL Server creates, stores and maintains statistics  Understand how the optimizer consumes statistics to arrive at an execution plan  Learn various methods for controlling statistics to take more direct control of your queries Grant Fritchey | www.ScaryDBA.com
    55. 55. Resources  Understanding SQL Server Cardinality Estimations  Fixing Cardinality Estimation Errors  Statistics Used by the Optimizer  First look at the query_optimizer_estimate_cardinality XE Event  Changes to automatic update statistics inSQL Server – traceflag 2371  Cardinality Estimation for Multiple Predicates Grant Fritchey | www.ScaryDBA.com 55
    56. 56. Questions? Grant Fritchey | www.ScaryDBA.com 56
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×