SQL Server uses a phase during query optimization called cardinality estimation (CE). This process makes estimates based on the statistics as to how many rows flow from one query plan iterator to the next. Knowing how CE generates these numbers, will enable you to write better TSQL code and in turn influence type of physical operations during a query execution. Based on that estimated rows, the query processor decides how to access an object, which physical join to use, how to sort the data. Do you know how the CE generates these numbers? If your query has only one predicate, the query optimizer will use the histogram to estimate how many rows will be qualified. What happen when you have multiple predicates, range predicates, variable values that are “NOT KNOWN” to the optimizer or you have predicate values increasing in ascending order? Do you know what will happen if your predicate is using a value that is outside of histogram range? In this session, I will show you how cardinality estimator estimates in all of these scenarios. You will walk out of this session with a clear understanding of how the CE generates its numbers and ready to tackle those nasty, hard to solve query plans.

1. Think Like the
Cardinality Estimator
Taiob M Ali

2. • 11 years as DBA
• MCSA SQL Server 2012/2014
• 10 years in Merchant Marine
Work
• Running—One 26.2 and Many 13.1
• Shuttling 3 kids
Outside Work
• Frequent speaker local user groups & SQL Saturdays
• Answering question at #sqlhelp & dba.stackexchange
• Blog at sqlworldwide.com
Giving Back

3. Agenda
Predicate, Density, Predicate Selectivity
What is Cardinality
Why Cardinality Matters
DBCC SHOW_STATISTICS
Magic Numbers

4. Predicate
Is an expression that evaluates to
TRUE, FALSE, or UNKNOWN
 Join
 Filter
 Where
 Having

5. Predicate
Join Predicate
Filter Predicate
Where/Having

6. Density
How often duplicate values occur in a column.
1__________________________________
[# of distinct values in a column]
Density
663 distinct customerID
1/663=0.00150830

7. Predicate Selectivity
Most commonly used to describe a predicate.
[ # rows that pass the predicate ]__________________________________
[total number of rows]
75 rows for
customerID 577
73595 total number
of rows.
75/73595=.0010

8. Cardinality
• Number of rows returned by a query operator.
For Us
• Uniqueness of data values contained in a particular
column (attribute) of a database table.
Structured Query Language
• Cardinality of a set is a measure of the “number
of elements of the set”.
Math

9. Cardinality
Estimated
Actual

10. Why Cardinality Matters
Before
Parse
+
Normalize
Cardinality
Estimation
Derive
Cardinality
+
Create
Many Plans
Downstream
Optimization
Downstream
Execute
Plan

11. Why Cardinality Matters
Cost
Parallel
Serial
Memory
Grant
In Memory
Spill to Disk
Access
Method
Seek
Scan
Seek + Scan
Algorithm
Join
Aggregate
Sort

12. DBCC SHOW_STATISTICS
• Meta data about the statistics.
Header
• How many unique values are present within a
column or columns?
DensityVector
• Frequency of data within the first key column
of the statistics.
Histogram

13. DBCC SHOW_STATISTICS
STAT_HEADER
DENSITY_VECTOR
HISTOGRAM

14. STAT_HEADER
Deprecated

15. DENSITY_VECTOR
1/Number of
distinct values in
column
Names of columns
in the prefix

16. Histogram

17. Histogram
RANGE_HI_KEY RANGE
_ROWS
EQ_ROWS DISTINCT_RANGE
_ROWS
AVG_RANGE
_ROWS
1055 126
Between
1056 and 1062
356 3 118.6667
1063 139

18. Magic Numbers
Single Predicate
 Histogram direct hit
 Histogram intra step
 Scaling
 Distinct

19. Magic Numbers
Multiple Predicates
Conjunction
Disjunction
Parameter Sniffing
Unknown
Ascending Key

21. Adaptive Query Processing
Announced on April 19, 2017
Joe Sack (Microsoft Program Manager)
Blog
Video

22. Resource
▪ QueryTuning Fundamentals
▪ DBCC SHOW_STATISTICS (Transact-SQL)
▪ 13ThingsYou Should Know About Statistics and the Query
Optimizer
▪ Cardinality Estimation for Multiple Predicates
▪ NewTrace Flag to FixTableVariable Performance
▪ Ascending key IssueTF 2389 and 2390
▪ Optimizing Query Plans with the SQL Server 2014
Cardinality Estimator

23. Resource
▪ QueryTuning Fundamentals
▪ DBCC SHOW_STATISTICS (Transact-SQL)
▪ 13ThingsYou Should Know About Statistics and the Query
Optimizer
▪ Cardinality Estimation for Multiple Predicates
▪ NewTrace Flag to FixTableVariable Performance
▪ Ascending key IssueTF 2389 and 2390
▪ Optimizing Query Plans with the SQL Server 2014
Cardinality Estimator

24. Resource-Adaptive Query Processing
▪ Blog-Introducing Batch Mode Adaptive Joins
▪ Video-SQL Server 2017: Adaptive Query Processing

25. @SqlWorldWide
linkedin.com/in/taiobali
sqlworldwide.com
taiob@sqlworldwide.com