EXPLAIN ANALYZE is a new query profiling tool first released in MySQL 8.0.18. This presentation covers how this new feature works, both on the surface and on the inside, and how you can use it to better understand your queries, to improve them and make them go faster. This presentation is for everyone who has ever had to understand why a query is executed slower than anticipated, and for everyone who wants to learn more about query plans and query execution in MySQL.

1. Norvald H. Ryeng
Software Development Director
MySQL Optimizer Team
January 30, 2020
MySQL 8.0
EXPLAIN ANALYZE
New in
8.0.18

2. Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.

3. Program agenda
1 Quick demo
2 The MySQL query executor
3 EXPLAIN FORMAT=TREE
4 EXPLAIN ANALYZE

4. Quick demo
MySQL 8.0.19

6. The MySQL query executor
TurtlesIterators all the way down

7. Refactoring the executor
 Refactor iterator interfaces
From many to one interface
From C style function pointers to C++ classes
 Possible because phases were separated
 Much more modular exeuctor
One common iterator interface for all operations
Each operation is contained within an iterator
 Able to put together plans in new ways
Immediate benefit: Removes temporary tables in some cases
 Join is just an iterator
Nested loop join is just an iterator
Hash join is just an iterator
Your favorite join method is just an iterator
Parse
Prepare
Optimize
Execute
SQL
Resolve
Transform
Abstract syntax tree
Logical plan
Physical plan

8. MySQL iterator executor
 Each operation is an iterator
 Execution loop reads from root node
Row by row
May trigger multiple read calls further down
 Common interface
Init()
Read()
HashJoinIterator
TableScanIterator TableScanIterator
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
t1 t2

9. Old MySQL executor vs. iterator executor
Old executor
 Nested loop focused
 Hard to extend
 Code for one operation spread out
 Different interfaces for each operation
 Combination of operations hard coded
Iterator executor
 Modular
 Easy to extend
 Each iterator encapsulates one operation
 Same interface for all iterators
 All operations can be connected

10. MySQL 8.0 features based on the iterator executor
 Hash join
Just another iterator type
 EXPLAIN FORMAT=TREE
Print the iterator tree
 EXPLAIN ANALYZE
1. Insert intstrumentation nodes in the tree
2. Execute the query
3. Print the iterator tree
Parse
Prepare
Optimize
Execute
SQL
Resolve
Transform
Abstract syntax tree
Logical plan
Physical plan

11. EXPLAIN FORMAT=TREE
 Old EXPLAIN implementation
Analyzes query plan and prints its interpretation of it
Duplicates plan interpretation
EXPLAIN interpretation
Executor interpretation
Still used for FORMAT=TRADITIONAL and FORMAT=JSON
Hope to change to a single EXPLAIN implementation
 EXPLAIN FORMAT=TREE
Direct dump of execution structures
A single plan interpretation for both EXPLAIN and execution
Plan to reuse same implementation for all formats
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN FORMAT=TREE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1)
-> Table scan on t2 (cost=0.35 rows=1)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1)

12. EXPLAIN FORMAT=TREE
 Old EXPLAIN implementation
Analyzes query plan and prints its interpretation of it
Duplicates plan interpretation
EXPLAIN interpretation
Executor interpretation
Still used for FORMAT=TRADITIONAL and FORMAT=JSON
Hope to change to a single EXPLAIN implementation
 EXPLAIN FORMAT=TREE
Direct dump of execution structures
A single plan interpretation for both EXPLAIN and execution
Plan to reuse same implementation for all formats
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN FORMAT=TREE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1)
-> Table scan on t2 (cost=0.35 rows=1)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1)

13. EXPLAIN ANALYZE

14. TimingIteratorTimingIterator
TimingIterator
MySQL EXPLAIN ANALYZE
 Wrap iterators in instrumentation nodes
 Measurements
Time (in ms) to first row
Time (in ms) to last row
Number of rows
Number of loops
 Execute the query and dump the stats
 Built on EXPLAIN FORMAT=TREE
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN ANALYZE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1)
-> Table scan on t2 (cost=0.35 rows=1) (never executed)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)

15. TimingIteratorTimingIterator
TimingIterator
MySQL EXPLAIN ANALYZE
 Profiling of query execution
Where does the executor spend time?
What are the actual row counts?
 Low overhead
Timing is close to normal execution
But considered too expensive to enable for
all queries
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN ANALYZE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1)
-> Table scan on t2 (cost=0.35 rows=1) (never executed)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)

16. 467x
1520x>1400x
1332x
968x
What's wrong with Q2?
SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address,
s_phone, s_comment
FROM part, supplier, partsupp, nation, region
WHERE
p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND p_size = 4
AND p_type LIKE '%TIN' AND s_nationkey = n_nationkey
AND n_regionkey = r_regionkey AND r_name = 'AMERICA'
AND ps_supplycost = (
SELECT min(ps_supplycost)
FROM partsupp, supplier, nation, region
WHERE
p_partkey = ps_partkey AND s_suppkey = ps_suppkey
AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey
AND r_name = 'AMERICA'
)
ORDER BY s_acctbal DESC, n_name, s_name, p_partkey
LIMIT 100;
MySQL EXPLAIN ANALYZE to the rescue!

17. -> Limit: 100 row(s) (actual time=591739.000..591739.018 rows=100 loops=1)
-> Sort: <temporary>.s_acctbal DESC, <temporary>.n_name, <temporary>.s_name, <temporary>.p_partkey, limit input to 100 row(s) per chunk (actual time
-> Stream results (actual time=591738.599..591738.772 rows=462 loops=1)
-> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost=2074295.37 rows=98) (actual
-> Table scan on nation (cost=0.00 rows=25) (actual time=0.024..0.026 rows=25 loops=1)
-> Hash
-> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=2074041.10 rows=98) (actual time=591735.554..591738.311 rows=46
-> Table scan on supplier (cost=0.06 rows=9760) (actual time=0.068..2.024 rows=10000 loops=1)
-> Hash
-> Filter: (partsupp.ps_supplycost = (select #2)) (cost=1977898.52 rows=98) (actual time=1282.855..591733.987 rows=462 loop
-> Inner hash join (partsupp.ps_partkey = part.p_partkey) (cost=1977898.52 rows=98) (actual time=84.827..271.307 rows=3
-> Table scan on partsupp (cost=3.54 rows=796168) (actual time=0.034..108.684 rows=800000 loops=1)
-> Hash
-> Inner hash join (cost=20353.91 rows=24) (actual time=0.274..83.964 rows=780 loops=1)
-> Filter: ((part.p_size = 4) and (part.p_type like '%TIN')) (cost=20353.16 rows=2204) (actual time=0.212..
-> Table scan on part (cost=20353.16 rows=198401) (actual time=0.160..63.957 rows=200000 loops=1)
-> Hash
-> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.029..0.036 rows=1 loops=1)
-> Table scan on region (cost=0.75 rows=5) (actual time=0.021..0.030 rows=5 loops=1)
-> Select #2 (subquery in condition; dependent)
-> Aggregate: min(partsupp.ps_supplycost) (actual time=189.566..189.566 rows=1 loops=3120)
-> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost
-> Table scan on nation (cost=0.00 rows=25) (actual time=0.005..0.007 rows=25 loops=3120)
-> Hash
-> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=77789257.28 rows=79617) (actual tim
-> Table scan on supplier (cost=0.02 rows=9760) (actual time=0.014..1.237 rows=10000 loops=3120)
-> Hash
-> Filter: (part.p_partkey = partsupp.ps_partkey) (cost=81757.41 rows=79617) (actual time=92.08
-> Inner hash join (cost=81757.41 rows=79617) (actual time=0.018..155.118 rows=800000 loops
-> Table scan on partsupp (cost=10101.54 rows=796168) (actual time=0.011..97.353 rows=8
-> Hash
-> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.004..0.005
-> Table scan on region (cost=0.75 rows=5) (actual time=0.003..0.003 rows=5 loo
Time is in milliseconds

18. MySQL 8.0 query analysis toolbox
Analysis:
 EXPLAIN
Plan analysis
Cost estimates
 EXPLAIN ANALYZE
Query profiling
Actual statistics
 Optimizer trace
Debug tracing
Optimizer decisions
Mitigation:
 Optimizer hints
Per query workarounds
Enable/disable specific optimizations
Ask for specific plan choices
Access methods
Join order
 Optimizer switch
Per session workarounds
Enable/disable specific optimizations
New in
8.0.18

19. Feature descriptions and design details
directly from the source
https://mysqlserverteam.com/

20. Thank you
Norvald H. Ryeng
Software Development Director
MySQL Optimizer Team