SlideShare a Scribd company logo
1 of 20
Download to read offline
Norvald H. Ryeng
Software Development Director
MySQL Optimizer Team
January 30, 2020
MySQL 8.0
EXPLAIN ANALYZE
New in
8.0.18
Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
Program agenda
1 Quick demo
2 The MySQL query executor
3 EXPLAIN FORMAT=TREE
4 EXPLAIN ANALYZE
Quick demo
MySQL 8.0.19
The MySQL query executor
TurtlesIterators all the way down
Refactoring the executor
 Refactor iterator interfaces
From many to one interface
From C style function pointers to C++ classes
 Possible because phases were separated
 Much more modular exeuctor
One common iterator interface for all operations
Each operation is contained within an iterator
 Able to put together plans in new ways
Immediate benefit: Removes temporary tables in some cases
 Join is just an iterator
Nested loop join is just an iterator
Hash join is just an iterator
Your favorite join method is just an iterator
Parse
Prepare
Optimize
Execute
SQL
Resolve
Transform
Abstract syntax tree
Logical plan
Physical plan
MySQL iterator executor
 Each operation is an iterator
 Execution loop reads from root node
Row by row
May trigger multiple read calls further down
 Common interface
Init()
Read()
HashJoinIterator
TableScanIterator TableScanIterator
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
t1 t2
Old MySQL executor vs. iterator executor
Old executor
 Nested loop focused
 Hard to extend
 Code for one operation spread out
 Different interfaces for each operation
 Combination of operations hard coded
Iterator executor
 Modular
 Easy to extend
 Each iterator encapsulates one operation
 Same interface for all iterators
 All operations can be connected
MySQL 8.0 features based on the iterator executor
 Hash join
Just another iterator type
 EXPLAIN FORMAT=TREE
Print the iterator tree
 EXPLAIN ANALYZE
1. Insert intstrumentation nodes in the tree
2. Execute the query
3. Print the iterator tree
Parse
Prepare
Optimize
Execute
SQL
Resolve
Transform
Abstract syntax tree
Logical plan
Physical plan
EXPLAIN FORMAT=TREE
 Old EXPLAIN implementation
Analyzes query plan and prints its interpretation of it
Duplicates plan interpretation
EXPLAIN interpretation
Executor interpretation
Still used for FORMAT=TRADITIONAL and FORMAT=JSON
Hope to change to a single EXPLAIN implementation
 EXPLAIN FORMAT=TREE
Direct dump of execution structures
A single plan interpretation for both EXPLAIN and execution
Plan to reuse same implementation for all formats
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN FORMAT=TREE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1)
-> Table scan on t2 (cost=0.35 rows=1)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1)
EXPLAIN FORMAT=TREE
 Old EXPLAIN implementation
Analyzes query plan and prints its interpretation of it
Duplicates plan interpretation
EXPLAIN interpretation
Executor interpretation
Still used for FORMAT=TRADITIONAL and FORMAT=JSON
Hope to change to a single EXPLAIN implementation
 EXPLAIN FORMAT=TREE
Direct dump of execution structures
A single plan interpretation for both EXPLAIN and execution
Plan to reuse same implementation for all formats
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN FORMAT=TREE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1)
-> Table scan on t2 (cost=0.35 rows=1)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1)
EXPLAIN ANALYZE
TimingIteratorTimingIterator
TimingIterator
MySQL EXPLAIN ANALYZE
 Wrap iterators in instrumentation nodes
 Measurements
Time (in ms) to first row
Time (in ms) to last row
Number of rows
Number of loops
 Execute the query and dump the stats
 Built on EXPLAIN FORMAT=TREE
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN ANALYZE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1)
-> Table scan on t2 (cost=0.35 rows=1) (never executed)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
TimingIteratorTimingIterator
TimingIterator
MySQL EXPLAIN ANALYZE
 Profiling of query execution
Where does the executor spend time?
What are the actual row counts?
 Low overhead
Timing is close to normal execution
But considered too expensive to enable for
all queries
HashJoinIterator
TableScanIterator TableScanIterator
t1 t2
EXPLAIN ANALYZE
SELECT * FROM t1 JOIN t2 ON t1.a = t2.a;
-> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1)
-> Table scan on t2 (cost=0.35 rows=1) (never executed)
-> Hash
-> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
467x
1520x>1400x
1332x
968x
What's wrong with Q2?
SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address,
s_phone, s_comment
FROM part, supplier, partsupp, nation, region
WHERE
p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND p_size = 4
AND p_type LIKE '%TIN' AND s_nationkey = n_nationkey
AND n_regionkey = r_regionkey AND r_name = 'AMERICA'
AND ps_supplycost = (
SELECT min(ps_supplycost)
FROM partsupp, supplier, nation, region
WHERE
p_partkey = ps_partkey AND s_suppkey = ps_suppkey
AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey
AND r_name = 'AMERICA'
)
ORDER BY s_acctbal DESC, n_name, s_name, p_partkey
LIMIT 100;
MySQL EXPLAIN ANALYZE to the rescue!
-> Limit: 100 row(s) (actual time=591739.000..591739.018 rows=100 loops=1)
-> Sort: <temporary>.s_acctbal DESC, <temporary>.n_name, <temporary>.s_name, <temporary>.p_partkey, limit input to 100 row(s) per chunk (actual time
-> Stream results (actual time=591738.599..591738.772 rows=462 loops=1)
-> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost=2074295.37 rows=98) (actual
-> Table scan on nation (cost=0.00 rows=25) (actual time=0.024..0.026 rows=25 loops=1)
-> Hash
-> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=2074041.10 rows=98) (actual time=591735.554..591738.311 rows=46
-> Table scan on supplier (cost=0.06 rows=9760) (actual time=0.068..2.024 rows=10000 loops=1)
-> Hash
-> Filter: (partsupp.ps_supplycost = (select #2)) (cost=1977898.52 rows=98) (actual time=1282.855..591733.987 rows=462 loop
-> Inner hash join (partsupp.ps_partkey = part.p_partkey) (cost=1977898.52 rows=98) (actual time=84.827..271.307 rows=3
-> Table scan on partsupp (cost=3.54 rows=796168) (actual time=0.034..108.684 rows=800000 loops=1)
-> Hash
-> Inner hash join (cost=20353.91 rows=24) (actual time=0.274..83.964 rows=780 loops=1)
-> Filter: ((part.p_size = 4) and (part.p_type like '%TIN')) (cost=20353.16 rows=2204) (actual time=0.212..
-> Table scan on part (cost=20353.16 rows=198401) (actual time=0.160..63.957 rows=200000 loops=1)
-> Hash
-> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.029..0.036 rows=1 loops=1)
-> Table scan on region (cost=0.75 rows=5) (actual time=0.021..0.030 rows=5 loops=1)
-> Select #2 (subquery in condition; dependent)
-> Aggregate: min(partsupp.ps_supplycost) (actual time=189.566..189.566 rows=1 loops=3120)
-> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost
-> Table scan on nation (cost=0.00 rows=25) (actual time=0.005..0.007 rows=25 loops=3120)
-> Hash
-> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=77789257.28 rows=79617) (actual tim
-> Table scan on supplier (cost=0.02 rows=9760) (actual time=0.014..1.237 rows=10000 loops=3120)
-> Hash
-> Filter: (part.p_partkey = partsupp.ps_partkey) (cost=81757.41 rows=79617) (actual time=92.08
-> Inner hash join (cost=81757.41 rows=79617) (actual time=0.018..155.118 rows=800000 loops
-> Table scan on partsupp (cost=10101.54 rows=796168) (actual time=0.011..97.353 rows=8
-> Hash
-> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.004..0.005
-> Table scan on region (cost=0.75 rows=5) (actual time=0.003..0.003 rows=5 loo
Time is in milliseconds
MySQL 8.0 query analysis toolbox
Analysis:
 EXPLAIN
Plan analysis
Cost estimates
 EXPLAIN ANALYZE
Query profiling
Actual statistics
 Optimizer trace
Debug tracing
Optimizer decisions
Mitigation:
 Optimizer hints
Per query workarounds
Enable/disable specific optimizations
Ask for specific plan choices
Access methods
Join order
 Optimizer switch
Per session workarounds
Enable/disable specific optimizations
New in
8.0.18
Feature descriptions and design details
directly from the source
https://mysqlserverteam.com/
Thank you
Norvald H. Ryeng
Software Development Director
MySQL Optimizer Team

More Related Content

What's hot

MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6
MYXPLAIN
 

What's hot (20)

ProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQLProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQL
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
 
PostgreSql query planning and tuning
PostgreSql query planning and tuningPostgreSql query planning and tuning
PostgreSql query planning and tuning
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
Nginx Internals
Nginx InternalsNginx Internals
Nginx Internals
 
MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6
 
High Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseHigh Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouse
 
OpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQLOpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQL
 
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
 
All about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdfAll about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdf
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
 
Advanced MySQL Query Tuning
Advanced MySQL Query TuningAdvanced MySQL Query Tuning
Advanced MySQL Query Tuning
 
MongoDB Fundamentals
MongoDB FundamentalsMongoDB Fundamentals
MongoDB Fundamentals
 
Automated master failover
Automated master failoverAutomated master failover
Automated master failover
 
MyRocks Deep Dive
MyRocks Deep DiveMyRocks Deep Dive
MyRocks Deep Dive
 
PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)PGConf.ASIA 2017 Logical Replication Internals (English)
PGConf.ASIA 2017 Logical Replication Internals (English)
 
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
A Practical Introduction to Handling Log Data in ClickHouse, by Robert Hodges...
 
How to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleHow to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScale
 
Introduction to PostgreSQL
Introduction to PostgreSQLIntroduction to PostgreSQL
Introduction to PostgreSQL
 

Similar to MySQL 8.0 EXPLAIN ANALYZE

Oracle Diagnostics : Joins - 1
Oracle Diagnostics : Joins - 1Oracle Diagnostics : Joins - 1
Oracle Diagnostics : Joins - 1
Hemant K Chitale
 
Problem 1 Show the comparison of runtime of linear search and binar.pdf
Problem 1 Show the comparison of runtime of linear search and binar.pdfProblem 1 Show the comparison of runtime of linear search and binar.pdf
Problem 1 Show the comparison of runtime of linear search and binar.pdf
ebrahimbadushata00
 
Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007
paulguerin
 

Similar to MySQL 8.0 EXPLAIN ANALYZE (20)

Optimizer features in recent releases of other databases
Optimizer features in recent releases of other databasesOptimizer features in recent releases of other databases
Optimizer features in recent releases of other databases
 
Deep Dive Into Catalyst: Apache Spark 2.0’s Optimizer
Deep Dive Into Catalyst: Apache Spark 2.0’s OptimizerDeep Dive Into Catalyst: Apache Spark 2.0’s Optimizer
Deep Dive Into Catalyst: Apache Spark 2.0’s Optimizer
 
Oracle Diagnostics : Joins - 1
Oracle Diagnostics : Joins - 1Oracle Diagnostics : Joins - 1
Oracle Diagnostics : Joins - 1
 
Deep Dive Into Catalyst: Apache Spark 2.0'S Optimizer
Deep Dive Into Catalyst: Apache Spark 2.0'S OptimizerDeep Dive Into Catalyst: Apache Spark 2.0'S Optimizer
Deep Dive Into Catalyst: Apache Spark 2.0'S Optimizer
 
MySQL 8.0 New Features -- September 27th presentation for Open Source Summit
MySQL 8.0 New Features -- September 27th presentation for Open Source SummitMySQL 8.0 New Features -- September 27th presentation for Open Source Summit
MySQL 8.0 New Features -- September 27th presentation for Open Source Summit
 
Linuxfest Northwest 2022 - MySQL 8.0 Nre Features
Linuxfest Northwest 2022 - MySQL 8.0 Nre FeaturesLinuxfest Northwest 2022 - MySQL 8.0 Nre Features
Linuxfest Northwest 2022 - MySQL 8.0 Nre Features
 
Introduction into MySQL Query Tuning
Introduction into MySQL Query TuningIntroduction into MySQL Query Tuning
Introduction into MySQL Query Tuning
 
Problem 1 Show the comparison of runtime of linear search and binar.pdf
Problem 1 Show the comparison of runtime of linear search and binar.pdfProblem 1 Show the comparison of runtime of linear search and binar.pdf
Problem 1 Show the comparison of runtime of linear search and binar.pdf
 
SQL: Query optimization in practice
SQL: Query optimization in practiceSQL: Query optimization in practice
SQL: Query optimization in practice
 
A Deep Dive into Spark SQL's Catalyst Optimizer with Yin Huai
A Deep Dive into Spark SQL's Catalyst Optimizer with Yin HuaiA Deep Dive into Spark SQL's Catalyst Optimizer with Yin Huai
A Deep Dive into Spark SQL's Catalyst Optimizer with Yin Huai
 
Functional Programming in Swift
Functional Programming in SwiftFunctional Programming in Swift
Functional Programming in Swift
 
Understand the Query Plan to Optimize Performance with EXPLAIN and EXPLAIN AN...
Understand the Query Plan to Optimize Performance with EXPLAIN and EXPLAIN AN...Understand the Query Plan to Optimize Performance with EXPLAIN and EXPLAIN AN...
Understand the Query Plan to Optimize Performance with EXPLAIN and EXPLAIN AN...
 
Mysql
MysqlMysql
Mysql
 
Mysql
MysqlMysql
Mysql
 
Mysql
MysqlMysql
Mysql
 
Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007
 
MySQL Query tuning 101
MySQL Query tuning 101MySQL Query tuning 101
MySQL Query tuning 101
 
Using-Python-Libraries.9485146.powerpoint.pptx
Using-Python-Libraries.9485146.powerpoint.pptxUsing-Python-Libraries.9485146.powerpoint.pptx
Using-Python-Libraries.9485146.powerpoint.pptx
 
Dive into EXPLAIN - PostgreSql
Dive into EXPLAIN  - PostgreSqlDive into EXPLAIN  - PostgreSql
Dive into EXPLAIN - PostgreSql
 
Top 10 Oracle SQL tuning tips
Top 10 Oracle SQL tuning tipsTop 10 Oracle SQL tuning tips
Top 10 Oracle SQL tuning tips
 

More from Norvald Ryeng

More from Norvald Ryeng (9)

JSON Array Indexes in MySQL
JSON Array Indexes in MySQLJSON Array Indexes in MySQL
JSON Array Indexes in MySQL
 
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZEMySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
 
Spatial Support in MySQL
Spatial Support in MySQLSpatial Support in MySQL
Spatial Support in MySQL
 
LATERAL Derived Tables in MySQL 8.0
LATERAL Derived Tables in MySQL 8.0LATERAL Derived Tables in MySQL 8.0
LATERAL Derived Tables in MySQL 8.0
 
More SQL in MySQL 8.0
More SQL in MySQL 8.0More SQL in MySQL 8.0
More SQL in MySQL 8.0
 
MySQL 8.0: What Is New in Optimizer and Executor?
MySQL 8.0: What Is New in Optimizer and Executor?MySQL 8.0: What Is New in Optimizer and Executor?
MySQL 8.0: What Is New in Optimizer and Executor?
 
How to Take Advantage of Optimizer Improvements in MySQL 8.0
How to Take Advantage of Optimizer Improvements in MySQL 8.0How to Take Advantage of Optimizer Improvements in MySQL 8.0
How to Take Advantage of Optimizer Improvements in MySQL 8.0
 
MySQL 8.0 GIS Overview
MySQL 8.0 GIS OverviewMySQL 8.0 GIS Overview
MySQL 8.0 GIS Overview
 
MySQL 8.0: GIS — Are you ready?
MySQL 8.0: GIS — Are you ready?MySQL 8.0: GIS — Are you ready?
MySQL 8.0: GIS — Are you ready?
 

Recently uploaded

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
Alluxio, Inc.
 

Recently uploaded (20)

5 Reasons Driving Warehouse Management Systems Demand
5 Reasons Driving Warehouse Management Systems Demand5 Reasons Driving Warehouse Management Systems Demand
5 Reasons Driving Warehouse Management Systems Demand
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
 
10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf
 
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
 
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
 
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdfMicrosoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
 
What need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java DevelopersWhat need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java Developers
 
How to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabberHow to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabber
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
 
The Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion ProductionThe Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion Production
 
Workforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdfWorkforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdf
 
Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024
 
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAGAI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdf
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 

MySQL 8.0 EXPLAIN ANALYZE

  • 1. Norvald H. Ryeng Software Development Director MySQL Optimizer Team January 30, 2020 MySQL 8.0 EXPLAIN ANALYZE New in 8.0.18
  • 2. Safe harbor statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
  • 3. Program agenda 1 Quick demo 2 The MySQL query executor 3 EXPLAIN FORMAT=TREE 4 EXPLAIN ANALYZE
  • 5.
  • 6. The MySQL query executor TurtlesIterators all the way down
  • 7. Refactoring the executor  Refactor iterator interfaces From many to one interface From C style function pointers to C++ classes  Possible because phases were separated  Much more modular exeuctor One common iterator interface for all operations Each operation is contained within an iterator  Able to put together plans in new ways Immediate benefit: Removes temporary tables in some cases  Join is just an iterator Nested loop join is just an iterator Hash join is just an iterator Your favorite join method is just an iterator Parse Prepare Optimize Execute SQL Resolve Transform Abstract syntax tree Logical plan Physical plan
  • 8. MySQL iterator executor  Each operation is an iterator  Execution loop reads from root node Row by row May trigger multiple read calls further down  Common interface Init() Read() HashJoinIterator TableScanIterator TableScanIterator SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; t1 t2
  • 9. Old MySQL executor vs. iterator executor Old executor  Nested loop focused  Hard to extend  Code for one operation spread out  Different interfaces for each operation  Combination of operations hard coded Iterator executor  Modular  Easy to extend  Each iterator encapsulates one operation  Same interface for all iterators  All operations can be connected
  • 10. MySQL 8.0 features based on the iterator executor  Hash join Just another iterator type  EXPLAIN FORMAT=TREE Print the iterator tree  EXPLAIN ANALYZE 1. Insert intstrumentation nodes in the tree 2. Execute the query 3. Print the iterator tree Parse Prepare Optimize Execute SQL Resolve Transform Abstract syntax tree Logical plan Physical plan
  • 11. EXPLAIN FORMAT=TREE  Old EXPLAIN implementation Analyzes query plan and prints its interpretation of it Duplicates plan interpretation EXPLAIN interpretation Executor interpretation Still used for FORMAT=TRADITIONAL and FORMAT=JSON Hope to change to a single EXPLAIN implementation  EXPLAIN FORMAT=TREE Direct dump of execution structures A single plan interpretation for both EXPLAIN and execution Plan to reuse same implementation for all formats HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN FORMAT=TREE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) -> Table scan on t2 (cost=0.35 rows=1) -> Hash -> Table scan on t1 (cost=0.35 rows=1)
  • 12. EXPLAIN FORMAT=TREE  Old EXPLAIN implementation Analyzes query plan and prints its interpretation of it Duplicates plan interpretation EXPLAIN interpretation Executor interpretation Still used for FORMAT=TRADITIONAL and FORMAT=JSON Hope to change to a single EXPLAIN implementation  EXPLAIN FORMAT=TREE Direct dump of execution structures A single plan interpretation for both EXPLAIN and execution Plan to reuse same implementation for all formats HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN FORMAT=TREE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) -> Table scan on t2 (cost=0.35 rows=1) -> Hash -> Table scan on t1 (cost=0.35 rows=1)
  • 14. TimingIteratorTimingIterator TimingIterator MySQL EXPLAIN ANALYZE  Wrap iterators in instrumentation nodes  Measurements Time (in ms) to first row Time (in ms) to last row Number of rows Number of loops  Execute the query and dump the stats  Built on EXPLAIN FORMAT=TREE HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN ANALYZE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1) -> Table scan on t2 (cost=0.35 rows=1) (never executed) -> Hash -> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
  • 15. TimingIteratorTimingIterator TimingIterator MySQL EXPLAIN ANALYZE  Profiling of query execution Where does the executor spend time? What are the actual row counts?  Low overhead Timing is close to normal execution But considered too expensive to enable for all queries HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN ANALYZE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1) -> Table scan on t2 (cost=0.35 rows=1) (never executed) -> Hash -> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
  • 16. 467x 1520x>1400x 1332x 968x What's wrong with Q2? SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, partsupp, nation, region WHERE p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND p_size = 4 AND p_type LIKE '%TIN' AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey AND r_name = 'AMERICA' AND ps_supplycost = ( SELECT min(ps_supplycost) FROM partsupp, supplier, nation, region WHERE p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey AND r_name = 'AMERICA' ) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 100; MySQL EXPLAIN ANALYZE to the rescue!
  • 17. -> Limit: 100 row(s) (actual time=591739.000..591739.018 rows=100 loops=1) -> Sort: <temporary>.s_acctbal DESC, <temporary>.n_name, <temporary>.s_name, <temporary>.p_partkey, limit input to 100 row(s) per chunk (actual time -> Stream results (actual time=591738.599..591738.772 rows=462 loops=1) -> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost=2074295.37 rows=98) (actual -> Table scan on nation (cost=0.00 rows=25) (actual time=0.024..0.026 rows=25 loops=1) -> Hash -> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=2074041.10 rows=98) (actual time=591735.554..591738.311 rows=46 -> Table scan on supplier (cost=0.06 rows=9760) (actual time=0.068..2.024 rows=10000 loops=1) -> Hash -> Filter: (partsupp.ps_supplycost = (select #2)) (cost=1977898.52 rows=98) (actual time=1282.855..591733.987 rows=462 loop -> Inner hash join (partsupp.ps_partkey = part.p_partkey) (cost=1977898.52 rows=98) (actual time=84.827..271.307 rows=3 -> Table scan on partsupp (cost=3.54 rows=796168) (actual time=0.034..108.684 rows=800000 loops=1) -> Hash -> Inner hash join (cost=20353.91 rows=24) (actual time=0.274..83.964 rows=780 loops=1) -> Filter: ((part.p_size = 4) and (part.p_type like '%TIN')) (cost=20353.16 rows=2204) (actual time=0.212.. -> Table scan on part (cost=20353.16 rows=198401) (actual time=0.160..63.957 rows=200000 loops=1) -> Hash -> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.029..0.036 rows=1 loops=1) -> Table scan on region (cost=0.75 rows=5) (actual time=0.021..0.030 rows=5 loops=1) -> Select #2 (subquery in condition; dependent) -> Aggregate: min(partsupp.ps_supplycost) (actual time=189.566..189.566 rows=1 loops=3120) -> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost -> Table scan on nation (cost=0.00 rows=25) (actual time=0.005..0.007 rows=25 loops=3120) -> Hash -> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=77789257.28 rows=79617) (actual tim -> Table scan on supplier (cost=0.02 rows=9760) (actual time=0.014..1.237 rows=10000 loops=3120) -> Hash -> Filter: (part.p_partkey = partsupp.ps_partkey) (cost=81757.41 rows=79617) (actual time=92.08 -> Inner hash join (cost=81757.41 rows=79617) (actual time=0.018..155.118 rows=800000 loops -> Table scan on partsupp (cost=10101.54 rows=796168) (actual time=0.011..97.353 rows=8 -> Hash -> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.004..0.005 -> Table scan on region (cost=0.75 rows=5) (actual time=0.003..0.003 rows=5 loo Time is in milliseconds
  • 18. MySQL 8.0 query analysis toolbox Analysis:  EXPLAIN Plan analysis Cost estimates  EXPLAIN ANALYZE Query profiling Actual statistics  Optimizer trace Debug tracing Optimizer decisions Mitigation:  Optimizer hints Per query workarounds Enable/disable specific optimizations Ask for specific plan choices Access methods Join order  Optimizer switch Per session workarounds Enable/disable specific optimizations New in 8.0.18
  • 19. Feature descriptions and design details directly from the source https://mysqlserverteam.com/
  • 20. Thank you Norvald H. Ryeng Software Development Director MySQL Optimizer Team