Mysql Explain Explained

Jeremy Coates
Jeremy CoatesFounder & Chief Executive Officer (CEO) at Magma Digital Ltd
MySQL EXPLAIN Explained
   Quick and Easy Query Optimisation




   Adrian Hardy <talks@fuzzee.co.uk>
Before we begin...
What you need to know
    How and why we add indexes to tables
●


    The benefits of correct field typing
●


    Understanding of the ideals of 3NF
●


    Basic understanding of SQL JOINs
●




This presentation
    Very quick introduction to EXPLAIN
●



    Improve understanding of MySQL and indexing
●


    Simplified examples / results
●
Introduction - Using MySQL EXPLAIN
  Prefix a SELECT query with EXPLAIN
      MySQL won't actually execute the query, just analyse it
  ●


      EXPLAIN helps us understand how and when MySQL
  ●


      will use indexes
      EXPLAIN returns a table of data from which you identify
  ●


      potential improvements
      Optimise queries in three ways
  ●


           Modify or create indexes
       ●


           Modify query structure
       ●


           Modify data structure
       ●


      Optimised queries = faster results, lower server load...
  ●
Introduction - Review of Indexing
    Fast, compact structure for identifying row locations
●


    Keep indexes in memory by trimming the fat:
●


        Can I reduce the characters in that VARCHAR index?
    ●



        Can I use a TINYINT instead of a BIGINT?
    ●



        Can I use an INTEGER to describe a status or flag (rather
    ●


        than a textual description)?
    Chop down your result set as quickly as possible
●


    MySQL will only use one index per query/table – it cannot
●

    combine two separate indexes to make a useful one *


Understanding and preparation brings about Indexing Strategy



                                  * Not strictly true - look up “Index Merge” operations
Booking application schema

attendees
  attendee_id      surname      conference_id   registration_status
 INTEGER (PK)     VARCHAR       INTEGER (FK)         TINYINT




conferences
 conference_id    location_id     topic_id             date
 INTEGER (PK)    INTEGER (FK)   INTEGER (FK)          DATE
EXPLAIN – Worked Example
     EXPLAIN SELECT * FROM attendees WHERE
conference_id = 123 AND registration_status > 0

           table       possible_keys          key              rows
         attendees         NULL              NULL              14052


   The three most important columns returned by EXPLAIN
    1)Possible keys
         All the possible indexes which MySQL could have used
     ●



         Based on a series of very quick lookups and
     ●

         calculations
    2)Chosen key
    3)Rows scanned
         Indication of effort required to identify your result set
     ●
EXPLAIN – Worked Example
     EXPLAIN SELECT * FROM attendees WHERE
conference_id = 123 AND registration_status > 0

             table      possible_keys          key            rows
           attendees        NULL               NULL       14052


   Interpreting the results
       No suitable indexes for this query
   ●



           MySQL had to do a full table scan
       ●



       Full table scans are almost always the slowest query
   ●



       Full table scans, while not always bad, are usually an
   ●

       indication that an index is required
EXPLAIN – Worked Example
   ALTER TABLE ADD INDEX conf (conference_id);
 ALTER TABLE ADD INDEX reg (registration_status);

EXPLAIN SELECT * FROM attendees WHERE conference_id
         = 123 AND registration_status > 1;

            table       possible_keys           key      rows
          attendees        conf, reg            conf      331


      MySQL had two indexes to choose from, but discarded “reg”
  ●



      “reg” isn't sufficiently unique
  ●



          The spread of values can also be a factor (e.g when 99% of
      ●

          rows contain the same value)
      Index “uniqueness” is called cardinality
  ●



      There is scope for some performance increase...
  ●



          Lower server load, quicker response
      ●
EXPLAIN – Worked Example
           ALTER TABLE ADD INDEX reg_conf_index
          (registration_status, conference_id);

EXPLAIN SELECT * FROM attendees WHERE conference_id =
           123 AND registration_status > 1;

          table     possible_keys         key         rows
                       reg, conf,
        attendees                    reg_conf_index   204
                    reg_conf_index

        reg_conf_index is a much better choice
    ●



     Note that the other two keys are still available, just
    ●

    not as effective
        Our query is now served well by the new index
    ●
EXPLAIN – Worked Example
              DELETE INDEX conf; DELETE INDEX reg;
EXPLAIN SELECT * FROM attendees WHERE conference_id = 123

           table     possible_keys        key           rows
         attendees       NULL            NULL           14052

       Without the “conf” index, we're back to square one
   ●


       The order in which fields were defined in a composite index
   ●


       affects whether it is available for use in a query
       ● Remember,     we defined our index : (registration_status,
         conference_id)
   Potential workaround:
EXPLAIN SELECT * FROM attendees WHERE conference_id = 123
                AND registration_status >= -1

           table      possible_keys         key          rows
         attendees    reg_conf_index   reg_conf_index     204
EXPLAIN – Example 2
EXPLAIN SELECT * FROM attendees WHERE surname LIKE 'har%';

         table        possible_keys      key           rows
       attendees        surname        surname             234


         MySQL uses an index on surname – which is good.



EXPLAIN SELECT * FROM attendees WHERE surname LIKE '%har%';

         table        possible_keys      key           rows
       attendees          NULL          NULL           14052


                 MySQL doesn't even try to use an index!
EXPLAIN – Example 3
EXPLAIN SELECT * FROM conferences WHERE location_id = 2 OR
                    topic_id IN (4,6,1)

         table      possible_keys         key         rows
                     location_id,
      conferences                        NULL         5043
                       topic_id

           MySQL doesn't use an index, because of the OR

 ALTER TABLE ADD INDEX location_topic (location_id,
                     topic_id);

 EXPLAIN SELECT * FROM conferences WHERE location_id = 2
                  OR topic_id IN (4,6,1)

         table      possible_keys         key         rows
                     location_id,
      conferences                    location_topic    15
                       topic_id,
                    location_topic

     Full table scan avoided – could also use UNION (ALL) trick
EXPLAIN – Example 4
EXPLAIN SELECT * FROM attendees WHERE MD5(conference_id) =
                         MD5(123)
         table      possible_keys      key           rows
       attendees        NULL          NULL          14052

         Understandably, MySQL has to do a full table scan



                     A more realistic example?
          EXPLAIN SELECT * FROM conferences WHERE
               DATE_FORMAT(date,'%a') = 'Sat'

         table      possible_keys      key           rows
      conferences       NULL          NULL           5043

    A good candidate for Optimisation #3 – Modify Data Structure
JOINs
    JOINing together large data sets (>= 100,000) is really
●


    where EXPLAIN becomes useful
    Each JOIN in a query gets its own row in EXPLAIN
●


        Make sure each JOIN condition is FAST
    ●


    Make sure each joined table is getting to its result set
●

    as quickly as possible
    ● The benefits compound if each join requires less

      effort
JOINs – Simple Example
                  EXPLAIN SELECT * FROM
conferences INNER JOIN attendees USING (conference_id)
         WHERE conferences.location_id = 2 AND
           conferences.topic_id IN (4,6,1) AND
            attendees.registration_status > 1


   table            type       possible_keys         key              rows
conferences          ref      conference_topic conference_topic        15

 attendees          ALL            NULL             NULL              14052


           Looks like I need an index on attendees.conference_id

      There are 13 different values for “type”
        ● Another indication of effort, aside from rows scanned

        ● Here, “ALL” is bad – we should be aiming for “ref”

          ● Common values are “const”, “ref”, and “all”

        ● http://dev.mysql.com/doc/refman/5.0/en/using-explain.html
The “extra” column
  With every EXPLAIN, you get an “extra” column, which
  shows additional operations invoked to get your result set.
  table     possible_keys     key          rows           extra
                                                      Using where
attendees         conf       conf          331
                                                      Using filesort

  Some example “extra” values:

        Using   where
    ●


        Using   temporary table
    ●


        Using   filesort
    ●


        Using   index
    ●




  There are many more “extra” values which are discussed in
  the MySQL manual.
“Using filesort”
Avoid, because:
● Doesn't use an index

  ● Involves a full scan of your result set


  ● Employs a generic (i.e. one size fits all)

    algorithm
● Uses the filesystem (eeek)


● Will get slower with more data



It's not all bad...
    Perfectly acceptable provided you get to your
●

    result set as quickly as possible, and keep it
    predictably small
    Sometimes unavoidable - ORDER BY RAND()
●


    ORDER BY operations can use indexes to do the
●


    sorting!
“Using filesort” – Example
 EXPLAIN SELECT * FROM attendees WHERE conference_id = 123
                     ORDER BY surname
    table     possible_keys         key             rows         Extra
  attendees    conference_id    conference_id       331       Using filesort


       MySQL is using an index, but it's sorting the results slowly

ALTER TABLE attendees ADD INDEX conf_surname (conference_id,
                          surname);
 EXPLAIN SELECT * FROM attendees WHERE conference_id = 123
                      ORDER BY surname
    table     possible_keys         key             rows         Extra
               conference_id,
  attendees                     conf_surname        331
               conf_surname

                         We've avoided a filesort
“Using index”
    Celebrate, because:
 MySQL got your results just by consulting the index,
●


  ● Which could well have been sat in memory


●MySQL didn't need to even look at the table to get you your

results
  ● Opening a table can be an expensive operation.


●MySQL can answer the next query more quickly


●The fastest way for you to get your data?




    Particularly useful...
 When you're just interested in a single date or an id
●


●Or the COUNT(), SUM(), AVG() etc. of a field
“Using index” – Example
EXPLAIN SELECT AVG(age) FROM attendees WHERE conference_id
                           = 123
   table         possible_keys         key           rows             Extra
 attendees        conference_id    conference_id     331


  Nothing is actually wrong with this query – it could just be quicker!

 ALTER TABLE attendees ADD INDEX conf_age (conference_id,
                           age);
EXPLAIN SELECT AVG(age) FROM attendees WHERE conference_id
                          = 123
    table        possible_keys          key          rows             Extra
                  conference_id,
  attendees                        conf_surname       331         Using index
                  conf_surname

             Outside of caching, the fastest way to get your data *
                                                                  *Not a guarantee
Moving forward...

Just because your queries are fast now, doesn't mean that they will stay
that way forever

Enable MySQL's Slow Query Log
 ● --log-slow-queries=/var/lib/mysql/slow-query.log

 ● Defaults to logging queries which take more than 10 seconds

 ● --long_query_time=1

 ● Use Percona's “microslow” patch for values < 1 second

 ● Find the query in the log, EXPLAIN it, improve it, rinse and repeat
Moving forward...
Use the command line to identify more general problems
 ● mysqladmin -u dbuser -p -r -i 10 extended-status

 ● Figures are relative, updated every 10 seconds

      ● Slow_queries = number of slow queries in last period

      ● Select_Scan = full table scans

      ● Select_full_join = full scans to complete join operations

      ● Created_tmp_disk_tables = filesorts

      ● Key_read_requests/Key_write_requests

        ● Determine write/read weighting of our application and alter your

          indexes accordingly
MySQL Resources

    http://dev.mysql.com/doc/refman/5.0/en/using-explain.html
●



    High Performance MySQL - Baron Schwartz
●



          ISBN 0596101716
      –

          £20 (Money well spent)
      –

    http://www.mysqlperformanceblog.com
●



              Regular posts
          –
1 of 23

Recommended

MySQL Query And Index Tuning by
MySQL Query And Index TuningMySQL Query And Index Tuning
MySQL Query And Index TuningManikanda kumar
17.9K views46 slides
MySQL: Indexing for Better Performance by
MySQL: Indexing for Better PerformanceMySQL: Indexing for Better Performance
MySQL: Indexing for Better Performancejkeriaki
5.7K views26 slides
Explain that explain by
Explain that explainExplain that explain
Explain that explainFabrizio Parrella
5.1K views43 slides
The MySQL Query Optimizer Explained Through Optimizer Trace by
The MySQL Query Optimizer Explained Through Optimizer TraceThe MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer Traceoysteing
1K views55 slides
Advanced MySQL Query Tuning by
Advanced MySQL Query TuningAdvanced MySQL Query Tuning
Advanced MySQL Query TuningAlexander Rubin
1.2K views67 slides
MySQL partitions tutorial by
MySQL partitions tutorialMySQL partitions tutorial
MySQL partitions tutorialGiuseppe Maxia
112.8K views118 slides

More Related Content

What's hot

How to Use JSON in MySQL Wrong by
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongKarwin Software Solutions LLC
27.3K views73 slides
MySQL Performance Schema in Action: the Complete Tutorial by
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialSveta Smirnova
2.1K views174 slides
Indexing the MySQL Index: Key to performance tuning by
Indexing the MySQL Index: Key to performance tuningIndexing the MySQL Index: Key to performance tuning
Indexing the MySQL Index: Key to performance tuningOSSCube
16.9K views40 slides
MySQL Query Optimization by
MySQL Query OptimizationMySQL Query Optimization
MySQL Query OptimizationMorgan Tocker
1.8K views51 slides
How to Analyze and Tune MySQL Queries for Better Performance by
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
1.9K views119 slides
More mastering the art of indexing by
More mastering the art of indexingMore mastering the art of indexing
More mastering the art of indexingYoshinori Matsunobu
27.7K views54 slides

What's hot(20)

MySQL Performance Schema in Action: the Complete Tutorial by Sveta Smirnova
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete Tutorial
Sveta Smirnova2.1K views
Indexing the MySQL Index: Key to performance tuning by OSSCube
Indexing the MySQL Index: Key to performance tuningIndexing the MySQL Index: Key to performance tuning
Indexing the MySQL Index: Key to performance tuning
OSSCube16.9K views
MySQL Query Optimization by Morgan Tocker
MySQL Query OptimizationMySQL Query Optimization
MySQL Query Optimization
Morgan Tocker1.8K views
How to Analyze and Tune MySQL Queries for Better Performance by oysteing
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
oysteing1.9K views
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013 by Jaime Crespo
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Jaime Crespo20.3K views
MySQL Performance Tuning: Top 10 Tips by OSSCube
MySQL Performance Tuning: Top 10 TipsMySQL Performance Tuning: Top 10 Tips
MySQL Performance Tuning: Top 10 Tips
OSSCube55.6K views
Using Optimizer Hints to Improve MySQL Query Performance by oysteing
Using Optimizer Hints to Improve MySQL Query PerformanceUsing Optimizer Hints to Improve MySQL Query Performance
Using Optimizer Hints to Improve MySQL Query Performance
oysteing5.6K views
How to Analyze and Tune MySQL Queries for Better Performance by oysteing
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
oysteing1.7K views
SQL Joins and Query Optimization by Brian Gallagher
SQL Joins and Query OptimizationSQL Joins and Query Optimization
SQL Joins and Query Optimization
Brian Gallagher25.4K views

Similar to Mysql Explain Explained

Introduction to Databases - query optimizations for MySQL by
Introduction to Databases - query optimizations for MySQLIntroduction to Databases - query optimizations for MySQL
Introduction to Databases - query optimizations for MySQLMárton Kodok
735 views32 slides
MySQL Performance Optimization by
MySQL Performance OptimizationMySQL Performance Optimization
MySQL Performance OptimizationMindfire Solutions
506 views28 slides
Scaling MySQL Strategies for Developers by
Scaling MySQL Strategies for DevelopersScaling MySQL Strategies for Developers
Scaling MySQL Strategies for DevelopersJonathan Levin
4.5K views165 slides
Goldilocks and the Three MySQL Queries by
Goldilocks and the Three MySQL QueriesGoldilocks and the Three MySQL Queries
Goldilocks and the Three MySQL QueriesDave Stokes
507 views43 slides
Myth busters - performance tuning 101 2007 by
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007paulguerin
527 views28 slides
Why Use EXPLAIN FORMAT=JSON? by
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON? Sveta Smirnova
1.8K views47 slides

Similar to Mysql Explain Explained(20)

Introduction to Databases - query optimizations for MySQL by Márton Kodok
Introduction to Databases - query optimizations for MySQLIntroduction to Databases - query optimizations for MySQL
Introduction to Databases - query optimizations for MySQL
Márton Kodok735 views
Scaling MySQL Strategies for Developers by Jonathan Levin
Scaling MySQL Strategies for DevelopersScaling MySQL Strategies for Developers
Scaling MySQL Strategies for Developers
Jonathan Levin4.5K views
Goldilocks and the Three MySQL Queries by Dave Stokes
Goldilocks and the Three MySQL QueriesGoldilocks and the Three MySQL Queries
Goldilocks and the Three MySQL Queries
Dave Stokes507 views
Myth busters - performance tuning 101 2007 by paulguerin
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007
paulguerin527 views
Why Use EXPLAIN FORMAT=JSON? by Sveta Smirnova
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON?
Sveta Smirnova1.8K views
Advanced MySQL Query Optimizations by Dave Stokes
Advanced MySQL Query OptimizationsAdvanced MySQL Query Optimizations
Advanced MySQL Query Optimizations
Dave Stokes796 views
Optimizing Queries with Explain by MYXPLAIN
Optimizing Queries with ExplainOptimizing Queries with Explain
Optimizing Queries with Explain
MYXPLAIN2K views
Top 10 Oracle SQL tuning tips by Nirav Shah
Top 10 Oracle SQL tuning tipsTop 10 Oracle SQL tuning tips
Top 10 Oracle SQL tuning tips
Nirav Shah3.8K views
Explaining the Postgres Query Optimizer (Bruce Momjian) by Ontico
Explaining the Postgres Query Optimizer (Bruce Momjian)Explaining the Postgres Query Optimizer (Bruce Momjian)
Explaining the Postgres Query Optimizer (Bruce Momjian)
Ontico1.3K views
My SQL Skills Killed the Server by devObjective
My SQL Skills Killed the ServerMy SQL Skills Killed the Server
My SQL Skills Killed the Server
devObjective340 views
Understanding query-execution806 by yubao fu
Understanding query-execution806Understanding query-execution806
Understanding query-execution806
yubao fu888 views
Understanding Query Execution by webhostingguy
Understanding Query ExecutionUnderstanding Query Execution
Understanding Query Execution
webhostingguy1.1K views
MySQL Indexing : Improving Query Performance Using Index (Covering Index) by Hemant Kumar Singh
MySQL Indexing : Improving Query Performance Using Index (Covering Index)MySQL Indexing : Improving Query Performance Using Index (Covering Index)
MySQL Indexing : Improving Query Performance Using Index (Covering Index)
Introduction to MySQL Query Tuning for Dev[Op]s by Sveta Smirnova
Introduction to MySQL Query Tuning for Dev[Op]sIntroduction to MySQL Query Tuning for Dev[Op]s
Introduction to MySQL Query Tuning for Dev[Op]s
Sveta Smirnova1.1K views
Introduction into MySQL Query Tuning by Sveta Smirnova
Introduction into MySQL Query TuningIntroduction into MySQL Query Tuning
Introduction into MySQL Query Tuning
Sveta Smirnova674 views

More from Jeremy Coates

Cyber Security and GDPR by
Cyber Security and GDPRCyber Security and GDPR
Cyber Security and GDPRJeremy Coates
3.9K views28 slides
Aspect Oriented Programming by
Aspect Oriented ProgrammingAspect Oriented Programming
Aspect Oriented ProgrammingJeremy Coates
887 views13 slides
Why is PHP Awesome by
Why is PHP AwesomeWhy is PHP Awesome
Why is PHP AwesomeJeremy Coates
7.5K views37 slides
Testing with Codeception by
Testing with CodeceptionTesting with Codeception
Testing with CodeceptionJeremy Coates
20.6K views23 slides
An introduction to Phing the PHP build system (PHPDay, May 2012) by
An introduction to Phing the PHP build system (PHPDay, May 2012)An introduction to Phing the PHP build system (PHPDay, May 2012)
An introduction to Phing the PHP build system (PHPDay, May 2012)Jeremy Coates
3.2K views26 slides
An introduction to Phing the PHP build system by
An introduction to Phing the PHP build systemAn introduction to Phing the PHP build system
An introduction to Phing the PHP build systemJeremy Coates
5K views25 slides

More from Jeremy Coates(17)

Cyber Security and GDPR by Jeremy Coates
Cyber Security and GDPRCyber Security and GDPR
Cyber Security and GDPR
Jeremy Coates3.9K views
Aspect Oriented Programming by Jeremy Coates
Aspect Oriented ProgrammingAspect Oriented Programming
Aspect Oriented Programming
Jeremy Coates887 views
Testing with Codeception by Jeremy Coates
Testing with CodeceptionTesting with Codeception
Testing with Codeception
Jeremy Coates20.6K views
An introduction to Phing the PHP build system (PHPDay, May 2012) by Jeremy Coates
An introduction to Phing the PHP build system (PHPDay, May 2012)An introduction to Phing the PHP build system (PHPDay, May 2012)
An introduction to Phing the PHP build system (PHPDay, May 2012)
Jeremy Coates3.2K views
An introduction to Phing the PHP build system by Jeremy Coates
An introduction to Phing the PHP build systemAn introduction to Phing the PHP build system
An introduction to Phing the PHP build system
Jeremy Coates5K views
Hudson Continuous Integration for PHP by Jeremy Coates
Hudson Continuous Integration for PHPHudson Continuous Integration for PHP
Hudson Continuous Integration for PHP
Jeremy Coates2.7K views
The Uncertainty Principle by Jeremy Coates
The Uncertainty PrincipleThe Uncertainty Principle
The Uncertainty Principle
Jeremy Coates2.3K views
Exploiting Php With Php by Jeremy Coates
Exploiting Php With PhpExploiting Php With Php
Exploiting Php With Php
Jeremy Coates32.6K views
What's new, what's hot in PHP 5.3 by Jeremy Coates
What's new, what's hot in PHP 5.3What's new, what's hot in PHP 5.3
What's new, what's hot in PHP 5.3
Jeremy Coates9.6K views
Introduction to Version Control by Jeremy Coates
Introduction to Version ControlIntroduction to Version Control
Introduction to Version Control
Jeremy Coates13K views

Recently uploaded

Measuring User on the web with the core web vitals - by @theafolayan.pptx by
Measuring User on the web with the core web vitals - by @theafolayan.pptxMeasuring User on the web with the core web vitals - by @theafolayan.pptx
Measuring User on the web with the core web vitals - by @theafolayan.pptxOluwaseun Raphael Afolayan
14 views13 slides
CryptoBotsAI by
CryptoBotsAICryptoBotsAI
CryptoBotsAIchandureddyvadala199
42 views5 slides
LLMs in Production: Tooling, Process, and Team Structure by
LLMs in Production: Tooling, Process, and Team StructureLLMs in Production: Tooling, Process, and Team Structure
LLMs in Production: Tooling, Process, and Team StructureAggregage
65 views77 slides
AIM102-S_Cognizant_CognizantCognitive by
AIM102-S_Cognizant_CognizantCognitiveAIM102-S_Cognizant_CognizantCognitive
AIM102-S_Cognizant_CognizantCognitivePhilipBasford
23 views36 slides
Measurecamp Brussels - Synthetic data.pdf by
Measurecamp Brussels - Synthetic data.pdfMeasurecamp Brussels - Synthetic data.pdf
Measurecamp Brussels - Synthetic data.pdfHuman37
27 views14 slides
"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell by
"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell
"Node.js vs workers — A comparison of two JavaScript runtimes", James M SnellFwdays
14 views30 slides

Recently uploaded(20)

LLMs in Production: Tooling, Process, and Team Structure by Aggregage
LLMs in Production: Tooling, Process, and Team StructureLLMs in Production: Tooling, Process, and Team Structure
LLMs in Production: Tooling, Process, and Team Structure
Aggregage65 views
AIM102-S_Cognizant_CognizantCognitive by PhilipBasford
AIM102-S_Cognizant_CognizantCognitiveAIM102-S_Cognizant_CognizantCognitive
AIM102-S_Cognizant_CognizantCognitive
PhilipBasford23 views
Measurecamp Brussels - Synthetic data.pdf by Human37
Measurecamp Brussels - Synthetic data.pdfMeasurecamp Brussels - Synthetic data.pdf
Measurecamp Brussels - Synthetic data.pdf
Human37 27 views
"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell by Fwdays
"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell
"Node.js vs workers — A comparison of two JavaScript runtimes", James M Snell
Fwdays14 views
Discover Aura Workshop (12.5.23).pdf by Neo4j
Discover Aura Workshop (12.5.23).pdfDiscover Aura Workshop (12.5.23).pdf
Discover Aura Workshop (12.5.23).pdf
Neo4j20 views
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays38 views
AI + Memoori = AIM by Memoori
AI + Memoori = AIMAI + Memoori = AIM
AI + Memoori = AIM
Memoori15 views
Adopting Karpenter for Cost and Simplicity at Grafana Labs.pdf by MichaelOLeary82
Adopting Karpenter for Cost and Simplicity at Grafana Labs.pdfAdopting Karpenter for Cost and Simplicity at Grafana Labs.pdf
Adopting Karpenter for Cost and Simplicity at Grafana Labs.pdf
MichaelOLeary8213 views
The Role of Patterns in the Era of Large Language Models by Yunyao Li
The Role of Patterns in the Era of Large Language ModelsThe Role of Patterns in the Era of Large Language Models
The Role of Patterns in the Era of Large Language Models
Yunyao Li104 views
Digital Personal Data Protection (DPDP) Practical Approach For CISOs by Priyanka Aash
Digital Personal Data Protection (DPDP) Practical Approach For CISOsDigital Personal Data Protection (DPDP) Practical Approach For CISOs
Digital Personal Data Protection (DPDP) Practical Approach For CISOs
Priyanka Aash171 views
The Power of Generative AI in Accelerating No Code Adoption.pdf by Saeed Al Dhaheri
The Power of Generative AI in Accelerating No Code Adoption.pdfThe Power of Generative AI in Accelerating No Code Adoption.pdf
The Power of Generative AI in Accelerating No Code Adoption.pdf
Saeed Al Dhaheri44 views
Redefining the book supply chain: A glimpse into the future - Tech Forum 2023 by BookNet Canada
Redefining the book supply chain: A glimpse into the future - Tech Forum 2023Redefining the book supply chain: A glimpse into the future - Tech Forum 2023
Redefining the book supply chain: A glimpse into the future - Tech Forum 2023
BookNet Canada46 views
GDSC GLAU Info Session.pptx by gauriverrma4
GDSC GLAU Info Session.pptxGDSC GLAU Info Session.pptx
GDSC GLAU Info Session.pptx
gauriverrma415 views
Bronack Skills - Risk Management and SRE v1.0 12-3-2023.pdf by ThomasBronack
Bronack Skills - Risk Management and SRE v1.0 12-3-2023.pdfBronack Skills - Risk Management and SRE v1.0 12-3-2023.pdf
Bronack Skills - Risk Management and SRE v1.0 12-3-2023.pdf
ThomasBronack31 views
Initiating and Advancing Your Strategic GIS Governance Strategy by Safe Software
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
Safe Software198 views

Mysql Explain Explained

  • 1. MySQL EXPLAIN Explained Quick and Easy Query Optimisation Adrian Hardy <talks@fuzzee.co.uk>
  • 2. Before we begin... What you need to know How and why we add indexes to tables ● The benefits of correct field typing ● Understanding of the ideals of 3NF ● Basic understanding of SQL JOINs ● This presentation Very quick introduction to EXPLAIN ● Improve understanding of MySQL and indexing ● Simplified examples / results ●
  • 3. Introduction - Using MySQL EXPLAIN Prefix a SELECT query with EXPLAIN MySQL won't actually execute the query, just analyse it ● EXPLAIN helps us understand how and when MySQL ● will use indexes EXPLAIN returns a table of data from which you identify ● potential improvements Optimise queries in three ways ● Modify or create indexes ● Modify query structure ● Modify data structure ● Optimised queries = faster results, lower server load... ●
  • 4. Introduction - Review of Indexing Fast, compact structure for identifying row locations ● Keep indexes in memory by trimming the fat: ● Can I reduce the characters in that VARCHAR index? ● Can I use a TINYINT instead of a BIGINT? ● Can I use an INTEGER to describe a status or flag (rather ● than a textual description)? Chop down your result set as quickly as possible ● MySQL will only use one index per query/table – it cannot ● combine two separate indexes to make a useful one * Understanding and preparation brings about Indexing Strategy * Not strictly true - look up “Index Merge” operations
  • 5. Booking application schema attendees attendee_id surname conference_id registration_status INTEGER (PK) VARCHAR INTEGER (FK) TINYINT conferences conference_id location_id topic_id date INTEGER (PK) INTEGER (FK) INTEGER (FK) DATE
  • 6. EXPLAIN – Worked Example EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 AND registration_status > 0 table possible_keys key rows attendees NULL NULL 14052 The three most important columns returned by EXPLAIN 1)Possible keys All the possible indexes which MySQL could have used ● Based on a series of very quick lookups and ● calculations 2)Chosen key 3)Rows scanned Indication of effort required to identify your result set ●
  • 7. EXPLAIN – Worked Example EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 AND registration_status > 0 table possible_keys key rows attendees NULL NULL 14052 Interpreting the results No suitable indexes for this query ● MySQL had to do a full table scan ● Full table scans are almost always the slowest query ● Full table scans, while not always bad, are usually an ● indication that an index is required
  • 8. EXPLAIN – Worked Example ALTER TABLE ADD INDEX conf (conference_id); ALTER TABLE ADD INDEX reg (registration_status); EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 AND registration_status > 1; table possible_keys key rows attendees conf, reg conf 331 MySQL had two indexes to choose from, but discarded “reg” ● “reg” isn't sufficiently unique ● The spread of values can also be a factor (e.g when 99% of ● rows contain the same value) Index “uniqueness” is called cardinality ● There is scope for some performance increase... ● Lower server load, quicker response ●
  • 9. EXPLAIN – Worked Example ALTER TABLE ADD INDEX reg_conf_index (registration_status, conference_id); EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 AND registration_status > 1; table possible_keys key rows reg, conf, attendees reg_conf_index 204 reg_conf_index reg_conf_index is a much better choice ● Note that the other two keys are still available, just ● not as effective Our query is now served well by the new index ●
  • 10. EXPLAIN – Worked Example DELETE INDEX conf; DELETE INDEX reg; EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 table possible_keys key rows attendees NULL NULL 14052 Without the “conf” index, we're back to square one ● The order in which fields were defined in a composite index ● affects whether it is available for use in a query ● Remember, we defined our index : (registration_status, conference_id) Potential workaround: EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 AND registration_status >= -1 table possible_keys key rows attendees reg_conf_index reg_conf_index 204
  • 11. EXPLAIN – Example 2 EXPLAIN SELECT * FROM attendees WHERE surname LIKE 'har%'; table possible_keys key rows attendees surname surname 234 MySQL uses an index on surname – which is good. EXPLAIN SELECT * FROM attendees WHERE surname LIKE '%har%'; table possible_keys key rows attendees NULL NULL 14052 MySQL doesn't even try to use an index!
  • 12. EXPLAIN – Example 3 EXPLAIN SELECT * FROM conferences WHERE location_id = 2 OR topic_id IN (4,6,1) table possible_keys key rows location_id, conferences NULL 5043 topic_id MySQL doesn't use an index, because of the OR ALTER TABLE ADD INDEX location_topic (location_id, topic_id); EXPLAIN SELECT * FROM conferences WHERE location_id = 2 OR topic_id IN (4,6,1) table possible_keys key rows location_id, conferences location_topic 15 topic_id, location_topic Full table scan avoided – could also use UNION (ALL) trick
  • 13. EXPLAIN – Example 4 EXPLAIN SELECT * FROM attendees WHERE MD5(conference_id) = MD5(123) table possible_keys key rows attendees NULL NULL 14052 Understandably, MySQL has to do a full table scan A more realistic example? EXPLAIN SELECT * FROM conferences WHERE DATE_FORMAT(date,'%a') = 'Sat' table possible_keys key rows conferences NULL NULL 5043 A good candidate for Optimisation #3 – Modify Data Structure
  • 14. JOINs JOINing together large data sets (>= 100,000) is really ● where EXPLAIN becomes useful Each JOIN in a query gets its own row in EXPLAIN ● Make sure each JOIN condition is FAST ● Make sure each joined table is getting to its result set ● as quickly as possible ● The benefits compound if each join requires less effort
  • 15. JOINs – Simple Example EXPLAIN SELECT * FROM conferences INNER JOIN attendees USING (conference_id) WHERE conferences.location_id = 2 AND conferences.topic_id IN (4,6,1) AND attendees.registration_status > 1 table type possible_keys key rows conferences ref conference_topic conference_topic 15 attendees ALL NULL NULL 14052 Looks like I need an index on attendees.conference_id There are 13 different values for “type” ● Another indication of effort, aside from rows scanned ● Here, “ALL” is bad – we should be aiming for “ref” ● Common values are “const”, “ref”, and “all” ● http://dev.mysql.com/doc/refman/5.0/en/using-explain.html
  • 16. The “extra” column With every EXPLAIN, you get an “extra” column, which shows additional operations invoked to get your result set. table possible_keys key rows extra Using where attendees conf conf 331 Using filesort Some example “extra” values: Using where ● Using temporary table ● Using filesort ● Using index ● There are many more “extra” values which are discussed in the MySQL manual.
  • 17. “Using filesort” Avoid, because: ● Doesn't use an index ● Involves a full scan of your result set ● Employs a generic (i.e. one size fits all) algorithm ● Uses the filesystem (eeek) ● Will get slower with more data It's not all bad... Perfectly acceptable provided you get to your ● result set as quickly as possible, and keep it predictably small Sometimes unavoidable - ORDER BY RAND() ● ORDER BY operations can use indexes to do the ● sorting!
  • 18. “Using filesort” – Example EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 ORDER BY surname table possible_keys key rows Extra attendees conference_id conference_id 331 Using filesort MySQL is using an index, but it's sorting the results slowly ALTER TABLE attendees ADD INDEX conf_surname (conference_id, surname); EXPLAIN SELECT * FROM attendees WHERE conference_id = 123 ORDER BY surname table possible_keys key rows Extra conference_id, attendees conf_surname 331 conf_surname We've avoided a filesort
  • 19. “Using index” Celebrate, because: MySQL got your results just by consulting the index, ● ● Which could well have been sat in memory ●MySQL didn't need to even look at the table to get you your results ● Opening a table can be an expensive operation. ●MySQL can answer the next query more quickly ●The fastest way for you to get your data? Particularly useful... When you're just interested in a single date or an id ● ●Or the COUNT(), SUM(), AVG() etc. of a field
  • 20. “Using index” – Example EXPLAIN SELECT AVG(age) FROM attendees WHERE conference_id = 123 table possible_keys key rows Extra attendees conference_id conference_id 331 Nothing is actually wrong with this query – it could just be quicker! ALTER TABLE attendees ADD INDEX conf_age (conference_id, age); EXPLAIN SELECT AVG(age) FROM attendees WHERE conference_id = 123 table possible_keys key rows Extra conference_id, attendees conf_surname 331 Using index conf_surname Outside of caching, the fastest way to get your data * *Not a guarantee
  • 21. Moving forward... Just because your queries are fast now, doesn't mean that they will stay that way forever Enable MySQL's Slow Query Log ● --log-slow-queries=/var/lib/mysql/slow-query.log ● Defaults to logging queries which take more than 10 seconds ● --long_query_time=1 ● Use Percona's “microslow” patch for values < 1 second ● Find the query in the log, EXPLAIN it, improve it, rinse and repeat
  • 22. Moving forward... Use the command line to identify more general problems ● mysqladmin -u dbuser -p -r -i 10 extended-status ● Figures are relative, updated every 10 seconds ● Slow_queries = number of slow queries in last period ● Select_Scan = full table scans ● Select_full_join = full scans to complete join operations ● Created_tmp_disk_tables = filesorts ● Key_read_requests/Key_write_requests ● Determine write/read weighting of our application and alter your indexes accordingly
  • 23. MySQL Resources http://dev.mysql.com/doc/refman/5.0/en/using-explain.html ● High Performance MySQL - Baron Schwartz ● ISBN 0596101716 – £20 (Money well spent) – http://www.mysqlperformanceblog.com ● Regular posts –