Capturing, Analyzing &
       Optimizing your SQL
       Padraig O'Sullivan - http://posulliv.com/
     Ronald Bradford - ...
Capturing your SQL

ODTUG – June 2010
MySQL Provided Options
    Slow Query Log
    General Query Log
    Binary Log
    Connectors
    Processlist
    Engine s...
Other Techniques
    MySQL Proxy
    TCP/IP Capture
    Dtrace/SystemTap
    Application Management




ODTUG – June 2010
Slow Query Log
    Not enabled by default
        log-slow-queries = /path/to/file (5.0)
        slow_query_log/slow_query...
Slow Query Log Example
# Time: 100518 19:34:31
# User@Host: user[db] @ localhost []
# Query_time: 4  Lock_time: 0  Rows_se...
Slow Query Table Example

mysql> select start_time, query_time, rows_sent, rows_examined, db, sql_text
    ­> from mysql.s...
General Query Log
    Not enabled by default
        log = /path/to/file
        general_log/general_log_file (5.1)
    Lo...
General Query Log Example
    Output using Wordpress


 26 Query SELECT * FROM wp_users WHERE user_login = 'xxxx'
 26 Quer...
General Query Log Example (2)
    Framework new connection


    23 Connect     user@host
    23 Query       SELECT 1
    ...
General Query Log Example (3)
     Finding framework overhead
         ROLLBACK after every SELECT
         SELECT 1 – con...
Binary Log
    Not enabled by default
        log-bin = /path/to/file
        expire_log_days = 5
    View with mysqlbinlo...
Binary Log Example
    mysqlbinlog /path/to/log-bin.000000
 # at 2461
 #100604 10:49:43 server id 1  end_log_pos 2632 Quer...
Connector/J
    logSlowQueries
    slowQueryThresholdMillis
    SlowQueryThresholdNanos




    http://dev.mysql.com/doc/r...
Processlist
    Current connection/thread
    No filtering options
    Ideal for locking


     mysql> SHOW [FULL] PROCESS...
Processlist Example
mysql [localhost] {root} ((none)) > show processlist;
+­­­­­­­+­­­­­­+­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­...
Processlist Example (2)


 | Id   | User    | Host     | db  | Command | Time | State              | Info

 | 2692 | app_u...
Engine Status
    Provides engine specific information
    Vendor implemented
    InnoDB
        Running threads
        L...
Engine Status



           mysql> SHOW ENGINE INNODB STATUS;




                    http://dev.mysql.com/doc/refman/5.1/...
Engine Status Example
  ­­­­­­­­­­­­
  TRANSACTIONS
  ­­­­­­­­­­­­
  Trx id counter 0 480206585
  Purge done for trx's n:o...
Engine Status Example (2)
------------------------
LATEST DETECTED DEADLOCK
------------------------
090624 14:32:18
*** (...
Status Variables
    Shows incrementing internal counters
    Volume of SQL
    Read/Write ratio




ODTUG – June 2010
Status Variables


      mysql> SHOW GLOBAL STATUS;

      $ mysqladmin extended­status

      mysql> SELECT * FROM GLOBAL...
Status Variables Example

         | Com_select                        | 553971     |

         | Com_insert              ...
Status Variables Example (2)
         | Bytes_received                    | 635778091  |
         | Bytes_sent            ...
Status Variables Example (3)
  ●$ mysqladmin ­u ­p ­r ­s 1 extended status | grep ­v “| 0”
  +­­­­­­­­­­­­­­­­­­­­­­­­­­­­...
MySQL Proxy
    Included histogram.lua
        Gives read/write breakdown per table
        Shows top executing queries by...
MySQL Proxy Example
    mysql> select * from histogram.tables; 
    +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­+ 
   ...
MySQL Proxy Example (2)
    mysql> select * from histogram.queries; 
    *************************** 8. row **************...
TCP/IP Capture
    Captures network packets




                         http://www.tcpdump.org/tcpdump_man.html

ODTUG – ...
TCP/IP Example
   tcpdump -i any -s 65535 -x -nn -q -tttt port 3306

  2010­05­18 19:08:09.291455 IP 999.67.245.14.3338 > ...
DTrace/SystemTap
    Observability technologies
    Enable the behavior of running systems
    and applications to be inve...
SystemTap Example
 # stap query_trace.sty /home/posulliv/repos/mysql/mysql-5.5.2-m2/5.5.2/libexec/mysqld
 select @@version...
Analyzing your SQL

ODTUG – June 2010
Bulk Analysis Tools
    Your eyes
    Maatkit
        mk-query-digest




ODTUG – June 2010
Slow Query Log
    mk-query-digest --type slowlog /path/to/slow.log


    Best practice to roll/analyze daily




        ...
Slow Query Analysis Example
  ●   $ mk­query­digest ­­type slowlog /var/log/mysql/slow.log
  ●

  # Query 1: 0.00 QPS, 2.7...
Slow Query Analysis Example

  # Tables
  #    SHOW TABLE STATUS FROM `xxxx` LIKE 'gg'G
  #    SHOW CREATE TABLE `xxxx`.`g...
General Query Log
  ●   $ mk­query­digest ­­type genlog ­­report­format profile /opt/mysql_sandboxes/latest/data/msandbox....
General Query Log Analysis
$ mk­query­digest ­­type genlog /opt/mysql_sandboxes/latest/data/msandbox.log

# Query 20: 0 QP...
Binary Log



    mysql> SHOW GLOBAL VARIABLES LIKE 'log_bin';
    mysql> SHOW MASTER STATUS;
    mysql> SHOW MASTER LOGS;...
Aggregating Binlog Output
    mysqlbinlog /path/to/mysql­bin.000999 |  
      grep ­i ­e "^update" ­e "^insert" ­e "^delet...
Maatkit
$ mysqlbinlog /opt/mysql_sandboxes/latest/data/msandbox.000001 
> | ./mk­query­digest ­­type binlog ­­report­forma...
Status variables Analysis
    Gathering global status per time interval
    Reporting options
        statpack




       ...
statpack Example
  ====================================================================================================
  ...
TCP/IP Dump Analysis
    Non-intrusive
    Close to full sample
    Near realtime


      $ tcpdump ­i eth0 port 3306 ­s 6...
TCP/IP Dump Summary
  # 4.1s user time, 85.9ms system time, 17.85M rss, 18.46M vsz
  # Overall: 1.99k total, 86 unique, 32...
TCP/IP Dump Query Review
  # Query 6: 23.02 QPS, 0.01x concurrency, ID 0x8D86ED007917BCE6 at byte ­2
  # This item is incl...
mycheckpoint
    SQL orientated monitoring utility for
    MySQL
    Measures various metrics
    Stores measurements in d...
SQL Analysis Tools
    EXPLAIN
    mk-visual-explain
    SHOW CREATE TABLE
    SHOW INDEXES
    SHOW TABLE STATUS
    I_S ...
Explain
    Returns information on query plan
    Query not actually executed (unless
    query contains sub-query)
    1 ...
Explain Examples
mysql> explain select 1;
+­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­...
Explain Examples
    mysql> explain
        ­> select 
        ­>   avg(length(w.variables)) / 1048576
        ­> from 
  ...
mk-visual-explain
    Reverse engineers explain output from
    MySQL
    Formats MySQL explain output as a left-
    deep...
mk-visual-explain Examples
echo "explain SELECT DISTINCT b.* FROM blocks b LEFT JOIN blocks_roles r ON b.module = r.module...
Show Create Table
    Shows full definition
    Includes all keys
    Shows storage engine



            SHOW CREATE TABL...
Show Create Table Example

   mysql> SHOW CREATE TABLE wp_postmetaG
   *************************** 1. row ****************...
Show Indexes
    Provides cardinality



           SHOW INDEX[ES] [FROM|IN] [schema.]tablename




                    ht...
Show Index Example


mysql> SHOW INDEXES FROM wp_postmeta;
+­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­...
Table Status



        SHOW TABLE STATUS [IN schema] LIKE 'tablename'




                http://dev.mysql.com/doc/refman...
Table Status Example
  mysql> SHOW TABLE STATUS LIKE 'wp_postmeta'G
  *************************** 1. row *****************...
Information Schema
    INFORMATION_SCHEMA
        TABLES
        COLUMNS
    Meta data
        Simple view of DBA_TABLES/C...
Information Schema Example
      SELECT   table_schema,
               table_name,
               engine,
               r...
Information Schema Example


  +­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­...
Show Session Status
    MySQL Internal counters




                    http://dev.mysql.com/doc/refman/5.1/en/show-status...
Show Session Status Example
                    SHOW SESSION STATUS LIKE 'Created%';
                    +­­­­­­­­­­­­­­­­...
Show Profile

    Provides internal message calls
    Descriptions not always accurate


                    SET PROFILING...
Show Profile Example

     mysql> SHOW PROFILESG
     *************************** 1. row ***************************
     ...
Show Profile Example (2)
  mysql> SHOW PROFILE SOURCE FOR QUERY 1;
  +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­...
Show Profile Example (3)

  mysql> SHOW PROFILES;
  +­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­...
Optimizing your
             SQL

ODTUG – June 2010
About the MySQL Optimizer
    Cost based optimizer
    No way to pin QEP
    One index per table rule (with exceptions)
  ...
SQL Optimization Goals
    Identify the best indexes
    Reduce unnecessary data access
    Minimize kernel internals




...
Explain Signs
    No index used
    Large rows
    Excessive possible indexes
    SQL for composite/covering indexes
    U...
MySQL Index Types
    Column indexes
    Composite indexes
    Covering indexes
    Partial indexes
    Spatial indexes
  ...
Column Index – Sub-optimal Design
  CREATE TABLE `content` (
    `ContentID` int(11) NOT NULL auto_increment,
  ....
    `...
Composite Indexes Example

         CREATE TABLE relationship (
           relation_id bigint NOT NULL auto_increment,
   ...
Composite Indexes

explain SELECT user_2 FROM relationship WHERE user_1 = '1580980221' AND user_2 IN (508534334,521894993)...
Covering Indexes Example
      CREATE TABLE logging (
        source_addr int(10) unsigned NOT NULL,
        source_port i...
Covering Indexes
SELECT source_port, SUM(size) FROM logging WHERE log_time BETWEEN 1 AND 2 
GROUP BY source_port ORDER BY ...
Partial Indexes
  CREATE TABLE parts(
   ...
    brand_id INT UNSIGNED NOT NULL,
    part_number VARCHAR(30) NOT NULL,
   ...
QEP – Using Filesort
EXPLAIN SELECT itemid FROM items WHERE productid=999 GROUP BY itemid;
+­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­...
QEP – Using Filesort

mysql> show profile source for query 1;
+­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­+­­­...
MySQL Index Sizes
    NULL columns
    Bigint vs int
    InnoDB secondary index PK value




ODTUG – June 2010
NULL Index Columns

CREATE TABLE child(
  child_id INT UNSIGNED NOT NULL,
  parent_id INT UNSIGNED,
PRIMARY KEY(child_id),...
NULL Index Columns (2)

  mysql> SELECT parent_id FROM CHILD PROCEDURE ANALYSE(5,2000)G
  *************************** 1. r...
NULL Index Columns (3)



    ALTER TABLE child MODIFY parent_id INT UNSIGNED NOT NULL;
    +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­...
BIGINT vs INT
explain SELECT user_2 FROM relationship 
WHERE user_1 = '1580980221' AND user_2 IN (508534334,521894993); 

...
MySQL Index Sizes
    NULL columns
    Bigint vs int
    InnoDB secondary index PK value




ODTUG – June 2010
When Indexes Don't Help
  CREATE TABLE `items` (
    `itemid` int(11) NOT NULL auto_increment,
    `productid` int(11) NOT...
When Indexes Don't Help

EXPLAIN
SELECT DISTINCT(categories.categoryid) 
FROM categories, category_product, items
WHERE ca...
When Indexes Don't Help
     Adding an index can reduce this full table scan



mysql> alter table items drop index produc...
Optimizing SQL for Locking
    DML is a blocking statement for MyISAM
    Long running cleanup scripts

DELETE FROM fts_da...
Optimizing SQL for Locking
    Rewriting statement into 2 statements
    can reduce locking time to ~10 ms
  CREATE TEMPOR...
Don't Forget Update
UPDATE T 
SET STATE = 'MISFIRED' 
WHERE (STATE = 'WAITING' OR STATE = 'WAITING') 
AND NEXT_FIRE_TIME <...
Remove Duplicate Indexes
  CREATE TABLE `accounts` (
    `accountid` int(11) NOT NULL auto_increment,
  ...
    `retail_da...
The Most Optimal
  SQL is the SQL You
    Can Eliminate
ODTUG – June 2010
Eliminating SQL – Example 1
    Following executed 6,000 times in 5 min period

 SELECT pages_id, 
        pages_livestats...
Eliminating SQL – Example 2
    Following executed on single page load
 5 Query   SELECT *  FROM `artist`
 5 Query   SELEC...
Eliminating SQL – Example 3
Row at a time (RAT) processing

SELECT * FROM activities_theme WHERE theme_parent_id=0
SELECT ...
Conclusion

ODTUG – June 2010
Capture/Analysis/Optimization
    Improvement is a reoccurring process
    Ideally you should analyze all SQL
    Changes ...
Upcoming SlideShare
Loading in …5
×

Capturing, Analyzing and Optimizing MySQL

6,052 views

Published on

Published in: Technology, Business
1 Comment
12 Likes
Statistics
Notes
No Downloads
Views
Total views
6,052
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
241
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

Capturing, Analyzing and Optimizing MySQL

  1. 1. Capturing, Analyzing & Optimizing your SQL Padraig O'Sullivan - http://posulliv.com/ Ronald Bradford - http://ronaldbradford.com/ ODTUG – June 2010
  2. 2. Capturing your SQL ODTUG – June 2010
  3. 3. MySQL Provided Options Slow Query Log General Query Log Binary Log Connectors Processlist Engine status Status variables ODTUG – June 2010
  4. 4. Other Techniques MySQL Proxy TCP/IP Capture Dtrace/SystemTap Application Management ODTUG – June 2010
  5. 5. Slow Query Log Not enabled by default log-slow-queries = /path/to/file (5.0) slow_query_log/slow_query_log_file (5.1) Granularity of seconds Microsecond patch by percona Default slow time is 10 seconds long_query_time = 2 Log output to file or table (5.1) log_output = FILE | TABLE | BOTH http://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html ODTUG – June 2010
  6. 6. Slow Query Log Example # Time: 100518 19:34:31 # User@Host: user[db] @ localhost [] # Query_time: 4  Lock_time: 0  Rows_sent: 1155  Rows_examined: 1663671 select distinct visitorid from log where  timestamp like '20100518%' and accountid!=1 and  accountid!=37 and file not like '%admin%'; # Time: 100518 19:35:37 # User@Host: user[db] @ localhost [] # Query_time: 4  Lock_time: 0  Rows_sent: 6  Rows_examined: 1662525 select distinct accountid from log  where timestamp>='20100518132033' and accountid NOT IN ('0','1','37','1635'); # Time: 100518 19:35:41 # User@Host: user[db] @ localhost [] # Query_time: 4  Lock_time: 0  Rows_sent: 2477  Rows_examined: 1664996 select timestamp, file from log  where timestamp>='20100518000000' and timestamp<='end' and accountid!=1 and accountid!=37  order by timestamp desc; # Time: 100518 19:35:44 # User@Host: user[db] @ localhost [] # Query_time: 3  Lock_time: 0  Rows_sent: 1157  Rows_examined: 1663676 select distinct visitorid from log  where timestamp like '20100518%' and accountid!=1 and accountid!=37  and file not like '%admin%'; ODTUG – June 2010
  7. 7. Slow Query Table Example mysql> select start_time, query_time, rows_sent, rows_examined, db, sql_text     ­> from mysql.slow_log; +­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | start_time          | query_time | rows_sent | rows_examined | db     | sql_text                          | +­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | 2010­06­14 13:43:37 | 00:03:45   |         1 |      31290045 | drupal | select count(*) from search_index |  +­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ 1 row in set (0.00 sec) mysql> ODTUG – June 2010
  8. 8. General Query Log Not enabled by default log = /path/to/file general_log/general_log_file (5.1) Log output to file or table (5.1) log_output = FILE | TABLE | BOTH Never use in production Ideal for development, unit testing environments http://dev.mysql.com/doc/refman/5.1/en/query-log.html ODTUG – June 2010
  9. 9. General Query Log Example Output using Wordpress 26 Query SELECT * FROM wp_users WHERE user_login = 'xxxx' 26 Query SELECT meta_key, meta_value FROM wp_usermeta WHERE user_id = 2 26 Query SELECT option_value FROM wp_options WHERE option_name = 'sidebars_widgets' LIMIT 1 26 Query SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private') ORDER BY wp_posts.post_date DESC LIMIT 0, 10 26 Query SELECT FOUND_ROWS() 26 Query SELECT t.*, tt.*, tr.object_id FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy IN ('category', 'post_tag') AND tr.object_id IN (2849, 2842, 2836, 2824, 2812, 2680, 2813, 2800, 2770, 2784) ORDER BY t.name ASC 26 Query SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (2849,2842,2836,2824,2812,2680,2813,2800,2770,2784) ODTUG – June 2010
  10. 10. General Query Log Example (2) Framework new connection     23 Connect     user@host     23 Query       SELECT 1     23 Query       SET NAMES utf8     23 Query       SET character_set_results = NULL     23 Query       SHOW VARIABLES     23 Query       SHOW COLLATION     23 Query       SET autocommit=1     23 Query       SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED     23 Query       SET autocommit=0 ODTUG – June 2010
  11. 11. General Query Log Example (3) Finding framework overhead ROLLBACK after every SELECT SELECT 1 – connection ping test     23 Query       select count(*) from collection where account_name = 'xxx'                     and user_id = '313'     23 Query       rollback     23 Query       select 1     23 Query       select count(*) from collection where account_name = 'xxx'                     and user_id = '346'     23 Query       rollback     23 Query       select 1     23 Query       select name,user_id,date_format(`register_time`,'%Y%m%d%H%i%S') as                     createTime   from account where name = 'xxx' and                     user_id = '346'  and `status` = '1'     23 Query       rollback     23 Query       select 1     23 Query       select max(mdate)  from received where account_name = 'xxx'                     And user_id = '346' order by mdate desc     23 Query       rollback ODTUG – June 2010
  12. 12. Binary Log Not enabled by default log-bin = /path/to/file expire_log_days = 5 View with mysqlbinlog http://dev.mysql.com/doc/refman/5.1/en/binary-log.html ODTUG – June 2010
  13. 13. Binary Log Example mysqlbinlog /path/to/log-bin.000000 # at 2461 #100604 10:49:43 server id 1  end_log_pos 2632 Query thread_id=6  exec_time=0 error_code=0 use blog/*!*/; SET TIMESTAMP=1275662983/*!*/; UPDATE `wp_postmeta` SET `meta_value` = '1275662983' WHERE  `meta_key` = '_edit_lock'  AND `post_id` = 2849 /*!*/; # at 21107 #100604 10:58:49 server id 1  end_log_pos 21469 Query thread_id=62 exec_time=0  error_code=0 use monitoringmysql/*!*/; SET TIMESTAMP=1275663529/*!*/; SET @@session.time_zone='SYSTEM'/*!*/; INSERT INTO wp_redirection_logs  (url,sent_to,created,agent,redirection_id,ip,referrer,module_id,group_id)  VALUES ('/mysql­monitoring/dbtuna',NULL,NOW(),'Mozilla/5.0  (compatible; Yahoo! Slurp/3.0; http ://help.yahoo.com/help/us/ysearch/slurp)',NULL, '67.195.114.227', NULL, 3, NULL) /*!*/; ODTUG – June 2010
  14. 14. Connector/J logSlowQueries slowQueryThresholdMillis SlowQueryThresholdNanos http://dev.mysql.com/doc/refman/5.1/en/connector-j-reference-configuration-properties.html ODTUG – June 2010
  15. 15. Processlist Current connection/thread No filtering options Ideal for locking mysql> SHOW [FULL] PROCESSLIST $ mysqladmin [­v] processlist mysql> SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST (5.1)) http://dev.mysql.com/doc/refman/5.1/en/show-processlist.html ODTUG – June 2010
  16. 16. Processlist Example mysql [localhost] {root} ((none)) > show processlist; +­­­­­­­+­­­­­­+­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ ­­­­­+ | Id    | User | Host      | db   | Command | Time | State             | Info                                                                                                  | +­­­­­­­+­­­­­­+­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ ­­­­­+ |     7 | root | localhost | NULL | Query   |    0 | NULL              | show processlist                                                                                      | | 20017 | root | localhost | oltp | Query   |    0 | creating table    | create table bench1 (a  int NOT NULL,i int,s char(10),primary key (a))                                | | 20020 | root | localhost | oltp | Query   |    1 | copy to tmp table | ALTER TABLE bench ADD   i919 integer,ADD  i920 integer,ADD  i921 integer,ADD  i922 integer,ADD  i923  | +­­­­­­­+­­­­­­+­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ ­­­­­+ 3 rows in set (0.00 sec) ODTUG – June 2010
  17. 17. Processlist Example (2) | Id   | User    | Host     | db  | Command | Time | State              | Info | 2692 | app_usr | db:44376 | db1 | Query   |    3 | Sending data       | SELECT ... | 2805 | app_usr | db:44703 | db1 | Query   |    0 | statistics         | SELECT ... FOR UPDATE | 2829 | app_usr | db:44812 | db1 | Query   |    0 | init               | UPDATE ... | 2845 | app_usr | db:44866 | db1 | Sleep   |    2 |                    | NULL                   | 2846 | app_usr | db:44866 | db1 | Sleep   |    2 |                    | NULL   | 2915 | app_usr | db:56751 | db1 | Query   |    0 | Updating           | UPDATE ... | 2847 | app_usr | db:44868 | db1 | Query   |    0 | query end          | SELECT * FROM ... | 2928 | app_usr | db:56792 | db1 | Query   |    0 | logging slow query | INSERT INTO ... ODTUG – June 2010
  18. 18. Engine Status Provides engine specific information Vendor implemented InnoDB Running threads Last deadlock Last foreign key error Other engine types include NDB, PBXT, TOKUDB ODTUG – June 2010
  19. 19. Engine Status mysql> SHOW ENGINE INNODB STATUS; http://dev.mysql.com/doc/refman/5.1/en/show-engine.html ODTUG – June 2010
  20. 20. Engine Status Example ­­­­­­­­­­­­ TRANSACTIONS ­­­­­­­­­­­­ Trx id counter 0 480206585 Purge done for trx's n:o < 0 480205682 undo n:o < 0 0 History list length 5 LIST OF TRANSACTIONS FOR EACH SESSION: ­­­TRANSACTION 0 0, not started, process no 27230, OS thread id 1182685536 MySQL thread id 427, query id 123563  10.8.160.241 dba SHOW ENGINE INNODB STATUS ­­­TRANSACTION 0 0, not started, process no 27230, OS thread id 1169639776 MySQL thread id 44, query id 123445  10.8.160.241 dba ­­­TRANSACTION 0 480206305, ACTIVE 67 sec, process no 27230, OS thread id 1192270176 mysql tables in use 1, locked 1 1 lock struct(s), heap size 368, 0 row lock(s), undo log entries 1 MySQL thread id 370, query id 123171  10.120.229.37 user query end INSERT INTO `table` .... ­­­TRANSACTION 0 480206290, ACTIVE 70 sec, process no 27230, OS thread id 1178159456 mysql tables in use 1, locked 1 1 lock struct(s), heap size 368, 0 row lock(s), undo log entries 1 MySQL thread id 372, query id 123150  10.120.229.37 user query end INSERT INTO `table` .... ­­­TRANSACTION 0 480206289, ACTIVE 70 sec, process no 27230, OS thread id 1178425696 mysql tables in use 1, locked 1 1 lock struct(s), heap size 368, 0 row lock(s), undo log entries 1 MySQL thread id 373, query id 123149  10.120.229.37 user query end INSERT INTO `table` .... .... ODTUG – June 2010
  21. 21. Engine Status Example (2) ------------------------ LATEST DETECTED DEADLOCK ------------------------ 090624 14:32:18 *** (1) TRANSACTION: TRANSACTION 0 39639074, ACTIVE 0 sec, process no 27807, OS thread id 1176562016 inserting mysql tables in use 1, locked 1 LOCK WAIT 7 lock struct(s), heap size 1216 MySQL thread id 1225, query id 4695283 10.120.19.204 db update INSERT INTO table VALUES(254381,638,1,57) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 6371457 n bits 248 index `PRIMARY` of table `db/table` trx id 0 39639074 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) TRANSACTION: TRANSACTION 0 39639079, ACTIVE 0 sec, process no 27807, OS thread id 1179224416 inserting, thread declared inside InnoDB 500 mysql tables in use 1, locked 1 7 lock struct(s), heap size 1216 MySQL thread id 1229, query id 4695295 prodgos23.eao.abn-iad.ea.com 10.120.19.204 madden_2010 update INSERT INTO table VALUES(254382,746,1,57) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 0 page no 6371457 n bits 248 index `PRIMARY` of table `madden_2010_franchise_xbl2/franchise_draft_board` trx id 0 39639079 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 6371457 n bits 248 index `PRIMARY` of table `db/table` trx id 0 39639079 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; *** WE ROLL BACK TRANSACTION (2) ODTUG – June 2010
  22. 22. Status Variables Shows incrementing internal counters Volume of SQL Read/Write ratio ODTUG – June 2010
  23. 23. Status Variables mysql> SHOW GLOBAL STATUS; $ mysqladmin extended­status mysql> SELECT * FROM GLOBAL_STATUS (5.1) http://dev.mysql.com/doc/refman/5.1/en/show-status.html ODTUG – June 2010
  24. 24. Status Variables Example | Com_select                        | 553971     | | Com_insert                        | 64068      | | Com_insert_select                 | 6521       | | Com_update                        | 190627     | | Com_update_multi                  | 4809       | | Com_delete                        | 95197      | | Com_delete_multi                  | 1408       | | Com_replace                       | 1549       | | Com_replace_select                | 0          | ODTUG – June 2010
  25. 25. Status Variables Example (2) | Bytes_received                    | 635778091  | | Bytes_sent                        | 4207810294 | Throughput | Bytes_received                    | 635778091  | | Bytes_sent                        | 4207810294 | Load | Threads_connected                 | 22         | | Threads_connected                 | 22         | | Threads_running                   | 3          | | Threads_running                   | 3          | Server Running | Uptime                            | 5951       | ODTUG – June 2010
  26. 26. Status Variables Example (3) ●$ mysqladmin ­u ­p ­r ­s 1 extended status | grep ­v “| 0” +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+ | Variable_name                     | Value      | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+ | Bytes_received                    | 8888       | | Bytes_sent                        | 151482     | | Com_select                        | 114        | | Com_show_status                   | 1          | | Created_tmp_tables                | 2          | | Handler_read_key                  | 194        | | Handler_read_next                 | 273        | | Handler_read_rnd                  | 165        | | Handler_read_rnd_next             | 47416      | | Handler_write                     | 289        | | Key_read_requests                 | 1555       | | Questions                         | 116        | | Select_range                      | 23         | | Select_scan                       | 3          | | Sort_rows                         | 165        | | Sort_scan                         | 3          | | Table_locks_immediate             | 115        | | Threads_connected                 | ­1         | | Threads_running                   | ­1         | | Uptime                            | 1          | | Uptime_since_flush_status         | 1          | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­+ ODTUG – June 2010
  27. 27. MySQL Proxy Included histogram.lua Gives read/write breakdown per table Shows top executing queries by frequency Shows max and avg execution time http://dev.mysql.com/doc/refman/5.1/en/mysql-proxy.html https://launchpad.net/mysql-proxy http://ronaldbradford.com/blog/sql-query-analysis-with-mysql-proxy-2009-09-02/ ODTUG – June 2010
  28. 28. MySQL Proxy Example mysql> select * from histogram.tables;  +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­+  | table                        | reads | writes |  +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­+  | schema1.dotCommLookup        |   991 |      0 |  | schema2xxxxxx.dotCommClient  |     6 |      0 |  | schema2xxxxxx.ReleaseNumbers |     3 |      6 |  | schema2xxxxxx.PAYCYCLE       |     1 |      0 |  | schema1.dotCommClient        |  1505 |      0 |  | schema1.Job                  |   131 |      0 |  | schema2xxxxxx.CondMod        |     0 |     28 |  | schema2xxxxxx.SPINFO         |     3 |      0 |  | schema1.Poll                 |  1184 |    633 |  | schema2xxxxxx.LaborSchedules |     0 |     96 |  | schema1.JobRequest           |   192 |      0 |  | schema1.PollTasks            |   454 |   1048 |  | schema1.Tasks                |  1048 |      0 |  | schema1.auth                 |   141 |     59 |  | schema2xxxxxx.dotComm        |     2 |      0 |  | schema2xxxxxx.MenuItem       |     0 |   2446 |  | schema1.Location             |    19 |    118 |  | schema2xxxxxx.CONCASH        |     2 |      0 |  | schema1.dotComm              |   156 |      0 |  | schema1.Client               |    47 |      0 |  +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­+  20 rows in set (0.00 sec) ODTUG – June 2010
  29. 29. MySQL Proxy Example (2) mysql> select * from histogram.queries;  *************************** 8. row ***************************           query: SELECT `type` , COUNT( * ) AS `count` FROM `schema1` . `table1`  `a` WHERE `clientID` = ? AND `bocDate` = ? AND `locationID` IN( ? ) GROUP BY ?  ORDER BY ?           count: 8  max_query_time: 12932  avg_query_time: 4154.125  *************************** 35. row ***************************           query: SELECT `empl_first_nam` , `empl_last_nam` , `empl_mid_initial`  FROM `schema2` . `table2` WHERE `clientID` = ? AND `empl_ssn` = ? AND  `locationID` = ? LIMIT ?           count: 84  max_query_time: 61974  avg_query_time: 2501.3095238095  *************************** 305. row ***************************           query: INSERT INTO `schema3` . `table3` SET `eventTime` = NOW( ) ,  `event` = ? , `userID` = ? , `details` = ?           count: 59  max_query_time: 433610  avg_query_time: 38760.983050847  *************************** 302. row ***************************           query: SELECT * FROM `table4` WHERE `jobID` = ? AND `taskID` = ?           count: 1056  max_query_time: 17564  avg_query_time: 672.67708333333 ODTUG – June 2010
  30. 30. TCP/IP Capture Captures network packets http://www.tcpdump.org/tcpdump_man.html ODTUG – June 2010
  31. 31. TCP/IP Example tcpdump -i any -s 65535 -x -nn -q -tttt port 3306 2010­05­18 19:08:09.291455 IP 999.67.245.14.3338 > 999.174.90.144.3306: tcp 0 0x0000:  4500 0030 fdd0 4000 7106 e466 4b43 f50e  E..0..@.q..fKC.. 0x0010:  8cae 5a90 0d0a 0cea e1cd 4d71 0000 0000  ..Z.......Mq.... 0x0020:  7002 ffff 125c 0000 0204 05b4 0101 0402  p.............. 2010­05­18 19:08:09.291519 IP 999.174.90.144.3306 > 999.67.245.14.3338: tcp 0 0x0000:  4500 0030 0000 4000 4006 1338 8cae 5a90  E..0..@.@..8..Z. 0x0010:  4b43 f50e 0cea 0d0a 69af 73bf e1cd 4d72  KC......i.s...Mr 0x0020:  7012 16d0 1e0c 0000 0204 05b4 0101 0402  p............... 2010­05­18 19:08:09.316990 IP 999.67.245.14.3338 > 999.174.90.144.3306: tcp 0 0x0000:  4500 0028 fdd1 4000 7106 e46d 4b43 f50e  E..(..@.q..mKC.. 0x0010:  8cae 5a90 0d0a 0cea e1cd 4d72 69af 73c0  ..Z.......Mri.s. 0x0020:  5010 ffff 61a0 0000                      P...a... 2010­05­18 19:08:09.322483 IP 999.174.90.144.3306 > 999.67.245.14.3338: tcp 67 0x0000:  4508 006b 7eed 4000 4006 9407 8cae 5a90  E..k~.@.@.....Z. 0x0010:  4b43 f50e 0cea 0d0a 69af 73c0 e1cd 4d72  KC......i.s...Mr 0x0020:  5018 16d0 574b 0000 3f00 0000 0a35 2e30  P...WK..?....5.0 0x0030:  2e35 3161 2d63 6f6d 6d75 6e69 7479 0036  .51a­community.6 0x0040:  9800 0057 7462 6f28 732f 2e00 2ca2 0802  ...Wtbo(s/..,... 0x0050:  0000 0000 0000 0000 0000 0000 0000 386f  ..............8o 0x0060:  4858 615f 3838 5274 7055 00              HXa_88RtpU. ODTUG – June 2010
  32. 32. DTrace/SystemTap Observability technologies Enable the behavior of running systems and applications to be investigated Zero probe effect when not enabled http://www.sun.com/bigadmin/content/dtrace/index.jsp http://sourceware.org/systemtap/ ODTUG – June 2010
  33. 33. SystemTap Example # stap query_trace.sty /home/posulliv/repos/mysql/mysql-5.5.2-m2/5.5.2/libexec/mysqld select @@version_comment limit 1 Total: 287us Locks: 0us Engine: 0us Network: 6us Filesort: 0us select USER() Total: 127us Locks: 0us Engine: 0us Network: 6us Filesort: 0us SELECT DATABASE() Total: 562us Locks: 0us Engine: 0us Network: 21us Filesort: 0us show databases Total: 642us Locks: 0us Engine: 5us Network: 6us Filesort: 0us show tables Total: 1148us Locks: 0us Engine: 6us Network: 6us Filesort: 0us show tables Total: 6201us Locks: 0us Engine: 19us Network: 23us Filesort: 0us SELECT DATABASE() Total: 564us Locks: 0us Engine: 0us Network: 21us Filesort: 0us show databases Total: 81us Locks: 0us Engine: 5us Network: 6us Filesort: 0us show tables Total: 293us Locks: 0us Engine: 5us Network: 6us Filesort: 0us select * from t1 Total: 1202us Locks: 32us Engine: 20us Network: 23us Filesort: 0us select * from t1 where a = 1 Total: 1134us Locks: 30us Engine: 56us Network: 21us Filesort: 0us select * from t1 order by a desc Total: 4727us Locks: 26us Engine: 20us Network: 22us Filesort: 3182us insert into t1 values (3, 'tomas') Total: 3406us Locks: 146us Engine: 858us Network: 30us Filesort: 0us select * from t1 order by a desc Total: 1340us Locks: 27us Engine: 20us Network: 21us Filesort: 228us ^C ODTUG – June 2010
  34. 34. Analyzing your SQL ODTUG – June 2010
  35. 35. Bulk Analysis Tools Your eyes Maatkit mk-query-digest ODTUG – June 2010
  36. 36. Slow Query Log mk-query-digest --type slowlog /path/to/slow.log Best practice to roll/analyze daily http://ronaldbradford.com/blog/the-correct-approach-to-rolling-mysql-logs-2010-02-22/ ODTUG – June 2010
  37. 37. Slow Query Analysis Example ● $ mk­query­digest ­­type slowlog /var/log/mysql/slow.log ● # Query 1: 0.00 QPS, 2.74x concurrency, ID 0x036A4959F0EBB3BD at byte ­1 _ #              pct   total     min     max     avg     95%  stddev  median # Count          1     395 # Exec time     28 478528s     11s   4733s   1211s   4168s   1431s    511s # Lock time     20  31777s       0   2769s     80s    685s    291s       0 # Rows sent      0   3.86k      10      10      10      10       0      10 # Rows exam      3 193.22M 500.73k 500.96k 500.90k 485.50k       0 485.50k # Users                  1  xxx # Hosts                  5 app5.xxxxx... (128), app1.xxxx... (85)... 3 more # Databases              1  xxxx # Time range 2009­05­18 10:23:18 to 2009­05­20 10:50:14 # bytes          0 184.67k     476     479  478.73  463.90       0  463.90 # Query_time distribution #   1us #  10us # 100us #   1ms #  10ms # 100ms #    1s #  10s+  ################################################################ ODTUG – June 2010
  38. 38. Slow Query Analysis Example # Tables #    SHOW TABLE STATUS FROM `xxxx` LIKE 'gg'G #    SHOW CREATE TABLE `xxxx`.`gg`G #    SHOW TABLE STATUS FROM `xxxx ` LIKE 'gr'G #    SHOW CREATE TABLE `xxxx `.`gr`G #    SHOW TABLE STATUS FROM `xxxx ` LIKE 'tt'G #    SHOW CREATE TABLE `xxxx `.`tt`G # EXPLAIN SELECT tt.*,         (SELECT COUNT(*) FROM gg WHERE gg.tt_user_id = tt.id) AS g_c,         (SELECT COUNT(*) FROM  WHERE gr.tt_user_id = tt.id ) AS r_c         FROM tt AS tt         WHERE 1 AND tt.user_type = 'CUSTOMER'         GROUP BY tt.id HAVING g_c > 1 OR r_c > 1         ORDER BY tt.id desc, g_c DESC , r_c DESC         LIMIT 10 OFFSET 0G ODTUG – June 2010
  39. 39. General Query Log ● $ mk­query­digest ­­type genlog ­­report­format profile /opt/mysql_sandboxes/latest/data/msandbox.log # 2.4s user time, 40ms system time, 37.40M rss, 78.70M vsz # Current date: Fri Jun 11 14:17:22 2010 # Files: /opt/mysql_sandboxes/latest/data/msandbox.log # Overall: 17.11k total, 2.64k unique, 0.71 QPS, 0x concurrency __________ #                    total     min     max     avg     95%  stddev  median # Exec time              0       0       0       0       0       0       0 # Time range        2010­06­11 07:35:44 to 2010­06­11 14:17:01 # bytes            793.75k      11  26.50k   47.52   54.21  295.84   38.53 # Profile # Rank Query ID           Response time    Calls R/Call   Item # ==== ================== ================ ===== ======== ================ #    1 0xC9EEB2D51C356A33     0.0000  0.0%  3622   0.0000 SET #    2 0x6C099B0B73EA7633     0.0000  0.0%   608   0.0000 USE #    3 0xAC5C18B304F9DE87     0.0000  0.0%   595   0.0000 SELECT #    4 0x30FE5C2032672827     0.0000  0.0%   595   0.0000 SHOW TABLE STATUS #    5 0x1D7CBA0BE5BAA758     0.0000  0.0%   595   0.0000 SHOW TRIGGERS #    6 0xAEA9A980826923AF     0.0000  0.0%   595   0.0000 SET #    7 0x10AB55EA465447D0     0.0000  0.0%   565   0.0000 ADMIN FIELD #    8 0x5D51E5F01B88B79E     0.0000  0.0%   522   0.0000 ADMIN CONNECT #    9 0xAA353644DE4C4CB4     0.0000  0.0%   438   0.0000 ADMIN QUIT #   10 0xE3A3649C5FAC418D     0.0000  0.0%   410   0.0000 SELECT #   11 0x28FC5B5D583E2DA6     0.0000  0.0%   402   0.0000 SHOW STATUS #   12 0x3AEAAD0E15D725B5     0.0000  0.0%   107   0.0000 SET #   13 0x3607184B9D9C3A96     0.0000  0.0%   107   0.0000 SELECT #   14 0x44001A2E53C1AB70     0.0000  0.0%   107   0.0000 SET #   15 0xA2750AF24EA2AEE6     0.0000  0.0%   107   0.0000 SHOW COLLATION #   16 0x38B3D80280BBFA2A     0.0000  0.0%   107   0.0000 SET #   17 0xC69B6ED2C47380A4     0.0000  0.0%   107   0.0000 SHOW VARIABLES #   18 0xE4CF7146873CCC28     0.0000  0.0%   107   0.0000 SET #   19 0x5CBA2034458B5BC9     0.0000  0.0%    95   0.0000 SHOW DATABASES #   20 0x2DE40AAFA6F4B715     0.0000  0.0%    46   0.0000 INSERT SELECT TEMP.task ODTUG – June 2010
  40. 40. General Query Log Analysis $ mk­query­digest ­­type genlog /opt/mysql_sandboxes/latest/data/msandbox.log # Query 20: 0 QPS, 0x concurrency, ID 0x2DE40AAFA6F4B715 at byte 56687 ___ # This item is included in the report because it matches ­­limit. #              pct   total     min     max     avg     95%  stddev  median # Count          0      46 # Exec time      0       0       0       0       0       0       0       0 # Databases              1 anonymous # Time range 2010­06­11 09:33:04 to 2010­06­11 09:33:04 # bytes          3  25.72k     404   1.12k  572.57  833.10  157.30  511.45 # Query_time distribution #   1us #  10us # 100us #   1ms #  10ms # 100ms #    1s #  10s+ # Tables #    SHOW TABLE STATUS FROM `TEMP` LIKE 'task'G #    SHOW CREATE TABLE `TEMP`.`task`G insert into TEMP.task(    task_type,     state,     user_table_schema,     user_table_table, user _table_depth,     artifact_schema,     artifact_table,     command )  values(    'GenerateFinalBySort',    'waiting',    'drupal',    'node_revisions',    '0',     'TEMP',    'node_revisions$final',    'create table `TEMP`.`node_revisions$final`(`nid` int,  `vid` int, `uid` int, `title` varchar(255), `body` longtext, `teaser` longtext, `log` longtext,  `timestamp` int, `format` int) select `nid`, `vid`, `uid`, `title`, `body`, `teaser`, `log`,  `timestamp`, `format` from `drupal`.`node_revisions` order by `vid`')G ODTUG – June 2010
  41. 41. Binary Log mysql> SHOW GLOBAL VARIABLES LIKE 'log_bin'; mysql> SHOW MASTER STATUS; mysql> SHOW MASTER LOGS; ODTUG – June 2010
  42. 42. Aggregating Binlog Output  mysqlbinlog /path/to/mysql­bin.000999 |      grep ­i ­e "^update" ­e "^insert" ­e "^delete" ­e "^replace" ­e "^alter"  |     cut ­c1­100 | tr '[A­Z]' '[a­z]' |      sed ­e "s/t/ /g;s/`//g;s/(.*$//;s/ set .*$//;s/ as .*$//" |     sed ­e "s/ where .*$//" |      sort | uniq ­c | sort ­nr     33389 update e_acc   17680 insert into r_b   17680 insert into e_rec   14332 insert into rcv_c   13543 update e_rec   10805 update loc    3339 insert into r_att    2781 insert into o_att ODTUG – June 2010
  43. 43. Maatkit $ mysqlbinlog /opt/mysql_sandboxes/latest/data/msandbox.000001  > | ./mk­query­digest ­­type binlog ­­report­format profile # Profile # Rank Query ID           Response time    Calls R/Call   Item # ==== ================== ================ ===== ======== ================ #    1 0xBE1EAF34D33DE48B     0.0000  0.0%     3   0.0000 CREATE TABLE coitest_?k.customers  `bulkload`.`customers$final` #    2 0x0A44B866EBFF94FC     0.0000  0.0%     3   0.0000 DROP #    3 0x7B747B0DEABCA018     0.0000  0.0%     3   0.0000 CREATE TABLE coitest_?k.orders  `bulkload`.`orders$final` #    4 0xB4E2D6D6475B00FF     0.0000  0.0%     3   0.0000 DROP #    5 0xF5DAC1762A3DE61F     0.0000  0.0%     3   0.0000 DROP #    6 0x42EF521F930E2DB7     0.0000  0.0%     3   0.0000 CREATE TABLE coitest_?k.orders  `bulkload`.`orders$parent` #    7 0xE4D87F83229BB741     0.0000  0.0%    18   0.0000 UPDATE bulkload.task #    8 0x5A7960FD3F39E5F8     0.0000  0.0%     3   0.0000 CREATE TABLE bulkload.progress #    9 0x813031B8BBC3B329     0.0000  0.0% 60141   0.0000 COMMIT #   10 0x854DE5DBCC259DFB     0.0000  0.0% 30000   0.0000 INSERT bulkload.items$parent #   11 0x3FBD919BC7E69704     0.0000  0.0%    87   0.0000 INSERT bulkload.progress #   12 0xE50052BA71B4D17C     0.0000  0.0%     3   0.0000 DROP TABLE if #   13 0xD645DADD65C8DD01     0.0000  0.0%    18   0.0000 INSERT SELECT bulkload.task #   14 0x5B6EB50967F9BAB8     0.0000  0.0%    18   0.0000 UPDATE bulkload.task #   15 0x277902B2380B46B4     0.0000  0.0%     1   0.0000 DROP #   16 0x15174D089E600489     0.0000  0.0%     3   0.0000 CREATE TABLE bulkload.task #   17 0x6F1EE7DAFC74D9AF     0.0000  0.0%     3   0.0000 CREATE TABLE coitest_?k.items  `bulkload`.`items$child` #   18 0x85FFF5AA78E5FF6A     0.0000  0.0% 60141   0.0000 BEGIN #   19 0x9EE6FD43C5546BA1     0.0000  0.0%     3   0.0000 CREATE TABLE bulkload.items$parent  `bulkload`.`items$parent` #   20 0x51DCCB95FA917A6F     0.0000  0.0%     3   0.0000 DROP TABLE if ODTUG – June 2010
  44. 44. Status variables Analysis Gathering global status per time interval Reporting options statpack http://ronaldbradford.com/blog/using-statpack-with-show-status-2009-06-18/ ODTUG – June 2010
  45. 45. statpack Example ====================================================================================================               Uptime: 17 days 17 hours 22 mins Snapshot Period 1: 59 minute interval ====================================================================================================                     Variable Delta/Percentage      Per Second   Total =================================================================================... ====================================================================================================                                          Statement Activity ====================================================================================================                      SELECT:       13,503,876                 3,798.56            4,298,170,239 (94.25%)                      INSERT:           91,101                    25.63               25,327,062 (0.56%)                      UPDATE:          782,004                   219.97              220,640,296 (4.84%)                      DELETE:            9,674                     2.72                2,485,643 (0.05%)                      COMMIT:           46,422                    13.06               13,700,478 (0.30%)                    ROLLBACK:                0                     0.00                       13 (0.00%) ... ====================================================================================================                                           Temporary Space ====================================================================================================   tmp_table_size Efficiency:           2.10%          Memory Temp Tables:           26,467                     7.45                9,335,030            Disk Temp Tables:           25,951                     7.30                9,138,705                  Temp Files:                2                     0.00                      601 ODTUG – June 2010
  46. 46. TCP/IP Dump Analysis Non-intrusive Close to full sample Near realtime $ tcpdump ­i eth0 port 3306 ­s 65535 ­x  > ­n ­q ­tttt ­c 1000 > tcpdump.out $ cat tcpdump.out | mk­query­digest ­­type tcpdump ODTUG – June 2010
  47. 47. TCP/IP Dump Summary # 4.1s user time, 85.9ms system time, 17.85M rss, 18.46M vsz # Overall: 1.99k total, 86 unique, 321.13 QPS, 0.21x concurrency _________ #                    total     min     max     avg     95%  stddev  median # Exec time             1s    39us    70ms   649us   839us     4ms   152us # Time range        2010­03­30 17:52:42.569092 to 2010­03­30 17:52:48.772254 # bytes            186.80k      13   1.61k   96.03  166.51  196.84   40.45 # Rows affe             59       0       2    0.03       0    0.18       0 # Warning c              5       0       2    0.00       0    0.07       0 #   4% (98)   No_index_used # Profile # Rank Query ID           Response time    Calls R/Call   Item # ==== ================== ================ ===== ======== ================ #    3 0xC9ECBBF2C88C2336     0.0875  7.5%   568   0.0002 SELECT r_c #    4 0x448E4AEB7E02AF72     0.0785  6.7%   566   0.0001 SELECT r_t #    5 0xCF3A86CFCC509800     0.0596  5.1%     5   0.0119 SELECT h_c_i #    6 0x8D86ED007917BCE6     0.0480  4.1%   131   0.0004 SELECT loc #    7 0xDB8614598CD1ED36     0.0416  3.6%     1   0.0416 SELECT lml r #    8 0x9BA140CF9CE950C8     0.0316  2.7%     1   0.0316 SELECT bsc #    9 0x633F85619F198C33     0.0219  1.9%    23   0.0010 SELECT hlm mt  #   10 0xADCCD37E265C8300     0.0197  1.7%    79   0.0002 SELECT h_l_l ODTUG – June 2010
  48. 48. TCP/IP Dump Query Review # Query 6: 23.02 QPS, 0.01x concurrency, ID 0x8D86ED007917BCE6 at byte ­2 # This item is included in the report because it matches ­­limit. #              pct   total     min     max     avg     95%  stddev  median # Count          6     131 # Exec time      3    48ms   160us     5ms   366us   690us   477us   247us # Hosts                  5 10.0.0.77 (86), 10.0.0.36 (21), 10.0.0.79 (20)... 2 more # Time range 2010­03­30 17:52:43.034021 to 2010­03­30 17:52:48.724815 # bytes          8  15.93k     123     125  124.54  124.25       1  124.25 # Errors                 1    none # Rows affe      0       0       0       0       0       0       0       0 # Warning c      0       0       0       0       0       0       0       0 # Query_time distribution #   1us #  10us # 100us  ################################################################ #   1ms  ## #  10ms # 100ms #    1s #  10s+ # Tables #    SHOW TABLE STATUS LIKE 'loc'G #    SHOW CREATE TABLE `loc`G # EXPLAIN SELECT l.* FROM loc p, loc l WHERE p.id=2090 AND l.lft BETWEEN p.lft AND p.rgt AND  l.depth=p.depth+1 ORDER BY l.lftG ODTUG – June 2010
  49. 49. mycheckpoint SQL orientated monitoring utility for MySQL Measures various metrics Stores measurements in database Metrics can be accessed through SQL Can generate HTML reports and Google charts easily http://code.openark.org/forge/mycheckpoint/documentation ODTUG – June 2010
  50. 50. SQL Analysis Tools EXPLAIN mk-visual-explain SHOW CREATE TABLE SHOW INDEXES SHOW TABLE STATUS I_S Table/Index size SHOW SESSION STATUS SHOW PROFILE ODTUG – June 2010
  51. 51. Explain Returns information on query plan Query not actually executed (unless query contains sub-query) 1 row in output per table EXPLAIN EXTENDED shows query actually executed http://dev.mysql.com/doc/refman/5.1/en/explain.html ODTUG – June 2010
  52. 52. Explain Examples mysql> explain select 1; +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­+ | id | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra          | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | No tables used |  +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­+ 1 row in set (0.00 sec) mysql> explain select 1G *************************** 1. row ***************************            id: 1   select_type: SIMPLE         table: NULL          type: NULL possible_keys: NULL           key: NULL       key_len: NULL           ref: NULL          rows: NULL         Extra: No tables used 1 row in set (0.00 sec) mysql> ODTUG – June 2010
  53. 53. Explain Examples mysql> explain     ­> select      ­>   avg(length(w.variables)) / 1048576     ­> from      ­>   watchdog w     ­> G *************************** 1. row ***************************            id: 1   select_type: SIMPLE         table: w          type: ALL possible_keys: NULL           key: NULL       key_len: NULL           ref: NULL          rows: 69248         Extra:  1 row in set (0.00 sec) mysql> ODTUG – June 2010
  54. 54. mk-visual-explain Reverse engineers explain output from MySQL Formats MySQL explain output as a left- deep tree Can take piped input or can process explain plans from input file http://www.maatkit.org/doc/mk-visual-explain.html ODTUG – June 2010
  55. 55. mk-visual-explain Examples echo "explain SELECT DISTINCT b.* FROM blocks b LEFT JOIN blocks_roles r ON b.module = r.module AND  b.delta = r.delta WHERE b.theme = 'garland' AND b.status = 1 AND (r.rid IN (2) OR r.rid IS NULL)  ORDER BY b.region, b.weight, b.module" |  mysql ­u root drupal |  mk­visual­explain Filesort +­ TEMPORARY    table          temporary(b,r)    +­ JOIN       +­ Distinct/Not­Exists       |  +­ Filter with WHERE       |     +­ Index lookup       |        key            r­>PRIMARY       |        possible_keys  PRIMARY       |        key_len        292       |        ref            drupal.b.module,drupal.b.delta       |        rows           3       +­ Filter with WHERE          +­ Bookmark lookup             +­ Table             |  table          b             |  possible_keys  tmd,list             +­ Index lookup                key            b­>tmd                possible_keys  tmd,list                key_len        194                ref            con ODTUG – June 2010
  56. 56. Show Create Table Shows full definition Includes all keys Shows storage engine SHOW CREATE TABLE [schema.]tablename http://dev.mysql.com/doc/refman/5.1/en/show-create-table.html ODTUG – June 2010
  57. 57. Show Create Table Example mysql> SHOW CREATE TABLE wp_postmetaG *************************** 1. row ***************************        Table: wp_postmeta Create Table: CREATE TABLE `wp_postmeta` (   `meta_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,   `post_id` bigint(20) unsigned NOT NULL DEFAULT '0',   `meta_key` varchar(255) DEFAULT NULL,   `meta_value` longtext,   PRIMARY KEY (`meta_id`),   KEY `post_id` (`post_id`),   KEY `meta_key` (`meta_key`) ) ENGINE=MyISAM AUTO_INCREMENT=1745 DEFAULT CHARSET=utf8 1 row in set (0.00 sec) ODTUG – June 2010
  58. 58. Show Indexes Provides cardinality SHOW INDEX[ES] [FROM|IN] [schema.]tablename http://dev.mysql.com/doc/refman/5.1/en/show-index.html ODTUG – June 2010
  59. 59. Show Index Example mysql> SHOW INDEXES FROM wp_postmeta; +­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­+­­­­­­+­­­­­­­­­­­­+ | Table       | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type |  +­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­+­­­­­­+­­­­­­­­­­­­+ | wp_postmeta |          0 | PRIMARY  |            1 | meta_id     | A         |        1351 |     NULL | NULL   |      | BTREE      |     | wp_postmeta |          1 | post_id  |            1 | post_id     | A         |         450 |     NULL | NULL   |      | BTREE      |     | wp_postmeta |          1 | meta_key |            1 | meta_key    | A         |           7 |     NULL | NULL   | YES  | BTREE      |     +­­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­+­­­­­­+­­­­­­­­­­­­+ ODTUG – June 2010
  60. 60. Table Status SHOW TABLE STATUS [IN schema] LIKE 'tablename' http://dev.mysql.com/doc/refman/5.1/en/show-table-status.html ODTUG – June 2010
  61. 61. Table Status Example mysql> SHOW TABLE STATUS LIKE 'wp_postmeta'G *************************** 1. row ***************************            Name: wp_postmeta          Engine: MyISAM         Version: 10      Row_format: Dynamic            Rows: 1351  Avg_row_length: 46     Data_length: 62960 Max_data_length: 281474976710655    Index_length: 58368       Data_free: 76  Auto_increment: 1745     Create_time: 2009­09­08 09:57:51     Update_time: 2010­06­04 15:06:47      Check_time: 2010­06­01 11:12:54       Collation: utf8_general_ci        Checksum: NULL  Create_options:          Comment:  1 row in set (0.00 sec) ODTUG – June 2010
  62. 62. Information Schema INFORMATION_SCHEMA TABLES COLUMNS Meta data Simple view of DBA_TABLES/COLUMNS http://dev.mysql.com/doc/refman/5.1/en/information-schema.html ODTUG – June 2010
  63. 63. Information Schema Example SELECT   table_schema,          table_name,          engine,          row_format,           table_rows,           avg_row_length,          (data_length+index_length)/1024/1024 as total_mb,           (data_length)/1024/1024 as data_mb,           (index_length)/1024/1024 as index_mb,          CURDATE() AS today FROM     information_schema.tables  WHERE    table_schema=DATABASE() ORDER BY 7 DESC; http://ronaldbradford.com/mysql-dba/#perschema ODTUG – June 2010
  64. 64. Information Schema Example +­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+ | table_name       | engine | row_format | table_rows | avg_row_length | total_mb     | data_mb      | index_mb     | +­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+ | xxxxxxx          | InnoDB | Compact    |    1778523 |            314 | 658.39062500 | 533.84375000 | 124.54687500 | | xxxxxxxxx        | InnoDB | Compact    |     553266 |            846 | 472.25000000 | 446.75000000 |  25.50000000 | | xxxxxxx          | InnoDB | Compact    |     435892 |            884 | 392.25000000 | 367.81250000 |  24.43750000 | | xxxxxxxxxxxxxx   | InnoDB | Compact    |    1106547 |             65 | 133.26562500 |  68.59375000 |  64.67187500 | | xxxxxxxxxxxxxxxx | InnoDB | Compact    |      58281 |            531 |  30.34375000 |  29.51562500 |   0.82812500 | | xxxxxxxxxx       | InnoDB | Compact    |      68721 |            298 |  28.12500000 |  19.54687500 |   8.57812500 | ... ODTUG – June 2010
  65. 65. Show Session Status MySQL Internal counters http://dev.mysql.com/doc/refman/5.1/en/show-status.html ODTUG – June 2010
  66. 66. Show Session Status Example SHOW SESSION STATUS LIKE 'Created%'; +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ | Variable_name           | Value | +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ | Created_tmp_disk_tables | 1     |  | Created_tmp_files       | 5     |  | Created_tmp_tables      | 3     |  +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ SELECT ...; SHOW SESSION STATUS LIKE 'Created%'; +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ | Variable_name           | Value | +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ | Created_tmp_disk_tables | 2     |  | Created_tmp_files       | 5     |  | Created_tmp_tables      | 3     |  +­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­+ ODTUG – June 2010
  67. 67. Show Profile Provides internal message calls Descriptions not always accurate SET PROFILING = 1; SHOW PROFILES; SHOW PROFILE [FOR QUERY n]; http://dev.mysql.com/doc/refman/5.1/en/show-profiles.html ODTUG – June 2010
  68. 68. Show Profile Example mysql> SHOW PROFILESG *************************** 1. row *************************** Query_ID: 1 Duration: 0.00081600    Query: SELECT t.*, tt.*, tr.object_id FROM wp_terms                AS t INNER JOIN wp_term_taxonomy AS tt ON                tt.term_id = t.term_id INNER JOIN                wp_term_relationships AS tr ON                 tr.term_taxonomy_id = tt.term_taxonomy_id                WHERE tt.taxonomy IN               ('category', 'post_tag') AND                tr.object_id IN (2487)                ORDER BY t.name ASC ODTUG – June 2010
  69. 69. Show Profile Example (2) mysql> SHOW PROFILE SOURCE FOR QUERY 1; +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ | Status                         | Duration | Source_function  | Source_file   | Source_line | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ | starting                       | 0.000049 | NULL             | NULL          |        NULL | | checking query cache for query | 0.000074 | unknown function | sql_cache.cc  |        1446 | | Opening tables                 | 0.000013 | unknown function | sql_base.cc   |        4495 | | System lock                    | 0.000005 | unknown function | lock.cc       |         258 | | Table lock                     | 0.000032 | unknown function | lock.cc       |         269 | | init                           | 0.000038 | unknown function | sql_select.cc |        2484 | | optimizing                     | 0.000014 | unknown function | sql_select.cc |         820 | | statistics                     | 0.000135 | unknown function | sql_select.cc |        1011 | | preparing                      | 0.000017 | unknown function | sql_select.cc |        1033 | | Creating tmp table             | 0.000155 | unknown function | sql_select.cc |        1524 | | executing                      | 0.000002 | unknown function | sql_select.cc |        1755 | | Copying to tmp table           | 0.000142 | unknown function | sql_select.cc |        1902 | | Sorting result                 | 0.000026 | unknown function | sql_select.cc |        2176 | | Sending data                   | 0.000051 | unknown function | sql_select.cc |        2309 | | end                            | 0.000002 | unknown function | sql_select.cc |        2530 | | removing tmp table             | 0.000063 | unknown function | sql_select.cc |       10776 | | end                            | 0.000003 | unknown function | sql_select.cc |       10801 | | query end                      | 0.000002 | unknown function | sql_parse.cc  |        4978 | | freeing items                  | 0.000017 | unknown function | sql_parse.cc  |        6004 | | storing result in query cache  | 0.000005 | unknown function | sql_cache.cc  |         983 | | logging slow query             | 0.000001 | unknown function | sql_parse.cc  |        1679 | | cleaning up                    | 0.000002 | unknown function | sql_parse.cc  |        1647 | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ 22 rows in set (0.00 sec) ODTUG – June 2010
  70. 70. Show Profile Example (3) mysql> SHOW PROFILES; +­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ |        1 | 0.00084800 | SELECT t.*, tt.*, tr.object_id FROM wp_terms AS t ... |        2 | 0.00006000 | SELECT t.*, tt.*, tr.object_id FROM wp_terms AS t .... +­­­­­­­­­­+­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ 2 rows in set (0.00 sec) mysql> SHOW PROFILE SOURCE FOR QUERY 2; +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ | starting                       | 0.000034 | NULL             | NULL         |        NULL | | checking query cache for query | 0.000008 | unknown function | sql_cache.cc |        1446 | | checking privileges on cached  | 0.000004 | unknown function | sql_cache.cc |        1534 | | sending cached result to clien | 0.000010 | unknown function | sql_cache.cc |        1629 | | logging slow query             | 0.000002 | unknown function | sql_parse.cc |        1679 | | cleaning up                    | 0.000002 | unknown function | sql_parse.cc |        1647 | +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ ODTUG – June 2010
  71. 71. Optimizing your SQL ODTUG – June 2010
  72. 72. About the MySQL Optimizer Cost based optimizer No way to pin QEP One index per table rule (with exceptions) The lack of good hints No statistics granularity No parallelism http://forge.mysql.com/wiki/How_does_the_MySQL_Optimizer_work ODTUG – June 2010
  73. 73. SQL Optimization Goals Identify the best indexes Reduce unnecessary data access Minimize kernel internals ODTUG – June 2010
  74. 74. Explain Signs No index used Large rows Excessive possible indexes SQL for composite/covering indexes Using Temporary Using filesort Derived tables ODTUG – June 2010
  75. 75. MySQL Index Types Column indexes Composite indexes Covering indexes Partial indexes Spatial indexes Fulltext indexes Clustered indexes Hash indexes http://ronaldbradford.com/blog/understanding-different-mysql-index-implementations-2009-07-22/ ODTUG – June 2010
  76. 76. Column Index – Sub-optimal Design CREATE TABLE `content` (   `ContentID` int(11) NOT NULL auto_increment, ....   `Promo` tinyint(1) NOT NULL,   `Theme` float NOT NULL,   `ForDelete` tinyint(1) NOT NULL,   PRIMARY KEY  (`ContentID`),   KEY `CategoryID` (`CategoryID`),   KEY `Verified` (`Verified`),   KEY `FileName` (`FileName`(10)),   KEY `FileDuration` (`FileDuration`),   KEY `Votes` (`Votes`),   KEY `FileDir` (`FileDir`),   KEY `ForDelete` (`ForDelete`),   KEY `UserID` (`UserID`),   KEY `Type` (`Type`),   KEY `CategoryType` (`CategoryType`),   KEY `DateAdded` (`DateAdded`) )  ODTUG – June 2010
  77. 77. Composite Indexes Example CREATE TABLE relationship (   relation_id bigint NOT NULL auto_increment,   type_id bigint default NULL,   user_1 bigint NOT NULL,   user_2 bigint NOT NULL, ...   PRIMARY KEY  (relation_id),   KEY user_1 (user_1),   KEY user_2 (user_2), ); ODTUG – June 2010
  78. 78. Composite Indexes explain SELECT user_2 FROM relationship WHERE user_1 = '1580980221' AND user_2 IN (508534334,521894993); +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ | id | select_type | table         | type  | possible_keys | key    | key_len | ref  | rows | Extra       | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ |  1 | SIMPLE      | relationship  | range | user_1,user_2 | user_2 | 8       | NULL |    2 | Using where |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ ALTER TABLE relationship DROP INDEX user_2, ADD INDEX user_2 (user_2, user_1); +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table         | type  | possible_keys | key    | key_len | ref  | rows | Extra                   | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | relationship  | range | user_1,user_2 | user_2 | 16      | NULL |    2 | Using where;Using Index |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ ODTUG – June 2010
  79. 79. Covering Indexes Example CREATE TABLE logging (   source_addr int(10) unsigned NOT NULL,   source_port int(9) unsigned NOT NULL,   destn_addr int(10) unsigned NOT NULL,   destn_port int(9) unsigned NOT NULL,   protocol int(9) unsigned NOT NULL,   log_time int(11) unsigned NOT NULL,   size int(11) unsigned NOT NULL,   KEY log_time (log_time),   KEY size (size),   KEY source_port (src_port),   KEY destn_port (destn_port),   KEY source_addr (source_addr),   KEY destn_addr (dstn_addr) ) ENGINE=MyISAM; ODTUG – June 2010
  80. 80. Covering Indexes SELECT source_port, SUM(size) FROM logging WHERE log_time BETWEEN 1 AND 2  GROUP BY source_port ORDER BY SUM(size) DESC LIMIT 0,20 +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table   | type  | possible_keys | key  | key_len | ref  | rows  | Extra                                        | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | logging | range | log_time      | log_t| 4       | NULL | 69997 | Using where; Using temporary; Using filesort |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ By changing the index we achieve an index only scan. ALTER TABLE logging DROP INDEX log_time, DROP INDEX source_port, ADD INDEX (log_time, source_port, size ); +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table   | type  | possible_keys | key  | key_len | ref  | rows  | Extra                                                     | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | logging | range | log_time      | log_t| 4       | NULL | 69977 | Using where; Using index; Using temporary; Using filesort |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ ODTUG – June 2010
  81. 81. Partial Indexes CREATE TABLE parts(  ...   brand_id INT UNSIGNED NOT NULL,   part_number VARCHAR(30) NOT NULL,   manufacturer_name VARCHAR(100) NOT NULL, ...   INDEX (brand_id, part_number,manufacturer_name) ) ALTER TABLE parts DROP INDEX brand_id, ADD KEY (brand_id,          interchange_part_number,          manufacturer_name(8)); Reduced Index size by 30%. ODTUG – June 2010
  82. 82. QEP – Using Filesort EXPLAIN SELECT itemid FROM items WHERE productid=999 GROUP BY itemid; +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table | type | possible_keys | key       | key_len | ref   | rows | Extra                        | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | items | ref  | productid     | productid | 4       | const |    1 | Using where; Using filesort Using filesort  |  +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ 1 row in set (0.00 sec) EXPLAIN SELECT DISTINCT DISTINCT itemid FROM items WHERE productid=999 +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ | id | select_type | table | type | possible_keys | key       | key_len | ref   | rows | Extra | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ |  1 | SIMPLE      | items | ref  | productid     | productid | 4       | const |    1 |       |  +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ ODTUG – June 2010
  83. 83. QEP – Using Filesort mysql> show profile source for query 1; +­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ | Status             | Duration | Source_function       | Source_file   | Source_line | +­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­­­+ | (initialization)   | 0.000003 | open_tables           | sql_base.cc   |        ... | preparing          | 0.000021 | exec                  | sql_select.cc |        1594 |  | executing          | 0.000005 | exec                  | sql_select.cc |        1990 |  | Sorting result     | 0.000121 | exec                  | sql_select.cc |        2114 |  | Sending data       | 0.000033 | mysql_select          | sql_select.cc |        2318 |  | end                | 0.000007 | mysql_execute_command | sql_parse.cc  |        5141 |  | query end          | 0.000005 | mysql_parse           | sql_parse.cc  |        6142 |  ... ODTUG – June 2010
  84. 84. MySQL Index Sizes NULL columns Bigint vs int InnoDB secondary index PK value ODTUG – June 2010
  85. 85. NULL Index Columns CREATE TABLE child(   child_id INT UNSIGNED NOT NULL,   parent_id INT UNSIGNED, PRIMARY KEY(child_id), INDEX(parent_id)); mysql> explain select * from child where parent_id=42; +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­+ | id | select_type | table | type | possible_keys | key       | key_len | ref   | rows | Extra       | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­+ |  1 | SIMPLE      | child | ref  | parent_id     | parent_id | 5       | const |    1 | Using where | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­+ ODTUG – June 2010
  86. 86. NULL Index Columns (2) mysql> SELECT parent_id FROM CHILD PROCEDURE ANALYSE(5,2000)G *************************** 1. row ***************************              Field_name: odtug.CHILD.parent_id               Min_value: 1               Max_value: 13              Min_length: 1              Max_length: 2        Empties_or_zeros: 0                   Nulls: 0 Avg_value_or_avg_length: 7.0000                     Std: 3.7417       Optimal_fieldtype: TINYINT(2) UNSIGNED NOT NULL 1 row in set (0.00 sec) ODTUG – June 2010
  87. 87. NULL Index Columns (3) ALTER TABLE child MODIFY parent_id INT UNSIGNED NOT NULL; +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ | id | select_type | table | type | possible_keys | key       | key_len | ref   | rows | Extra | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ |  1 | SIMPLE      | child | ref  | parent_id     | parent_id | 4       | const |    1 |       | +­­­­+­­­­­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­+­­­­­­+­­­­­­­+ ODTUG – June 2010
  88. 88. BIGINT vs INT explain SELECT user_2 FROM relationship  WHERE user_1 = '1580980221' AND user_2 IN (508534334,521894993);  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table         | type  | possible_keys | key    | key_len | ref  | rows | Extra                   | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | relationship  | range | user_1,user_2 | user_2 | 16      | NULL |    2 | Using where;Using Index |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ ALTER TABLE relationship MODIFY user_1 INT NOT NULL, MODIFY user_2 INT NOT NULL); explain SELECT user_2 FROM relationship  WHERE user_1 = '1580980221' AND user_2 IN (508534334,521894993);  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table         | type  | possible_keys | key    | key_len | ref  | rows | Extra                   | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | relationship  | range | user_1,user_2 | user_2 | 8       | NULL |    2 | Using where;Using Index |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­+ ODTUG – June 2010
  89. 89. MySQL Index Sizes NULL columns Bigint vs int InnoDB secondary index PK value ODTUG – June 2010
  90. 90. When Indexes Don't Help CREATE TABLE `items` (   `itemid` int(11) NOT NULL auto_increment,   `productid` int(11) NOT NULL default '0', ...   `itemlive` enum('true','false') NOT NULL  default 'true', ...   PRIMARY KEY  (`itemid`),   KEY `productid` (`productid`), KEY `productid` (`productid`)   KEY `packaging_cost` (`packaging_cost`) ) ODTUG – June 2010
  91. 91. When Indexes Don't Help EXPLAIN SELECT DISTINCT(categories.categoryid)  FROM categories, category_product, items WHERE categories.categoryname!=''  AND categories.categoryid=category_product.categoryid AND category_product.productid=items.productid  AND category_product.productid=items.productid AND items.itemlive='true';  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table            | type   | possible_keys        | key       | key_len | ref                                | rows | Extra                        | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ NULL      | NULL    | NULL                               | 2906 | Using where; Using temporary |  |  1 | SIMPLE      | items            | ALL    | productid            | NULL |  1 | SIMPLE      | category_product | ref    | categoryid,productid | productid | 4       | baris5.items.productid             |    1 |                              |  |  1 | SIMPLE      | categories       | eq_ref | PRIMARY              | PRIMARY   | 4       | baris5.category_product.categoryid |    1 | Using where                  |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ 3 rows in set (0.00 sec) ODTUG – June 2010
  92. 92. When Indexes Don't Help Adding an index can reduce this full table scan mysql> alter table items drop index productid, add index (productid,itemlive); +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ | id | select_type | table            | type   | possible_keys        | key       | key_len | ref                                | rows | Extra                                      | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ |  1 | SIMPLE      | items            | index  | productid            | productid | 5       | NULL                               | 2180 | Using  where; Using index; Using temporary |  |  1 | SIMPLE      | category_product | ref    | categoryid,productid | productid | 4       | ronald.items.productid             |    1 |                                            |  |  1 | SIMPLE      | categories       | eq_ref | PRIMARY              | PRIMARY   | 4       | baris5.category_product.categoryid |    1 | Using  where                               |  +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­ +­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+ ODTUG – June 2010
  93. 93. Optimizing SQL for Locking DML is a blocking statement for MyISAM Long running cleanup scripts DELETE FROM fts_datacache WHERE productid NOT IN ( SELECT productid FROM xcart_products_categories WHERE categoryid IN (297,1356,1366,1361,1202,...) GROUP BY productid 3+ minutes to run SELECT ODTUG – June 2010
  94. 94. Optimizing SQL for Locking Rewriting statement into 2 statements can reduce locking time to ~10 ms CREATE TEMPORARY TABLE delete_cache ( productid INT NOT NULL,  primary key (productid)) ENGINE=MEMORY SELECT DISTINCT productid FROM retailbl_xcart.xcart_products_categories WHERE categoryid IN (297,1356,1366,1361,1202,...); DELETE FROM fts_datacache WHERE productid NOT IN  (SELECT productid FROM delete_cache); ODTUG – June 2010
  95. 95. Don't Forget Update UPDATE T  SET STATE = 'MISFIRED'  WHERE (STATE = 'WAITING' OR STATE = 'WAITING')  AND NEXT_FIRE_TIME < 263352299544 EXPLAIN SELECT NAME,STATE FROM T WHERE (STATE = 'WAITING' OR STATE = 'WAITING')  AND NEXT_FIRE_TIME < 263350625813; +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ | id | select_type | table         | type | possible_keys | key  | key_len | ref  | rows | Extra       | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ |  1 | SIMPLE      | QRTZ_TRIGGERS | ALL  | NULL          | NULL | NULL    | NULL | 1185 | Using where | +­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­­­­­­­+­­­­­­+­­­­­­­­­+­­­­­­+­­­­­­+­­­­­­­­­­­­­+ ODTUG – June 2010
  96. 96. Remove Duplicate Indexes CREATE TABLE `accounts` (   `accountid` int(11) NOT NULL auto_increment, ...   `retail_date` date NOT NULL,   `customer_survey` tinyint(1) NOT NULL default '1',   PRIMARY KEY  (`accountid`),   KEY `accountid` (`accountid`) ); CREATE TABLE `vehicle_applications` (  PRIMARY KEY  (`id`),   KEY `mcp_index` (`product_id`,`make`,`model`,`year`),   KEY `status_idx` (`status`),   KEY `sid_idx` (`status_identifier`),   KEY `product_id_idx` (`product_id`), ) ODTUG – June 2010
  97. 97. The Most Optimal SQL is the SQL You Can Eliminate ODTUG – June 2010
  98. 98. Eliminating SQL – Example 1 Following executed 6,000 times in 5 min period SELECT pages_id,         pages_livestats_code,         pages_title,         pages_parent,         pages_exhibid,         pages_theme,         pages_accession_num  FROM pages  WHERE pages_id = 0 0 is an invalid pages_id ODTUG – June 2010
  99. 99. Eliminating SQL – Example 2 Following executed on single page load 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` WHERE (ArtistID = 196 ) 5 Query   SELECT *  FROM `artist` WHERE (ArtistID = 2188) 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` 5 Query   SELECT *  FROM `artist` ODTUG – June 2010
  100. 100. Eliminating SQL – Example 3 Row at a time (RAT) processing SELECT * FROM activities_theme WHERE theme_parent_id=0 SELECT * FROM activities_theme WHERE theme_parent_id=1 SELECT * FROM activities_theme WHERE theme_parent_id=2 SELECT * FROM activities_theme WHERE theme_parent_id=11 SELECT * FROM activities_theme WHERE theme_parent_id=16 Chunk at a time (CAT) processing SELECT *  FROM   activities_theme  WHERE  theme_parent_id in  (0,1,2,11,16)  ODTUG – June 2010
  101. 101. Conclusion ODTUG – June 2010
  102. 102. Capture/Analysis/Optimization Improvement is a reoccurring process Ideally you should analyze all SQL Changes in data size and result set size over time can affect SQL performance ODTUG – June 2010

×