Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Title                              EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
Title                              Mastering MySQL                              Performance Tuning                        ...
OBJECTIVE                       Identify key MySQL instrumentation                       Detail the MySQL terms to know an...
ABOUT AUTHOR                                                             Ronald BRADFORD                       12 years wi...
MySQL                              Instrumentation                              EffectiveMySQL.com - Its all about Perform...
Key sources                       Configuration                              my.cnf      (e.g. /etc/my.cnf, /etc/mysql/my.c...
STATUS                SHOW STATUS EXAMPLE                INFORMATION_SCHEMA.STATUS Example                              Ef...
Key Sources                       Running Threads                              SHOW FULL PROCESSLIST                      ...
PROCESSLIST                PROCESSLIST EXAMPLE                INFORMATION_SCHEMA.PROCESSLIST Example                      ...
KEY areas                       System performance                       SQL performance                              Effe...
Know your system                                 bottleneck                               EffectiveMySQL.com - Its all abo...
BOTTLENECK                       Memory                       Disk                       CPU                       Network...
MEMORY                       Basic configuration (Global use)                       Basic configuration (Thread use)        ...
Memory                       Global Buffer configuration                              innodb_buffer_pool_size              ...
InnoDB Size                                                                                            EXAMPLE            ...
InnoDB Size                                                                                            EXAMPLE            ...
MyISAM Size                                                                                            EXAMPLE            ...
MyISAM Size                                                                                            EXAMPLE            ...
memory                       Thread buffer configuration                              read_buffer_size                     ...
memory                       In Memory Tables                              max_heap_table_size                       Inter...
Memory                       mysqld memory                              Unbounded                              Linux proce...
MEMORY                       PROCESSLIST                              Copying to tmp table                               E...
Disk                       fsync configuration                              flush_log_at_trx_commit                        ...
MySQL DISK                       Status Variables                  mysql> show global status like innodb%write%;          ...
CPU                       Query sorting (ORDER BY/GROUP BY)                              PROCESSLIST                      ...
NETWORK                       STATUS                              Bytes_sent                              Bytes_received  ...
SQL                              EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
SQL performance                       Throughput                              Scalability is about increasing             ...
THROUgHPUT                       Status                       Performance Schema                       TCP/IP             ...
MySQL STATUS                                                                                          EXAMPLE     $ status...
MySQL STATUS                                                                                          EXAMPLE     $ status...
TCP/IP                TCP/IP Example                              EffectiveMySQL.com - Its all about Performance and Scala...
Latency                       EXPLAIN                              SHOW CREATE TABLE                              SHOW IND...
EXPLAin                EXPLAIN Example                              EffectiveMySQL.com - Its all about Performance and Sca...
LATENCY                       Optimizer Trace (Since 5.6.3)                              Shows all optimizer actions      ...
OPTIMIZER TRACE                Optimizer Trace Example                              EffectiveMySQL.com - Its all about Per...
latency                       performance_schema                              Introduced in 5.5                           ...
Performance schema                Performance Schema example                              EffectiveMySQL.com - Its all abo...
throughput                       Repeating queries                       Redundant queries                       Unnecessa...
throughput                       MySQL Query Cache                              Great for high read                       ...
The art of doing more                    starts with doing less                              EffectiveMySQL.com - Its all ...
MYSQL STATUS                                                                                          EXAMPLE=============...
Row Processing                                                                                          EXAMPLE   SELECT n...
Row Processing                                                                                          EXAMPLE   SELECT n...
Row Processing                                                                                          EXAMPLE    SET PRO...
Row Processing                                                                                          EXAMPLE    SET PRO...
resources                       Many principles are straight forward                       for experienced Oracle resoures...
Architecture                              EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October...
ARCHITECTure                       Storage Engines                       Reducing SQL                       Monitoring    ...
replication                                                                                               Weakest Link    ...
conclusion                       MySQL is a relational database                              Internals are very different ...
Available NOW  http://effectivemysql.com/book/optimizing-sql-statements/                 EffectiveMySQL.com - Its all abou...
Ronald Bradford                                  http://effectiveMySQL.com                              EffectiveMySQL.com...
Upcoming SlideShare
Loading in …5
×

[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

2,457 views

Published on

Published in: Technology
  • Be the first to comment

[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

  1. 1. Title EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  2. 2. Title Mastering MySQL Performance Tuning Ronald Bradford http://ronaldbradford.com Insight Out DB Showcase October 2011 EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  3. 3. OBJECTIVE Identify key MySQL instrumentation Detail the MySQL terms to know and use Provide references to further materials EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  4. 4. ABOUT AUTHOR Ronald BRADFORD 12 years with MySQL / 22 years with RDBMS Senior Consultant at MySQL Inc (06-08) Consultant for Oracle Corporation (96-99) 7 years presenting MySQL content All time top MySQL blogger Available NOW Published author for consulting Oracle ACE Director http://RonaldBradford.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  5. 5. MySQL Instrumentation EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  6. 6. Key sources Configuration my.cnf (e.g. /etc/my.cnf, /etc/mysql/my.cnf) SHOW [SESSION|GLOBAL] VARIABLES INFORMATION_SCHEMA tables Status SHOW [SESSION|GLOBAL] STATUS INFORMATION_SCHEMA tables EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  7. 7. STATUS SHOW STATUS EXAMPLE INFORMATION_SCHEMA.STATUS Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  8. 8. Key Sources Running Threads SHOW FULL PROCESSLIST INFORMATION_SCHEMA.PROCESSLIST Storage Engines SHOW ENGINE INNODB STATUS Performance Schema PERFORMANCE_SCHEMA EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  9. 9. PROCESSLIST PROCESSLIST EXAMPLE INFORMATION_SCHEMA.PROCESSLIST Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  10. 10. KEY areas System performance SQL performance EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  11. 11. Know your system bottleneck EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  12. 12. BOTTLENECK Memory Disk CPU Network EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  13. 13. MEMORY Basic configuration (Global use) Basic configuration (Thread use) Memory Tables Thread Temporary tables http://dev.mysql.com/doc/refman/5.5/en/memory-use.html EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  14. 14. Memory Global Buffer configuration innodb_buffer_pool_size key_buffer_size query_cache_size EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  15. 15. InnoDB Size EXAMPLE innodb_buffer_pool_size Was 1 GB, should be 300M+ AWS small instance with 1.7G RAM mysql> source sql/innodb_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total InnoDB Size | 278 | 171 | 107 | +-------------------+----------+---------+----------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  16. 16. InnoDB Size EXAMPLE innodb_buffer_pool_size Was 1 GB, should be 300M+ AWS small instance with 1.7G RAM mysql> source sql/innodb_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total InnoDB Size | 278 | 171 | 107 | +-------------------+----------+---------+----------+ Rule of 75-80% of RAM is WRONG! EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  17. 17. MyISAM Size EXAMPLE key_buffer_size For MyISAM Indexes ONLY Was default in RDS, should be 700M+ Github mysql> source sql/myisam_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total MyISAM Size | 6140 | 5492 | 648 | +-------------------+----------+---------+----------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  18. 18. MyISAM Size EXAMPLE key_buffer_size For MyISAM Indexes ONLY Was default in RDS, should be 700M+ Github mysql> source sql/myisam_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total MyISAM Size | 6140 | 5492 | 648 | +-------------------+----------+---------+----------+ Needs 1M min for all InnoDB system EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  19. 19. memory Thread buffer configuration read_buffer_size read_rnd_buffer_size sort_buffer_size join_buffer_size EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  20. 20. memory In Memory Tables max_heap_table_size Internal Temporary Tables MIN(tmp_table_size,max_heap_table_size) EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  21. 21. Memory mysqld memory Unbounded Linux process list (ps) $ $ ps -eopid,fname,vsz,rss,user,command | grep -e "RSS" -e "mysql" PID COMMAND VSZ RSS USER COMMAND 13628 grep 3352 820 ubuntu grep -e RSS -e mysql 28082 mysqld 631076 405660 mysql /usr/sbin/mysqld $ ps -eopid,fname,vsz,rss,user,command | grep " mysqld " | grep -v grep | awk {print $3,$4} 631076 405660 http://ronaldbradford.com/blog/are-you-monitoring-rss-vsz-2009-03-08/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  22. 22. MEMORY PROCESSLIST Copying to tmp table EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  23. 23. Disk fsync configuration flush_log_at_trx_commit sync_binlog innodb_file_per_table EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  24. 24. MySQL DISK Status Variables mysql> show global status like innodb%write%; +-----------------------------------+-----------+ Volumes | Innodb_buffer_pool_write_requests | 821053284 | | Innodb_data_pending_writes | 0 | | Innodb_data_writes | 198400090 | | Innodb_dblwr_writes | 547847 | | Innodb_log_write_requests | 64860531 | | Innodb_log_writes | 188609653 | | Innodb_os_log_pending_writes | 0 | +-----------------------------------+-----------+ mysql> show global status like %sync%; +------------------------------+---------+ True Costs | Innodb_data_fsyncs | 6835646 | | Innodb_data_pending_fsyncs | 0 | | Innodb_os_log_fsyncs | 5681432 | | Innodb_os_log_pending_fsyncs | 0 | +------------------------------+---------+ mysql> show global status like created_tmp_disk_tables; +-------------------------+-------+ | Created_tmp_disk_tables | 87 | +-------------------------+-------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  25. 25. CPU Query sorting (ORDER BY/GROUP BY) PROCESSLIST Sorting result EXPLAIN Using Filesort EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  26. 26. NETWORK STATUS Bytes_sent Bytes_received PROCESSLIST Sending data EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  27. 27. SQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  28. 28. SQL performance Throughput Scalability is about increasing throughout Latency Performance is about improving latency EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  29. 29. THROUgHPUT Status Performance Schema TCP/IP EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  30. 30. MySQL STATUS EXAMPLE $ status_now.sh --------------------------+------------+ | Variable_name | Value | +--------------------------+------------+ | Bytes_received | 142721 | | Bytes_sent | 85170 | Temporary disk table | Com_admin_commands | 160 | writes per second | Com_delete | 4 | | Com_insert | 12 | | Com_select | 331 | | Com_show_status | 1 | | Com_update | 5 | | Created_tmp_disk_tables | 52 | | Created_tmp_tables | 53 | ... | Qcache_inserts | 331 | | Qcache_hits | 382 | | Qcache_lowmem_prunes | 319 | | Qcache_free_memory | -8600 | ... | Uptime | 1 | +--------------------------+------------+ https://github.com/ronaldbradford/EffectiveMySQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  31. 31. MySQL STATUS EXAMPLE $ status_now.sh What is happening this second? --------------------------+------------+ | Variable_name | Value | +--------------------------+------------+ | Bytes_received | 142721 | | Bytes_sent | 85170 | Temporary disk table | Com_admin_commands | 160 | writes per second | Com_delete | 4 | | Com_insert | 12 | | Com_select | 331 | | Com_show_status | 1 | | Com_update | 5 | | Created_tmp_disk_tables | 52 | | Created_tmp_tables | 53 | ... | Qcache_inserts | 331 | | Qcache_hits | 382 | | Qcache_lowmem_prunes | 319 | | Qcache_free_memory | -8600 | ... | Uptime | 1 | +--------------------------+------------+ https://github.com/ronaldbradford/EffectiveMySQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  32. 32. TCP/IP TCP/IP Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  33. 33. Latency EXPLAIN SHOW CREATE TABLE SHOW INDEXES EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  34. 34. EXPLAin EXPLAIN Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  35. 35. LATENCY Optimizer Trace (Since 5.6.3) Shows all optimizer actions Rows statistics Tree pruning EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  36. 36. OPTIMIZER TRACE Optimizer Trace Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  37. 37. latency performance_schema Introduced in 5.5 Much improved 5.6.3 EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  38. 38. Performance schema Performance Schema example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  39. 39. throughput Repeating queries Redundant queries Unnecessary complex queries EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  40. 40. throughput MySQL Query Cache Great for high read Poor for read/write EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  41. 41. The art of doing more starts with doing less EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  42. 42. MYSQL STATUS EXAMPLE==================================================================================================== Uptime: 12 hours 17 mins Snapshot Period 1: 1 minute interval==================================================================================================== Variable Delta/Percentage Per Second Total Why are you doing crazy==================================================================================================== Statement Activity things Mr Framework?==================================================================================================== SELECT: 16,042 267.37 8,177,050 (46.03%) INSERT: 5,838 97.30 1,826,616 (10.28%) UPDATE: 1,109 18.48 738,546 (4.16%) DELETE: 2,018 33.63 1,374,983 (7.74%) REPLACE: 0 0.00 0 (0.00%) INSERT ... SELECT: 0 0.00 27 (0.00%) REPLACE ... SELECT: 0 0.00 0 (0.00%) Multi UPDATE: 0 0.00 0 (0.00%) Multi DELETE: 0 0.00 0 (0.00%) COMMIT: 5,708 95.13 2,161,232 (12.17%) ROLLBACK: 5,746 95.77 3,485,828 (19.62%) EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  43. 43. Row Processing EXAMPLE SELECT name FROM firms WHERE id=727; SELECT name FROM firms WHERE id=758; SELECT name FROM firms WHERE id=857; SELECT name FROM firms WHERE id=740; SELECT id, name SELECT name FROM firms WHERE id=849; FROM firms SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839, SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868); SELECT name FROM firms WHERE id=867; SELECT name FROM firms WHERE id=829; SELECT name FROM firms WHERE id=812; SELECT name FROM firms WHERE id=868; SELECT name FROM firms WHERE id=723; http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/ http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  44. 44. Row Processing EXAMPLE SELECT name FROM firms WHERE id=727; Classic N+1 problem SELECT name FROM firms WHERE id=758; SELECT name FROM firms WHERE id=857; SELECT name FROM firms WHERE id=740; SELECT id, name SELECT name FROM firms WHERE id=849; FROM firms SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839, SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868); SELECT name FROM firms WHERE id=867; SELECT name FROM firms WHERE id=829; SELECT name FROM firms WHERE id=812; SELECT name FROM firms WHERE id=868; SELECT name FROM firms WHERE id=723; http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/ http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  45. 45. Row Processing EXAMPLE SET PROFILING=1; SELECT … 4X longer processing for SHOW PROFILES; every page load +----------+------------+--------------------------------------------------------- | Query_ID | Duration | Query +----------+------------+--------------------------------------------------------- | 1 | 0.00030400 | SELECT name FROM firms WHERE id=727 SELECT Sum Individual Queries AS txt,SUM(DURATI | 2 | 0.00014400 | SELECT name FROM firms WHERE id=758 INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE | 3 | 0.00014300 | SELECT name FROM firms WHERE id=857 UNION | 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query,SUM(DURATION) FROM INFORM SELECT Combined | 5 | 0.00012300 | SELECT name FROM firms WHERE 13; QUERY_ID = id=849 | 6 | 0.00012200 | SELECT name FROM firms WHERE id=839 +------------------------+------------+ | 7 | 0.00011600 | SELECT name FROM firms WHERE id=847 | txt | total_time | | 8 | 0.00014300 | SELECT name FROM firms WHERE id=867 +------------------------+------------+ | 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries | | Sum Individual 0.001311 | | 10 | 0.00014000 | SELECT name FROM firms WHERE Query | Combined id=812 | 0.000311 | | 11 | 0.00012800 | SELECT name FROM firms WHERE id=868 +------------------------+------------+ | 12 | 0.00011700 | SELECT name FROM firms WHERE id=723 | 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ... +----------+------------+--------------------------------------------------------- EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  46. 46. Row Processing EXAMPLE SET PROFILING=1; SELECT … 4X longer processing for SHOW PROFILES; every page load +----------+------------+--------------------------------------------------------- | Query_ID | Duration | Query +----------+------------+--------------------------------------------------------- | 1 | 0.00030400 | SELECT name FROM firms WHERE id=727 SELECT Sum Individual Queries AS txt,SUM(DURATI | 2 | 0.00014400 | SELECT name FROM firms WHERE id=758 INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE | 3 | 0.00014300 | SELECT name FROM firms WHERE id=857 UNION | 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query,SUM(DURATION) FROM INFORM SELECT Combined | 5 | 0.00012300 | SELECT name FROM firms WHERE 13; QUERY_ID = id=849 | 6 | 0.00012200 | SELECT name FROM firms WHERE id=839 +------------------------+------------+ | 7 | 0.00011600 | SELECT name FROM firms WHERE id=847 | txt | total_time | | 8 | 0.00014300 | SELECT name FROM firms WHERE id=867 +------------------------+------------+ | 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries | | Sum Individual 0.001311 | | 10 | 0.00014000 | SELECT name FROM firms WHERE Query | Combined id=812 | 0.000311 | | 11 | 0.00012800 | SELECT name FROM firms WHERE id=868 +------------------------+------------+ | 12 | 0.00011700 | SELECT name FROM firms WHERE id=723 | 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ... +----------+------------+--------------------------------------------------------- Instant DB Scalability EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  47. 47. resources Many principles are straight forward for experienced Oracle resoures Open Source projects Less skilled workforce EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  48. 48. Architecture EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  49. 49. ARCHITECTure Storage Engines Reducing SQL Monitoring Replication EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  50. 50. replication Weakest Link EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  51. 51. conclusion MySQL is a relational database Internals are very different Instrumentation is improving in new versions 5.5 GA 5.6 DMR Planet MySQL http://planet.mysql.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  52. 52. Available NOW http://effectivemysql.com/book/optimizing-sql-statements/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  53. 53. Ronald Bradford http://effectiveMySQL.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011

×