[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)
Upcoming SlideShare
Loading in...5
×
 

[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

on

  • 962 views

 

Statistics

Views

Total Views
962
Views on SlideShare
962
Embed Views
0

Actions

Likes
1
Downloads
32
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford) [INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford) Presentation Transcript

  • Title EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Title Mastering MySQL Performance Tuning Ronald Bradford http://ronaldbradford.com Insight Out DB Showcase October 2011 EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • OBJECTIVE Identify key MySQL instrumentation Detail the MySQL terms to know and use Provide references to further materials EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • ABOUT AUTHOR Ronald BRADFORD 12 years with MySQL / 22 years with RDBMS Senior Consultant at MySQL Inc (06-08) Consultant for Oracle Corporation (96-99) 7 years presenting MySQL content All time top MySQL blogger Available NOW Published author for consulting Oracle ACE Director http://RonaldBradford.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MySQL Instrumentation EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Key sources Configuration my.cnf (e.g. /etc/my.cnf, /etc/mysql/my.cnf) SHOW [SESSION|GLOBAL] VARIABLES INFORMATION_SCHEMA tables Status SHOW [SESSION|GLOBAL] STATUS INFORMATION_SCHEMA tables EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • STATUS SHOW STATUS EXAMPLE INFORMATION_SCHEMA.STATUS Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Key Sources Running Threads SHOW FULL PROCESSLIST INFORMATION_SCHEMA.PROCESSLIST Storage Engines SHOW ENGINE INNODB STATUS Performance Schema PERFORMANCE_SCHEMA EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • PROCESSLIST PROCESSLIST EXAMPLE INFORMATION_SCHEMA.PROCESSLIST Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • KEY areas System performance SQL performance EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Know your system bottleneck EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • BOTTLENECK Memory Disk CPU Network EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MEMORY Basic configuration (Global use) Basic configuration (Thread use) Memory Tables Thread Temporary tables http://dev.mysql.com/doc/refman/5.5/en/memory-use.html EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Memory Global Buffer configuration innodb_buffer_pool_size key_buffer_size query_cache_size EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • InnoDB Size EXAMPLE innodb_buffer_pool_size Was 1 GB, should be 300M+ AWS small instance with 1.7G RAM mysql> source sql/innodb_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total InnoDB Size | 278 | 171 | 107 | +-------------------+----------+---------+----------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • InnoDB Size EXAMPLE innodb_buffer_pool_size Was 1 GB, should be 300M+ AWS small instance with 1.7G RAM mysql> source sql/innodb_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total InnoDB Size | 278 | 171 | 107 | +-------------------+----------+---------+----------+ Rule of 75-80% of RAM is WRONG! EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MyISAM Size EXAMPLE key_buffer_size For MyISAM Indexes ONLY Was default in RDS, should be 700M+ Github mysql> source sql/myisam_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total MyISAM Size | 6140 | 5492 | 648 | +-------------------+----------+---------+----------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MyISAM Size EXAMPLE key_buffer_size For MyISAM Indexes ONLY Was default in RDS, should be 700M+ Github mysql> source sql/myisam_size.sql +-------------------+----------+---------+----------+ | title | total_mb | data_mb | index_mb | +-------------------+----------+---------+----------+ | Total MyISAM Size | 6140 | 5492 | 648 | +-------------------+----------+---------+----------+ Needs 1M min for all InnoDB system EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • memory Thread buffer configuration read_buffer_size read_rnd_buffer_size sort_buffer_size join_buffer_size EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • memory In Memory Tables max_heap_table_size Internal Temporary Tables MIN(tmp_table_size,max_heap_table_size) EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Memory mysqld memory Unbounded Linux process list (ps) $ $ ps -eopid,fname,vsz,rss,user,command | grep -e "RSS" -e "mysql" PID COMMAND VSZ RSS USER COMMAND 13628 grep 3352 820 ubuntu grep -e RSS -e mysql 28082 mysqld 631076 405660 mysql /usr/sbin/mysqld $ ps -eopid,fname,vsz,rss,user,command | grep " mysqld " | grep -v grep | awk {print $3,$4} 631076 405660 http://ronaldbradford.com/blog/are-you-monitoring-rss-vsz-2009-03-08/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MEMORY PROCESSLIST Copying to tmp table EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Disk fsync configuration flush_log_at_trx_commit sync_binlog innodb_file_per_table EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MySQL DISK Status Variables mysql> show global status like innodb%write%; +-----------------------------------+-----------+ Volumes | Innodb_buffer_pool_write_requests | 821053284 | | Innodb_data_pending_writes | 0 | | Innodb_data_writes | 198400090 | | Innodb_dblwr_writes | 547847 | | Innodb_log_write_requests | 64860531 | | Innodb_log_writes | 188609653 | | Innodb_os_log_pending_writes | 0 | +-----------------------------------+-----------+ mysql> show global status like %sync%; +------------------------------+---------+ True Costs | Innodb_data_fsyncs | 6835646 | | Innodb_data_pending_fsyncs | 0 | | Innodb_os_log_fsyncs | 5681432 | | Innodb_os_log_pending_fsyncs | 0 | +------------------------------+---------+ mysql> show global status like created_tmp_disk_tables; +-------------------------+-------+ | Created_tmp_disk_tables | 87 | +-------------------------+-------+ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • CPU Query sorting (ORDER BY/GROUP BY) PROCESSLIST Sorting result EXPLAIN Using Filesort EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • NETWORK STATUS Bytes_sent Bytes_received PROCESSLIST Sending data EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • SQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • SQL performance Throughput Scalability is about increasing throughout Latency Performance is about improving latency EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • THROUgHPUT Status Performance Schema TCP/IP EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MySQL STATUS EXAMPLE $ status_now.sh --------------------------+------------+ | Variable_name | Value | +--------------------------+------------+ | Bytes_received | 142721 | | Bytes_sent | 85170 | Temporary disk table | Com_admin_commands | 160 | writes per second | Com_delete | 4 | | Com_insert | 12 | | Com_select | 331 | | Com_show_status | 1 | | Com_update | 5 | | Created_tmp_disk_tables | 52 | | Created_tmp_tables | 53 | ... | Qcache_inserts | 331 | | Qcache_hits | 382 | | Qcache_lowmem_prunes | 319 | | Qcache_free_memory | -8600 | ... | Uptime | 1 | +--------------------------+------------+ https://github.com/ronaldbradford/EffectiveMySQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MySQL STATUS EXAMPLE $ status_now.sh What is happening this second? --------------------------+------------+ | Variable_name | Value | +--------------------------+------------+ | Bytes_received | 142721 | | Bytes_sent | 85170 | Temporary disk table | Com_admin_commands | 160 | writes per second | Com_delete | 4 | | Com_insert | 12 | | Com_select | 331 | | Com_show_status | 1 | | Com_update | 5 | | Created_tmp_disk_tables | 52 | | Created_tmp_tables | 53 | ... | Qcache_inserts | 331 | | Qcache_hits | 382 | | Qcache_lowmem_prunes | 319 | | Qcache_free_memory | -8600 | ... | Uptime | 1 | +--------------------------+------------+ https://github.com/ronaldbradford/EffectiveMySQL EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • TCP/IP TCP/IP Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Latency EXPLAIN SHOW CREATE TABLE SHOW INDEXES EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • EXPLAin EXPLAIN Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • LATENCY Optimizer Trace (Since 5.6.3) Shows all optimizer actions Rows statistics Tree pruning EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • OPTIMIZER TRACE Optimizer Trace Example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • latency performance_schema Introduced in 5.5 Much improved 5.6.3 EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Performance schema Performance Schema example EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • throughput Repeating queries Redundant queries Unnecessary complex queries EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • throughput MySQL Query Cache Great for high read Poor for read/write EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • The art of doing more starts with doing less EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • MYSQL STATUS EXAMPLE==================================================================================================== Uptime: 12 hours 17 mins Snapshot Period 1: 1 minute interval==================================================================================================== Variable Delta/Percentage Per Second Total Why are you doing crazy==================================================================================================== Statement Activity things Mr Framework?==================================================================================================== SELECT: 16,042 267.37 8,177,050 (46.03%) INSERT: 5,838 97.30 1,826,616 (10.28%) UPDATE: 1,109 18.48 738,546 (4.16%) DELETE: 2,018 33.63 1,374,983 (7.74%) REPLACE: 0 0.00 0 (0.00%) INSERT ... SELECT: 0 0.00 27 (0.00%) REPLACE ... SELECT: 0 0.00 0 (0.00%) Multi UPDATE: 0 0.00 0 (0.00%) Multi DELETE: 0 0.00 0 (0.00%) COMMIT: 5,708 95.13 2,161,232 (12.17%) ROLLBACK: 5,746 95.77 3,485,828 (19.62%) EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Row Processing EXAMPLE SELECT name FROM firms WHERE id=727; SELECT name FROM firms WHERE id=758; SELECT name FROM firms WHERE id=857; SELECT name FROM firms WHERE id=740; SELECT id, name SELECT name FROM firms WHERE id=849; FROM firms SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839, SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868); SELECT name FROM firms WHERE id=867; SELECT name FROM firms WHERE id=829; SELECT name FROM firms WHERE id=812; SELECT name FROM firms WHERE id=868; SELECT name FROM firms WHERE id=723; http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/ http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Row Processing EXAMPLE SELECT name FROM firms WHERE id=727; Classic N+1 problem SELECT name FROM firms WHERE id=758; SELECT name FROM firms WHERE id=857; SELECT name FROM firms WHERE id=740; SELECT id, name SELECT name FROM firms WHERE id=849; FROM firms SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839, SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868); SELECT name FROM firms WHERE id=867; SELECT name FROM firms WHERE id=829; SELECT name FROM firms WHERE id=812; SELECT name FROM firms WHERE id=868; SELECT name FROM firms WHERE id=723; http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/ http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Row Processing EXAMPLE SET PROFILING=1; SELECT … 4X longer processing for SHOW PROFILES; every page load +----------+------------+--------------------------------------------------------- | Query_ID | Duration | Query +----------+------------+--------------------------------------------------------- | 1 | 0.00030400 | SELECT name FROM firms WHERE id=727 SELECT Sum Individual Queries AS txt,SUM(DURATI | 2 | 0.00014400 | SELECT name FROM firms WHERE id=758 INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE | 3 | 0.00014300 | SELECT name FROM firms WHERE id=857 UNION | 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query,SUM(DURATION) FROM INFORM SELECT Combined | 5 | 0.00012300 | SELECT name FROM firms WHERE 13; QUERY_ID = id=849 | 6 | 0.00012200 | SELECT name FROM firms WHERE id=839 +------------------------+------------+ | 7 | 0.00011600 | SELECT name FROM firms WHERE id=847 | txt | total_time | | 8 | 0.00014300 | SELECT name FROM firms WHERE id=867 +------------------------+------------+ | 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries | | Sum Individual 0.001311 | | 10 | 0.00014000 | SELECT name FROM firms WHERE Query | Combined id=812 | 0.000311 | | 11 | 0.00012800 | SELECT name FROM firms WHERE id=868 +------------------------+------------+ | 12 | 0.00011700 | SELECT name FROM firms WHERE id=723 | 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ... +----------+------------+--------------------------------------------------------- EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Row Processing EXAMPLE SET PROFILING=1; SELECT … 4X longer processing for SHOW PROFILES; every page load +----------+------------+--------------------------------------------------------- | Query_ID | Duration | Query +----------+------------+--------------------------------------------------------- | 1 | 0.00030400 | SELECT name FROM firms WHERE id=727 SELECT Sum Individual Queries AS txt,SUM(DURATI | 2 | 0.00014400 | SELECT name FROM firms WHERE id=758 INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE | 3 | 0.00014300 | SELECT name FROM firms WHERE id=857 UNION | 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query,SUM(DURATION) FROM INFORM SELECT Combined | 5 | 0.00012300 | SELECT name FROM firms WHERE 13; QUERY_ID = id=849 | 6 | 0.00012200 | SELECT name FROM firms WHERE id=839 +------------------------+------------+ | 7 | 0.00011600 | SELECT name FROM firms WHERE id=847 | txt | total_time | | 8 | 0.00014300 | SELECT name FROM firms WHERE id=867 +------------------------+------------+ | 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries | | Sum Individual 0.001311 | | 10 | 0.00014000 | SELECT name FROM firms WHERE Query | Combined id=812 | 0.000311 | | 11 | 0.00012800 | SELECT name FROM firms WHERE id=868 +------------------------+------------+ | 12 | 0.00011700 | SELECT name FROM firms WHERE id=723 | 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ... +----------+------------+--------------------------------------------------------- Instant DB Scalability EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • resources Many principles are straight forward for experienced Oracle resoures Open Source projects Less skilled workforce EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Architecture EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • ARCHITECTure Storage Engines Reducing SQL Monitoring Replication EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • replication Weakest Link EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • conclusion MySQL is a relational database Internals are very different Instrumentation is improving in new versions 5.5 GA 5.6 DMR Planet MySQL http://planet.mysql.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Available NOW http://effectivemysql.com/book/optimizing-sql-statements/ EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011
  • Ronald Bradford http://effectiveMySQL.com EffectiveMySQL.com - Its all about Performance and ScalabilityWednesday, October 19, 2011