Your SlideShare is downloading. ×
0
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
jacobs_tuuri_performance
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

jacobs_tuuri_performance

1,838

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,838
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
81
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. InnoDB Designing Applications and Configuring for Best Performance Heikki Tuuri CEO Innobase Oy Vice President, Development Oracle Corporation 1 InnoDB
  • 2. Today’s Topics <ul><li>Application Design and Performance </li></ul><ul><li>Basic InnoDB Configuration </li></ul><ul><li>Maximizing CPU Efficiency </li></ul><ul><li>Maximizing Disk I/O Efficiency </li></ul><ul><li>Maximizing Transaction Throughput </li></ul><ul><li>Some Operating System Tips </li></ul><ul><li>Speeding Bulk Data Operations </li></ul>
  • 3. Application Design and Performance <ul><li>Create appropriate indexes, so that SELECTs, UPDATEs and DELETEs do not require table scans to find the rows </li></ul><ul><ul><li>index the most-often referenced columns in your tables </li></ul></ul><ul><ul><li>try different column orderings in composite (multi-column) keys </li></ul></ul><ul><ul><li>Note: too many indexes can slow INSERT, UPDATE, DELETE </li></ul></ul><ul><li>Use EXPLAIN SELECT ... to check that query plans look sensible and there are good indexes for them </li></ul><ul><li>Consider “fat indexes” (a secondary index including extra columns that queries need), to avoid clustered index lookup </li></ul><ul><ul><li>Note: InnoDB&apos;s secondary index records always contain the clustered index columns (usually the table’s PRIMARY KEY); consider storage implications of long keys! </li></ul></ul>Proper application design is the most important part of performance tuning
  • 4. Application Design and Performance <ul><li>Use SQL statements that process sets of rows at a time, rather than just one row-at-a-time </li></ul><ul><li>Enforce referential integrity within the server, not at the application level </li></ul><ul><li>Use transactions to group operations </li></ul><ul><ul><li>Avoid excessive commits (e.g., avoid autocommit) </li></ul></ul><ul><ul><li>But don’t make transactions to large, or too long-lasting </li></ul></ul>Proper application design is the most important part of performance tuning
  • 5. Basic InnoDB Configuration <ul><li>[mysqld] </li></ul><ul><li># You can write your other MySQL server options here </li></ul><ul><li># ... </li></ul><ul><li># Data files must be able to hold your data and indexes. </li></ul><ul><li># Make sure that you have enough free disk space. </li></ul><ul><li>innodb_data_file_path = ibdata1:10M:autoextend </li></ul><ul><li># </li></ul><ul><li># Set buffer pool size to 50-80% of your computer&apos;s memory </li></ul><ul><li>innodb_buffer_pool_size=70M </li></ul><ul><li>innodb_additional_mem_pool_size=10M </li></ul><ul><li># </li></ul><ul><li># Set the log file size to about 25% of the buffer pool size </li></ul><ul><li>innodb_log_file_size=20M </li></ul><ul><li>innodb_log_buffer_size=8M </li></ul><ul><li># </li></ul><ul><li>innodb_flush_log_at_trx_commit=1 </li></ul>
  • 6. Analyzing InnoDB Performance Problems <ul><li>Print several consecutive SHOW INNODB STATUSG outputs during high workload </li></ul><ul><li>Print SHOW STATUS </li></ul><ul><li>Enable the MySQL &apos;slow query log&apos; </li></ul><ul><li>In Unix/Linux, use &apos;top&apos;, vmstat, iostat </li></ul><ul><li>In Windows use the Task Manager </li></ul>
  • 7. CPU or Disk-Bound Workload? <ul><li>Database throughput can be determined by the processor speed or by the disk access time </li></ul><ul><li>Both CPU and disk use need to be optimized </li></ul><ul><ul><li>first tune the more limited resource, the one limiting throughput </li></ul></ul><ul><li>Unix/Linux shell command ‘ top ’ and Windows Task Manager show Total CPU usage </li></ul>CPU usage &lt; 70 % suggests a disk-bound workload
  • 8. Save CPU Time <ul><li>Use multi-row inserts: INSERT INTO t VALUES (1, 2), (2, 2), (3, 2); </li></ul><ul><li>Use stored procedures </li></ul><ul><li>Instead of many small SELECT queries, try to use one bigger query </li></ul><ul><li>Use the MySQL query cache: query_cache_type=ON query_cache_size=100M </li></ul><ul><li>Save CPU time by minimizing communications </li></ul><ul><ul><ul><ul><li>between the client and the server </li></ul></ul></ul></ul>
  • 9. Reduce Disk Access Bottleneck <ul><li>Keep your hot data set small; delete obsolete data </li></ul><ul><li>Run OPTIMIZE TABLE ... </li></ul><ul><ul><li>Rebuilds the table to eliminate fragmentation -&gt; smaller table! </li></ul></ul><ul><ul><li>Requires taking the database offline </li></ul></ul><ul><li>Create &apos;fat&apos; secondary indexes with extra columns </li></ul><ul><li>Use a large buffer pool to cache more data </li></ul><ul><ul><li>set innodb_buffer_pool_size up to 80% of computer&apos;s RAM </li></ul></ul><ul><li>Use sufficiently large log files </li></ul><ul><ul><li>set innodb_log_file_size to approx 25% of the buffer pool (assumes 2 log files with combined size not more than 4 GB) </li></ul></ul>
  • 10. Make Your InnoDB Tables Smaller <ul><li>Use new (default) COMPACT InnoDB table format in MySQL 5.0 </li></ul><ul><ul><li>typically saves ~20 % of space vs. old REDUNDANT table format </li></ul></ul><ul><li>Use a short PRIMARY KEY , or surrogate key </li></ul><ul><ul><li>InnoDB stores the primary key in every secondary index record </li></ul></ul><ul><ul><li>InnoDB creates 6-byte internal ROW_ID for tables with no primary key or UNIQUE, NOT NULL index </li></ul></ul><ul><li>Use VARCHAR instead of CHAR(n) where possible </li></ul><ul><ul><li>InnoDB always reserves n bytes for fixed-size CHAR( n ) </li></ul></ul><ul><ul><li>Note: UPDATEs to VARCHAR columns can cause fragmentation </li></ul></ul>Smaller tables use fewer disk blocks, thus requiring less i/o
  • 11. Reduce Transaction Commit Disk I/O <ul><li>InnoDB must flush its log to a durable medium at a COMMIT </li></ul><ul><li>Log flush is not normally a bottleneck with a battery-backed disk controller and write-back cache enabled </li></ul><ul><li>But … log flush to a physical disk can be a very serious bottleneck </li></ul><ul><ul><li>Log flush typically takes 4 milliseconds on a fast disk </li></ul></ul><ul><li>Minimize log flushes by wrapping several individual INSERT s, UPDATE s, DELETE s in one transaction </li></ul><ul><li>OR, allow transactions to commit without flush </li></ul><ul><ul><li>Set innodb_flush_log_at_trx_commit=2 in my.cnf </li></ul></ul><ul><ul><li>WARNING : can lose 1 second’s worth of transactions that occurred prior to an operating system crash </li></ul></ul>
  • 12. Spread Disk I/O to Several Disks <ul><li>Two InnoDB modes for storing tables in files: </li></ul><ul><ul><li>one file/table ( innodb_file_per_table in my.cnf ) </li></ul></ul><ul><ul><li>multiple tables per ibdata file </li></ul></ul><ul><li>With one InnoDB file/table on Unix/Linux </li></ul><ul><ul><li>Symlink MySQL database directories to separate disk drives </li></ul></ul><ul><ul><li>Also can symlink .ibd (data) files on different drives </li></ul></ul><ul><ul><li>Note: ALTER TABLE and CREATE INDEX relocate the . ibd file because the table is recreated </li></ul></ul><ul><li>With InnoDB data in multiple ibdata files, create the files on different disk drives </li></ul><ul><ul><li>InnoDB fills ibdata files linearly, starting from first listed in the my.cnf </li></ul></ul><ul><li>Though maybe the easiest solution is to buy a RAID disk array! </li></ul>Store InnoDB data on multiple physical disk drives
  • 13. Avoid O/S Double Buffering <ul><li>InnoDB itself buffers database pages. Using the operating system file cache is a waste of memory and CPU time </li></ul><ul><ul><li>the o/s can page out parts of the InnoDB buffer pool </li></ul></ul><ul><ul><li>use ~80% of physical RAM for InnoDB, and turn off o/s file caching </li></ul></ul><ul><li>Linux : set innodb_flush_method=O_DIRECT in my.cnf to advise o/s not to buffer InnoDB files </li></ul><ul><li>Solaris : use a direct i/o UFS filesystem </li></ul><ul><ul><li>use mount option forcedirectio ; see mount_ufs(1M) </li></ul></ul><ul><ul><li>with Veritas file system VxFS, use the mount option convosync=direct </li></ul></ul><ul><li>Windows : InnoDB always uses unbuffered async i/o. No action required </li></ul>Devote physical RAM to InnoDB, not the o/s file cache!
  • 14. Avoid Slow O/S File Flushing <ul><li>Linux/Unix use fsync() to flush data to disk </li></ul><ul><li>fsync() is extremely slow on some old Linux and Unix versions </li></ul><ul><ul><li>NetBSD was mentioned by a user as one such platform </li></ul></ul><ul><li>Setting innodb_flush_method=O_DSYNC in my.cnf may help </li></ul>
  • 15. Avoid “Thread Thrashing” in High Concurrency Environments <ul><li>With too many threads (on Linux), performance can degrade badly </li></ul><ul><ul><li>SHOW INNODB STATUSG shows many threads waiting for a semaphore </li></ul></ul><ul><ul><li>queries pile up, and little work gets done </li></ul></ul><ul><ul><li>throughput can drop to 1/1000 of the normal </li></ul></ul><ul><li>Workaround: set innodb_thread_concurrency to 8 , or even to 1 in my.cnf </li></ul><ul><li>We are working on removing this problem </li></ul><ul><li>MySQL-5.1.9 behaves somewhat better in this respect </li></ul>Limit the number of threads executing inside InnoDB at once
  • 16. Avoid Transaction Deadlocks <ul><li>Use short transactions, which are less prone to collide and deadlock </li></ul><ul><li>Appropriate indexes reduce table scans and reduce the number of locks taken </li></ul><ul><li>Access tables and rows in the same predefined order throughout your application </li></ul>Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently
  • 17. Avoid Transaction Deadlocks (2) <ul><li>With MySQL V5.0 or before, use the my.cnf option innodb_locks_unsafe_for_binlog to minimize next-key locking </li></ul><ul><ul><li>This is safe only if you do not use binlogging for replication, </li></ul></ul><ul><ul><li>OR, if your application is not prone to &apos;phantom row&apos; problems </li></ul></ul><ul><li>Beginning with MySQL-5.1.xx , if you use row-based replication, you can safely reduce next-key locking by … </li></ul><ul><ul><li>setting innodb_locks_unsafe_for_binlog </li></ul></ul><ul><ul><li>use TRANSACTION ISOLATION LEVEL READ COMMITTED </li></ul></ul>Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently
  • 18. Avoid Transaction Deadlocks (3) <ul><li>Last resort: use table locks to serialize transactions </li></ul><ul><ul><li>table locks eliminate deadlocks, but reduce throughput </li></ul></ul><ul><li>You have to SET AUTOCOMMIT=0 to make them work properly </li></ul>Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently SET AUTOCOMMIT=0; LOCK TABLES t1 WRITE, t2 READ, ...; &lt;INSERTs, UPDATEs, DELETEs&gt; …; UNLOCK TABLES; COMMIT;
  • 19. Speed up Table Imports to InnoDB <ul><li>Execute many INSERT s per transaction, not one transaction per row! </li></ul><ul><li>Before loading, sort the rows in primary key order to reduce random disk seeks </li></ul><ul><li>For tables with foreign key constraints, turn off foreign key checks with SET FOREIGN_KEY_CHECKS=0 </li></ul><ul><ul><li>Do this only if you know your data conforms to the constraints </li></ul></ul><ul><ul><li>Use only for the duration of the import session </li></ul></ul><ul><li>For tables with unique secondary indexes, , turn off uniqueness checking with SET UNIQUE_CHECKS=0 </li></ul><ul><ul><li>Do this only if the data is unique in required columns </li></ul></ul><ul><ul><li>Use only for the duration of the import session </li></ul></ul>Pre-process your data and know whether it is valid before loading
  • 20. Avoid Big Deletes or Rollbacks <ul><li>Use TRUNCATE TABLE to empty a table, rather than DELETE all rows </li></ul><ul><li>Beware of big rollbacks … use smaller transactions! </li></ul><ul><ul><li>InnoDB speeds INSERT s with an insert buffer, but … </li></ul></ul><ul><ul><li>Rollback occurs row-by-row – can be 30 times slower than inserts </li></ul></ul><ul><ul><li>If you end up with a runaway rollback, drop the table if you can afford losing the data </li></ul></ul>
  • 21. Summary: InnoDB Performance Tuning <ul><li>Design SQL and Indexes with care </li></ul><ul><li>Follow buffer pool, log &amp; data file size guidelines </li></ul><ul><li>Use performance problem diagnostic tools </li></ul><ul><li>Minimize client-server, MySQL-InnoDB traffic </li></ul><ul><li>Reduce i/o w/ small data, large buffers, good indexes </li></ul><ul><li>Spread i/o to multiple physical disk drives </li></ul><ul><li>Avoid o/s double buffering, slow fsync(), “thread thrashing” </li></ul><ul><li>Reduce or eliminate transaction deadlocks </li></ul><ul><li>Pre-process your data and know its validity </li></ul><ul><li>Avoid big deletes or rollbacks </li></ul>
  • 22. A Q &amp; Q U E S T I O N S A N S W E R S
  • 23. 1 InnoDB

×