jacobs_tuuri_performance
Upcoming SlideShare
Loading in...5
×
 

jacobs_tuuri_performance

on

  • 2,270 views

 

Statistics

Views

Total Views
2,270
Views on SlideShare
2,270
Embed Views
0

Actions

Likes
1
Downloads
79
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

jacobs_tuuri_performance jacobs_tuuri_performance Presentation Transcript

  • InnoDB Designing Applications and Configuring for Best Performance Heikki Tuuri CEO Innobase Oy Vice President, Development Oracle Corporation 1 InnoDB
  • Today’s Topics
    • Application Design and Performance
    • Basic InnoDB Configuration
    • Maximizing CPU Efficiency
    • Maximizing Disk I/O Efficiency
    • Maximizing Transaction Throughput
    • Some Operating System Tips
    • Speeding Bulk Data Operations
  • Application Design and Performance
    • Create appropriate indexes, so that SELECTs, UPDATEs and DELETEs do not require table scans to find the rows
      • index the most-often referenced columns in your tables
      • try different column orderings in composite (multi-column) keys
      • Note: too many indexes can slow INSERT, UPDATE, DELETE
    • Use EXPLAIN SELECT ... to check that query plans look sensible and there are good indexes for them
    • Consider “fat indexes” (a secondary index including extra columns that queries need), to avoid clustered index lookup
      • Note: InnoDB's secondary index records always contain the clustered index columns (usually the table’s PRIMARY KEY); consider storage implications of long keys!
    Proper application design is the most important part of performance tuning
  • Application Design and Performance
    • Use SQL statements that process sets of rows at a time, rather than just one row-at-a-time
    • Enforce referential integrity within the server, not at the application level
    • Use transactions to group operations
      • Avoid excessive commits (e.g., avoid autocommit)
      • But don’t make transactions to large, or too long-lasting
    Proper application design is the most important part of performance tuning
  • Basic InnoDB Configuration
    • [mysqld]
    • # You can write your other MySQL server options here
    • # ...
    • # Data files must be able to hold your data and indexes.
    • # Make sure that you have enough free disk space.
    • innodb_data_file_path = ibdata1:10M:autoextend
    • #
    • # Set buffer pool size to 50-80% of your computer's memory
    • innodb_buffer_pool_size=70M
    • innodb_additional_mem_pool_size=10M
    • #
    • # Set the log file size to about 25% of the buffer pool size
    • innodb_log_file_size=20M
    • innodb_log_buffer_size=8M
    • #
    • innodb_flush_log_at_trx_commit=1
  • Analyzing InnoDB Performance Problems
    • Print several consecutive SHOW INNODB STATUSG outputs during high workload
    • Print SHOW STATUS
    • Enable the MySQL 'slow query log'
    • In Unix/Linux, use 'top', vmstat, iostat
    • In Windows use the Task Manager
  • CPU or Disk-Bound Workload?
    • Database throughput can be determined by the processor speed or by the disk access time
    • Both CPU and disk use need to be optimized
      • first tune the more limited resource, the one limiting throughput
    • Unix/Linux shell command ‘ top ’ and Windows Task Manager show Total CPU usage
    CPU usage < 70 % suggests a disk-bound workload
  • Save CPU Time
    • Use multi-row inserts: INSERT INTO t VALUES (1, 2), (2, 2), (3, 2);
    • Use stored procedures
    • Instead of many small SELECT queries, try to use one bigger query
    • Use the MySQL query cache: query_cache_type=ON query_cache_size=100M
    • Save CPU time by minimizing communications
          • between the client and the server
  • Reduce Disk Access Bottleneck
    • Keep your hot data set small; delete obsolete data
    • Run OPTIMIZE TABLE ...
      • Rebuilds the table to eliminate fragmentation -> smaller table!
      • Requires taking the database offline
    • Create 'fat' secondary indexes with extra columns
    • Use a large buffer pool to cache more data
      • set innodb_buffer_pool_size up to 80% of computer's RAM
    • Use sufficiently large log files
      • set innodb_log_file_size to approx 25% of the buffer pool (assumes 2 log files with combined size not more than 4 GB)
  • Make Your InnoDB Tables Smaller
    • Use new (default) COMPACT InnoDB table format in MySQL 5.0
      • typically saves ~20 % of space vs. old REDUNDANT table format
    • Use a short PRIMARY KEY , or surrogate key
      • InnoDB stores the primary key in every secondary index record
      • InnoDB creates 6-byte internal ROW_ID for tables with no primary key or UNIQUE, NOT NULL index
    • Use VARCHAR instead of CHAR(n) where possible
      • InnoDB always reserves n bytes for fixed-size CHAR( n )
      • Note: UPDATEs to VARCHAR columns can cause fragmentation
    Smaller tables use fewer disk blocks, thus requiring less i/o
  • Reduce Transaction Commit Disk I/O
    • InnoDB must flush its log to a durable medium at a COMMIT
    • Log flush is not normally a bottleneck with a battery-backed disk controller and write-back cache enabled
    • But … log flush to a physical disk can be a very serious bottleneck
      • Log flush typically takes 4 milliseconds on a fast disk
    • Minimize log flushes by wrapping several individual INSERT s, UPDATE s, DELETE s in one transaction
    • OR, allow transactions to commit without flush
      • Set innodb_flush_log_at_trx_commit=2 in my.cnf
      • WARNING : can lose 1 second’s worth of transactions that occurred prior to an operating system crash
  • Spread Disk I/O to Several Disks
    • Two InnoDB modes for storing tables in files:
      • one file/table ( innodb_file_per_table in my.cnf )
      • multiple tables per ibdata file
    • With one InnoDB file/table on Unix/Linux
      • Symlink MySQL database directories to separate disk drives
      • Also can symlink .ibd (data) files on different drives
      • Note: ALTER TABLE and CREATE INDEX relocate the . ibd file because the table is recreated
    • With InnoDB data in multiple ibdata files, create the files on different disk drives
      • InnoDB fills ibdata files linearly, starting from first listed in the my.cnf
    • Though maybe the easiest solution is to buy a RAID disk array!
    Store InnoDB data on multiple physical disk drives
  • Avoid O/S Double Buffering
    • InnoDB itself buffers database pages. Using the operating system file cache is a waste of memory and CPU time
      • the o/s can page out parts of the InnoDB buffer pool
      • use ~80% of physical RAM for InnoDB, and turn off o/s file caching
    • Linux : set innodb_flush_method=O_DIRECT in my.cnf to advise o/s not to buffer InnoDB files
    • Solaris : use a direct i/o UFS filesystem
      • use mount option forcedirectio ; see mount_ufs(1M)
      • with Veritas file system VxFS, use the mount option convosync=direct
    • Windows : InnoDB always uses unbuffered async i/o. No action required
    Devote physical RAM to InnoDB, not the o/s file cache!
  • Avoid Slow O/S File Flushing
    • Linux/Unix use fsync() to flush data to disk
    • fsync() is extremely slow on some old Linux and Unix versions
      • NetBSD was mentioned by a user as one such platform
    • Setting innodb_flush_method=O_DSYNC in my.cnf may help
  • Avoid “Thread Thrashing” in High Concurrency Environments
    • With too many threads (on Linux), performance can degrade badly
      • SHOW INNODB STATUSG shows many threads waiting for a semaphore
      • queries pile up, and little work gets done
      • throughput can drop to 1/1000 of the normal
    • Workaround: set innodb_thread_concurrency to 8 , or even to 1 in my.cnf
    • We are working on removing this problem
    • MySQL-5.1.9 behaves somewhat better in this respect
    Limit the number of threads executing inside InnoDB at once
  • Avoid Transaction Deadlocks
    • Use short transactions, which are less prone to collide and deadlock
    • Appropriate indexes reduce table scans and reduce the number of locks taken
    • Access tables and rows in the same predefined order throughout your application
    Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently
  • Avoid Transaction Deadlocks (2)
    • With MySQL V5.0 or before, use the my.cnf option innodb_locks_unsafe_for_binlog to minimize next-key locking
      • This is safe only if you do not use binlogging for replication,
      • OR, if your application is not prone to 'phantom row' problems
    • Beginning with MySQL-5.1.xx , if you use row-based replication, you can safely reduce next-key locking by …
      • setting innodb_locks_unsafe_for_binlog
      • use TRANSACTION ISOLATION LEVEL READ COMMITTED
    Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently
  • Avoid Transaction Deadlocks (3)
    • Last resort: use table locks to serialize transactions
      • table locks eliminate deadlocks, but reduce throughput
    • You have to SET AUTOCOMMIT=0 to make them work properly
    Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently SET AUTOCOMMIT=0; LOCK TABLES t1 WRITE, t2 READ, ...; <INSERTs, UPDATEs, DELETEs> …; UNLOCK TABLES; COMMIT;
  • Speed up Table Imports to InnoDB
    • Execute many INSERT s per transaction, not one transaction per row!
    • Before loading, sort the rows in primary key order to reduce random disk seeks
    • For tables with foreign key constraints, turn off foreign key checks with SET FOREIGN_KEY_CHECKS=0
      • Do this only if you know your data conforms to the constraints
      • Use only for the duration of the import session
    • For tables with unique secondary indexes, , turn off uniqueness checking with SET UNIQUE_CHECKS=0
      • Do this only if the data is unique in required columns
      • Use only for the duration of the import session
    Pre-process your data and know whether it is valid before loading
  • Avoid Big Deletes or Rollbacks
    • Use TRUNCATE TABLE to empty a table, rather than DELETE all rows
    • Beware of big rollbacks … use smaller transactions!
      • InnoDB speeds INSERT s with an insert buffer, but …
      • Rollback occurs row-by-row – can be 30 times slower than inserts
      • If you end up with a runaway rollback, drop the table if you can afford losing the data
  • Summary: InnoDB Performance Tuning
    • Design SQL and Indexes with care
    • Follow buffer pool, log & data file size guidelines
    • Use performance problem diagnostic tools
    • Minimize client-server, MySQL-InnoDB traffic
    • Reduce i/o w/ small data, large buffers, good indexes
    • Spread i/o to multiple physical disk drives
    • Avoid o/s double buffering, slow fsync(), “thread thrashing”
    • Reduce or eliminate transaction deadlocks
    • Pre-process your data and know its validity
    • Avoid big deletes or rollbacks
  • A Q & Q U E S T I O N S A N S W E R S
  • 1 InnoDB