• Save
Operational Wins in MySQL 5.5
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Operational Wins in MySQL 5.5

on

  • 951 views

MySQL Release 5.5 offers many improvements—such as making InnoDB the default storage engine and giving more visibility into server performance—in areas that have been consistent pain points for ...

MySQL Release 5.5 offers many improvements—such as making InnoDB the default storage engine and giving more visibility into server performance—in areas that have been consistent pain points for operational DBAs. They are the beneficiaries of some great new features and lots of cleanup in this release of the MySQL server. Replication and partitioning both benefited from attention, and it's clear that Oracle has been hearing about headaches operational DBAs are kept up with late at night. This presentation highlights which changes have enabled the speaker's staff to sleep more and why everyone should consider upgrading to Release 5.5.

Statistics

Views

Total Views
951
Views on SlideShare
949
Embed Views
2

Actions

Likes
1
Downloads
0
Comments
0

2 Embeds 2

http://us-w1.rockmelt.com 1
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • MySQL 3.23 … managing 100s of DBs
  • MySQL 3.23 … managing 100s of DBs
  • Baraccuda  steve ballmer’s developers developers developers.
  • Plugin Starting with MySQL 5.5 and InnoDB 1.1, the built-in InnoDB storage engine within MySQL is upgraded to the full feature set and performance of the former InnoDB Plugin.
  • Faster. InnoDB has historically protected the internal state of a read/write lock with an InnoDB mutex. InnoDB implements mutexes and read/write locks with the built-in functions provided by the GNU Compiler Collection (GCC) for atomic memory access instead of using the Pthreads approach previously used. More specifically, an InnoDB that is compiled with GCC version 4.1.2 or later uses the atomic builtins instead of a pthread_mutex_t to implement InnoDB mutexes and read/write locks.
  • in earlier versions, InnoDB implemented its own memory allocator in the mem subsystem. This allocator was guarded by a single mutex, which became a bottleneck. InnoDB also implements a wrapper interface around the system allocator (malloc and free) that was likewise guarded by a single mutex. now use -- system memory allocator – better scaled for multi-core boxes
  • Atomic operations can often be used to synchronize the actions of multiple threads more efficiently than Pthreads. Each operation to acquire or release a lock can be done in fewer CPU instructions, and thus result in less wasted time when threads are contending for access to shared data structures. This in turn means greater scalability on multi-core platforms.
  • With MySQL 5.5 and higher, or MySQL 5.1 with the InnoDB Plugin, creating and dropping secondary indexes for InnoDB tables is much faster than before. Historically, adding or dropping an index on a table with existing data could be very slow. The CREATE INDEX and DROP INDEX statements worked by creating a new, empty table defined with the requested set of indexes, then copying the existing rows to the new table one-by-one, updating the indexes as the rows are inserted. After all rows from the original table were copied, the old table was dropped and the copy was renamed with the name of the original table.
  • user transaction threads -- Before limiting the number of concurrently executing threads, review configuration options that may improve the performance of InnoDB on multi-core and multi-processor computers, such as innodb_use_sys_malloc and innodb_adaptive_hash_index.
  • Barracuda default innodb_file_format >5.5.0 -- requires innodb_file_per_table
  • InnoDB compression applies both to user data and to indexes . In many cases, indexes can constitute 40-50% or more of the total database size, so this difference is significant smaller database can in turn lead to a reduction in I/O, and an increase in throughput, at a modest cost in terms of increased CPU utilization.
  • long column values are stored fully off-page, and the clustered index record contains only a 20-byte pointer to the overflow page.
  • The cleanup activities that occur when InnoDB is started again after a crash. Changes that were committed before the crash, but not yet written into the tablespace files, are reconstructed from the doublewrite buffer. When the database is shut down normally, this type of activity is performed during shutdown by the purge operation. During normal operation, committed data can be stored in the insert buffer for a period of time before being written to the tablespace files. There is always a tradeoff between keeping the tablespace files up-to-date, which introduces performance overhead during normal operation, and buffering the data, which can make shutdown and crash recovery take longer.
  • In particular, scanning the redo log and applying the redo log are faster than in MySQL 5.1 and earlier, due to improved algorithms for memory management. You do not need to take any actions to take advantage of this performance enhancement. If you kept the size of your redo log files artificially low because recovery took a long time, you can consider increasing the file size. redo logs - A set of files, typically named ib_logfile0 and ib_logfile1, that record statements that attempt to change data in InnoDB tables. These statements are replayed automatically to correct data written by incomplete transactions, on startup following a crash. The passage of data through the redo logs is represented by the ever-increasing LSN value. The 4GB limit on maximum size for the redo log is raised in MySQL 5.6. The disk layout of the redo log is influenced by the configuration options innodb_log_file_size, innodb_log_group_home_dir, and (rarely) innodb_log_files_in_group. The performance of redo log operations is also affected by the log buffer, which is controlled by the innodb_log_buffer_size configuration option.
  • opaque. show engine innodb status
  • For every blocked transaction, INNODB_LOCKS contains one row that describes each lock the transaction has requested, and for which it is waiting. INNODB_LOCKS also contains one row for each lock that is blocking another transaction, whatever the state of the transaction that holds the lock ('RUNNING', 'LOCK WAIT', 'ROLLING BACK' or 'COMMITTING'). The lock that is blocking a transaction is always held in a mode (read vs. write, shared vs. exclusive) incompatible with the mode of requested lock. Using this table, you can tell which transactions are waiting for a given lock, or for which lock a given transaction is waiting. This table contains one or more rows for each blocked transaction, indicating the lock it has requested and the lock(s) that is (are) blocking that request.
  • For every blocked transaction, INNODB_LOCKS contains one row that describes each lock the transaction has requested, and for which it is waiting. INNODB_LOCKS also contains one row for each lock that is blocking another transaction, whatever the state of the transaction that holds the lock ('RUNNING', 'LOCK WAIT', 'ROLLING BACK' or 'COMMITTING'). The lock that is blocking a transaction is always held in a mode (read vs. write, shared vs. exclusive) incompatible with the mode of requested lock. Using this table, you can tell which transactions are waiting for a given lock, or for which lock a given transaction is waiting. This table contains one or more rows for each blocked transaction, indicating the lock it has requested and the lock(s) that is (are) blocking that request.
  • •❑ disk•❑memory•❑up/down•❑cachehit -- baseline defiinition•❑replication •❑ mktablesync•❑mkchecksum

Operational Wins in MySQL 5.5 Presentation Transcript

  • 1. operational wins in MySQL 5.5 (why we love the new version) sarah novotny – sarah@bluegecko.net MySQL and LAMP services www .BlueGecko . net
  • 2. www .BlueGecko . net
  • 3. www .BlueGecko . net
  • 4.
    • InnoDB
    www .BlueGecko . net InnoDB InnoDB
  • 5. www .BlueGecko . net plug in
  • 6.
    • InnoDB
    www .BlueGecko . net speed up
  • 7. the need for speed
    • bottlenecking mutexes
      • MUTual EXclusion locks. This locking primitive is simpler and semantically tighter than the others, and hence is easier to make faster, and to prove correct. Some constraints are; lock has one owner at a time, the locker, who must also be the unlocker. – kernelnewbies.org
    www .BlueGecko . net
  • 8. the need for speed
    • atomic operations
      • In concurrent programming, an operation is atomic, linearizable, indivisible or uninterruptible if it appears to the rest of the system to occur instantaneously. Atomicity is a guarantee of isolation from concurrent processes.
    www .BlueGecko . net
  • 9. www .BlueGecko . net fast index creation
  • 10. www .BlueGecko . net transaction threads
  • 11. www .BlueGecko . net InnoDB thread concurrency changes from MySQL documentation
  • 12. www .BlueGecko . net barracuda
  • 13. innodb file format compatibility
    • began with InnoDB 1.0.1
    • @ startup
      • innodb_file_format_check = ON
    • on table open
      • All tables using any file format supported
      • innodb_file_format = $FORMAT
    www .BlueGecko . net
  • 14. innodb file format compatibility
    • @ startup - innodb_file_format_check = ON
      • InnoDB: Error: the system tablespace is in afile format that this version doesn't support
    • on table open
      • at the commandline
        • ERROR 1146 (42S02): Table 'test.t1' doesn't exist
      • in the error log
        • InnoDB: table test/t1: unknown table type 33
    www .BlueGecko . net
  • 15. faster processors + larger datasets + data-driven applications – minimal cost effective disk innovation => i/o wait the problem
  • 16. www .BlueGecko . net a short digression - Artur Bergman on SSDs $$/GB/IOPS
  • 17. www .BlueGecko . net compressed rows
  • 18. www .BlueGecko . net off page columns
  • 19. the problem slow crash recovery
  • 20. www .BlueGecko . net crash recovery
  • 21. crash recovery optimizations
    • improved algorithms
      • faster redo scanning and application
    • no longer need artificially small redo logs
    www .BlueGecko . net
  • 22. www .BlueGecko . net InnoDB transparency problem
  • 23.
    • mysql> SHOW ENGINE INNODB STATUSG
    • *************************** 1. row ***************************
    • Status:
    • =====================================
    • 030709 13:00:59 INNODB MONITOR OUTPUT
    • =====================================
    • Per second averages calculated from the last 18 seconds
    • ----------
    • SEMAPHORES
    • ----------
    • OS WAIT ARRAY INFO: reservation count 413452, signal count 378357
    • --Thread 32782 has waited at btr0sea.c line 1477 for 0.00 seconds the
    www .BlueGecko . net
  • 24.
    • semaphore: X-lock on RW-latch at 41a28668 created in file btr0sea.c line 135
    • a writer (thread id 32782) has reserved it in mode wait exclusive
    • number of readers 1, waiters flag 1
    • Last time read locked in file btr0sea.c line 731
    • Last time write locked in file btr0sea.c line 1347
    • Mutex spin waits 0, rounds 0, OS waits 0
    • RW-shared spins 108462, OS waits 37964; RW-excl spins 681824, OS waits
    • 375485
    • ------------------------
    • LATEST FOREIGN KEY ERROR
    • ------------------------
    • 030709 13:00:59 Transaction:
    • TRANSACTION 0 290328284, ACTIVE 0 sec, process no 3195, OS thread id 34831
    • inserting
    www .BlueGecko . net
  • 25.
    • 15 lock struct(s), heap size 2496, undo log entries 9
    • MySQL thread id 25, query id 4668733 localhost heikki update
    • insert into ibtest11a (D, B, C) values (5, 'khDk' ,'khDk')
    • Foreign key constraint fails for table test/ibtest11a:
    • ,
    • CONSTRAINT `0_219242` FOREIGN KEY (`A`, `D`) REFERENCES `ibtest11b` (`A`,
    • `D`) ON DELETE CASCADE ON UPDATE CASCADE
    • Trying to add in child table, in index PRIMARY tuple:
    • 0: len 4; hex 80000101; asc ....;; 1: len 4; hex 80000005; asc ....;; 2:
    • len 4; hex 6b68446b; asc khDk;; 3: len 6; hex 0000114e0edc; asc ...N..;; 4:
    • len 7; hex 00000000c3e0a7; asc .......;; 5: len 4; hex 6b68446b; asc khDk;;
    • But in parent table test/ibtest11b, in index PRIMARY,
    • the closest match we can find is record:
    www .BlueGecko . net
  • 26.
    • RECORD: info bits 0 0: len 4; hex 8000015b; asc ...[;; 1: len 4; hex
    • 80000005; asc ....;; 2: len 3; hex 6b6864; asc khd;; 3: len 6; hex
    • 0000111ef3eb; asc ......;; 4: len 7; hex 800001001e0084; asc .......;; 5:
    • len 3; hex 6b6864; asc khd;;
    www .BlueGecko . net
  • 27.
    • ------------------------
    • LATEST DETECTED DEADLOCK
    • ------------------------
    • 030709 12:59:58
    • *** (1) TRANSACTION:
    • TRANSACTION 0 290252780, ACTIVE 1 sec, process no 3185, OS thread id 30733
    • inserting
    • LOCK WAIT 3 lock struct(s), heap size 320, undo log entries 146
    • MySQL thread id 21, query id 4553379 localhost heikki update
    • INSERT INTO alex1 VALUES(86, 86, 794,'aA35818','bb','c79166','d4766t',
    • 'e187358f','g84586','h794',date_format('2001-04-03 12:54:22','%Y-%m-%d
    • %H:%i'),7
    www .BlueGecko . net
  • 28.
    • *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
    • RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
    • symbole trx id 0 290252780 lock mode S waiting
    • Record lock, heap no 324 RECORD: info bits 0 0: len 7; hex 61613335383138;
    • asc aa35818;; 1:
    • *** (2) TRANSACTION:
    • TRANSACTION 0 290251546, ACTIVE 2 sec, process no 3190, OS thread id 32782
    • inserting
    • 130 lock struct(s), heap size 11584, undo log entries 437
    • MySQL thread id 23, query id 4554396 localhost heikki update
    • REPLACE INTO alex1 VALUES(NULL, 32, NULL,'aa3572','','c3572','d6012t','',
    • NULL,'h396', NULL, NULL, 7.31,7.31,7.31,200)
    www .BlueGecko . net
  • 29.
    • *** (2) HOLDS THE LOCK(S):
    • RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
    • symbole trx id 0 290251546 lock_mode X locks rec but not gap
    • Record lock, heap no 324 RECORD: info bits 0 0: len 7; hex 61613335383138;
    • asc aa35818;; 1:
    • *** (2) WAITING FOR THIS LOCK TO BE GRANTED:
    • RECORD LOCKS space id 0 page no 48310 n bits 568 table test/alex1 index
    • symbole trx id 0 290251546 lock_mode X locks gap before rec insert intention
    • waiting
    • Record lock, heap no 82 RECORD: info bits 0 0: len 7; hex 61613335373230;
    • asc aa35720;; 1:
    • *** WE ROLL BACK TRANSACTION (1)
    www .BlueGecko . net
  • 30.
    • ------------
    • TRANSACTIONS
    • ------------
    • Trx id counter 0 290328385
    • Purge done for trx's n:o < 0 290315608 undo n:o < 0 17
    • History list length 20
    • Total number of lock structs in row lock hash table 70
    • LIST OF TRANSACTIONS FOR EACH SESSION:
    • ---TRANSACTION 0 0, not started, process no 3491, OS thread id 42002
    • MySQL thread id 32, query id 4668737 localhost heikki
    • show innodb status
    www .BlueGecko . net
  • 31.
    • ---TRANSACTION 0 290328384, ACTIVE 0 sec, process no 3205, OS thread id
    • 38929 inserting
    • 1 lock struct(s), heap size 320
    • MySQL thread id 29, query id 4668736 localhost heikki update
    • insert into speedc values (1519229,1, 'hgjhjgghggjgjgjgjgjggjgjgjgjgjgggjgjg
    • jlhhgghggggghhjhghgggggghjhghghghghghhhhghghghjhhjghjghjkghjghjghjghjfhjfh
    • ---TRANSACTION 0 290328383, ACTIVE 0 sec, process no 3180, OS thread id
    • 28684 committing
    • 1 lock struct(s), heap size 320, undo log entries 1
    • MySQL thread id 19, query id 4668734 localhost heikki update
    • insert into speedcm values (1603393,1, 'hgjhjgghggjgjgjgjgjggjgjgjgjgjgggjgj
    • gjlhhgghggggghhjhghgggggghjhghghghghghhhhghghghjhhjghjghjkghjghjghjghjfhjf
    www .BlueGecko . net
  • 32.
    • ---TRANSACTION 0 290328327, ACTIVE 0 sec, process no 3200, OS thread id
    • 36880 starting index read
    • LOCK WAIT 2 lock struct(s), heap size 320
    • MySQL thread id 27, query id 4668644 localhost heikki Searching rows for
    • update
    • update ibtest11a set B = 'kHdkkkk' where A = 89572
    • ------- TRX HAS BEEN WAITING 0 SEC FOR THIS LOCK TO BE GRANTED:
    • RECORD LOCKS space id 0 page no 65556 n bits 232 table test/ibtest11a index
    • PRIMARY trx id 0 290328327 lock_mode X waiting
    • Record lock, heap no 1 RECORD: info bits 0 0: len 9; hex 73757072656d756d00;
    • asc supremum.;;
    • ------------------
    www .BlueGecko . net
  • 33.
    • ---TRANSACTION 0 290328284, ACTIVE 0 sec, process no 3195, OS thread id
    • 34831 rollback of SQL statement
    • ROLLING BACK 14 lock struct(s), heap size 2496, undo log entries 9
    • MySQL thread id 25, query id 4668733 localhost heikki update
    • insert into ibtest11a (D, B, C) values (5, 'khDk' ,'khDk')
    • ---TRANSACTION 0 290327208, ACTIVE 1 sec, process no 3190, OS thread id
    • 32782
    • 58 lock struct(s), heap size 5504, undo log entries 159
    • MySQL thread id 23, query id 4668732 localhost heikki update
    • REPLACE INTO alex1 VALUES(86, 46, 538,'aa95666','bb','c95666','d9486t',
    • 'e200498f','g86814','h538',date_format('2001-04-03 12:54:22','%Y-%m-%d
    • %H:%i'),
    www .BlueGecko . net
  • 34.
    • ---TRANSACTION 0 290323325, ACTIVE 3 sec, process no 3185, OS thread id
    • 30733 inserting
    • 4 lock struct(s), heap size 1024, undo log entries 165
    • MySQL thread id 21, query id 4668735 localhost heikki update
    • INSERT INTO alex1 VALUES(NULL, 49, NULL,'aa42837','','c56319','d1719t','',
    • NULL,'h321', NULL, NULL, 7.31,7.31,7.31,200)
    www .BlueGecko . net
  • 35.
    • --------
    • FILE I/O
    • --------
    • I/O thread 0 state: waiting for i/o request (insert buffer thread)
    • I/O thread 1 state: waiting for i/o request (log thread)
    • I/O thread 2 state: waiting for i/o request (read thread)
    • I/O thread 3 state: waiting for i/o request (write thread)
    • Pending normal aio reads: 0, aio writes: 0,
    • ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
    • Pending flushes (fsync) log: 0; buffer pool: 0
    • 151671 OS file reads, 94747 OS file writes, 8750 OS fsyncs
    • 25.44 reads/s, 18494 avg bytes/read, 17.55 writes/s, 2.33 fsyncs/s
    www .BlueGecko . net
  • 36.
    • -------------------------------------
    • INSERT BUFFER AND ADAPTIVE HASH INDEX
    • -------------------------------------
    • Ibuf for space 0: size 1, free list len 19, seg size 21,
    • 85004 inserts, 85004 merged recs, 26669 merges
    • Hash table size 207619, used cells 14461, node heap has 16 buffer(s)
    • 1877.67 hash searches/s, 5121.10 non-hash searches/s
    • ---
    • LOG
    • ---
    • Log sequence number 18 1212842764
    • Log flushed up to 18 1212665295
    • Last checkpoint at 18 1135877290
    • 0 pending log writes, 0 pending chkp writes
    • 4341 log i/o's done, 1.22 log i/o's/second
    www .BlueGecko . net
  • 37.
    • ----------------------
    • BUFFER POOL AND MEMORY
    • ----------------------
    • Total memory allocated 84966343; in additional pool allocated 1402624
    • Buffer pool size 3200
    • Free buffers 110
    • Database pages 3074
    • Modified db pages 2674
    • Pending reads 0
    • Pending writes: LRU 0, flush list 0, single page 0
    • Pages read 171380, created 51968, written 194688
    • 28.72 reads/s, 20.72 creates/s, 47.55 writes/s
    • Buffer pool hit rate 999 / 1000
    www .BlueGecko . net
  • 38.
    • --------------
    • ROW OPERATIONS
    • --------------
    • 0 queries inside InnoDB, 0 queries in queue
    • Main thread process no. 3004, id 7176, state: purging
    • Number of rows inserted 3738558, updated 127415, deleted 33707, read 755779
    • 1586.13 inserts/s, 50.89 updates/s, 28.44 deletes/s, 107.88 reads/s
    • ----------------------------
    • END OF INNODB MONITOR OUTPUT
    • ============================
    www .BlueGecko . net
  • 39. www .BlueGecko . net InnoDB Information Schema
  • 40. new information_schema tables
    • INNODB_TRX
      • every transaction currently executing inside InnoDB
    • INNODB_LOCKS
      • Each transaction in InnoDB that is waiting for another transaction to release a lock
    • INNODB_LOCK_WAITS
      • which transactions are waiting for a given lock
    www .BlueGecko . net
  • 41. demo workload
    • User A:
      • BEGIN;
      • SELECT a FROM t FOR UPDATE;
      • SELECT SLEEP(100);
    • User B:
      • SELECT b FROM t FOR UPDATE;
    • User C:
      • SELECT c FROM t FOR UPDATE;
    www .BlueGecko . net
  • 42. information_schema query
    • SELECT r.trx_id waiting_trx_id,
    • r.trx_mysql_thread_id waiting_thread,
    • r.trx_query waiting_query,
    • b.trx_id blocking_trx_id,
    • b.trx_mysql_thread_id blocking_thread,
    • b.trx_query blocking_query
    • FROM information_schema.innodb_lock_waits w
    • INNER JOIN information_schema.innodb_trx b ON
    • b.trx_id = w.blocking_trx_id
    • INNER JOIN information_schema.innodb_trx r ON
    • r.trx_id = w.requesting_trx_id;
    www .BlueGecko . net
  • 43. tada! www .BlueGecko . net waiting trx id waiting thread waiting query blocking trx id blocking thread blocking query A4 6 SELECT b FROM t FOR UPDATE A3 5 SELECT SLEEP(100) A5 7 SELECT c FROM t FOR UPDATE A3 5 SELECT SLEEP(100) A6 SELECT c FROM t FOR UPDATE A4 6 SELECT b FROM t FOR UPDATE
  • 44.
    • other tools we use to
    • manage MySQL
    • for our customers
    • Innotop
    • Percona Toolkit
    • cacti templates
    • nagios
    www .BlueGecko . net
  • 45. credits
    • flickr
    • license plate – severud
    • CDC3800 – nostri-imago
    • plug -- 39747297@N05
    • Tokyo train – johnmueller
    • index cards -- paulk
    • threads – wendypiersall
    • barracuda – diverslog
    • funnelball – ejk
    • Artur Bergman – O’Reilly Media
    • other
    • special thanks to BG staff and clients for suggestions on content and slides
    www .BlueGecko . net flickr
    • compressed car - marcovdz
    • bins -- joost-ijmuiden
    • domo – elzey
    • crash – srboisvert
    • MySQL mints – jimwinstead
    • clouds -- liberato
    • tools – meanestindian
  • 46. Blue Gecko and contact info
    • [email_address]
    • [email_address]
    • @sarahnovotny
    • @bluegecko
    • senk on #mysql
    www .BlueGecko . net Blue Gecko provides Remote DBA services for companies around the world 7x24x365 support including monitoring, performance analysis, proactive maintenance and architectural guidance for small and large datasets.