SlideShare a Scribd company logo
InnoDB Performance and
    Usability Patches
           MySQL CE 2009

           Vadim Tkachenko,
           Ewen Fortune

           Percona Inc
           MySQLPerformanceBlog.com
Who are we ?
•  Vadim Tkachenko
  –  Co-Founder of Percona Inc
     •  Lead of R&D department
  –  Co-Author MySQLPerformanceBlog.com
  –  Co-Author “High Performance MySQL” 2nd edition book
•  Ewen Fortune
  –  Consultant, Percona Inc
•  Special Thanks Yasufumi Kinoshita
  –  Not here, but author of most patches
What is this talk about?
•  Patches made by Percona for InnoDB Storage
   Engine
•  Two main focuses
  –  Performance improvement patches
  –  “Usability” patches
     •  Make InnoDB a bit more friendly 
     •  World changed since time of Pentium 100MHz and 8MB of RAM
         –  But many such assumptions still in InnoDB code
Why we do it
•  Most requirements and changes come from
   practical work with customers
•  We need InnoDB fully utilizing modern hardware
   today
  –  16 cores
  –  RAIDs
  –  SSD / FusionIO / other storage technologies
•  InnoDB team is “conservative” in making
   improvements in this area
Future
•  Why patches ? Why it can’t be included in InnoDB
   ?
  –  We are often asked about, but actually question is to
     InnoDB team
•  (empty space due to uncertainty of MySQL future in
   Oracle)




•  Anyway we will continue our work
Versions
•  5.0
   –  Set of patches
   –  SHOW PATCHES to see full list
•  5.1
   –  Storage engine XtraDB
   –  Based on InnoDB + patches, not real competitor of
      InnoDB, but drop-in enhanced version
Performance Patches
Scalability
•  Enhanced read_write locks
  –  Improves InnoDB scalability on systems with 8-16 cores
  –  Similar on Google implementation, InnoDB-plugin-1.0.3
  –  Our implementation is alternative
     •  Topic to research which one is better
     •  InnoDB-plugin may be preferred, InnoDB team made hard job
        porting it to many platforms
  –  And now in 5.4
•  Split buffer_pool mutex even more
  –  Additional split of buffer_pool mutex to 5.0.33
IO patches
•  InnoDB IO patches
  –  Part similar to Google’s InnoDB IO patches, but again
     alternative 
  –  Several parts – some of them now in 5.4
IO – multiple threads
•  Read_io_threads
  –  Number of threads for reads requests (by default 1)
  –  Not really useful as used only for read-ahead requests
•  Write_io_threads
  –  Number of threads for write requests (by default 1)
  –  This is one you may want to use on system with multiple
     disks
•  Io_capacity
  –  Amount of IO operations per second InnoDB assumes
     server can do (by default 100, which is not right
     assumptions for modern systems)
IO – Adaptive checkpoint
•  InnoDB flushing of dirty buffer_pool pages may be
   intensive
•  Lack of free pages may be controlled by
   innodb_max_dirty_pages_pct
•  Flushing at the moment of checkpoint is not
   controllable, intensive and may hurt
Adaptive checkpointing
InnoDB default behavior, hiccups during buffer_pool flushing
Adaptive checkpoint
•  What we do:
•  Flush pages more intensive
  –  the closer checkpoint the more intensive
Adaptive_checkpoint
•  Adaptive_checkpoint=1
IO Control of Insert buffer
•  Ibuf_max_size – maximal size of insert buffer (by
   default can be half of buffer_pool)
•  Ibuf_accel_rate – IO rate for background thread,
   works in pair with io_capacity
IO – multiple pages
•  Read_ahead = (both | linear | random)
  –  Control to use or not internal InnoDB read-ahead logic
•  Flush_neighbor_pages = (yes|no)
  –  By default InnoDB also writes neighborhoods of flushing
     pages
•  All these operations were made for disks with
   expensive (in time sense) random reads – may be
   not needed for SSD / FusionIO / other devices with
   cheap random reads
Extra rollback segments
•  By default InnoDB uses single segment protected
   by mutex
•  Sensitive in intensive parallel insert load
Fix group commit
•  “Broken” in 5.0
   –  Problem appears on slow disks with enabled binary-logs
Benchmark
•    Tpcc-like workload
•    100 Warehouses (about 10GB of data)
•    Buffer_pool=5GB
•    System: Dell PowerEdge R900, RAID 10 on 8
     disks, RAM 32GB
     –  O_DIRECT for InnoDB, xfs filesystem, mounted with
        nobarrier
•  5.0.77 vs 5.0.77-percona
     –  Had no chance to test 5.4 yet
Benchmark
Usability patches
Microslow
•  InnoDB part
  InnoDB_IO_r_ops: 1 InnoDB_IO_r_bytes: 16384
  InnoDB_IO_r_wait: 0.028487 # InnoDB_rec_lock_wait:
  0.000000 InnoDB_queue_wait: 0.000000 #
  InnoDB_pages_distinct: 5
Limit data dictionary
•  Problem:
   –  Data dictionary entry of once opened table kept in
      memory forever (or while DELETE table)
   –  Is not problem for regular usage (100-1000 tables)
   –  Is problem for instances with 10K+ tables
      •  10GB+ of memory just allocated for datadictionary entries
•  Our solution:
   –  LRU based datadictionary entries
   –  Remove from memory oldest entries if limit reached
IO access pattern
     Show pattern of pages on disk accessed

•    mysql> select INDEX_ID,TABLE_NAME,INDEX_NAME,sum(N_READ),sum(N_WRITE) from
     INFORMATION_SCHEMA.INNODB_ALL_PAGE_IO group
•    by INDEX_ID;
•    +------------+------------------+-------------------+-------------+--------------+
•    | INDEX_ID    | TABLE_NAME      | INDEX_NAME        | sum(N_READ) | sum(N_WRITE) |
•    +------------+------------------+-------------------+-------------+--------------+
•    |          30 | tpcc/item       | PRIMARY           |         547 |            0|
•    |          32 | tpcc/district   | PRIMARY           |           1|             1|
•    |          36 | tpcc/history    | GEN_CLUST_INDEX   |          11 |            5|
•    |          37 | tpcc/history    | fkey_history_1    |         166 |          163 |
•    |          38 | tpcc/history    | fkey_history_2    |          37 |           30 |
•    |          39 | tpcc/new_orders | PRIMARY           |          76 |           76 |
•    |          43 | tpcc/order_line | PRIMARY           |         218 |          189 |
•    |          44 | tpcc/order_line | fkey_order_line_2 |        1040 |         1040 |
•    |          46 | tpcc/stock      | PRIMARY           |        3137 |         1764 |
•    |          47 | tpcc/stock      | fkey_stock_2      |         269 |            0|
•    |          48 | tpcc/customer   | PRIMARY           |         960 |          580 |
•    |          49 | tpcc/customer   | idx_customer      |         171 |            0|
•    |          50 | tpcc/orders     | PRIMARY           |          94 |           70 |
•    |          51 | tpcc/orders     | idx_orders        |         142 |          129 |
Show buffer pool content
•  What is in buffer_pool
  select space,offset, RECORDS, DATASIZE, INDEX_NAME,TABLE_SCHEMA,TABLE_NAME from
  information_schema.INNODB_BUFFER_POOL_CONTENT limit 10;
  +-------+---------+---------+----------+------------+--------------+-------------+
  | space | offset | RECORDS | DATASIZE | INDEX_NAME | TABLE_SCHEMA | TABLE_NAME |
  +-------+---------+---------+----------+------------+--------------+-------------+
  | 1584 | 640643 |         9|     10312 | PRIMARY    | art104       | article104 |
  | 1648 |     2100 |     135 |    15226 | PRIMARY    | art114       | author114   |
  | 1492 |     4507 |     158 |    15130 | PRIMARY    | art87        | author87    |
  | 1406 |    17498 |     141 |    16056 | img_status | art52        | img_out52   |
  | 1466 |    47632 |      49 |    15140 | PRIMARY    | art62        | img_out62   |
  | 1470 | 1395457 |       24 |    14769 | PRIMARY    | art84        | article84   |
  | 1460 |    16025 |      62 |    15174 | PRIMARY    | art61        | img_out61   |
  | 1458 | 560956 |        20 |    14977 | PRIMARY    | art61        | article61   |
  | 1466 |    67953 |      56 |    15182 | PRIMARY    | art62        | img_out62   |
  | 1621 | 162962 |        46 |    15134 | PRIMARY    | art110       | link_out110 |
  +-------+---------+---------+----------+------------+--------------+-------------+
Show memory usage
•  Extended information about memory consuming
 ---------------------- BUFFER POOL AND MEMORY
 ----------------------
 Total memory allocated 328830416; in additional pool allocated
 2117120
 + Internal hash tables (constant factor + variable factor)
 + Adaptive hash index 4839388 (4425628 + 413760) + Page hash
 138716
 + Dictionary cache 3383508 (3320220 + 63288)
 + File system 41848 (41336 + 512)
 + Lock system 332788 (332468 + 320)
 + Recovery system 0 (0 + 0)
 + Threads 41900 (41348 + 552)
 Buffer pool size 16384
 + Buffer pool size, bytes 268435456
 Free buffers 12396
Show locks held
•    ---TRANSACTION 0 163390, ACTIVE 0 sec, process no 15571, OS thread id 1159485776
     inserting
•    mysql tables in use 1, locked 1
•    7 lock struct(s), heap size 1216, undo log entries 4
•    MySQL thread id 15, query id 15455 127.0.0.1 root update
•    INSERT INTO history(h_c_d_id, h_c_w_id, h_c_id, h_d_id, h_w_id, h_date, h_amount, h_data)
     VALUES(?, ?, ?, ?, ?, ?,
•    ?, ?)
•    Trx read view will not see trx with id >= 0 163391, sees < 0 163086
•    TABLE LOCK table `test/warehouse` trx id 0 163390 lock mode IX
•    RECORD LOCKS space id 10 page no 3 n bits 168 index `PRIMARY` of table `test/warehouse`
     trx id 0 163390 lock_mode X
•     locks rec but not gap
•    TABLE LOCK table `test/district` trx id 0 163390 lock mode IX
•  RECORD LOCKS space id 18 page no 7 n bits 216 index `PRIMARY` of
   table `test/district` trx id 0 163390 lock_mode X locks rec but not
   gap
•    TABLE LOCK table `test/customer` trx id 0 163390 lock mode IX
•    RECORD LOCKS space id 19 page no 57918 n bits 96 index `PRIMARY` of table `test/customer`
     trx id 0 163390 lock_mode X locks rec but not gap
•    TABLE LOCK table `test/history` trx id 0 163390 lock mode IX
Extra undo slots
•  By default 1024 slots to store transaction undo
   information, that may limit count of concurrent
   transactions to 512
•  We increase to 4072
  –  Only on 5.1 XtraDB
  –  Use it only if you need, breaks compatibility with InnoDB
TransactionalReplication
•  Similar to Google’s patch
•  Information in relay-log.info is not consistent with
   InnoDB state.
   –  When server crash MySQL will repeat several transaction
      •  You are lucky if replication fails on “Duplicate key error”
      •  In worst case you will have several transactions executed twice
•  Our solution: store information of binary log name
   and position and relay-log name and position in
   InnoDB transactional log file
Plans
•  Still hunt performance improvements
•  Operations tasks:
  –  Fast recovery
     •  There is reported bug http://bugs.mysql.com/bug.php?id=29847
  –  Preload table / index into buffer_pool.
  –  Copy single .ibd table from one server to different
  –  Open InnoDB tables in parallel
     •  Currently serialized
  –  Different improvements on statistics
     •  Some patches already published (not by us)
To finalize
•  Most of patches is not rocket science
  –  Could be developed or included in official tree long time
     ago
     •  Even more, for some patches we just only uncommented few
        lines of code
  –  Expect most of them in MariaDB 5.1
Questions ?
•  Thank you for coming!

More Related Content

What's hot

Drilling Deep Into Exadata Performance
Drilling Deep Into Exadata PerformanceDrilling Deep Into Exadata Performance
Drilling Deep Into Exadata PerformanceEnkitec
 
Stacki and Chef at Pardot
Stacki and Chef at PardotStacki and Chef at Pardot
Stacki and Chef at Pardot
Salesforce Engineering
 
MySQL Monitoring 101
MySQL Monitoring 101MySQL Monitoring 101
MySQL Monitoring 101
Ronald Bradford
 
Introduction to PgBench
Introduction to PgBenchIntroduction to PgBench
Introduction to PgBench
Joshua Drake
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentals
Chris Adkin
 
Salesforce at Stacki Atlanta Meetup February 2016
Salesforce at Stacki Atlanta Meetup February 2016Salesforce at Stacki Atlanta Meetup February 2016
Salesforce at Stacki Atlanta Meetup February 2016
StackIQ
 
OOW 2013: Where did my CPU go
OOW 2013: Where did my CPU goOOW 2013: Where did my CPU go
OOW 2013: Where did my CPU goKristofferson A
 
Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016
StackIQ
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton inserts
Chris Adkin
 
Introduction To Navicat MySql GUI
Introduction To Navicat MySql GUIIntroduction To Navicat MySql GUI
Introduction To Navicat MySql GUI
chadrobertson75
 
Towards Scalable Service Composition on Multicores
Towards Scalable Service Composition on MulticoresTowards Scalable Service Composition on Multicores
Towards Scalable Service Composition on Multicores
Cesare Pautasso
 
Capstone HEDIS Database Documentation
Capstone HEDIS Database DocumentationCapstone HEDIS Database Documentation
Capstone HEDIS Database Documentation
Stephen Cain Jr.
 
4 Sessions
4 Sessions4 Sessions
4 Sessions
Marian Marinov
 
Prod java-error
Prod java-errorProd java-error
Prod java-error
Ramakrishna Chava
 
RedGateWebinar - Where did my CPU go?
RedGateWebinar - Where did my CPU go?RedGateWebinar - Where did my CPU go?
RedGateWebinar - Where did my CPU go?Kristofferson A
 
RMOUG 2013 - Where did my CPU go?
RMOUG 2013 - Where did my CPU go?RMOUG 2013 - Where did my CPU go?
RMOUG 2013 - Where did my CPU go?Kristofferson A
 
PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Portland Performance Practice Project - Database Test 2 TuningPostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
Mark Wong
 
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
Nuno Alves
 
MySQL Cluster 7.3 Performance Tuning - Severalnines Slides
MySQL Cluster 7.3 Performance Tuning - Severalnines SlidesMySQL Cluster 7.3 Performance Tuning - Severalnines Slides
MySQL Cluster 7.3 Performance Tuning - Severalnines Slides
Severalnines
 

What's hot (19)

Drilling Deep Into Exadata Performance
Drilling Deep Into Exadata PerformanceDrilling Deep Into Exadata Performance
Drilling Deep Into Exadata Performance
 
Stacki and Chef at Pardot
Stacki and Chef at PardotStacki and Chef at Pardot
Stacki and Chef at Pardot
 
MySQL Monitoring 101
MySQL Monitoring 101MySQL Monitoring 101
MySQL Monitoring 101
 
Introduction to PgBench
Introduction to PgBenchIntroduction to PgBench
Introduction to PgBench
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentals
 
Salesforce at Stacki Atlanta Meetup February 2016
Salesforce at Stacki Atlanta Meetup February 2016Salesforce at Stacki Atlanta Meetup February 2016
Salesforce at Stacki Atlanta Meetup February 2016
 
OOW 2013: Where did my CPU go
OOW 2013: Where did my CPU goOOW 2013: Where did my CPU go
OOW 2013: Where did my CPU go
 
Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton inserts
 
Introduction To Navicat MySql GUI
Introduction To Navicat MySql GUIIntroduction To Navicat MySql GUI
Introduction To Navicat MySql GUI
 
Towards Scalable Service Composition on Multicores
Towards Scalable Service Composition on MulticoresTowards Scalable Service Composition on Multicores
Towards Scalable Service Composition on Multicores
 
Capstone HEDIS Database Documentation
Capstone HEDIS Database DocumentationCapstone HEDIS Database Documentation
Capstone HEDIS Database Documentation
 
4 Sessions
4 Sessions4 Sessions
4 Sessions
 
Prod java-error
Prod java-errorProd java-error
Prod java-error
 
RedGateWebinar - Where did my CPU go?
RedGateWebinar - Where did my CPU go?RedGateWebinar - Where did my CPU go?
RedGateWebinar - Where did my CPU go?
 
RMOUG 2013 - Where did my CPU go?
RMOUG 2013 - Where did my CPU go?RMOUG 2013 - Where did my CPU go?
RMOUG 2013 - Where did my CPU go?
 
PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Portland Performance Practice Project - Database Test 2 TuningPostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
 
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
Wp intelli cache_reduction_iops_xd5.6_fp1_xs6.1
 
MySQL Cluster 7.3 Performance Tuning - Severalnines Slides
MySQL Cluster 7.3 Performance Tuning - Severalnines SlidesMySQL Cluster 7.3 Performance Tuning - Severalnines Slides
MySQL Cluster 7.3 Performance Tuning - Severalnines Slides
 

Similar to Inno Db Performance And Usability Patches

Oracle Database In-Memory Option in Action
Oracle Database In-Memory Option in ActionOracle Database In-Memory Option in Action
Oracle Database In-Memory Option in Action
Tanel Poder
 
In Memory Database In Action by Tanel Poder and Kerry Osborne
In Memory Database In Action by Tanel Poder and Kerry OsborneIn Memory Database In Action by Tanel Poder and Kerry Osborne
In Memory Database In Action by Tanel Poder and Kerry Osborne
Enkitec
 
Apache Cassandra at Macys
Apache Cassandra at MacysApache Cassandra at Macys
Apache Cassandra at Macys
DataStax Academy
 
Alto Desempenho com Java
Alto Desempenho com JavaAlto Desempenho com Java
Alto Desempenho com Javacodebits
 
php & performance
 php & performance php & performance
php & performance
simon8410
 
Varnish @ Velocity Ignite
Varnish @ Velocity IgniteVarnish @ Velocity Ignite
Varnish @ Velocity IgniteArtur Bergman
 
How to build a state-of-the-art rails cluster
How to build a state-of-the-art rails clusterHow to build a state-of-the-art rails cluster
How to build a state-of-the-art rails cluster
Tim Lossen
 
Webinar: Untethering Compute from Storage
Webinar: Untethering Compute from StorageWebinar: Untethering Compute from Storage
Webinar: Untethering Compute from Storage
Avere Systems
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
Morgan Tocker
 
PHP & Performance
PHP & PerformancePHP & Performance
PHP & Performance
毅 吕
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015PostgreSQL-Consulting
 
MySQLinsanity
MySQLinsanityMySQLinsanity
MySQLinsanity
Stanley Huang
 
6 tips for improving ruby performance
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performance
Engine Yard
 
New features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in actionNew features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in action
Sveta Smirnova
 
Highload Perf Tuning
Highload Perf TuningHighload Perf Tuning
Highload Perf TuningHighLoad2009
 
Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic Stack
Jakub Hajek
 
Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek
PROIDEA
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012
Wim Godden
 
介绍 Percona 服务器 XtraDB 和 Xtrabackup
介绍 Percona 服务器 XtraDB 和 Xtrabackup介绍 Percona 服务器 XtraDB 和 Xtrabackup
介绍 Percona 服务器 XtraDB 和 Xtrabackup
YUCHENG HU
 

Similar to Inno Db Performance And Usability Patches (20)

Oracle Database In-Memory Option in Action
Oracle Database In-Memory Option in ActionOracle Database In-Memory Option in Action
Oracle Database In-Memory Option in Action
 
In Memory Database In Action by Tanel Poder and Kerry Osborne
In Memory Database In Action by Tanel Poder and Kerry OsborneIn Memory Database In Action by Tanel Poder and Kerry Osborne
In Memory Database In Action by Tanel Poder and Kerry Osborne
 
Apache Cassandra at Macys
Apache Cassandra at MacysApache Cassandra at Macys
Apache Cassandra at Macys
 
Alto Desempenho com Java
Alto Desempenho com JavaAlto Desempenho com Java
Alto Desempenho com Java
 
php & performance
 php & performance php & performance
php & performance
 
Varnish @ Velocity Ignite
Varnish @ Velocity IgniteVarnish @ Velocity Ignite
Varnish @ Velocity Ignite
 
How to build a state-of-the-art rails cluster
How to build a state-of-the-art rails clusterHow to build a state-of-the-art rails cluster
How to build a state-of-the-art rails cluster
 
Webinar: Untethering Compute from Storage
Webinar: Untethering Compute from StorageWebinar: Untethering Compute from Storage
Webinar: Untethering Compute from Storage
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
 
PHP & Performance
PHP & PerformancePHP & Performance
PHP & Performance
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
MySQLinsanity
MySQLinsanityMySQLinsanity
MySQLinsanity
 
6 tips for improving ruby performance
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performance
 
New features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in actionNew features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in action
 
MySQL Tuning
MySQL TuningMySQL Tuning
MySQL Tuning
 
Highload Perf Tuning
Highload Perf TuningHighload Perf Tuning
Highload Perf Tuning
 
Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic Stack
 
Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek Docker Logging and analysing with Elastic Stack - Jakub Hajek
Docker Logging and analysing with Elastic Stack - Jakub Hajek
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012
 
介绍 Percona 服务器 XtraDB 和 Xtrabackup
介绍 Percona 服务器 XtraDB 和 Xtrabackup介绍 Percona 服务器 XtraDB 和 Xtrabackup
介绍 Percona 服务器 XtraDB 和 Xtrabackup
 

More from MySQLConference

Memcached Functions For My Sql Seemless Caching In My Sql
Memcached Functions For My Sql Seemless Caching In My SqlMemcached Functions For My Sql Seemless Caching In My Sql
Memcached Functions For My Sql Seemless Caching In My SqlMySQLConference
 
Using Open Source Bi In The Real World
Using Open Source Bi In The Real WorldUsing Open Source Bi In The Real World
Using Open Source Bi In The Real WorldMySQLConference
 
Partitioning Under The Hood
Partitioning Under The HoodPartitioning Under The Hood
Partitioning Under The HoodMySQLConference
 
Tricks And Tradeoffs Of Deploying My Sql Clusters In The Cloud
Tricks And Tradeoffs Of Deploying My Sql Clusters In The CloudTricks And Tradeoffs Of Deploying My Sql Clusters In The Cloud
Tricks And Tradeoffs Of Deploying My Sql Clusters In The CloudMySQLConference
 
D Trace Support In My Sql Guide To Solving Reallife Performance Problems
D Trace Support In My Sql Guide To Solving Reallife Performance ProblemsD Trace Support In My Sql Guide To Solving Reallife Performance Problems
D Trace Support In My Sql Guide To Solving Reallife Performance ProblemsMySQLConference
 
Writing Efficient Java Applications For My Sql Cluster Using Ndbj
Writing Efficient Java Applications For My Sql Cluster Using NdbjWriting Efficient Java Applications For My Sql Cluster Using Ndbj
Writing Efficient Java Applications For My Sql Cluster Using NdbjMySQLConference
 
My Sql Performance On Ec2
My Sql Performance On Ec2My Sql Performance On Ec2
My Sql Performance On Ec2MySQLConference
 
My Sql And Search At Craigslist
My Sql And Search At CraigslistMy Sql And Search At Craigslist
My Sql And Search At CraigslistMySQLConference
 
Solving Common Sql Problems With The Seq Engine
Solving Common Sql Problems With The Seq EngineSolving Common Sql Problems With The Seq Engine
Solving Common Sql Problems With The Seq EngineMySQLConference
 
Using Continuous Etl With Real Time Queries To Eliminate My Sql Bottlenecks
Using Continuous Etl With Real Time Queries To Eliminate My Sql BottlenecksUsing Continuous Etl With Real Time Queries To Eliminate My Sql Bottlenecks
Using Continuous Etl With Real Time Queries To Eliminate My Sql BottlenecksMySQLConference
 
Make Your Life Easier With Maatkit
Make Your Life Easier With MaatkitMake Your Life Easier With Maatkit
Make Your Life Easier With MaatkitMySQLConference
 
Getting The Most Out Of My Sql Enterprise Monitor 20
Getting The Most Out Of My Sql Enterprise Monitor 20Getting The Most Out Of My Sql Enterprise Monitor 20
Getting The Most Out Of My Sql Enterprise Monitor 20MySQLConference
 
Wide Open Spaces Using My Sql As A Web Mapping Service Backend
Wide Open Spaces Using My Sql As A Web Mapping Service BackendWide Open Spaces Using My Sql As A Web Mapping Service Backend
Wide Open Spaces Using My Sql As A Web Mapping Service BackendMySQLConference
 
Unleash The Power Of Your Data Using Open Source Business Intelligence
Unleash The Power Of Your Data Using Open Source Business IntelligenceUnleash The Power Of Your Data Using Open Source Business Intelligence
Unleash The Power Of Your Data Using Open Source Business IntelligenceMySQLConference
 
Inno Db Internals Inno Db File Formats And Source Code Structure
Inno Db Internals Inno Db File Formats And Source Code StructureInno Db Internals Inno Db File Formats And Source Code Structure
Inno Db Internals Inno Db File Formats And Source Code StructureMySQLConference
 
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin Express
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin ExpressMy Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin Express
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin ExpressMySQLConference
 

More from MySQLConference (17)

Memcached Functions For My Sql Seemless Caching In My Sql
Memcached Functions For My Sql Seemless Caching In My SqlMemcached Functions For My Sql Seemless Caching In My Sql
Memcached Functions For My Sql Seemless Caching In My Sql
 
Using Open Source Bi In The Real World
Using Open Source Bi In The Real WorldUsing Open Source Bi In The Real World
Using Open Source Bi In The Real World
 
Partitioning Under The Hood
Partitioning Under The HoodPartitioning Under The Hood
Partitioning Under The Hood
 
Tricks And Tradeoffs Of Deploying My Sql Clusters In The Cloud
Tricks And Tradeoffs Of Deploying My Sql Clusters In The CloudTricks And Tradeoffs Of Deploying My Sql Clusters In The Cloud
Tricks And Tradeoffs Of Deploying My Sql Clusters In The Cloud
 
D Trace Support In My Sql Guide To Solving Reallife Performance Problems
D Trace Support In My Sql Guide To Solving Reallife Performance ProblemsD Trace Support In My Sql Guide To Solving Reallife Performance Problems
D Trace Support In My Sql Guide To Solving Reallife Performance Problems
 
Writing Efficient Java Applications For My Sql Cluster Using Ndbj
Writing Efficient Java Applications For My Sql Cluster Using NdbjWriting Efficient Java Applications For My Sql Cluster Using Ndbj
Writing Efficient Java Applications For My Sql Cluster Using Ndbj
 
My Sql Performance On Ec2
My Sql Performance On Ec2My Sql Performance On Ec2
My Sql Performance On Ec2
 
My Sql And Search At Craigslist
My Sql And Search At CraigslistMy Sql And Search At Craigslist
My Sql And Search At Craigslist
 
The Smug Mug Tale
The Smug Mug TaleThe Smug Mug Tale
The Smug Mug Tale
 
Solving Common Sql Problems With The Seq Engine
Solving Common Sql Problems With The Seq EngineSolving Common Sql Problems With The Seq Engine
Solving Common Sql Problems With The Seq Engine
 
Using Continuous Etl With Real Time Queries To Eliminate My Sql Bottlenecks
Using Continuous Etl With Real Time Queries To Eliminate My Sql BottlenecksUsing Continuous Etl With Real Time Queries To Eliminate My Sql Bottlenecks
Using Continuous Etl With Real Time Queries To Eliminate My Sql Bottlenecks
 
Make Your Life Easier With Maatkit
Make Your Life Easier With MaatkitMake Your Life Easier With Maatkit
Make Your Life Easier With Maatkit
 
Getting The Most Out Of My Sql Enterprise Monitor 20
Getting The Most Out Of My Sql Enterprise Monitor 20Getting The Most Out Of My Sql Enterprise Monitor 20
Getting The Most Out Of My Sql Enterprise Monitor 20
 
Wide Open Spaces Using My Sql As A Web Mapping Service Backend
Wide Open Spaces Using My Sql As A Web Mapping Service BackendWide Open Spaces Using My Sql As A Web Mapping Service Backend
Wide Open Spaces Using My Sql As A Web Mapping Service Backend
 
Unleash The Power Of Your Data Using Open Source Business Intelligence
Unleash The Power Of Your Data Using Open Source Business IntelligenceUnleash The Power Of Your Data Using Open Source Business Intelligence
Unleash The Power Of Your Data Using Open Source Business Intelligence
 
Inno Db Internals Inno Db File Formats And Source Code Structure
Inno Db Internals Inno Db File Formats And Source Code StructureInno Db Internals Inno Db File Formats And Source Code Structure
Inno Db Internals Inno Db File Formats And Source Code Structure
 
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin Express
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin ExpressMy Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin Express
My Sql High Availability With A Punch Drbd 83 And Drbd For Dolphin Express
 

Recently uploaded

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
Jen Stirrup
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
RinaMondal9
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Pierluigi Pugliese
 
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
UiPathCommunity
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 

Recently uploaded (20)

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
 
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 

Inno Db Performance And Usability Patches

  • 1. InnoDB Performance and Usability Patches MySQL CE 2009 Vadim Tkachenko, Ewen Fortune Percona Inc MySQLPerformanceBlog.com
  • 2. Who are we ? •  Vadim Tkachenko –  Co-Founder of Percona Inc •  Lead of R&D department –  Co-Author MySQLPerformanceBlog.com –  Co-Author “High Performance MySQL” 2nd edition book •  Ewen Fortune –  Consultant, Percona Inc •  Special Thanks Yasufumi Kinoshita –  Not here, but author of most patches
  • 3. What is this talk about? •  Patches made by Percona for InnoDB Storage Engine •  Two main focuses –  Performance improvement patches –  “Usability” patches •  Make InnoDB a bit more friendly  •  World changed since time of Pentium 100MHz and 8MB of RAM –  But many such assumptions still in InnoDB code
  • 4. Why we do it •  Most requirements and changes come from practical work with customers •  We need InnoDB fully utilizing modern hardware today –  16 cores –  RAIDs –  SSD / FusionIO / other storage technologies •  InnoDB team is “conservative” in making improvements in this area
  • 5. Future •  Why patches ? Why it can’t be included in InnoDB ? –  We are often asked about, but actually question is to InnoDB team •  (empty space due to uncertainty of MySQL future in Oracle) •  Anyway we will continue our work
  • 6. Versions •  5.0 –  Set of patches –  SHOW PATCHES to see full list •  5.1 –  Storage engine XtraDB –  Based on InnoDB + patches, not real competitor of InnoDB, but drop-in enhanced version
  • 8. Scalability •  Enhanced read_write locks –  Improves InnoDB scalability on systems with 8-16 cores –  Similar on Google implementation, InnoDB-plugin-1.0.3 –  Our implementation is alternative •  Topic to research which one is better •  InnoDB-plugin may be preferred, InnoDB team made hard job porting it to many platforms –  And now in 5.4 •  Split buffer_pool mutex even more –  Additional split of buffer_pool mutex to 5.0.33
  • 9. IO patches •  InnoDB IO patches –  Part similar to Google’s InnoDB IO patches, but again alternative  –  Several parts – some of them now in 5.4
  • 10. IO – multiple threads •  Read_io_threads –  Number of threads for reads requests (by default 1) –  Not really useful as used only for read-ahead requests •  Write_io_threads –  Number of threads for write requests (by default 1) –  This is one you may want to use on system with multiple disks •  Io_capacity –  Amount of IO operations per second InnoDB assumes server can do (by default 100, which is not right assumptions for modern systems)
  • 11. IO – Adaptive checkpoint •  InnoDB flushing of dirty buffer_pool pages may be intensive •  Lack of free pages may be controlled by innodb_max_dirty_pages_pct •  Flushing at the moment of checkpoint is not controllable, intensive and may hurt
  • 12. Adaptive checkpointing InnoDB default behavior, hiccups during buffer_pool flushing
  • 13. Adaptive checkpoint •  What we do: •  Flush pages more intensive –  the closer checkpoint the more intensive
  • 15. IO Control of Insert buffer •  Ibuf_max_size – maximal size of insert buffer (by default can be half of buffer_pool) •  Ibuf_accel_rate – IO rate for background thread, works in pair with io_capacity
  • 16. IO – multiple pages •  Read_ahead = (both | linear | random) –  Control to use or not internal InnoDB read-ahead logic •  Flush_neighbor_pages = (yes|no) –  By default InnoDB also writes neighborhoods of flushing pages •  All these operations were made for disks with expensive (in time sense) random reads – may be not needed for SSD / FusionIO / other devices with cheap random reads
  • 17. Extra rollback segments •  By default InnoDB uses single segment protected by mutex •  Sensitive in intensive parallel insert load
  • 18. Fix group commit •  “Broken” in 5.0 –  Problem appears on slow disks with enabled binary-logs
  • 19. Benchmark •  Tpcc-like workload •  100 Warehouses (about 10GB of data) •  Buffer_pool=5GB •  System: Dell PowerEdge R900, RAID 10 on 8 disks, RAM 32GB –  O_DIRECT for InnoDB, xfs filesystem, mounted with nobarrier •  5.0.77 vs 5.0.77-percona –  Had no chance to test 5.4 yet
  • 22. Microslow •  InnoDB part InnoDB_IO_r_ops: 1 InnoDB_IO_r_bytes: 16384 InnoDB_IO_r_wait: 0.028487 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 5
  • 23. Limit data dictionary •  Problem: –  Data dictionary entry of once opened table kept in memory forever (or while DELETE table) –  Is not problem for regular usage (100-1000 tables) –  Is problem for instances with 10K+ tables •  10GB+ of memory just allocated for datadictionary entries •  Our solution: –  LRU based datadictionary entries –  Remove from memory oldest entries if limit reached
  • 24. IO access pattern Show pattern of pages on disk accessed •  mysql> select INDEX_ID,TABLE_NAME,INDEX_NAME,sum(N_READ),sum(N_WRITE) from INFORMATION_SCHEMA.INNODB_ALL_PAGE_IO group •  by INDEX_ID; •  +------------+------------------+-------------------+-------------+--------------+ •  | INDEX_ID | TABLE_NAME | INDEX_NAME | sum(N_READ) | sum(N_WRITE) | •  +------------+------------------+-------------------+-------------+--------------+ •  | 30 | tpcc/item | PRIMARY | 547 | 0| •  | 32 | tpcc/district | PRIMARY | 1| 1| •  | 36 | tpcc/history | GEN_CLUST_INDEX | 11 | 5| •  | 37 | tpcc/history | fkey_history_1 | 166 | 163 | •  | 38 | tpcc/history | fkey_history_2 | 37 | 30 | •  | 39 | tpcc/new_orders | PRIMARY | 76 | 76 | •  | 43 | tpcc/order_line | PRIMARY | 218 | 189 | •  | 44 | tpcc/order_line | fkey_order_line_2 | 1040 | 1040 | •  | 46 | tpcc/stock | PRIMARY | 3137 | 1764 | •  | 47 | tpcc/stock | fkey_stock_2 | 269 | 0| •  | 48 | tpcc/customer | PRIMARY | 960 | 580 | •  | 49 | tpcc/customer | idx_customer | 171 | 0| •  | 50 | tpcc/orders | PRIMARY | 94 | 70 | •  | 51 | tpcc/orders | idx_orders | 142 | 129 |
  • 25. Show buffer pool content •  What is in buffer_pool select space,offset, RECORDS, DATASIZE, INDEX_NAME,TABLE_SCHEMA,TABLE_NAME from information_schema.INNODB_BUFFER_POOL_CONTENT limit 10; +-------+---------+---------+----------+------------+--------------+-------------+ | space | offset | RECORDS | DATASIZE | INDEX_NAME | TABLE_SCHEMA | TABLE_NAME | +-------+---------+---------+----------+------------+--------------+-------------+ | 1584 | 640643 | 9| 10312 | PRIMARY | art104 | article104 | | 1648 | 2100 | 135 | 15226 | PRIMARY | art114 | author114 | | 1492 | 4507 | 158 | 15130 | PRIMARY | art87 | author87 | | 1406 | 17498 | 141 | 16056 | img_status | art52 | img_out52 | | 1466 | 47632 | 49 | 15140 | PRIMARY | art62 | img_out62 | | 1470 | 1395457 | 24 | 14769 | PRIMARY | art84 | article84 | | 1460 | 16025 | 62 | 15174 | PRIMARY | art61 | img_out61 | | 1458 | 560956 | 20 | 14977 | PRIMARY | art61 | article61 | | 1466 | 67953 | 56 | 15182 | PRIMARY | art62 | img_out62 | | 1621 | 162962 | 46 | 15134 | PRIMARY | art110 | link_out110 | +-------+---------+---------+----------+------------+--------------+-------------+
  • 26. Show memory usage •  Extended information about memory consuming ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 328830416; in additional pool allocated 2117120 + Internal hash tables (constant factor + variable factor) + Adaptive hash index 4839388 (4425628 + 413760) + Page hash 138716 + Dictionary cache 3383508 (3320220 + 63288) + File system 41848 (41336 + 512) + Lock system 332788 (332468 + 320) + Recovery system 0 (0 + 0) + Threads 41900 (41348 + 552) Buffer pool size 16384 + Buffer pool size, bytes 268435456 Free buffers 12396
  • 27. Show locks held •  ---TRANSACTION 0 163390, ACTIVE 0 sec, process no 15571, OS thread id 1159485776 inserting •  mysql tables in use 1, locked 1 •  7 lock struct(s), heap size 1216, undo log entries 4 •  MySQL thread id 15, query id 15455 127.0.0.1 root update •  INSERT INTO history(h_c_d_id, h_c_w_id, h_c_id, h_d_id, h_w_id, h_date, h_amount, h_data) VALUES(?, ?, ?, ?, ?, ?, •  ?, ?) •  Trx read view will not see trx with id >= 0 163391, sees < 0 163086 •  TABLE LOCK table `test/warehouse` trx id 0 163390 lock mode IX •  RECORD LOCKS space id 10 page no 3 n bits 168 index `PRIMARY` of table `test/warehouse` trx id 0 163390 lock_mode X •  locks rec but not gap •  TABLE LOCK table `test/district` trx id 0 163390 lock mode IX •  RECORD LOCKS space id 18 page no 7 n bits 216 index `PRIMARY` of table `test/district` trx id 0 163390 lock_mode X locks rec but not gap •  TABLE LOCK table `test/customer` trx id 0 163390 lock mode IX •  RECORD LOCKS space id 19 page no 57918 n bits 96 index `PRIMARY` of table `test/customer` trx id 0 163390 lock_mode X locks rec but not gap •  TABLE LOCK table `test/history` trx id 0 163390 lock mode IX
  • 28. Extra undo slots •  By default 1024 slots to store transaction undo information, that may limit count of concurrent transactions to 512 •  We increase to 4072 –  Only on 5.1 XtraDB –  Use it only if you need, breaks compatibility with InnoDB
  • 29. TransactionalReplication •  Similar to Google’s patch •  Information in relay-log.info is not consistent with InnoDB state. –  When server crash MySQL will repeat several transaction •  You are lucky if replication fails on “Duplicate key error” •  In worst case you will have several transactions executed twice •  Our solution: store information of binary log name and position and relay-log name and position in InnoDB transactional log file
  • 30. Plans •  Still hunt performance improvements •  Operations tasks: –  Fast recovery •  There is reported bug http://bugs.mysql.com/bug.php?id=29847 –  Preload table / index into buffer_pool. –  Copy single .ibd table from one server to different –  Open InnoDB tables in parallel •  Currently serialized –  Different improvements on statistics •  Some patches already published (not by us)
  • 31. To finalize •  Most of patches is not rocket science –  Could be developed or included in official tree long time ago •  Even more, for some patches we just only uncommented few lines of code –  Expect most of them in MariaDB 5.1
  • 32. Questions ? •  Thank you for coming!