Recommended
More Related Content
What's hot
What's hot (20)
Similar to MySQL Performance Tuning 101
Similar to MySQL Performance Tuning 101 (20)
Recently uploaded
Recently uploaded (20)
MySQL Performance Tuning 101
- 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Copyright © 2017, Oracle and/or its affiliates. All rights reserved. MySQL Performance Tuning 101 Hands-on-Lab Mirko Ortensi Senior Support Engineer MySQL Support @ Oracle
- 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Understand what are the most important configuration parameters to get started with MySQL 2 Find out what to look at when performance is not the expected Better hardware does not always mean scalability. Find out what are the typical bottlenecks Objectives
- 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | “The beginning is the most important part of the work” – Plato 3
- 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hands-On Lab Agenda Introduction Connections Threads model REDO log InnoDB buffer pool Execution plan 1 2 3 4 5 4 6
- 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Introduction Get ready for speed in 5 steps 5
- 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Introduction • shell$ yum install sysstat • shell$ wget https://tinyurl.com/employeesdb Setup
- 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Introduction TEST DATABASE The Employees sample database provides a combination of a large base of data (approximately 160MB) spread over six separate tables and consisting of 4 million records in total. BENCHMARKING strategies: single operation vs workload mysqlslap is a diagnostic program designed to emulate client load for a MySQL server and to report the timing of each stage. It works as if multiple clients are accessing the server. Test Database and Benchmarking InnoDB storage engine will be used for the following exercises
- 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections The more, the merrier 8
- 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections THREAD THREAD THREAD THREAD CLIENT CLIENT CLIENT CLIENT THREAD THREAD • Each client connection is associated to a thread • Thread handles authentication and the request • Manager thread handles creation of the thread
- 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections • max_connections – The maximum permitted number of simultaneous client connections. By default, this is 151. • Max_used_connections – The maximum number of connections that have been in use simultaneously since the server started. • Connection_errors_max_connections – The number of connections refused because the server max_connections limit was reached. Basic parameters and configuration
- 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections mysql> select @@max_connections; +-------------------+ | @@max_connections | +-------------------+ | 151 | +-------------------+ 1 row in set (0.00 sec) mysql> show global status like 'Max_used_connections'; +----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | Max_used_connections | 1 | +----------------------+-------+ 1 row in set (0.01 sec) Exercise 1 – READ CONFIGURATION
- 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections [hol@localhost ~]$ mysqlslap --auto-generate-sql --concurrency=100 --iterations=10 Benchmark Average number of seconds to run all queries: 0.758 seconds Minimum number of seconds to run all queries: 0.620 seconds Maximum number of seconds to run all queries: 0.967 seconds Number of clients running queries: 100 Average number of queries per client: 0 mysql> show global status like 'Max_used_connections'; +----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | Max_used_connections | 101 | +----------------------+-------+ 1 row in set (0.01 sec) Exercise 1 – ADD LOAD, 100 CONNECTIONS
- 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections [hol@localhost ~]$ mysqlslap --auto-generate-sql --concurrency=170 --iterations=10 mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections mysqlslap: Error when connecting to server: 1040 Too many connections Exercise 1 – MORE LOAD, 170 CONNECTIONS
- 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections mysql> show global status like 'Connection_errors_max_connections'; +-----------------------------------+-------+ | Variable_name | Value | +-----------------------------------+-------+ | Connection_errors_max_connections | 213 | +-----------------------------------+-------+ 1 row in set (0.00 sec) mysql> SET GLOBAL max_connections=200; Query OK, 0 rows affected (0.00 sec) Exercise 1 – ALLOW MORE CONNECTIONS
- 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections [hol@localhost ~]$ mysqlslap --auto-generate-sql --concurrency=170 --iterations=10 Benchmark Average number of seconds to run all queries: 1.563 seconds Minimum number of seconds to run all queries: 1.367 seconds Maximum number of seconds to run all queries: 1.800 seconds Number of clients running queries: 170 Average number of queries per client: 0 Exercise 1 – RETEST
- 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Connections • Connection pooling is a technique of creating and managing a pool of connections that are ready for use by any thread that needs them • Connection pooling can greatly increase the performance of your application, while reducing overall resource usage – Connector/J – Connector/Python – Connector/Net – Connector/ODBC Connection pooling
- 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model Reuse for good 17
- 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model 18 THREAD THREAD THREAD THREAD CLIENT CLIENT CLIENT CLIENT THREAD THREAD • Manager threads create a new thread when necessary but try to avoid doing so by consulting the thread cache • When a connection ends, its thread is returned to the thread cache if the cache is not full.
- 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model mysql> select @@thread_cache_size; +---------------------+ | @@thread_cache_size | +---------------------+ | 9 | +---------------------+ 1 row in set (0.00 sec) mysql> show global status like 'Threads_%'; +-------------------+-------+ | Variable_name | Value | +-------------------+-------+ | Threads_cached | 8 | | Threads_connected | 1 | | Threads_created | 2816 | | Threads_running | 1 | +-------------------+-------+ 4 rows in set (0.00 sec) Exercise 2 – UNDERSTAND THE ROLE OF THREAD CACHE
- 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model [hol@localhost ~]$ mysqlslap --auto-generate-sql --concurrency=100 --iterations=10 Benchmark Average number of seconds to run all queries: 0.682 seconds Minimum number of seconds to run all queries: 0.517 seconds Maximum number of seconds to run all queries: 0.838 seconds Number of clients running queries: 100 Average number of queries per client: 0 mysql> show global status like 'Threads_created'; +-----------------+-------+ | Variable_name | Value | +-----------------+-------+ | Threads_created | 3727 | +-----------------+-------+ 1 row in set (0.00 sec) Exercise 2 – CHECK THREADS CREATION
- 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model mysql> SET GLOBAL thread_cache_size=100; Query OK, 0 rows affected (0.00 sec) [hol@localhost ~]$ mysqlslap --auto-generate-sql --concurrency=100 --iterations=10 Benchmark Average number of seconds to run all queries: 0.732 seconds Minimum number of seconds to run all queries: 0.655 seconds Maximum number of seconds to run all queries: 0.818 seconds Number of clients running queries: 100 Average number of queries per client: 0 mysql> show global status like 'Threads_created'; +-----------------+-------+ | Variable_name | Value | +-----------------+-------+ | Threads_created | 3819 | +-----------------+-------+ 1 row in set (0.00 sec) Exercise 2 – INCREASE THREAD CACHE
- 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Threads model • The default thread-handling model in MySQL Server executes statements using one thread per client connection • As more clients connect to the server and execute statements, overall performance degrades • The thread pool addresses several problems of the one thread per connection model MySQL Enterprise Thread pool
- 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log Sustainable IO throughput 23
- 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log • The redo log is a disk-based data structure used during crash recovery • Redo log encodes requests to change InnoDB table data • Redo log is flushed before a transaction is committed • MySQL writes to the redo log files in a circular fashion 24 What is it? CLIENT BUFFER POOL REDO LOG CLIENT
- 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log • innodb_log_buffer_size – The size in bytes of the buffer that InnoDB uses to write to the log files on disk • innodb_log_file_size – The size in bytes of each log file in a log group • innodb_log_files_in_group – The number of log files in the log group • innodb_flush_log_at_trx_commit – ACID compliance for commit operations vs higher performance Configuration
- 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log • shell$ mysqlslap --auto-generate-sql -- concurrency=100 --iterations=100 • shell$ iostat -xdh /dev/sda 2 – Execute on other console session as mysqlslap is running Exercise 3 – SHOW REDO LOGGING IO IMPACT %util Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). w/s The number of write requests that were issued to the device per second.
- 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 263.93 0.00 742.08 0.0k 3.9M 10.79 0.35 0.48 0.00 0.48 0.47 34.97 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 251.87 0.00 708.56 0.0k 3.7M 10.72 0.34 0.49 0.00 0.49 0.48 34.01 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 246.52 0.00 693.05 0.0k 3.6M 10.76 0.34 0.49 0.00 0.49 0.48 33.58 Exercise 3 – IOSTAT OUTPUT
- 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log • mysql> set global innodb_flush_log_at_trx_commit=0; • shell$ mysqlslap --auto-generate-sql -- concurrency=100 --iterations=100 • shell$ iostat -xdh /dev/sda 2 – Execute on other console session as mysqlslap is running Exercise 3 – REDUCE IO PRESSURE With innodb_flush_log_at_trx_commit = 0 you can lose up to a second of transactions with any mysqld process crash. Use with test instances, restores, replication slaves...
- 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 135.45 0.00 124.87 0.0k 1.9M 31.49 0.08 0.61 0.00 0.61 0.61 7.57 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 129.95 0.00 121.93 0.0k 1.9M 31.79 0.09 0.74 0.00 0.74 0.72 8.77 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 102.12 0.00 97.88 0.0k 1.5M 31.70 0.06 0.63 0.00 0.63 0.61 5.98 Exercise 3 – AFTER innodb_flush_log_at_trx_commit
- 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | REDO log • mysql> show global status like 'Innodb_os_log_written'; • SYS schema: the host_summary_by_file_io_type view Exercise 3 – MEASURE LATENCY mysql> select * from sys.host_summary_by_file_io_type; +------------+--------------------------------------+-------+---------------+-------------+ | host | event_name | total | total_latency | max_latency | +------------+--------------------------------------+-------+---------------+-------------+ | background | wait/io/file/innodb/innodb_data_file | 1607 | 332.96 ms | 39.41 ms | | background | wait/io/file/innodb/innodb_log_file | 171 | 237.56 ms | 17.46 ms | | background | wait/io/file/sql/FRM | 2135 | 1.98 ms | 67.37 us | | background | wait/io/file/sql/casetest | 15 | 926.63 us | 833.10 us | | background | wait/io/file/sql/file_parser | 204 | 296.84 us | 3.58 us | | background | wait/io/file/myisam/kfile | 33 | 51.20 us | 5.19 us | | background | wait/io/file/sql/pid | 3 | 39.02 us | 27.48 us |
- 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool In memory it’s faster 31
- 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool 32 What is it? • InnoDB maintains a storage area called the buffer pool for caching data and indexes in memory • Used to keep frequently accessed data in memory CLIENT BUFFER POOL REDO LOG CLIENT
- 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool mysql> show global status like 'innodb_buffer_pool_pages_total'; mysql> show global status like 'innodb_buffer_pool_pages_free'; mysql> show global status like 'Innodb_buffer_pool_reads'; mysql> show global status like 'Innodb_buffer_pool_read_requests'; mysql> select * from information_schema.INNODB_BUFFER_POOL_STATSG; mysql> select * from sys.schema_table_statistics_with_bufferG; How to Monitor the Buffer Pool
- 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool mysqlslap --no-drop --create-schema=employees --query="select * from employees.employees;" --concurrency=1 --iterations=100 Benchmark Average number of seconds to run all queries: 0.321 seconds Minimum number of seconds to run all queries: 0.265 seconds Maximum number of seconds to run all queries: 0.591 seconds Number of clients running queries: 1 Average number of queries per client: 1 Exercise 4 – STRESS TEST
- 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool mysql> select * from information_schema.INNODB_BUFFER_POOL_STATSG; *************************** 1. row *************************** POOL_ID: 0 POOL_SIZE: 8191 FREE_BUFFERS: 5469 DATABASE_PAGES: 2720 […] PAGES_WRITTEN_RATE: 0 NUMBER_PAGES_GET: 23128215 HIT_RATE: 1000 YOUNG_MAKE_PER_THOUSAND_GETS: 0 NOT_YOUNG_MAKE_PER_THOUSAND_GETS: 0 NUMBER_PAGES_READ_AHEAD: 757 NUMBER_READ_AHEAD_EVICTED: 0 READ_AHEAD_RATE: 0 READ_AHEAD_EVICTED_RATE: 0 LRU_IO_TOTAL: 0 LRU_IO_CURRENT: 0 UNCOMPRESS_TOTAL: 0 UNCOMPRESS_CURRENT: 0 1 row in set (0.00 sec) Exercise 4 – CHECK BUFFER POOL HIT RATE
- 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool EDIT /ETC/MY.CNF AND RESTART THE SERVER 1. innodb_buffer_pool_chunk_size=1048576 2. innodb_buffer_pool_size=5242880 3. shell$ sudo service mysld restart Run mysqlslap and monitor the stats of InnoDB Buffer Pool on a a different console • shell$ mysqlslap --no-drop --create-schema=employees - -query="select * from employees.employees;" -- concurrency=1 --iterations=100 Exercise 4 – SHRINK BUFFER POOL
- 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Buffer Pool mysql> select * from information_schema.INNODB_BUFFER_POOL_STATSG; *************************** 1. row *************************** POOL_ID: 0 POOL_SIZE: 320 FREE_BUFFERS: 0 DATABASE_PAGES: 320 OLD_DATABASE_PAGES: 0 [...] PAGES_CREATE_RATE: 0 PAGES_WRITTEN_RATE: 0 NUMBER_PAGES_GET: 774836 HIT_RATE: 975 YOUNG_MAKE_PER_THOUSAND_GETS: 0 NOT_YOUNG_MAKE_PER_THOUSAND_GETS: 0 NUMBER_PAGES_READ_AHEAD: 19360 NUMBER_READ_AHEAD_EVICTED: 475 READ_AHEAD_RATE: 2639.34016495876 READ_AHEAD_EVICTED_RATE: 65.983504123969 LRU_IO_TOTAL: 1659 LRU_IO_CURRENT: 193 UNCOMPRESS_TOTAL: 0 UNCOMPRESS_CURRENT: 0 1 row in set (0.00 sec) Exercise 4 – CHECK AGAIN BUFFER POOL
- 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Execution plan Don’t waste your time 38
- 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Execution plan • Different aspects to consider to speed up a query – Concurrency, caching, locking, IO…and execution plan • Tools to investigate a slow query – SHOW ENGINE INNODB STATUS – SHOW PROCESSLIST – EXPLAIN – SLOW QUERY LOG – SYS SCHEMA 39
- 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Execution plan mysql> explain select * from employees.employees where adddate(hire_date,INTERVAL 18 YEAR)>=NOW(); +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ | 1 | SIMPLE | employees | NULL | ALL | NULL | NULL | NULL | NULL | 291892 | 100.00 | Using where | +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ 1 row in set, 1 warning (0.00 sec) mysql> create index hire_date_idx on employees(hire_date); Query OK, 0 rows affected (0.99 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> explain select * from employees.employees where adddate(hire_date,INTERVAL 18 YEAR)>=NOW(); +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ | 1 | SIMPLE | employees | NULL | ALL | NULL | NULL | NULL | NULL | 291892 | 100.00 | Using where | +----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+ 1 row in set, 1 warning (0.00 sec) EXERCISE 5 – HAVE A BAD EXECUTION PLAN?
- 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Execution plan mysql> ALTER TABLE employees ADD COLUMN past_date date GENERATED ALWAYS AS (adddate(hire_date,INTERVAL 18 YEAR)) VIRTUAL; Query OK, 0 rows affected (0.03 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE INDEX past_date_idx on employees(past_date); Query OK, 0 rows affected (1.01 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> explain select * from employees.employees where adddate(hire_date,INTERVAL 18 YEAR)>=NOW(); +----+-------------+-----------+------------+-------+---------------+---------------+---------+------+------+----------+------------------------ + | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-----------+------------+-------+---------------+---------------+---------+------+------+----------+------------------------ + | 1 | SIMPLE | employees | NULL | range | past_date_idx | past_date_idx | 4 | NULL | 111 | 100.00 | Using where; Using MRR | +----+-------------+-----------+------------+-------+---------------+---------------+---------+------+------+----------+------------------------ + 1 row in set, 1 warning (0.00 sec) EXERCISE 5 –FUNCTION GENERATED INDEX
- 42. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Wrapping up 1. Use connection pool 2. Tune thread cache / use Thread Pooling plugin (Enterprise version) 3. Configure REDO logging 4. Configure buffer pool to maximize memory usage 5. Analyze execution plan for queries
- 43. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Thank You • MySQL Documentation dev.mysql.com • Support Blog dev.mysql.com/support/blogs 43 Q&A
Editor's Notes
- 1
- I would like to discuss today about the main points a DBA has to consider when addressing the most important topic about service availability: performance. In partcular this will be a walkthrough the most important aspcts and configuration parameters to check, what should be analyzed when our Server is «slow», and understand that we may be stressing our system more than needed, so the perception that better hardware implies better performance, is not always true.
- You may have surely heard that quote saying «Premature optimization is the root of all evil». And this is actually true, as in the moment we deploy our service, we can’t know what will happen in the future, what levels of concurrency will we observe, what will be the first bottlenecks to appear. Nevertheless it’s good to know what parameters we can tune from the very beginning so to forecast future impacts and prevent them from degrading our service quality.
- Let’s have a quick look at the content for this session, after introducing you to the tools we’ll use to perform the Hands On Lab, we’ll show some working examples of configurations to scale connections. This topic will be tackled by addressing connections threshold and threading model. After that, I’ll show how I/O is impacted by the two main actors in a MySQL Server: REDO logging and InnoDB buffer pool. Finally, I’ll show how to improve execution plan for a query with a concrete example. Let’s go!
- In order to perform these simple exercises over Linux OS, I have installed sysstat, available from repository. I have also provisioned MySQL Server with employees DB, which is a reference open source DB useful for a wide set of benchmarks.
- Here’s all the details for Employees DB. Besides that, I will be using mysqlslap util, which is an emulation client to simulate concurrency and to benchmark a server. Following exercises and topics will be based on InnoDB. Server configuration is the default, so I did not edit my.cnf configuration file. I also recommend to restart the server before trying these exercises, so to reset all counters and statistics.
- Let’s have a quick look at the connection model in a MySQL Server. In the default configuration scenario, to one connection corresponds a thread. Thread is in charge of authentication and after that, performs all the needed tasks to execute the query and return results to the client. It is easy to understand where the bottleneck may be if you grow quick with number of connections, as every thread needs resources to execute.
- Remember to make configuration changes persistent to /etc/my.cnf
- We have seen how to increase the maximum number of connections allowed, MySQL Server won’t reject connections if existing concurrent connections will stay within the limits configured. But this is not a good practice, as creating an arbitrary number of connections will provoke rejection sooner or later. A client side solution for intelligent scalability, is using connection pooling. Connection pooling is a technique of creating and managing a pool of connections that are ready for use by any thread that needs them. Connection pooling can increase performance of your application, as pulling an existing connection from the pool will reduce overheads to create and dispose it and will allow for more concurrent clients to access the database.
- Now that we know that a connection needs a thread created on the server to manage authentication and process all the tasks related to query execution, let’s dig a little bit to understand how to improve thread management. One asset to improve concurrency and reduce overhead when a thread is created and disposed, is the thread cache. Manager threads create a new thread when necessary but try to avoid doing so by consulting the thread cache. But… how many threads can fit into the thread cache?
- Thread cache is configurable, and this can be done by setting thread_cache_size. There is also a set of parameters to monitor how thread cache is doing and understand if it is big enough. From previous execution, we see that almost 300 0 thousands threads have been created, this seems high, with the little uptime and number of operations I executed after restart.
- Let’s observe how threads are created when mysqlslap utility is launched again. Now Threads_created has increased by almost 1000!
- Let’s fix this situation and let’s choose a bigger size for our thread cache. With high concurrency, I expect all the threads in the cache will be used, therefore to higher concurrency, I should choose a bigger cache size. After setting a bigger cache size, let’s run again our test. This time I notice just a little increase of around 100 threads. These are the new threads created to be stored into the cache size and will be kept for future operations. Purpose of setting the size of thread cache size, is avoiding new threads are created. So as a rule of thumb, rememember to monitor threads_created when choosing the right size for thread cache.
- So far we have discovered how to increment connections limit and how to cache threads. These tunings can help expand our server capabilities and scale connections beyond what originally planned. Nevertheless, this is not efficient and a bottleneck will be hit after certain number of threads are executing. Solution to scalability passes through using connection pools client side, as mentioned, and adopting an improved threads model, server side. MySQL Enterprise Thread Pool addresses the typical pattern «one thread per connection» by allowing an improved threads model, reducing context switching and contention.
- Let’s talk now about the REDO log, a disk based data structure used during during crash recovery. In practical terms, it’s a set of files, 2 by default, written in circular fashion before a transaction is committed. This represents the «D» in ACID, that is durability. Every change done to the database, must be logged so the transaction can be recovered in case the database crash and changes have not been applied to the tablespace.
- Let’s take a look at REDO logging most important parameters. When talking about REDO, main concern should have the right size so to incur into checkpointing issue. To a too small REDO log, corresponds a high rotation rate and a forced buffer pool checkpoint: we cannot permit transactions logged in the REDO are overwritten when REDO rotates, so InnoDB forces a buffer pool checkpoint: during this phase, all pages not yet synchronized to disk, are flushed. This forced flush can represent a bottleneck, as transactions are stopped until flush completes. Recommendation is to increase REDO log file size accordingly to avoid this mechanism be triggered. Apart from this impact, I would like to show the effect REDO logging has on I/O. When commit-related I/O operations are rearranged and done in batches against REDO log, performance improves. innodb_flush_log_at_trx_commit allows controlling this behavior. When it is set to 1, at every transactions commit corresponds a REDO log flush. By setting to 0, transactions are written and flushed every second. This will reduce overhead dramatically, as will save overhead by doing a flush for every transaction commit. Let’s see the different impacts this tuning has in terms of I/O pressure.
- To show effects on I/O, let’s have two console sessions at hand, on the first, we will execute mysqlslap as usual, and on the second, let’s use iostat to monitor I/O usage. In particular, parameters chosen will set a report every 2 seconds on the storage device where REDO logs are stored.
- Here’s the output, notice the number of write requests and bandwith utilization of the device in the last column.
- Now let’s use a mysql client session to set innodb_flush_log_at_trx_commit=0. Please keep in mind that this exercise helps understand how REDO logging keeps I/O busy, but flushing transactions once per second may provoke up to one second of transactions loss in case of crash, so be careful about using it, typical useful scenarios may be replication slaves or test instances. Let’s run again in the first console mysqlslap. Let’s observe again output for iostat in the second console.
- You can notice how the bandwith, therefore I/O usage has dramatically decreased. When you are binlogging, REDO logging, UNDO logging and checkpointing buffer pool to disk, it is important to monitor I/O so it does not become our bottleneck.
- Just a couple of further details: you can monitor I/O by checking related status variable and also using sys schema, not only for REDO, but for all files written by InnoDB.
- We have seen how important it is to keep I/O usage under control. InnoDB buffer pool is the key layer to increase performance by reducing accesses to disk whenever data is requested by a statement. This storage area is in memory and is a cache to ensure fast access to data and avoid innecessary I/O reads.
- The bigger buffer pool, the better. So as a rule of thumb, buffer pool should be as big as possible to hold all data frequently accessed in memory. There are different rules around, like making buffer pool use 80% of your system memory, but in reality, better option to size buffer pool is monitoring how buffer pool is effectively used. To the purpose, there are different parameters, being statistics from information_schema the most important, we will refer to them in the following example.
- Let’s run once again our mysqlslap benchmark tool in a console session and consult in another session what is reported by information_schema. I can tell mysqlskap to stress my data base instance by using a customer query, so this time I chose a bad query, SELECT * FROM employees is doing a scan and requesting all table content, therefore it will read as much data as possible from the buffer pool and return it to the client.
- Let’s focus at HIT_RATE: as you see, value of 1000 means that all data is successfully retrieved from buffer pool. So far, so good. Our buffer pool with default configuration and this traffic model is good enough. I/O is not accessed to fetch records from pages.
- To show the role of buffer pool, let’s now shrink it to the minimum size, by acting on innodb_buffer_pool_chunk_size and innodb_buffer_pool_size I can resize it at will. After restarting the server (note that buffer pool can be changed online, without any restart, but when chunk size is reconfigured, restart is needed). Let’s run again our benchmarking query and check information_schema
- Here oyu can see pool_size is smaller, and hit_rate is not 1000 anymore. To the smaller buffer pool, more I/O accesses are needed.
- So the typical question we hear at Support is «My query is slow, why?» This problem may be addressed by redesigning a query and by using the right set of indexes so suggest MySQL optimizer the fast execution path to retrieve resulting records. In this case, using the right index for a query corresponds to reduced I/O accesses and better performance. Execution plan is key to understand what is the perfromance for a query. Let’s see it with an example. I have chosen a slighly different than the typical query, based on a function of the column in the where filter clause. Indeed the query may be rewritten for better efficiency, but let’s keep it for educational purposes. By checking the explain plan, a full table scan is performed, almost 300.000 thousands rows are read. Optimizer need an index to filter only those records we need. So why not adding an index on hire_date? We can certainly do that, but it won’t improve our execution plan. What’s happening? Let’s see in the next slide.
- When filter is on a function of the column, optimizer can’t kick in so we need to use an index on a generated column. This translates into: «let’s precalculate the function and add an index on it». After this change, optimizer will finally be able to get only the rows we need. Look at the difference, 300,000 rows versus 111!
- We have gone through some basic tuning to address any immediate performance problem, to wrap up: