Successfully reported this slideshow.
Your SlideShare is downloading. ×

MySQL Performance Best Practices

More Related Content

MySQL Performance Best Practices

  1. 1. MySQL Performance Best Practices Implementing best practices will get you the scalability you need 1 © 2020 Oracle MySQL Olivier Dasini MySQL Principal Solutions Architect EMEA olivier.dasini@oracle.com Twitter : @freshdaz Blogs : www.dasini.net/blog/en : www.dasini.net/blog/fr
  2. 2. Safe harbor statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. 2 © 2020 Oracle MySQL
  3. 3. 3 Me, Myself & I Olivier DASINI Copyright © 2020 Oracle and/or its affiliates. All rights reserved.  MySQL Geek  Addicted to MySQL for 15+ years  Playing with databases for 20+ years  MySQL Writer, Blogger and Speaker  Also former : DBA, Consultant, Architect, Trainer, ...  MySQL Principal Solutions Architect EMEA at Oracle  Stay up to date!  Twitter: @freshdaz  Blog: www.dasini.net/blog/en
  4. 4. Introduction Program agenda 1 2 3 4 Monitoring Configuration Schema Optimization 4 © 2020 Oracle MySQL Query Tuning5
  5. 5. Why MySQL ? 5 © 2020 Oracle MySQL Performance
  6. 6. 6 © 2020 Oracle MySQL The world’s most popular open source database
  7. 7. Some Of Our Customers  MySQL powers the most demanding Web, E-commerce, SaaS and Online Transaction Processing (OLTP) applications  It is a fully integrated transaction-safe, ACID compliant database with full commit, rollback, crash recovery and row level locking capabilities  MySQL delivers the ease of use, scalability, and performance to power: 7 © 2020 Oracle MySQL
  8. 8. Explosive Data Growth 8 © 2020 Oracle MySQL Source IDC, 2017 Source Domo, 2018
  9. 9. MySQL Performance Tuning 9 © 2020 Oracle MySQL Elementary Guidelines Performance
  10. 10. 10 © 2020 Oracle MySQL Dimitri Kravtchuk MySQL Performance Architect The MySQL Performance Best Practice #1 is… ??? USE YOUR BRAIN !!! ;-)
  11. 11. Performance Tuning - Guidelines 1/2  Think – consider what you are doing!  Monitor your system – There is no control without measurement! - MySQL Enterprise Monitor is your best ally – https://www.mysql.com/products/enterprise/monitor.html  Be wary about “best practices” – No two systems are the same – What was true earlier may no longer hold  Ensure you test your changes before deploying to production – The testing must reflect your production usage  Make incremental changes – One change at a time – Relatively small change 11 © 2020 Oracle MySQL
  12. 12. Performance Tuning - Guidelines 2/2  Be mindful of your requirements – Some options give the choice between performance or data safety – what do you need?  Often the default value is the best value - we’ve made huge improvement in MySQL 8.0  Ensure all tables have a PRIMARY KEY - very important for InnoDB  InnoDB organizes the data according to the PRIMARY KEY: – The PRIMARY KEY is included in all secondary indexes in order to be able to locate the actual row – Smaller PRIMARY KEY gives smaller secondary indexes – A mostly sequential PRIMARY KEY is in general recommended to avoid inserting rows between existing rows 12 © 2020 Oracle MySQL
  13. 13. 13 Hardware Schema Changes Data Growth Indexes SQL 90% of Performance Problems Source of Database Performance Problems Copyright © 2020 Oracle and/or its affiliates. All rights reserved.
  14. 14. 14 1 2 3 4 • Visually Identify Slow Queries • Correlation Graphs • Query Response Time index (QRTi) • Execution Statistics • Tune Queries • Add Indexes • Tune Schemas • Improve Caching • MySQL Explain Plan • Sample Query • Query Graphs Better Performance (hopefully) Solving Query Performance Problems Copyright © 2020 Oracle and/or its affiliates. All rights reserved.
  15. 15. Monitoring 15 © 2020 Oracle MySQL https://www.mysql.com/products/enterprise/monitor.html There is no control without measurement! Performance
  16. 16. Monitoring  Your database referential – Fact-based decision making  Real-time performance monitoring → What is happening right now – Take the right decision at the right time  Go back in time → What has happened – Useful to investigate performance issues  Predict the future → Allows you to proactively handle potential issues – Capacity planning  Visual Query Analysis – Examine the query execution plan, see sample queries, see the details of the query execution...  Helps you to enforce best practices  Easy to setup and configure 16 © 2020 Oracle MySQL Expectations
  17. 17. MySQL Enterprise Monitor – Query Analyzer  Real-time query performance  Visual correlation graphs  Find & fix expensive queries  Detailed query statistics  Query Response Time index (QRTi) – "Quality of Service" (QoS) measurement for each query – QoS measurement for a server, group, or every instance – Single metric for query performance 17 © 2020 Oracle MySQL Visual Query Analysis for troubleshooting “With the MySQL Query Analyzer, we were able to identify and analyze problematic SQL code, and triple our database performance. More importantly, we were able to accomplish this in three days, rather than taking weeks.” Keith Souhrada Software Development Engineer Big Fish Games https://www.mysql.com/products/enterprise/query.html
  18. 18. Query Response Time Index (QRTi)  “Quality of Service” (QoS) measurement for each query  QoS measurement for a server, group, or every instance  Single metric for query performance 18 © 2020 Oracle MySQL Identifies queries with unacceptable response times
  19. 19. Best Practice Advisors  Provide a set of rules – Designed to enforce best practices  14 Advisor categories  250+ Advisors  Threshold-based alerts – Exponential moving averages – Rate change detection  Expert problem resolution advice 19 © 2020 Oracle MySQL
  20. 20. MySQL Configuration 20 © 2020 Oracle MySQL Performance
  21. 21. Configuration – Paths  Lots of work has gone into improving the defaults since MySQL 5.6 – The work continues  Some important paths: – datadir – where the data lives by default – default location for: ● Data and indexes for non-InnoDB tables ● InnoDB file-per-table, general, system, and undo log tablespaces – innodb_data_home_dir – default location for the InnoDB system tablespace (ibdata1) – innodb_log_group_home_dir – path to the InnoDB redo log files – innodb_undo_directory – path to the InnoDB undo tablespaces – log_bin – dual functionality: enable binary logging and set path/file name prefix – log_error – makes sure you know where it is 21 © 2020 Oracle MySQL
  22. 22. Configuration – Data Caching  Most important options for performance: – innodb_buffer_pool_size for InnoDB workloads – key_buffer_size for MyISAM workloads 22 © 2020 Oracle MySQL
  23. 23. InnoDB Configuration – innodb_buffer_pool_size  Size of the buffer pool, memory area where InnoDB caches table and index data  A larger buffer pool requires less disk I/O to access the same table data more than once  Optimal size: – innodb_buffer_pool_size for InnoDB workloads – How much memory does the host have? – Subtract memory required by OS and other processes – Subtract memory required by MySQL other then the InnoDB buffer pool – Choose minimum of this and the size of the “working data set”  Since MySQL 5.7, innodb_buffer_pool_size can be changed dynamically 23 © 2020 Oracle MySQL
  24. 24. InnoDB Configuration – Checking the size of your working set 24 © 2020 Oracle MySQL MySQL Enterprise Monitor displays InnoDB Buffer Pool utilization
  25. 25. InnoDB Configuration – Checking the size of your working set 25 © 2020 Oracle MySQL https://lefred.be/content/mysql-and-memory-a-love-story-part-2/ SELECT CONCAT(FORMAT(A.num * 100.0 / B.num,2),"%") BufferPoolFullPct FROM ( SELECT variable_value num FROM performance_schema.global_status WHERE variable_name = 'Innodb_buffer_pool_pages_data' ) A INNER JOIN ( SELECT variable_value num FROM performance_schema.global_status WHERE variable_name = 'Innodb_buffer_pool_pages_total' ) B; Verify how much the InnoDB Buffer Pool is filled with data
  26. 26. InnoDB Configuration – innodb_buffer_pool_instances  Specifies how many instances to split the buffer pool into – Can reduce contention for concurrent workload ie improve the scalability on a busy server – This option takes effect only when you set innodb_buffer_pool_size to a size of 1GB or more. – For best efficiency, specify a combination of innodb_buffer_pool_instances & innodb_buffer_pool_size so that each buffer pool instance is at least 1GB – Max allowed value: 64  Dimitri suggestion – # BP instances = max ( 4, CPU cores / 2 )  But test with your workload!  Requires restart 26 © 2020 Oracle MySQL https://dev.mysql.com/doc/refman/8.0/en/innodb-multiple-buffer-pools.html
  27. 27. InnoDB Configuration – Saving and Restoring the Buffer Pool State  To reduce the warmup period after restarting the server, InnoDB saves a percentage of the most recently used pages for each buffer pool at server shutdown and restores these pages at server startup – innodb_buffer_pool_dump_at_shutdown - default: ON – innodb_buffer_pool_dump_pct - default: 25 (percentage of the most recently used pages) – Page loading happens in the background, and does not delay database startup  You can also save and restore the buffer pool state at any time, while the server is running – SET GLOBAL innodb_buffer_pool_dump_now=ON; -- Save the state of the buffer pool – SET GLOBAL innodb_buffer_pool_load_now=ON; -- Restore the buffer pool state – SET GLOBAL innodb_buffer_pool_load_abort=ON; -- Abort a buffer pool load operation – SHOW STATUS LIKE 'Innodb_buffer_pool_dump_status'; -- Display progress when saving the buffer pool state – SHOW STATUS LIKE 'Innodb_buffer_pool_load_status'; -- Display progress when loading the buffer pool 27 © 2020 Oracle MySQL https://dev.mysql.com/doc/refman/8.0/en/innodb-preload-buffer-pool.html
  28. 28. InnoDB Configuration – innodb_dedicated_server  When innodb_dedicated_server is enabled, InnoDB automatically configures the following variables: – innodb_buffer_pool_size – innodb_log_file_size – innodb_log_files_in_group – innodb_flush_method  Only consider enabling innodb_dedicated_server if the MySQL instance resides on a dedicated server where it can use all available system resources – e.g. Running MySQL Server in a Docker container or dedicated VM that only runs MySQL  Enabling innodb_dedicated_server is not recommended if the MySQL instance shares system resources with other applications 28 © 2020 Oracle MySQL Enabling Automatic Configuration for a Dedicated MySQL Server https://dev.mysql.com/doc/refman/8.0/en/innodb-dedicated-server.html
  29. 29. Configuration – Other  max_connections – Maximum permitted number of simultaneous client connections – Be careful setting this too large as each connection requires memory  table_open_cache – Number of open tables for all threads – Increasing this value increases the number of file descriptors that mysqld requires – Each table can be open more than once – You can check whether you need to increase the table cache by checking the Opened_tables status variable ● If the value of Opened_tables is large and you do not use FLUSH TABLES often, then you should increase the value of the table_open_cache variable  table_open_cache & max_connections affect the maximum number of files the server keeps open – If you increase one or both of these values, you may run up against a limit imposed by your operating system on the per-process number of open file descriptors – open_files_limit ● The number of file descriptors available to mysqld from the operating system ● On Unix, the value cannot be set greater than the value displayed by the ulimit -n command 29 © 2020 Oracle MySQL
  30. 30. Configuration – join_buffer_size 30 © 2020 Oracle MySQL  Minimum size of the buffer that is used for plain index scans, range index scans, joins that do not use indexes, hash joins and when Block Nested-Loop or Batched Key Access algorithm is used  It is better to keep the global setting small and change the session setting to a larger value only in sessions  Or change the setting on a per-query basis by using a SET_VAR optimizer hint  Memory allocation time can cause substantial performance drops if the global size is larger than needed by most queries that use it  Usually default value is OK
  31. 31. Configuration – sort_buffer_size 31 © 2020 Oracle MySQL  Used by each session that must perform a sort  At minimum the sort_buffer_size value must be large enough to accommodate 15 tuples in the sort buffer  If many sort_merge_passes per second, consider increasing the sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization or improved indexing  Best to keep the global setting small and change the session setting to a larger value only in sessions  Memory allocation time can cause substantial performance drops if the global size is larger than needed by most queries that use it
  32. 32. InnoDB Configuration – InnoDB Redo Log  Disk-based data structure used during crash recovery to correct data written by incomplete transactions  Physically represented on disk by two files named ib_logfile0 and ib_logfile1  Total redo log size defined by two options: innodb_log_file_size & innodb_log_files_in_group  Total size = innodb_log_file_size * innodb_log_files_in_group – Max supported total redo log size is just below 512G since MySQL 5.6  Should be large enough to avoid “excessive” checkpointing – Make your redo log files big, even as big as the buffer pool – The larger the value, the less checkpoint flush activity is required in the buffer pool, saving disk I/O – Larger log files also make crash recovery slower  With large write transactions increase the size of the log buffer - innodb_log_buffer_size – A large log buffer enables large transactions to run without a need to write the log to disk before the transactions commit – if you have transactions that update, insert, or delete many rows, making the log buffer larger saves disk I/O  Dedicated log writer threads can improve performance on high-concurrency systems, but for low- concurrency systems, disabling dedicated log writer threads provides better performance – Enable or disable log writer threads using : innodb_log_writer_threads . 32 © 2020 Oracle MySQL https://dev.mysql.com/doc/refman/8.0/en/optimizing-innodb-logging.html
  33. 33. InnoDB Configuration – InnoDB Redo Log 33 © 2020 Oracle MySQL MySQL Enterprise Monitor displays InnoDB Redo Log usage
  34. 34. InnoDB Configuration – Disabling Redo Logging  As of MySQL 8.0.21, you can disable redo logging using – ALTER INSTANCE DISABLE INNODB REDO_LOG  This functionality is intended for loading data into a new MySQL instance  Disabling redo logging speeds up data loading by avoiding redo log writes and doublewrite buffering  This feature is intended only for loading data into a new MySQL instance  Do not disable redo logging on a production system  Details: – https://dev.mysql.com/doc/refman/8.0/en/innodb-redo-log.html#innodb-disable-redo-logging 34 © 2020 Oracle MySQL
  35. 35. Data Consistency vs Performance 35 © 2020 Oracle MySQL Trading safety for performance? Performance
  36. 36. innodb_flush_log_at_trx_commit  Controls the balance between strict ACID compliance for commit operations & higher performance  You can achieve better performance by changing the default value but then you can lose transactions in a crash  Possible values in order of data safety: – 1: Logs are written and flushed to disk at each transaction commit ● Is theoretically the slowest, but with fast SSD it may be around as fast as 2 and 0 – 2: Logs are written after each transaction commit and flushed to disk once per second ● Transactions for which logs have not been flushed can be lost in a crash (host) – 0: NOT TO USE (basically nothing is written on any commit and log buffer is written when filled) ● You can easily loose everything...  Defaults to 1; Logs are written and flushed to disk at each transaction commit – Required for full ACID compliance - D in ACID – For this reason it is the recommended value  Log flushing frequency is controlled by innodb_flush_log_at_timeout 36 © 2020 Oracle MySQL
  37. 37. innodb_flush_log_at_trx_commit  What if innodb_flush_log_at_trx_commit = 1 is too slow and you want the D in ACID? – Make sure the redo logs are on separate disks ● Do not be tempted to have the individual redo logs on different disks – Consider SSD particularly if you have a high commit rate – Battery-backed disk cache also makes flushes cheaper 37 © 2020 Oracle MySQL
  38. 38. sync_binlog  Controls how often the MySQL server synchronizes the binary log to disk – 0: Disables synchronization of the binary log to disk ● MySQL relies on the operating system to flush the binary log to disk ● Provides the best performance, but in the event of a power failure or operating system crash, it is possible that the server has committed transactions that have not been synchronized to the binary log – 1: Enables synchronization of the binary log to disk before transactions are committed ● The safest setting but can have a negative impact on performance due to the increased number of disk writes ● Guarantees that no transaction is lost from the binary log – N: The binary log is synchronized to disk after N binary log commit groups have been collected ● Can have a negative impact on performance due to the increased number of disk writes ● A higher value improves performance, but with an increased risk of data loss  Default value: 1 38 © 2020 Oracle MySQL
  39. 39. sync_binlog  sync_binlog != 1 means that replicas most likely will have to be rebuild if the primary crashes – CLONE plugin, for cloning data from a MySQL server instance, will help  But sync_binlog = 0 must be the best performance wise, right? – By default: max_binlog_size = 1G – 1G is not a lot of memory nowadays – So the OS may buffer the entire binary log – When rotating the binary log, up to 1G will be flushed to disk ● Stalls all other commits until the binary log rotation is done  So in short: – sync_binlog = 0 may give the best throughput – sync_binlog = 1 may give the most predictable performance 39 © 2020 Oracle MySQL
  40. 40. Durability & consistency with MySQL Replication  For durability and consistency in a replication setup that uses InnoDB with transactions: – If binary logging is enabled, set sync_binlog=1 – Always set innodb_flush_log_at_trx_commit=1 40 © 2020 Oracle MySQL
  41. 41. Schema Optimization 41 © 2020 Oracle MySQL Most important factor in making a database application fast Performance
  42. 42. Design Questions 42 © 2020 Oracle MySQL  Starting with an efficient database design makes it easier for team members to write high-performing application code – Do the columns have the right data types & clauses? – Does each table have the appropriate columns for the type of work? – Do the tables have a primary key? – Are the right indexes in place to make queries efficient?
  43. 43. Schema Optimization Tips 1/2 43 © 2020 Oracle MySQL  Design your tables to minimize their space on the disk <=> use more caching, thus minimize disk seeks – Reduce the amount of data written to and read from disk – Smaller tables normally require less main memory – Smaller indexes that can be processed faster  Use the most efficient (smallest) data types possible – e.g. MEDIUMINT vs INT (MEDIUMINT uses 25% less space than INT) or BIGINT, TINYINT (UNSIGNED ?)  Declare columns to be NOT NULL if possible – It makes SQL operations faster, by enabling better use of indexes & eliminating overhead for testing whether each value is NULL – You also save some storage space Look for the most efficient way to organize your data
  44. 44. Schema Optimization Tips 2/2 44 © 2020 Oracle MySQL  The primary key of a table should be as short as possible – This makes identification of each row easy and efficient – For InnoDB tables, the PK columns are duplicated in each secondary index entry, so a short PK saves considerable space if you have many secondary indexes.  Create only the indexes that you need to improve query performance – Indexes are good for retrieval, but slow down insert and update operations – Indexes use disk & memory spaces – Sometime it can be beneficial to use column prefix key parts (especially for long string column)  Declare columns with identical information in different tables with identical data types, charsets & collations – Speed up joins based on the corresponding columns Look for the most efficient way to organize your data
  45. 45. MySQL Query Tuning 45 © 2020 Oracle MySQL Make the best use of your data Performance
  46. 46. MySQL Optimizer (simplified) 46 © 2020 Oracle MySQL
  47. 47. Input to Cost Model  IO-cost – Estimates from storage engine based on number of pages to read – Both index and data pages 47 © 2020 Oracle MySQL  Statistics – Number of records in table – Key distribution/Cardinality: ● Average number of records per key value ● Only for indexed columns ● Maintained by storage engine – Number of records in an index range – Percentage of table/index in InnoDB buffer pool  Schema – Length of records and keys – Uniqueness for indexes – Nullability
  48. 48. InnoDB Persistent Optimizer Statistics 48 © 2020 Oracle MySQL  Persisted across server restarts: innodb_stats_persistent (ON by default) – Greater plan stability – More accurate statistics – Consistent query performance – Less variance between servers  Automatically recalculates statistics after significant changes: innodb_stats_auto_recalc – Threshold value is 10% of the rows in the table – May turn off automatic recalculations  ANALYZE TABLE forces recalculation of statistics  May manage precision by changing number of samples: innodb_stats_persistent_sample_pages – If statistics are not accurate enough and the optimizer chooses suboptimal plans – If ANALYZE TABLE is too slow
  49. 49. Indexes 49 © 2020 Oracle MySQL  The best way to improve the performance of SELECT operations is to create relevant indexes on one or more of the columns that are tested in the query  The index entries act like pointers to the table rows, allowing the query to quickly determine which rows match a condition in the WHERE clause, and retrieve the other column values for those rows  All MySQL data types can be indexed  Although it can be tempting to create an indexes for every possible column used in a query, unnecessary indexes waste space and waste time for MySQL to determine which indexes to use  Indexes also add to the cost of INSERT, UPDATE & DELETE because each index must be updated  You must find the right balance to achieve fast queries using the optimal set of indexes
  50. 50. 50 © 2020 Oracle MySQL
  51. 51. Multi-Valued Indexes 51 © 2020 Oracle MySQL  InnoDB supports multi-valued indexes  A multi-valued index is a secondary index defined on a column that stores an array of values  A multi-valued index can have multiple index records for a single data record  Multi-valued indexes are intended for indexing JSON arrays  The optimizer uses a multi-valued index to fetch records when the following functions are specified in a WHERE clause: – MEMBER OF() – JSON_CONTAINS() – JSON_OVERLAPS() https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-multi-valued SELECT * FROM customers -> WHERE 94507 MEMBER OF(custinfo->'$.zipcode'); +----+---------------------+-------------------------------------------------------------------+ | id | modified | custinfo | +----+---------------------+-------------------------------------------------------------------+ | 2 | 2019-06-29 22:23:12 | {"user": "Jill", "user_id": 22, "zipcode": [94568, 94507, 94582]} | | 3 | 2019-06-29 22:23:12 | {"user": "Bob", "user_id": 31, "zipcode": [94477, 94507]} | | 5 | 2019-06-29 22:23:12 | {"user": "Ted", "user_id": 56, "zipcode": [94507, 94582]} | +----+---------------------+-------------------------------------------------------------------+
  52. 52. Unused & Redundant Indexes 52 © 2020 Oracle MySQL  These 2 views, from sys schema, will help you to spot index issues – schema_redundant_indexes: indexes that duplicate other indexes or are made redundant by them – schema_unused_indexes: indexes for which there are no events, which indicates that they are not being used ● Most useful when the server has been up and processing long enough that its workload is representative. Otherwise, presence of an index in this view may not be meaningful.  The next 2 indicates statements that have done full table scans: – schema_tables_with_full_table_scans: display which tables are being accessed with full table scans – statements_with_full_table_scans: display normalized statements that have done full table scans Unnecessary indexes waste space and waste time
  53. 53. Some Reasons Why Index can NOT be Used 53 © 2020 Oracle MySQL  Indexed column is used as argument to function – YEAR(o_orderdate) = 1997 – => Use functional indexes : https://dasini.net/blog/2019/03/14/mysql-functional-indexes  Looking for a suffix : – name LIKE '%son'  First column(s) of compound index NOT used – b = 10 when index defined over (a, b)  Type mismatch – my_string = 10  Character set / collation mismatch – t1 LEFT JOIN t2 ON t1.utf8_string = t2. latin1_string
  54. 54. Histogram 54 © 2020 Oracle MySQL  The server stores histogram statistics about column values, for use by the optimizer in constructing query execution plans  The optimizer applies histogram statistics to determine row estimates based on the selectivity (filtering effect) of column value comparisons against constant values  Histogram statistics are useful primarily for nonindexed columns – Adding an index to a column for which histogram statistics are applicable might also help the optimizer make row estimates – A histogram is created or updated only on demand, so it adds no overhead when table data is modified – On the other hand, the statistics become progressively more out of date when table modifications occur, until the next time they are updated  The optimizer prefers range optimizer row estimates to those obtained from histogram statistics – If the optimizer determines that the range optimizer applies, it does not use histogram statistics  To perform histogram management, use the ANALYZE TABLE https://dev.mysql.com/doc/refman/8.0/en/optimizer-statistics.html
  55. 55. Histogram Tips 55 © 2020 Oracle MySQL https://dev.mysql.com/doc/refman/8.0/en/analyze-table.html#analyze-table-histogram-statistics-analysis -- Update (or create) an histogram on column salaries.to_date ANALYZE TABLE salaries UPDATE HISTOGRAM ON to_date; -- Extract values from the histogram objects SELECT TABLE_NAME, COLUMN_NAME, HISTOGRAM->>'$."data-type"' AS 'data-type', JSON_LENGTH(HISTOGRAM->>'$."buckets"') AS 'bucket-count' FROM INFORMATION_SCHEMA.COLUMN_STATISTICS;
  56. 56. InnoDB Tips 56 © 2020 Oracle MySQL  InnoDB primary key is included in all secondary indexes in order to be able to locate the actual row – Smaller primary key gives smaller secondary indexes – A mostly sequential primary key is in general recommended to avoid inserting rows between existing rows – Alternative: create an AUTO_INCREMENT column as the primary key if your primary key is long, or index a prefix of a long VARCHAR column instead of the entire column  InnoDB automatically extends each secondary index by appending the primary key columns to it – The optimizer takes into account the primary key columns of the extended secondary index when determining how and whether to use that index – This can result in more efficient query execution plans and better performance – Could be disable with : SET optimizer_switch = 'use_index_extensions=off';  When you use a randomly generated value as a primary key in an InnoDB table, prefix it with an ascending value such as the current date and time if possible – When consecutive primary values are physically stored near each other, InnoDB can insert and retrieve them faster  It is usually better to create a small number of concatenated indexes rather than a large number of single-column indexes – If an index contains all the columns needed for the result set (ie covering index), the query might be able to avoid reading the table data at all
  57. 57. MySQL Query Tuning 57 © 2020 Oracle MySQL Analyzing Queries Performance
  58. 58. Analyzing Queries – MySQL Enterprise Monitor - Query Analyzer 58 © 2020 Oracle MySQL Visual Query Analysis for troubleshooting  Real-time query performance  Find & fix expensive queries  Detailed query statistics  Query Response Time index (QRTi)
  59. 59. Analyzing Queries – EXPLAIN 59 © 2020 Oracle MySQL  The set of operations that the optimizer chooses to perform the most efficient query is called the Query Execution Plan or EXPLAIN plan  MySQL explains how it would process the statement, including information about how tables are joined and in which order  Returns a row of information for each table used in the statement  Lists the tables in the output in the order that MySQL would read them while processing the statement – You can see where you should add indexes to tables so that the statement executes faster by using indexes to find rows – You can check whether the optimizer joins the tables in an optimal order  EXPLAIN works with SELECT, DELETE, INSERT, REPLACE, and UPDATE statements Provide information about how MySQL executes statements
  60. 60. 60 © 2020 Oracle MySQL
  61. 61. Analyzing Queries – EXPLAIN 61 © 2020 Oracle MySQL EXPLAIN SELECT distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000 *************************** 1. row *************************** id: 1 select_type: SIMPLE table: salaries partitions: NULL type: ALL possible_keys: PRIMARY key: NULL key_len: NULL ref: NULL rows: 2838426 filtered: 0.00 Extra: Using where; Using temporary *************************** 2. row *************************** id: 1 select_type: SIMPLE table: titles partitions: NULL type: ref possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: employees.salaries.emp_no rows: 1 filtered: 100.00 Extra: Using index Provide information about how MySQL executes statements No index used for table salaries Full table scan Almost 3M rows examined (estimation)
  62. 62. Analyzing Queries – MySQL Workbench Visual EXPLAIN 62 © 2020 Oracle MySQL Provide information about how MySQL executes statements Cost per table Table & index Rows per lookup Total query cost Accumulated cost
  63. 63. MySQL Workbench EE  Database migrations – From Microsoft SQL Server, PostgreSQL, Sybase ASE, Sybase SQL Anywhere, SQLite, Microsoft Access, and more  Manage migration projects  Source and target selection  Object migration  Data migration  MySQL version upgrades 63 © 2020 Oracle MySQL Database Migration Wizard for SQL Server, Sybase, SQLite, Sybase & PostgreSQL https://www.mysql.com/products/workbench/
  64. 64. Analyzing Queries – EXPLAIN ANALYZE 64 © 2020 Oracle MySQL  Run a statement and produces EXPLAIN output along with timing and additional, iterator-based, information about how the optimizer's expectations matched the actual execution  The following information is provided: – Estimated execution cost – Estimated number of returned rows – Time to return first row – Time to return all rows (actual cost), in milliseconds – Number of rows returned by the iterator – Number of loops  Can be used with SELECT statements, as well as with multi-table UPDATE and DELETE statements Provide information about how MySQL executes statements https://dev.mysql.com/doc/refman/8.0/en/explain.html
  65. 65. Analyzing Queries – EXPLAIN 65 © 2020 Oracle MySQL Provide information about how MySQL executes statements EXPLAIN ANALYZE SELECT distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000 *************************** 1. row *************************** EXPLAIN: -> Table scan on <temporary> (actual time=0.002..0.003 rows=2 loops=1) -> Temporary table with deduplication (cost=10.62 rows=10) (actual time=5.939..5.939 rows=2 loops=1) -> Nested loop inner join (cost=10.62 rows=10) (actual time=4.087..5.880 rows=12 loops=1) -> Filter: (salaries.salary > 155000) (cost=2.42 rows=7) (actual time=0.069..0.085 rows=7 loops=1) -> Index range scan on salaries using idx_salary (cost=2.42 rows=7) (actual time=0.065..0.075 rows=7 loops=1) -> Index lookup on titles using PRIMARY (emp_no=salaries.emp_no) (cost=1.04 rows=1) (actual time=0.823..0.826 rows=2 loops=7)
  66. 66. MySQL Query Tuning 66 © 2020 Oracle MySQL Controlling the Query Optimizer Performance
  67. 67. Controlling the Query Optimizer 67 © 2020 Oracle MySQL When the optimizer does not do what you think it should do  Affect how query plans are evaluated  Add indexes and/or performs a key distribution analysis: run ANALYZE TABLE  Optimizer Hints - can be specified within individual statements – Join-Order Optimizer Hints – Table-Level Optimizer Hints – Index-Level Optimizer Hints – Subquery Optimizer Hints – Statement Execution Time Optimizer Hints...  Switchable Optimizations: – optimizer_switch flags: enable control over optimizer behavior  Controlling Query Plan Evaluation – Control how exhaustive the optimizer is in its search for an optimal query evaluation plan – The fewer plans that are investigated by the optimizer, the less time it spends in compiling a query ● But it may miss finding an optimal plan – optimizer_prune_level: tells the optimizer to skip certain plans based on estimates of the number of rows accessed for each table – optimizer_search_depth: tells how far into the “future” of each incomplete plan the optimizer should look to evaluate whether it should be expanded further https://dev.mysql.com/doc/refman/8.0/en/controlling-optimizer.html
  68. 68. Index-Level Optimizer Hints - example 68 © 2020 Oracle MySQL https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html#optimizer-hints-index-level SELECT /*+ NO_INDEX(salaries idx_salary) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000; SELECT /*+ NO_INDEX(salaries PRIMARY) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000; SELECT /*+ NO_INDEX(salaries) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000; SELECT /*+ INDEX(salaries PRIMARY) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000; SELECT /*+ INDEX(salaries PRIMARY) NO_INDEX(titles PRIMARY) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000;
  69. 69. Variable-Setting Hint Syntax 69 © 2020 Oracle MySQL  The SET_VAR hint sets the session value of a system variable temporarily ie for the duration of a single statement  It permits setting only a single variable, but multiple hints can be given to set multiple variables https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html#optimizer-hints-set-var SELECT /*+ SET_VAR(join_buffer_size=524288) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000; SELECT /*+ SET_VAR(join_buffer_size=524288) SET_VAR(sort_buffer_size=786432) */ distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000;
  70. 70. And also... 70 © 2020 Oracle MySQL Performance
  71. 71. Query Rewrite Plugin 71 © 2020 Oracle MySQL  Rewrite problematic queries without the need to make application changes – Add hints – Modify join order...  Statements subject to rewriting: SELECT, INSERT, REPLACE, UPDATE & DELETE  Standalone statements and prepared statements are subject to rewriting  Statements occurring within view definitions or stored programs are not subject to rewriting  e.g. Adding Rewrite Rules: INSERT INTO query_rewrite.rewrite_rulesquery_rewrite.rewrite_rules (pattern, replacement ) VALUES ("SELECT sum(salary) FROM employees.salaries WHERE from_date BETWEEN ? AND ?", "SELECT /*+ INDEX(salaries idx_from_date) */ sum(salary) FROM employees.salaries WHERE from_date BETWEEN ? AND ?");  Information: – https://dev.mysql.com/doc/refman/8.0/en/rewriter-query-rewrite-plugin.html – www.dasini.net/blog/2016/02/25/30-mins-with-mysql-query-rewriter/
  72. 72. Consider the Whole Stack 72 © 2020 Oracle MySQL The issue may not be in MySQL, or may be an interaction issue Performance
  73. 73. Consider the Whole Stack  The issue may occur at any place in the stack, e.g.: – Application – Application host/hardware – Network between application and host with MySQL – MySQL host/hardware – MySQL  This should also be considered when monitoring  Consider the OS/hardware settings 73 © 2020 Oracle MySQL
  74. 74. Take away  MySQL performance tuning is similar to all other performance tuning: – Premature optimization is bad – Only change one thing at a time and don’t make too large changes – One size does not fit all – Base your decisions on measurements (monitoring) – Understand what an option does before changing it – Understand your system (data) – Understand what you need – Consider the whole stack  MySQL team can help you 74 © 2020 Oracle MySQL Performance
  75. 75. MySQL Enterprise Edition 75 © 2020 Oracle MySQL https://www.youtube.com/watch?v=CtRIc6Hsia8 We’ve Got You Covered Performance
  76. 76. MySQL Enterprise Support  Largest MySQL engineering and support organization  Backed by the MySQL developers  World-class support, in 29 languages  Hot fixes & maintenance releases  24x7x365  Unlimited incidents  Consultative support  Global scale and reach 76 © 2020 Oracle MySQL Get immediate help for any MySQL issue, plus expert advice Get immediate help for any MySQL issue, plus expert advice https://www.mysql.com/fr/support/
  77. 77. MySQL Enterprise Consultative Support  Remote troubleshooting  Replication review  Partitioning review  Schema review  Query review  Performance tuning  ...and more 77 © 2020 Oracle MySQL https://www.mysql.com/fr/support/consultative.html Make the Most of your Deployments
  78. 78. 78 © 2020 Oracle MySQL Questions? olivier.dasini@oracle.com
  79. 79. Resources – MySQL Blogs 79 © 2020 Oracle MySQL • MySQL 8.0 Optimization Documentation – https://dev.mysql.com/doc/refman/8.0/en/optimization.html • Optimization for InnoDB tables – https://dev.mysql.com/doc/refman/8.0/en/optimizing-innodb.html • MySQL Server Team – http://mysqlserverteam.com/ • MySQL Performance Blog – http://dimitrik.free.fr/blog/ • MySQL High Availability – http://mysqlhighavailability.com/ • MySQL Community Manager – https://lefred.be/ • MySQL Engineering Team – https://insidemysql.com/ • Oracle MySQL Blog – https://blogs.oracle.com/mysql/
  80. 80. Resources – MySQL Features 80 © 2020 Oracle MySQL • MySQL Document Store – https://dev.mysql.com/doc/x-devapi-userguide/en/ – https://github.com/mysql/mysql-connector-nodejs – https://www.npmjs.com/package/@mysql/xdevapi – https://dev.mysql.com/doc/refman/8.0/en/document-store.html – https://dev.mysql.com/doc/x-devapi-userguide/en/ – https://dev.mysql.com/doc/dev/connector-nodejs/8.0/ – https://www.mysql.com/news-and-events/web-seminars/mysql-document-store-and-node-js/ • MySQL Performance by Dimitri – http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-over-18m-qps-with-80-ga-on-2s-skylake.html – http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-80-and-utf8-impact.html • MySQL InnoDB Cluster – https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/ – https://mysqlserverteam.com/mysql-innodb-cluster-whats-new-in-the-8-0-ga-release/ – https://mysqlserverteam.com/mysql-innodb-cluster-8-0-a-hands-on-tutorial/
  81. 81. Resources – MySQL Features 81 © 2020 Oracle MySQL • Common Table Expressions (CTE) – http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/ – http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-two-how-to-generate-series/ – http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-three-hierarchies/ – http://mysqlserverteam.com/mysql-8-0-1-recursive-common-table-expressions-in-mysql-ctes-part-four-depth-first-or-breadth-first-traversal- transitive-closure-cycle-avoidance/ • Window Functions – http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/ – https://mysqlserverteam.com/row-numbering-ranking-how-to-use-less-user-variables-in-mysql-queries/ – https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777 • Security – https://www.mysql.com/news-and-events/web-seminars/checking-and-hardening-mysql-8-0-security-via-sql-and-without-os-access/ – https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/ – http://lefred.be/content/mysql-8-0-listing-roles/ – https://dev.mysql.com/doc/refman/8.0/en/password-management.html • Upgrade from 5.7 to 8.0 – https://mysqlserverteam.com/inplace-upgrade-from-mysql-5-7-to-mysql-8-0/ – https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/
  82. 82. Thank you 82 © 2020 Oracle MySQL Olivier Dasini MySQL Principal Solutions Architect EMEA olivier.dasini@oracle.com Twitter : @freshdaz Blog : www.dasini.net/blog/en

×