Morgan Tocker, profile picture
Uploaded byMorgan Tocker
PDF, PPTX2,805 views

MySQL 5.6 - Operations and Diagnostics Improvements

This document discusses MySQL 5.6 and its improvements to operational and diagnostic capabilities. Key enhancements include online DDL operations that do not block reads or writes, buffer pool dump and restore for faster startup, import/export of partitioned tables, and transportable tablespaces. Diagnostic tools were improved with EXPLAIN showing more details, the ability to EXPLAIN updates and deletes, optimizer tracing, and the performance schema providing detailed query level instrumentation and monitoring by default.

Embed presentation

Download as PDF, PPTX
<Insert Picture Here> MySQL 5.6 - Online Operations and Improved Diagnostics
 Morgan Tocker, MySQL Community Manager
 http://www.tocker.ca/

Safe Harbor Statement The  following  is  intended  to  outline  our  general  product  direction.   It  is  intended  for  information  purposes  only,  and  may  not  be   incorporated  into  any  contract.  It  is  not  a  commitment  to  deliver   any  material,  code,  or  functionality,  and  should  not  be  relied  upon   in  making  purchasing  decisions.   
 The  development,  release,  and  timing  of  any  features  or   functionality  described  for  Oracle’s  products  remains  at  the  sole   discretion  of  Oracle.
4 Years of MySQL Innovation MySQL Cluster 7.3 MySQL Migration Wizard MySQL Workbench 6.0 MySQL 5.6 MySQL 5.5 Windows installer & Tools MySQL 5.7 M y S Q L Cluster MySQL MySQL Enterprise Monitor 2.3 & e r M a n a g 3.0 Applier for Hadoop MySQL Enterprise Backup Security MySQL Utilities MySQL Workbench 5.2 & 6.0 Scalability MySQL Cluster 7.2 HA MySQL Enterprise MySQL Cluster 7.1 O r a c l e C e r t i f i c a t i o n s Audit
Agenda for Today • Quick Intro to MySQL 5.6 • Operational Enhancements • Diagnostic Enhancements
Quick  Intro  to  5.6 Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
The Path to MySQL 5.6 DMR1- DMR2 DMR3 DMR4 DMR5 RC April 2011 Oct 2011 Dec 2011 Aug 2012 Aug 2012 Nov 2012 Optimizer: Optimizer: Optimizer: Optimizer: InnoDB: MRR, ICP, File Sort BKA, New EXPLAIN, Traces Sub-Queries JSON EXPLAIN, Sub-Queries TT, Online DDL, Memcached API New Server Defaults InnoDB: InnoDB: Replication: Replication: Split Kernel Mutex, MT Purge Dump/Restore Buffer Pool GTIDs Binary Log Group Commit Replication: More P_S ! Crash-Safe, MultiThread Slave, Checksums Memcached API New P_S Partitioning Improvements ! ! InnoDB: Full Text Index, Read-Only Optimizations Condition Handling Fractional Seconds ! TIMESTAMP & DATESTAMP More P_S ! Password Mgmt More Partitioning ! More Partitioning More GTID, TT
By Lines of Code • “MySQL 5.6 is the largest MySQL code size increase in a MySQL version ever. The last time we saw anything like this was with the merging of MySQL Cluster in 4.1. At the very least, Oracle is paying people to write lines of code to extent that nobody has before.” Stewart Smith
By Bugs Fixed • 1991 Bugs Fixed in 5.6 • 3763 Bugs Fixed in Total Since MySQL 5.5 GA
By Major Features • • • • • • • • • Better Performance and Scalability Better Transactional Throughput Better Performance with Solid State Drives Better Query Execution Times and Diagnostics Better Application Availability NoSQL Access to InnoDB InnoDB Fulltext Search Improved Replication and High Availability Improved Performance Schema
By Actual Features / Enhancements Scalable Read Only Transactions Improvement to Buffer Pool Flushing Precise spatial operations in GIS Concurrent Innodb data file extension Subquery Optimizations Password hashes instead of plain passwords in Query Logs Non-Recursive Deadlock Detection More efficient Optimizer SHA256 hashing with Salt for Authentication Faster Locking Primitives Optimized ROW Based Replication Use obfuscated password storage for command line tools Improved Innodb Thread Concurrency Multi-Threaded Slave Policy Based password validation Multiple background Purge Threads Global Transaction Identifiers Plugin authentication support for Replication Improved Purge lag control (now works) Crash Safe Slave and Binlog INNODB_METRICS (I_S) Split of “Kernel Mutex” Replication Event Checksums Meta Data Information Tables (I_S) Data Dictionary Cache Time Delayed Replication Buffer Pool Information Tables (I_S) Improved Adaptive Flushing Server UUID Reduced Overhead (PS) Page Cleaner/Separate Flush Thread Improved Logging for Row based Replication Simplified Configuration (PS) Group Commit for Binary Log Replication Utilities for Failover and Admin Table Access instrumentation (PS) Fight Cache Coherence and False Sharing issues Separate Tablespaces for Innodb Undo Logs Statements instrumentation (PS) Reduced Innodb Memory Fragmentation Fast Restart – Preloading Innodb Buffer Pool Stages Instrumentation (PS) Reduced Locking for Partitioned tables Online DDL Aggregations by User, Host etc (PS) Reduced Contention for LOCK_open Import/Export for Partitioned Tables Network IO Instrumentation (PS) Support for multiple table_open_cache instances Remote Binlog Backup Show Host Cache Contents (PS) Large (over 4GB) redo logs support Innodb Transportable Tablespaces Improved File I/O Instrumentation (PS) Index Condition pushdown (ICP) New configuration files defaults Explain for UPDATE/DELETE queries Multi-Range-Read (MRR) User Defined DATA DIRECTORY for Innodb Tables JSON output with more information Faster ORDER BY nidxcol LIMIT N Connection Attributes Optimizer Tracing Persistent Statistics for Innodb MemcacheD API in Innodb Deadlock Logging Improvements to Innodb Compression Explicit Partition Selection in queries GET DIAGNOSTICS Fast Page Checksums (CRC32) Full Text Search index for Innodb 4K and 8K Page sizes for Innodb Microsecond TIME precision
Operational  Enhancements Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
InnoDB Online DDL The beginning All DDL requires the table to be recreated, along with all indexes. MySQL 5.1 Nov 2008 MySQL 5.5 Dec 2010 MySQL 5.6 Jan 2013 Fast ALTER TABLE added. Changes to ENUM/SET online. Fast index creation added. Adding indexes changes to SHARED lock, dropping indexes immediate* Online DDL. Many changes do not block readers or writers.
In-Place? Copies Table? Allows Concurrent DML? Allows Concurrent Query? CREATE INDEX,ADD INDEX Yes* No* Yes Yes ADD FULLTEXT INDEX MySQL 5.6 Online DDL Operation Yes No* No Yes DROP INDEX Yes No Yes Yes Set default value for a column Yes No Yes Yes Change auto-increment value for a column Yes No Yes Yes Add a foreign key constraint Yes* No* Yes Yes Drop a foreign key constraint Yes No Yes Yes Rename a column Yes* No* Yes* Yes Add a column Yes Yes Yes* Yes Drop a column Yes Yes Yes Yes Reorder columns Yes Yes Yes Yes Change ROW_FORMAT property Yes Yes Yes Yes Change KEY_BLOCK_SIZE property Yes Yes Yes Yes Make column NULL Yes Yes Yes Yes Make column NOT NULL Yes* Yes Yes Yes Change data type of column No Yes No Yes Add primary key Yes* Yes Yes Yes Drop primary key and add another Yes Yes Yes Yes Drop primary key No Yes No Yes Convert character set No Yes No Yes Specify character set No Yes No Yes Rebuild with FORCE option No Yes No Yes Via http://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html
InnoDB Online DDL (cont.) • Also supports additional syntax LOCK=NONE and ALGORITHM=INPLACE; mysql> ALTER TABLE a DROP PRIMARY KEY, LOCK=NONE;! ERROR 1846 (0A000): LOCK=NONE is not supported. ! Reason: Dropping a primary key is not allowed without ! also adding a new primary key. Try LOCK=SHARED.
Buffer Pool Dump and Restore • Automatic cache priming feature. • Saves LRU contents on shutdown (or on demand) and reloads pages into memory on startup. • Not enabled by default: • innodb_buffer_pool_load_at_startup = 1 • innodb_buffer_pool_dump_at_shutdown = 1
Import/Export Partitioned Tables • Very helpful feature for time-series data. • Swap partitions with regular tables: ALTER TABLE pt 
 EXCHANGE PARTITION p 
 WITH TABLE t;
Transportable Tablespaces • InnoDB data can be copied to a different server without using mysqldump: /* source */
 FLUSH TABLES tableName FOR EXPORT;
 
 /* destination */
 ALTER TABLE tableName IMPORT TABLESPACE;
Important Bugs Fixed • InnoDB not complaining when innodb_log_file_size has changed between restarts: 2013-08-18 20:49:29 17883 [Warning] InnoDB: Resizing redo log from 2*3072 to 2*8192 pages, LSN=5980091! 2013-08-18 20:49:29 17883 [Warning] InnoDB: Starting to delete and rewrite log files.! 2013-08-18 20:49:29 17883 [Note] InnoDB: Setting log file ./ ib_logfile101 size to 128 MB! InnoDB: Progress in MB: 100! 2013-08-18 20:49:30 17883 [Note] InnoDB: Setting log file ./ ib_logfile1 size to 128 MB! InnoDB: Progress in MB: 100! 2013-08-18 20:49:30 17883 [Note] InnoDB: Renaming log file ./ ib_logfile101 to ./ib_logfile0
Replication with GTIDs • Binary log coordinates can now be global - rather than per instance. • Which means failover/topology changes are now much easier.
Multi-threaded slaves! • On a per schema basis, replication supports parallel apply on slaves. • In 5.7 it will be intra-schema parallel.
Diagnostics  Enhancements Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
EXPLAIN FORMAT=JSON EXPLAIN: {! "query_block": {! "select_id": 1,! "nested_loop": [! {! "table": {! "table_name": "country",! "access_type": "ref",! "possible_keys": [! "PRIMARY",! "Population"! ],! "key": "Population",! "used_key_parts": [! "Population"!
EXPLAIN UPDATE/DELETE statements • Previously only SELECT statements could be explained. • Required one to rewrite UPDATE/DELETE statement to a SELECT in order to EXPLAIN.
Optimizer Trace "join_optimization": {! "select#": 1,! "steps": [! {! "condition_processing": {! "condition": "WHERE",! "original_condition": "(`test`.`alias1`.`pk` and ! (`test`.`alias2`.`pk` = `test`.`alias1`.`col_int_key`))",! "steps": [! {! "transformation": "equality_propagation",! "resulting_condition": "(`test`.`alias1`.`pk` and ! multiple equal(`test`.`alias2`.`pk`, ! `test`.`alias1`.`col_int_key`))"!
Optimizer Trace (cont.) • Incredibly Verbose output. • Shows why query plans were not chosen. • Explains transformations that applied.
Performance  Schema Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
“The OLD way” • Largely SHOW GLOBAL STATUS + SHOW ENGINE INNODB STATUS tuning. • Some perf tools external to MySQL provided by the operating system. • Barrier to entry sometimes higher.
Old way is not always instrumented to a useable detail… • With SHOW GLOBAL STATUS, you can see a ratio of Temp tables in memory versus Temp tables on disk. • OK, How many rows per temp table on disk? • If it’s a couple, it might just be logical IO.
Not Query-Oriented • SHOW STATUS shows created temporary tables as a counter, but: • Can’t find which query created.
Difficult for Tooling • InnoDB status contains a lot of magic numbers. Tools like cacti-templates require light weight parsers just to be able to read the information: ! ------------
 ! TRANSACTIONS
 ------------
 Trx !id counter 0 290328385
 Purge done for trx's n:o < 0 290315608 undo n:o < 0 17
 History list length 20
 Total number of lock structs in row lock hash table 70
 LIST OF TRANSACTIONS FOR EACH SESSION:
 ---TRANSACTION 0 0, not started, process no 3491
 MySQL thread id 32, query id 4668737 localhost heikki
 show innodb status!
The New Way - Performance Schema • Series of views which expose internal server performance data with timing information. • Similar to “Oracle Wait Interface”.
Performance Schema • Allows you to monitor server events/stages of execution. • Place “probes” at instrumentation points. • Focus is on low overhead/fast collection. • Uses fixed memory, designed to be able to use in production.
Performance Schema in MySQL 5.5 • Instrumented features which are mainly useful to developers: • File I/O • Mutexes • RW Locks • Did not yet include query level instrumentation :( • Was not enabled by default.
Performance Schema in MySQL 5.6 • • • • Network IO, Table IO, Stages of execution time. Brings number of instruments up to 545. Performance_schema is now 52 tables. Enabled by default.
PS in 5.6 accounts events_statements_summary_global_by_event_name rwlock_instances cond_instances events_waits_current session_account_connect_attrs events_stages_current events_waits_history session_connect_attrs events_stages_history events_waits_history_long setup_actors events_stages_history_long events_waits_summary_by_account_by_event_name setup_consumers events_stages_summary_by_account_by_event_nam e events_stages_summary_by_host_by_event_name events_waits_summary_by_host_by_event_name setup_instruments events_waits_summary_by_instance setup_objects events_stages_summary_by_thread_by_event_name events_waits_summary_by_thread_by_event_name setup_timers events_stages_summary_by_user_by_event_name events_waits_summary_by_user_by_event_name socket_instances events_stages_summary_global_by_event_name events_waits_summary_global_by_event_name socket_summary_by_event_name events_statements_current file_instances socket_summary_by_instance events_statements_history file_summary_by_event_name table_io_waits_summary_by_index_usage events_statements_history_long file_summary_by_instance table_io_waits_summary_by_table events_statements_summary_by_account_by_event_ name events_statements_summary_by_digest host_cache table_lock_waits_summary_by_table hosts threads events_statements_summary_by_host_by_event_na me events_statements_summary_by_thread_by_event_n ame events_statements_summary_by_user_by_event_na me mutex_instances users objects_summary_global_by_type performance_timers
Example
 How Idle are your connections? SELECT
 user,
 host,
 100 * (SUM(IF(event_name = 'idle', sum_timer_wait, 0))
 / SUM(sum_timer_wait)) pct_idle,
 ps_helper.format_time(
 SUM(IF(event_name = 'idle', sum_timer_wait, 0))
 ) total_idle
 FROM events_waits_summary_by_account_by_event_name
 WHERE host IS NOT NULL
 GROUP BY user, host;
 +---------+-----------------------------+----------+---------------+
 | user | host | pct_idle | total_idle |
 +---------+-----------------------------+----------+---------------+
 | root | localhost | 99.9919 | 00:18:55.3054 |
 | test_hc | TFARMER-MYSQL.wh.oracle.com | 100.0000 | 20.61 s |
 +---------+-----------------------------+----------+---------------+
 2 rows in set (0.02 sec)
Example
 Re-implement SHOW PROCESSLIST, but add interface SELECT
 p.*,
 CASE
 WHEN PORT = 0 AND IP = '' THEN 'Unix Socket'
 WHEN IP REGEXP '^(::ffff:)?[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$' THEN 'IPv4'
 WHEN PORT > 0 THEN 'IPv6'
 ELSE 'Undetermined'
 END AS interface
 FROM performance_schema.socket_instances si
 RIGHT JOIN performance_schema.threads t ON (t.thread_id = si.thread_id)
 JOIN information_schema.processlist p ON (t.processlist_id = p.id)G
 *************************** 1. row ***************************
 ID: 6
 USER: root
 HOST: localhost:2873
 DB: performance_schema
 COMMAND: Query
 TIME: 0
 STATE: executing
 INFO: SELECT /* snip */ ON (t.processlist_id = p.id)
 interface: IPv6
 1 row in set (0.13 sec)
Example
 More detailed PROCESSLIST (version 2.0) mysql> select * from processlist_full where conn_id is not nullG
 ...
 *************************** 8. row ***************************
 thd_id: 12400
 conn_id: 12379
 user: root@localhost
 db: ps_helper
 command: Query
 state: Copying to tmp table
 time: 0
 current_statement: selectfrom processlist_full where conn_id is not null
 last_statement: NULL
 last_statement_latency: NULL
 lock_latency: 1.00 ms
 rows_examined: 0
 rows_sent: 0
 rows_affected: 0
 tmp_tables: 1
 tmp_disk_tables: 0
 full_scan: YES
 last_wait: wait/synch/mutex/sql/THD::LOCK_thd_data
 last_wait_latency: 62.53 ns
 source: sql_class.h:3843
Example
 Which account fails to close connections? SELECT 
 ess.USER,
 ess.HOST,
 (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) - ess.COUNT_STAR not_closed,
 ((a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) - ess.COUNT_STAR) * 100 / 
 (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) pct_not_closed
 FROM! performance_schema.events_statements_summary_by_account_by_event_name ess! JOIN performance_schema.accounts a ON (ess.USER = a.USER AND ess.HOST = a.HOST)! WHERE
 ess.EVENT_NAME = 'statement/com/Quit'
 AND (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) > ess.COUNT_STAR;
 +------+-----------+------------+----------------+
 | USER | HOST | not_closed | pct_not_closed |
 +------+-----------+------------+----------------+
 | root | localhost | 4 | 44.4444 |
 | ODBC | localhost | 1 | 100.0000 |
 +------+-----------+------------+----------------+! 2 rows in set (0.00 sec)
Example
 How much time could I save in CI by using tmpfs? mysql> SELECT
 `a`.`EVENT_NAME`,
 `a`.`SUM_TIMER_WAIT`/1024/1024/1024 AS `total_latency_ms`,
 `b`.`SUM_NUMBER_OF_BYTES_READ` AS `total_bytes_read`,
 `b`.`SUM_NUMBER_OF_BYTES_WRITE` AS `total_bytes_written`
 FROM
 `performance_schema`.`events_waits_summary_global_by_event_name` `a`
 INNER JOIN `performance_schema`.`file_summary_by_event_name` `b` USING (event_name)
 WHERE
 `a`.`EVENT_NAME` like 'wait/io/file/%'
 AND `a`.`COUNT_STAR` > 0;
 +--------------------------------------+------------------+------------------+---------------------+
 | EVENT_NAME | total_latency_ms | total_bytes_read | total_bytes_written |
 +--------------------------------------+------------------+------------------+---------------------+
 | wait/io/file/sql/casetest | 0.208953609690 | 0 | 0 |
 | wait/io/file/sql/dbopt | 1.635324865580 | 0 | 65 |
 | wait/io/file/sql/ERRMSG | 0.103571338579 | 58982 | 0 |
 | wait/io/file/sql/file_parser | 265.539358388633 | 1450 | 92321 |
 | wait/io/file/sql/FRM | 9.076319346204 | 233702 | 0 |
 | wait/io/file/sql/global_ddl_log | 0.042241489515 | 0 | 0 |
 | wait/io/file/sql/misc | 0.130650339648 | 0 | 0 |
 | wait/io/file/sql/pid | 0.297200348228 | 0 | 5 |
 | wait/io/file/mysys/charset | 0.054673913866 | 18316 | 0 |
 | wait/io/file/mysys/cnf | 0.047869719565 | 56 | 0 |
 | wait/io/file/myisam/dfile | 8.178050385788 | 53640 | 33796 |
 | wait/io/file/myisam/kfile | 15.392133934423 | 9778 | 5138 |
 | wait/io/file/innodb/innodb_data_file | 15.472816837952 | 8929280 | 49152 |
 | wait/io/file/innodb/innodb_log_file | 9.008877178654 | 69632 | 2560 |
 +--------------------------------------+------------------+------------------+---------------------+
 14 rows in set (0.03 sec)
Example
 How much time is spend waiting where? mysql> select * from ps_helper.wait_classes_global_by_latency;
 +-------------------+--------------+---------------+-------------+-------------+-------------+
 | event_class | total_events | total_latency | min_latency | avg_latency | max_latency |
 +-------------------+--------------+---------------+-------------+-------------+-------------+
 | wait/io/file | 550470 | 46.01 s | 19.44 ns | 83.58 µs | 4.21 s |
 | wait/io/socket | 228833 | 2.71 s | 0 ps | 11.86 µs | 29.93 ms |
 | wait/io/table | 64063 | 1.89 s | 99.79 ns | 29.43 µs | 68.07 ms |
 | wait/lock/table | 76029 | 47.19 ms | 65.45 ns | 620.74 ns | 969.88 µs |
 | wait/synch/mutex | 635925 | 34.93 ms | 19.44 ns | 54.93 ns | 107.70 µs |
 | wait/synch/rwlock | 61287 | 7.62 ms | 21.38 ns | 124.37 ns | 34.65 µs |
 +-------------------+--------------+---------------+-------------+-------------+-------------+
Example
 Index usage statistics mysql> select * from schema_index_statistics limit 5G
 *************************** 1. row ***************************
 table_schema: world
 table_name: city
 index_name: Name
 rows_selected: 31
 select_latency: 37.45 s
 rows_inserted: 0
 insert_latency: 0 ps
 rows_updated: 0
 update_latency: 0 ps
 rows_deleted: 0
 delete_latency: 0 ps
 *************************** 2. row ***************************
 table_schema: world
 table_name: city
 index_name: PRIMARY
 rows_selected: 1
 select_latency: 21.05 us
 rows_inserted: 0
 insert_latency: 0 ps
 rows_updated: 4
 update_latency: 262.35 us
 rows_deleted: 0
 delete_latency: 0 ps
Example
 Statement Analysis mysql> select * from statement_analysis limit 5G
 *************************** 1. row ***************************
 query: UPDATE `City` SET NAME = ? WHERE NAME = ?
 full_scan:
 exec_count: 3
 err_count: 0
 warn_count: 0
 total_latency: 37.44 s
 max_latency: 37.44 s
 avg_latency: 12.48 s
 rows_sent: 0
 rows_sent_avg: 0
 rows_scanned: 6
 digest: 0ab3971f852eb3541dc88efc2c278be0
 *************************** 2. row ***************************
 query: INSERT INTO `City` VALUES (...)
 full_scan:
 exec_count: 4079
 err_count: 0
 warn_count: 0
 total_latency: 994.07 ms
 max_latency: 866.28 us
 avg_latency: 243.70 us
 rows_sent: 0!
More context, examples: • Mark Leith
 http://www.markleith.co.uk/ps_helper/ • Todd Farmer
 http://mysqlblog.fivefarmers.com
Information  Schema Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
Concept • Similar to Performance Schema, but designed for static, slower changing data. • Introduced in MySQL 5.0, but dramatically more meta data in MySQL 5.6.
New I_S Tables (5.6) INNODB_LOCKS INNODB_TRX INNODB_SYS_DATAFILES INNODB_LOCK_WAITS INNODB_SYS_TABLESTATS INNODB_CMP INNODB_METRICS INNODB_CMP_RESET INNODB_CMP_PER_INDEX INNODB_CMPMEM_RESET INNODB_FT_DELETED INNODB_BUFFER_PAGE_LRU INNODB_SYS_FOREIGN INNODB_SYS_COLUMNS INNODB_SYS_INDEXES INNODB_FT_DEFAULT_STOPWORD INNODB_SYS_FIELDS INNODB_CMP_PER_INDEX_RESET INNODB_BUFFER_PAGE INNODB_CMPMEM INNODB_FT_INDEX_TABLE INNODB_FT_BEING_DELETED INNODB_SYS_TABLESPACES INNODB_FT_INDEX_CACHE INNODB_SYS_FOREIGN_COLS INNODB_SYS_TABLES INNODB_BUFFER_POOL_STATS INNODB_FT_CONFIG
5.6 Example - Estimate Working Set • The five minute rule:
 If you load a page into memory, then require it again within 5 minutes it should stay in memory.
 
 
 • Pages in memory is in information_schema!
Estimate Working Set (cont.) • I have only a 128MB innodb_buffer_pool_size. • Example shows I need 100679 pages (16K each)
 = 1573MB of memory to be efficient. mysql> call test.estimate_working_set(10, 30);! .. lines omitted for brevity ..
 +----------------------+
 | pages_in_working_set |
 +----------------------+
 | 100679 |
 +----------------------+
 1 row in set (5 min 55.61 sec)
The  Future Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
MySQL 5.7 • More Performance Schema • Instrumentation for transactions and memory allocation. • More Online DDL • Extend VARCHAR • More EXPLAIN enhancements • EXPLAIN a running query. • Show cost information.
MySQL 5.7 (cont.) • More online usability • Online replication filter reconfiguration. • Improved error log configuration and verbosity. • More annoying bugs fixed. • Warning when creating duplicate indexes. • Control-C support in the client
PS Example
 Memory usage per user mysql> select * from memory_by_user_by_current_bytes;
 +------+--------------------+-------------------+------------------+-------------------+-----------------+
 | user | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
 +------+--------------------+-------------------+-------------------+-------------------+-----------------+
 | NULL | 2085 | 10.64 MiB | 5.23 KiB | 8.00 MiB | 30.16 MiB |
 | root | 1401 | 1.09 MiB | 815 bytes | 334.97 KiB | 42.73 MiB |
 | mark | 201 | 496.08 KiB | 2.47 KiB | 334.97 KiB | 5.50 MiB |
 +------+--------------------+-------------------+-------------------+-------------------+-----------------+
Cost Information
By Actual Features / Enhancements Scalable Read Only Transactions Improvement to Buffer Pool Flushing Precise spatial operations in GIS Concurrent Innodb data file extension Subquery Optimizations Password hashes instead of plain passwords in Query Logs Non-Recursive Deadlock Detection More efficient Optimizer SHA256 hashing with Salt for Authentication Faster Locking Primitives Optimized ROW Based Replication Use obfuscated password storage for command line tools Improved Innodb Thread Concurrency Multi-Threaded Slave Policy Based password validation Multiple background Purge Threads Global Transaction Identifiers Plugin authentication support for Replication Improved Purge lag control (now works) Crash Safe Slave and Binlog INNODB_METRICS (I_S) Split of “Kernel Mutex” Replication Event Checksums Meta Data Information Tables (I_S) Data Dictionary Cache Time Delayed Replication Buffer Pool Information Tables (I_S) Improved Adaptive Flushing Server UUID Reduced Overhead (PS) Page Cleaner/Separate Flush Thread Improved Logging for Row based Replication Simplified Configuration (PS) Group Commit for Binary Log Replication Utilities for Failover and Admin Table Access instrumentation (PS) Fight Cache Coherence and False Sharing issues Separate Tablespaces for Innodb Undo Logs Statements instrumentation (PS) Reduced Innodb Memory Fragmentation Fast Restart – Preloading Innodb Buffer Pool Stages Instrumentation (PS) Reduced Locking for Partitioned tables Online DDL Aggregations by User, Host etc (PS) Reduced Contention for LOCK_open Import/Export for Partitioned Tables Network IO Instrumentation (PS) Support for multiple table_open_cache instances Remote Binlog Backup Show Host Cache Contents (PS) Large (over 4GB) redo logs support Innodb Transportable Tablespaces Improved File I/O Instrumentation (PS) Index Condition pushdown (ICP) New configuration files defaults Explain for UPDATE/DELETE queries Multi-Range-Read (MRR) User Defined DATA DIRECTORY for Innodb Tables JSON output with more information Faster ORDER BY nidxcol LIMIT N Connection Attributes Optimizer Tracing Persistent Statistics for Innodb MemcacheD API in Innodb Deadlock Logging Improvements to Innodb Compression Explicit Partition Selection in queries GET DIAGNOSTICS Fast Page Checksums (CRC32) Full Text Search index for Innodb 4K and 8K Page sizes for Innodb Microsecond TIME precision

More Related Content

PDF
The InnoDB Storage Engine for MySQL
byMorgan Tocker
 
PDF
Upcoming changes in MySQL 5.7
byMorgan Tocker
 
PDF
MySQL NoSQL APIs
byMorgan Tocker
 
PDF
Using MySQL in Automated Testing
byMorgan Tocker
 
PDF
Mysql 57-upcoming-changes
byMorgan Tocker
 
PDF
MySQL Performance Metrics that Matter
byMorgan Tocker
 
PDF
MySQL 5.7 NEW FEATURES, BETTER PERFORMANCE, AND THINGS THAT WILL BREAK -- Mid...
byDave Stokes
 
PDF
Optimizing MySQL
byMorgan Tocker
 
The InnoDB Storage Engine for MySQL
byMorgan Tocker
 
Upcoming changes in MySQL 5.7
byMorgan Tocker
 
MySQL NoSQL APIs
byMorgan Tocker
 
Using MySQL in Automated Testing
byMorgan Tocker
 
Mysql 57-upcoming-changes
byMorgan Tocker
 
MySQL Performance Metrics that Matter
byMorgan Tocker
 
MySQL 5.7 NEW FEATURES, BETTER PERFORMANCE, AND THINGS THAT WILL BREAK -- Mid...
byDave Stokes
 
Optimizing MySQL
byMorgan Tocker
 

What's hot

PDF
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
byAurimas Mikalauskas
 
PPT
MySQL Performance Tuning at COSCUP 2014
byRyusuke Kajiyama
 
PDF
Upgrade to MySQL 5.7 and latest news planned for MySQL 8
byTed Wennmark
 
PDF
MySQL 5.7 in a Nutshell
byEmily Ikuta
 
PDF
InnoDB Architecture and Performance Optimization, Peter Zaitsev
byFuenteovejuna
 
PDF
MySQL 5.6 Performance
byMYXPLAIN
 
PDF
MySQL Server Defaults
byMorgan Tocker
 
PDF
MySQL Query Optimization
byMorgan Tocker
 
PDF
MySQL Usability Guidelines
byMorgan Tocker
 
PPT
Understanding MySql locking issues
byOm Vikram Thapa
 
PDF
MySQL 5.7: Focus on InnoDB
byMario Beck
 
PDF
MySQL 5.7: Core Server Changes
byMorgan Tocker
 
ODP
MySQL 5.7 - What's new and How to upgrade
byAbel Flórez
 
PDF
MySQL Manchester TT - Performance Tuning
byMark Swarbrick
 
PDF
My sql 5.7-upcoming-changes-v2
byMorgan Tocker
 
PDF
MySQL For Linux Sysadmins
byMorgan Tocker
 
PDF
MySQL Cloud Service Deep Dive
byMorgan Tocker
 
PDF
MySQL: From Single Instance to Big Data
byMorgan Tocker
 
PDF
Barcelona mysqlnd qc
byAnis Berejeb
 
PDF
MySQL 5.7 + JSON
byMorgan Tocker
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
byAurimas Mikalauskas
 
MySQL Performance Tuning at COSCUP 2014
byRyusuke Kajiyama
 
Upgrade to MySQL 5.7 and latest news planned for MySQL 8
byTed Wennmark
 
MySQL 5.7 in a Nutshell
byEmily Ikuta
 
InnoDB Architecture and Performance Optimization, Peter Zaitsev
byFuenteovejuna
 
MySQL 5.6 Performance
byMYXPLAIN
 
MySQL Server Defaults
byMorgan Tocker
 
MySQL Query Optimization
byMorgan Tocker
 
MySQL Usability Guidelines
byMorgan Tocker
 
Understanding MySql locking issues
byOm Vikram Thapa
 
MySQL 5.7: Focus on InnoDB
byMario Beck
 
MySQL 5.7: Core Server Changes
byMorgan Tocker
 
MySQL 5.7 - What's new and How to upgrade
byAbel Flórez
 
MySQL Manchester TT - Performance Tuning
byMark Swarbrick
 
My sql 5.7-upcoming-changes-v2
byMorgan Tocker
 
MySQL For Linux Sysadmins
byMorgan Tocker
 
MySQL Cloud Service Deep Dive
byMorgan Tocker
 
MySQL: From Single Instance to Big Data
byMorgan Tocker
 
Barcelona mysqlnd qc
byAnis Berejeb
 
MySQL 5.7 + JSON
byMorgan Tocker
 

Viewers also liked

PDF
MySQL 5.6 Replication Webinar
byMark Swarbrick
 
PPT
GoldenGate for MySQL 설치 시 필요한 사항
by정명훈 Jerry Jeong
 
ODP
The care and feeding of a MySQL database
byDave Stokes
 
PPT
MySQL Atchitecture and Concepts
byTuyen Vuong
 
PDF
MySQL Troubleshooting with the Performance Schema
bySveta Smirnova
 
PDF
The MySQL Performance Schema & New SYS Schema
byTed Wennmark
 
PDF
Inno Db Internals Inno Db File Formats And Source Code Structure
byMySQLConference
 
PPT
MySql slides (ppt)
bywebhostingguy
 
PDF
InnoDB Internal
bymysqlops
 
PPT
Recovery of lost or corrupted inno db tables(mysql uc 2010)
byAleksandr Kuzminsky
 
PDF
Performance Schema in MySQL (Danil Zburivsky)
byOntico
 
PDF
MySQL 5.6, news in 5.7 and our HA options
byTed Wennmark
 
PDF
Troubleshooting MySQL Performance add-ons
bySveta Smirnova
 
PDF
MySQL Oslayer performace optimization
byLouis liu
 
ODP
Mastering InnoDB Diagnostics
byguest8212a5
 
KEY
MySQL Performance - SydPHP October 2011
byGraham Weldon
 
PDF
Linux con europe_2014_full_system_rollback_btrfs_snapper_0
bysprdd
 
PDF
1&1 MySQL Experience at SkySQL Roadshow
byMariaDB Corporation
 
PPTX
Web vulnerability seminar2
bySakuya Izayoi
 
MySQL 5.6 Replication Webinar
byMark Swarbrick
 
GoldenGate for MySQL 설치 시 필요한 사항
by정명훈 Jerry Jeong
 
The care and feeding of a MySQL database
byDave Stokes
 
MySQL Atchitecture and Concepts
byTuyen Vuong
 
MySQL Troubleshooting with the Performance Schema
bySveta Smirnova
 
The MySQL Performance Schema & New SYS Schema
byTed Wennmark
 
Inno Db Internals Inno Db File Formats And Source Code Structure
byMySQLConference
 
MySql slides (ppt)
bywebhostingguy
 
InnoDB Internal
bymysqlops
 
Recovery of lost or corrupted inno db tables(mysql uc 2010)
byAleksandr Kuzminsky
 
Performance Schema in MySQL (Danil Zburivsky)
byOntico
 
MySQL 5.6, news in 5.7 and our HA options
byTed Wennmark
 
Troubleshooting MySQL Performance add-ons
bySveta Smirnova
 
MySQL Oslayer performace optimization
byLouis liu
 
Mastering InnoDB Diagnostics
byguest8212a5
 
MySQL Performance - SydPHP October 2011
byGraham Weldon
 
Linux con europe_2014_full_system_rollback_btrfs_snapper_0
bysprdd
 
1&1 MySQL Experience at SkySQL Roadshow
byMariaDB Corporation
 
Web vulnerability seminar2
bySakuya Izayoi
 

Similar to MySQL 5.6 - Operations and Diagnostics Improvements

PPTX
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
byGeir Høydalsvik
 
PDF
My sql5.7 whatsnew_presentedatgids2015
bySanjay Manwani
 
PDF
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
byInsight Technology, Inc.
 
PDF
MySQL 5.7 what's new
byRicky Setyawan
 
PDF
My sql 56_roadmap_april2012
bysqlhjalp
 
PDF
What's New in MySQL 5.7
byOlivier DASINI
 
PDF
MySQL 5.7: What's New, Nov. 2015
byMario Beck
 
PPTX
MySQL Tech Tour 2015 - 5.7 Whats new
byMark Swarbrick
 
PDF
Mysql User Camp : 20th June - Mysql New Features
byTarique Saleem
 
PDF
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
byMysql User Camp
 
PDF
MySQL :What's New #GIDS16
bySanjay Manwani
 
PDF
TWJUG August, What's new in MySQL 5.7 RC
byRyusuke Kajiyama
 
PPTX
MySQL Tech Tour 2015 - 5.7 InnoDB
byMark Swarbrick
 
PDF
MySQL 5.7 -- SCaLE Feb 2014
byDave Stokes
 
PDF
My sql indo_comm
byRicky Setyawan
 
PDF
Meetup my sql5.6_cluster
byLee Stigile
 
PDF
What's new in MySQL 5.5? FOSDEM 2011
byLenz Grimmer
 
PDF
MySQL 5.7 New Features to Exploit -- PHPTek/Chicago MySQL User Group May 2014
byDave Stokes
 
PDF
My sql 5.6&MySQL Cluster 7.3
byOleksii(Alexey) Porytskyi
 
PPT
State of the Dolphin, at db tech showcase Osaka 2014
byRyusuke Kajiyama
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
byGeir Høydalsvik
 
My sql5.7 whatsnew_presentedatgids2015
bySanjay Manwani
 
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
byInsight Technology, Inc.
 
MySQL 5.7 what's new
byRicky Setyawan
 
My sql 56_roadmap_april2012
bysqlhjalp
 
What's New in MySQL 5.7
byOlivier DASINI
 
MySQL 5.7: What's New, Nov. 2015
byMario Beck
 
MySQL Tech Tour 2015 - 5.7 Whats new
byMark Swarbrick
 
Mysql User Camp : 20th June - Mysql New Features
byTarique Saleem
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
byMysql User Camp
 
MySQL :What's New #GIDS16
bySanjay Manwani
 
TWJUG August, What's new in MySQL 5.7 RC
byRyusuke Kajiyama
 
MySQL Tech Tour 2015 - 5.7 InnoDB
byMark Swarbrick
 
MySQL 5.7 -- SCaLE Feb 2014
byDave Stokes
 
My sql indo_comm
byRicky Setyawan
 
Meetup my sql5.6_cluster
byLee Stigile
 
What's new in MySQL 5.5? FOSDEM 2011
byLenz Grimmer
 
MySQL 5.7 New Features to Exploit -- PHPTek/Chicago MySQL User Group May 2014
byDave Stokes
 
My sql 5.6&MySQL Cluster 7.3
byOleksii(Alexey) Porytskyi
 
State of the Dolphin, at db tech showcase Osaka 2014
byRyusuke Kajiyama
 

More from Morgan Tocker

PDF
MySQL 8.0 Optimizer Guide
byMorgan Tocker
 
PDF
TiDB Introduction
byMorgan Tocker
 
PDF
My First 90 days with Vitess
byMorgan Tocker
 
PDF
TiDB Introduction - San Francisco MySQL Meetup
byMorgan Tocker
 
PDF
Introducing Spirit - Online Schema Change
byMorgan Tocker
 
PDF
Locking and Concurrency Control
byMorgan Tocker
 
PDF
FOSDEM MySQL and Friends Devroom
byMorgan Tocker
 
PDF
Introducing TiDB - Percona Live Frankfurt
byMorgan Tocker
 
PDF
TiDB Introduction - Boston MySQL Meetup Group
byMorgan Tocker
 
MySQL 8.0 Optimizer Guide
byMorgan Tocker
 
TiDB Introduction
byMorgan Tocker
 
My First 90 days with Vitess
byMorgan Tocker
 
TiDB Introduction - San Francisco MySQL Meetup
byMorgan Tocker
 
Introducing Spirit - Online Schema Change
byMorgan Tocker
 
Locking and Concurrency Control
byMorgan Tocker
 
FOSDEM MySQL and Friends Devroom
byMorgan Tocker
 
Introducing TiDB - Percona Live Frankfurt
byMorgan Tocker
 
TiDB Introduction - Boston MySQL Meetup Group
byMorgan Tocker
 

MySQL 5.6 - Operations and Diagnostics Improvements

  • 1.
    <Insert Picture Here> MySQL5.6 - Online Operations and Improved Diagnostics
 Morgan Tocker, MySQL Community Manager
 http://www.tocker.ca/

  • 2.
    Safe Harbor Statement The  following  is  intended  to  outline  our  general  product  direction.   It  is  intended  for  information  purposes  only,  and  may  not  be   incorporated  into  any  contract.  It  is  not  a  commitment  to  deliver   any  material,  code,  or  functionality,  and  should  not  be  relied  upon   in  making  purchasing  decisions.   
 The  development,  release,  and  timing  of  any  features  or   functionality  described  for  Oracle’s  products  remains  at  the  sole   discretion  of  Oracle.
  • 3.
    4 Years ofMySQL Innovation MySQL Cluster 7.3 MySQL Migration Wizard MySQL Workbench 6.0 MySQL 5.6 MySQL 5.5 Windows installer & Tools MySQL 5.7 M y S Q L Cluster MySQL MySQL Enterprise Monitor 2.3 & e r M a n a g 3.0 Applier for Hadoop MySQL Enterprise Backup Security MySQL Utilities MySQL Workbench 5.2 & 6.0 Scalability MySQL Cluster 7.2 HA MySQL Enterprise MySQL Cluster 7.1 O r a c l e C e r t i f i c a t i o n s Audit
  • 4.
    Agenda for Today •Quick Intro to MySQL 5.6 • Operational Enhancements • Diagnostic Enhancements
  • 5.
    Quick  Intro  to  5.6 Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 6.
    The Path toMySQL 5.6 DMR1- DMR2 DMR3 DMR4 DMR5 RC April 2011 Oct 2011 Dec 2011 Aug 2012 Aug 2012 Nov 2012 Optimizer: Optimizer: Optimizer: Optimizer: InnoDB: MRR, ICP, File Sort BKA, New EXPLAIN, Traces Sub-Queries JSON EXPLAIN, Sub-Queries TT, Online DDL, Memcached API New Server Defaults InnoDB: InnoDB: Replication: Replication: Split Kernel Mutex, MT Purge Dump/Restore Buffer Pool GTIDs Binary Log Group Commit Replication: More P_S ! Crash-Safe, MultiThread Slave, Checksums Memcached API New P_S Partitioning Improvements ! ! InnoDB: Full Text Index, Read-Only Optimizations Condition Handling Fractional Seconds ! TIMESTAMP & DATESTAMP More P_S ! Password Mgmt More Partitioning ! More Partitioning More GTID, TT
  • 7.
    By Lines ofCode • “MySQL 5.6 is the largest MySQL code size increase in a MySQL version ever. The last time we saw anything like this was with the merging of MySQL Cluster in 4.1. At the very least, Oracle is paying people to write lines of code to extent that nobody has before.” Stewart Smith
  • 8.
    By Bugs Fixed •1991 Bugs Fixed in 5.6 • 3763 Bugs Fixed in Total Since MySQL 5.5 GA
  • 9.
    By Major Features • • • • • • • • • BetterPerformance and Scalability Better Transactional Throughput Better Performance with Solid State Drives Better Query Execution Times and Diagnostics Better Application Availability NoSQL Access to InnoDB InnoDB Fulltext Search Improved Replication and High Availability Improved Performance Schema
  • 10.
    By Actual Features/ Enhancements Scalable Read Only Transactions Improvement to Buffer Pool Flushing Precise spatial operations in GIS Concurrent Innodb data file extension Subquery Optimizations Password hashes instead of plain passwords in Query Logs Non-Recursive Deadlock Detection More efficient Optimizer SHA256 hashing with Salt for Authentication Faster Locking Primitives Optimized ROW Based Replication Use obfuscated password storage for command line tools Improved Innodb Thread Concurrency Multi-Threaded Slave Policy Based password validation Multiple background Purge Threads Global Transaction Identifiers Plugin authentication support for Replication Improved Purge lag control (now works) Crash Safe Slave and Binlog INNODB_METRICS (I_S) Split of “Kernel Mutex” Replication Event Checksums Meta Data Information Tables (I_S) Data Dictionary Cache Time Delayed Replication Buffer Pool Information Tables (I_S) Improved Adaptive Flushing Server UUID Reduced Overhead (PS) Page Cleaner/Separate Flush Thread Improved Logging for Row based Replication Simplified Configuration (PS) Group Commit for Binary Log Replication Utilities for Failover and Admin Table Access instrumentation (PS) Fight Cache Coherence and False Sharing issues Separate Tablespaces for Innodb Undo Logs Statements instrumentation (PS) Reduced Innodb Memory Fragmentation Fast Restart – Preloading Innodb Buffer Pool Stages Instrumentation (PS) Reduced Locking for Partitioned tables Online DDL Aggregations by User, Host etc (PS) Reduced Contention for LOCK_open Import/Export for Partitioned Tables Network IO Instrumentation (PS) Support for multiple table_open_cache instances Remote Binlog Backup Show Host Cache Contents (PS) Large (over 4GB) redo logs support Innodb Transportable Tablespaces Improved File I/O Instrumentation (PS) Index Condition pushdown (ICP) New configuration files defaults Explain for UPDATE/DELETE queries Multi-Range-Read (MRR) User Defined DATA DIRECTORY for Innodb Tables JSON output with more information Faster ORDER BY nidxcol LIMIT N Connection Attributes Optimizer Tracing Persistent Statistics for Innodb MemcacheD API in Innodb Deadlock Logging Improvements to Innodb Compression Explicit Partition Selection in queries GET DIAGNOSTICS Fast Page Checksums (CRC32) Full Text Search index for Innodb 4K and 8K Page sizes for Innodb Microsecond TIME precision
  • 11.
    Operational  Enhancements Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 12.
    InnoDB Online DDL Thebeginning All DDL requires the table to be recreated, along with all indexes. MySQL 5.1 Nov 2008 MySQL 5.5 Dec 2010 MySQL 5.6 Jan 2013 Fast ALTER TABLE added. Changes to ENUM/SET online. Fast index creation added. Adding indexes changes to SHARED lock, dropping indexes immediate* Online DDL. Many changes do not block readers or writers.
  • 13.
    In-Place? Copies Table? Allows Concurrent DML? AllowsConcurrent Query? CREATE INDEX,ADD INDEX Yes* No* Yes Yes ADD FULLTEXT INDEX MySQL 5.6 Online DDL Operation Yes No* No Yes DROP INDEX Yes No Yes Yes Set default value for a column Yes No Yes Yes Change auto-increment value for a column Yes No Yes Yes Add a foreign key constraint Yes* No* Yes Yes Drop a foreign key constraint Yes No Yes Yes Rename a column Yes* No* Yes* Yes Add a column Yes Yes Yes* Yes Drop a column Yes Yes Yes Yes Reorder columns Yes Yes Yes Yes Change ROW_FORMAT property Yes Yes Yes Yes Change KEY_BLOCK_SIZE property Yes Yes Yes Yes Make column NULL Yes Yes Yes Yes Make column NOT NULL Yes* Yes Yes Yes Change data type of column No Yes No Yes Add primary key Yes* Yes Yes Yes Drop primary key and add another Yes Yes Yes Yes Drop primary key No Yes No Yes Convert character set No Yes No Yes Specify character set No Yes No Yes Rebuild with FORCE option No Yes No Yes Via http://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html
  • 14.
    InnoDB Online DDL(cont.) • Also supports additional syntax LOCK=NONE and ALGORITHM=INPLACE; mysql> ALTER TABLE a DROP PRIMARY KEY, LOCK=NONE;! ERROR 1846 (0A000): LOCK=NONE is not supported. ! Reason: Dropping a primary key is not allowed without ! also adding a new primary key. Try LOCK=SHARED.
  • 15.
    Buffer Pool Dumpand Restore • Automatic cache priming feature. • Saves LRU contents on shutdown (or on demand) and reloads pages into memory on startup. • Not enabled by default: • innodb_buffer_pool_load_at_startup = 1 • innodb_buffer_pool_dump_at_shutdown = 1
  • 16.
    Import/Export Partitioned Tables •Very helpful feature for time-series data. • Swap partitions with regular tables: ALTER TABLE pt 
 EXCHANGE PARTITION p 
 WITH TABLE t;
  • 17.
    Transportable Tablespaces • InnoDBdata can be copied to a different server without using mysqldump: /* source */
 FLUSH TABLES tableName FOR EXPORT;
 
 /* destination */
 ALTER TABLE tableName IMPORT TABLESPACE;
  • 18.
    Important Bugs Fixed •InnoDB not complaining when innodb_log_file_size has changed between restarts: 2013-08-18 20:49:29 17883 [Warning] InnoDB: Resizing redo log from 2*3072 to 2*8192 pages, LSN=5980091! 2013-08-18 20:49:29 17883 [Warning] InnoDB: Starting to delete and rewrite log files.! 2013-08-18 20:49:29 17883 [Note] InnoDB: Setting log file ./ ib_logfile101 size to 128 MB! InnoDB: Progress in MB: 100! 2013-08-18 20:49:30 17883 [Note] InnoDB: Setting log file ./ ib_logfile1 size to 128 MB! InnoDB: Progress in MB: 100! 2013-08-18 20:49:30 17883 [Note] InnoDB: Renaming log file ./ ib_logfile101 to ./ib_logfile0
  • 19.
    Replication with GTIDs •Binary log coordinates can now be global - rather than per instance. • Which means failover/topology changes are now much easier.
  • 20.
    Multi-threaded slaves! • Ona per schema basis, replication supports parallel apply on slaves. • In 5.7 it will be intra-schema parallel.
  • 21.
    Diagnostics  Enhancements Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 22.
    EXPLAIN FORMAT=JSON EXPLAIN: {! "query_block":{! "select_id": 1,! "nested_loop": [! {! "table": {! "table_name": "country",! "access_type": "ref",! "possible_keys": [! "PRIMARY",! "Population"! ],! "key": "Population",! "used_key_parts": [! "Population"!
  • 23.
    EXPLAIN UPDATE/DELETE statements •Previously only SELECT statements could be explained. • Required one to rewrite UPDATE/DELETE statement to a SELECT in order to EXPLAIN.
  • 24.
    Optimizer Trace "join_optimization": {! "select#":1,! "steps": [! {! "condition_processing": {! "condition": "WHERE",! "original_condition": "(`test`.`alias1`.`pk` and ! (`test`.`alias2`.`pk` = `test`.`alias1`.`col_int_key`))",! "steps": [! {! "transformation": "equality_propagation",! "resulting_condition": "(`test`.`alias1`.`pk` and ! multiple equal(`test`.`alias2`.`pk`, ! `test`.`alias1`.`col_int_key`))"!
  • 25.
    Optimizer Trace (cont.) •Incredibly Verbose output. • Shows why query plans were not chosen. • Explains transformations that applied.
  • 26.
    Performance  Schema Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 27.
    “The OLD way” •Largely SHOW GLOBAL STATUS + SHOW ENGINE INNODB STATUS tuning. • Some perf tools external to MySQL provided by the operating system. • Barrier to entry sometimes higher.
  • 28.
    Old way isnot always instrumented to a useable detail… • With SHOW GLOBAL STATUS, you can see a ratio of Temp tables in memory versus Temp tables on disk. • OK, How many rows per temp table on disk? • If it’s a couple, it might just be logical IO.
  • 29.
    Not Query-Oriented • SHOWSTATUS shows created temporary tables as a counter, but: • Can’t find which query created.
  • 30.
    Difficult for Tooling •InnoDB status contains a lot of magic numbers. Tools like cacti-templates require light weight parsers just to be able to read the information: ! ------------
 ! TRANSACTIONS
 ------------
 Trx !id counter 0 290328385
 Purge done for trx's n:o < 0 290315608 undo n:o < 0 17
 History list length 20
 Total number of lock structs in row lock hash table 70
 LIST OF TRANSACTIONS FOR EACH SESSION:
 ---TRANSACTION 0 0, not started, process no 3491
 MySQL thread id 32, query id 4668737 localhost heikki
 show innodb status!
  • 31.
    The New Way- Performance Schema • Series of views which expose internal server performance data with timing information. • Similar to “Oracle Wait Interface”.
  • 32.
    Performance Schema • Allowsyou to monitor server events/stages of execution. • Place “probes” at instrumentation points. • Focus is on low overhead/fast collection. • Uses fixed memory, designed to be able to use in production.
  • 33.
    Performance Schema inMySQL 5.5 • Instrumented features which are mainly useful to developers: • File I/O • Mutexes • RW Locks • Did not yet include query level instrumentation :( • Was not enabled by default.
  • 34.
    Performance Schema inMySQL 5.6 • • • • Network IO, Table IO, Stages of execution time. Brings number of instruments up to 545. Performance_schema is now 52 tables. Enabled by default.
  • 35.
    PS in 5.6 accounts events_statements_summary_global_by_event_name rwlock_instances cond_instances events_waits_current session_account_connect_attrs events_stages_current events_waits_history session_connect_attrs events_stages_history events_waits_history_long setup_actors events_stages_history_long events_waits_summary_by_account_by_event_name setup_consumers events_stages_summary_by_account_by_event_nam e events_stages_summary_by_host_by_event_name events_waits_summary_by_host_by_event_name setup_instruments events_waits_summary_by_instance setup_objects events_stages_summary_by_thread_by_event_name events_waits_summary_by_thread_by_event_name setup_timers events_stages_summary_by_user_by_event_name events_waits_summary_by_user_by_event_name socket_instances events_stages_summary_global_by_event_name events_waits_summary_global_by_event_name socket_summary_by_event_name events_statements_current file_instances socket_summary_by_instance events_statements_history file_summary_by_event_name table_io_waits_summary_by_index_usage events_statements_history_long file_summary_by_instance table_io_waits_summary_by_table events_statements_summary_by_account_by_event_ name events_statements_summary_by_digest host_cache table_lock_waits_summary_by_table hosts threads events_statements_summary_by_host_by_event_na me events_statements_summary_by_thread_by_event_n ame events_statements_summary_by_user_by_event_na me mutex_instances users objects_summary_global_by_type performance_timers
  • 36.
    Example
 How Idle areyour connections? SELECT
 user,
 host,
 100 * (SUM(IF(event_name = 'idle', sum_timer_wait, 0))
 / SUM(sum_timer_wait)) pct_idle,
 ps_helper.format_time(
 SUM(IF(event_name = 'idle', sum_timer_wait, 0))
 ) total_idle
 FROM events_waits_summary_by_account_by_event_name
 WHERE host IS NOT NULL
 GROUP BY user, host;
 +---------+-----------------------------+----------+---------------+
 | user | host | pct_idle | total_idle |
 +---------+-----------------------------+----------+---------------+
 | root | localhost | 99.9919 | 00:18:55.3054 |
 | test_hc | TFARMER-MYSQL.wh.oracle.com | 100.0000 | 20.61 s |
 +---------+-----------------------------+----------+---------------+
 2 rows in set (0.02 sec)
  • 37.
    Example
 Re-implement SHOW PROCESSLIST,but add interface SELECT
 p.*,
 CASE
 WHEN PORT = 0 AND IP = '' THEN 'Unix Socket'
 WHEN IP REGEXP '^(::ffff:)?[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$' THEN 'IPv4'
 WHEN PORT > 0 THEN 'IPv6'
 ELSE 'Undetermined'
 END AS interface
 FROM performance_schema.socket_instances si
 RIGHT JOIN performance_schema.threads t ON (t.thread_id = si.thread_id)
 JOIN information_schema.processlist p ON (t.processlist_id = p.id)G
 *************************** 1. row ***************************
 ID: 6
 USER: root
 HOST: localhost:2873
 DB: performance_schema
 COMMAND: Query
 TIME: 0
 STATE: executing
 INFO: SELECT /* snip */ ON (t.processlist_id = p.id)
 interface: IPv6
 1 row in set (0.13 sec)
  • 38.
    Example
 More detailed PROCESSLIST(version 2.0) mysql> select * from processlist_full where conn_id is not nullG
 ...
 *************************** 8. row ***************************
 thd_id: 12400
 conn_id: 12379
 user: root@localhost
 db: ps_helper
 command: Query
 state: Copying to tmp table
 time: 0
 current_statement: selectfrom processlist_full where conn_id is not null
 last_statement: NULL
 last_statement_latency: NULL
 lock_latency: 1.00 ms
 rows_examined: 0
 rows_sent: 0
 rows_affected: 0
 tmp_tables: 1
 tmp_disk_tables: 0
 full_scan: YES
 last_wait: wait/synch/mutex/sql/THD::LOCK_thd_data
 last_wait_latency: 62.53 ns
 source: sql_class.h:3843
  • 39.
    Example
 Which account failsto close connections? SELECT 
 ess.USER,
 ess.HOST,
 (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) - ess.COUNT_STAR not_closed,
 ((a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) - ess.COUNT_STAR) * 100 / 
 (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) pct_not_closed
 FROM! performance_schema.events_statements_summary_by_account_by_event_name ess! JOIN performance_schema.accounts a ON (ess.USER = a.USER AND ess.HOST = a.HOST)! WHERE
 ess.EVENT_NAME = 'statement/com/Quit'
 AND (a.TOTAL_CONNECTIONS - a.CURRENT_CONNECTIONS) > ess.COUNT_STAR;
 +------+-----------+------------+----------------+
 | USER | HOST | not_closed | pct_not_closed |
 +------+-----------+------------+----------------+
 | root | localhost | 4 | 44.4444 |
 | ODBC | localhost | 1 | 100.0000 |
 +------+-----------+------------+----------------+! 2 rows in set (0.00 sec)
  • 40.
    Example
 How much timecould I save in CI by using tmpfs? mysql> SELECT
 `a`.`EVENT_NAME`,
 `a`.`SUM_TIMER_WAIT`/1024/1024/1024 AS `total_latency_ms`,
 `b`.`SUM_NUMBER_OF_BYTES_READ` AS `total_bytes_read`,
 `b`.`SUM_NUMBER_OF_BYTES_WRITE` AS `total_bytes_written`
 FROM
 `performance_schema`.`events_waits_summary_global_by_event_name` `a`
 INNER JOIN `performance_schema`.`file_summary_by_event_name` `b` USING (event_name)
 WHERE
 `a`.`EVENT_NAME` like 'wait/io/file/%'
 AND `a`.`COUNT_STAR` > 0;
 +--------------------------------------+------------------+------------------+---------------------+
 | EVENT_NAME | total_latency_ms | total_bytes_read | total_bytes_written |
 +--------------------------------------+------------------+------------------+---------------------+
 | wait/io/file/sql/casetest | 0.208953609690 | 0 | 0 |
 | wait/io/file/sql/dbopt | 1.635324865580 | 0 | 65 |
 | wait/io/file/sql/ERRMSG | 0.103571338579 | 58982 | 0 |
 | wait/io/file/sql/file_parser | 265.539358388633 | 1450 | 92321 |
 | wait/io/file/sql/FRM | 9.076319346204 | 233702 | 0 |
 | wait/io/file/sql/global_ddl_log | 0.042241489515 | 0 | 0 |
 | wait/io/file/sql/misc | 0.130650339648 | 0 | 0 |
 | wait/io/file/sql/pid | 0.297200348228 | 0 | 5 |
 | wait/io/file/mysys/charset | 0.054673913866 | 18316 | 0 |
 | wait/io/file/mysys/cnf | 0.047869719565 | 56 | 0 |
 | wait/io/file/myisam/dfile | 8.178050385788 | 53640 | 33796 |
 | wait/io/file/myisam/kfile | 15.392133934423 | 9778 | 5138 |
 | wait/io/file/innodb/innodb_data_file | 15.472816837952 | 8929280 | 49152 |
 | wait/io/file/innodb/innodb_log_file | 9.008877178654 | 69632 | 2560 |
 +--------------------------------------+------------------+------------------+---------------------+
 14 rows in set (0.03 sec)
  • 41.
    Example
 How much timeis spend waiting where? mysql> select * from ps_helper.wait_classes_global_by_latency;
 +-------------------+--------------+---------------+-------------+-------------+-------------+
 | event_class | total_events | total_latency | min_latency | avg_latency | max_latency |
 +-------------------+--------------+---------------+-------------+-------------+-------------+
 | wait/io/file | 550470 | 46.01 s | 19.44 ns | 83.58 µs | 4.21 s |
 | wait/io/socket | 228833 | 2.71 s | 0 ps | 11.86 µs | 29.93 ms |
 | wait/io/table | 64063 | 1.89 s | 99.79 ns | 29.43 µs | 68.07 ms |
 | wait/lock/table | 76029 | 47.19 ms | 65.45 ns | 620.74 ns | 969.88 µs |
 | wait/synch/mutex | 635925 | 34.93 ms | 19.44 ns | 54.93 ns | 107.70 µs |
 | wait/synch/rwlock | 61287 | 7.62 ms | 21.38 ns | 124.37 ns | 34.65 µs |
 +-------------------+--------------+---------------+-------------+-------------+-------------+
  • 42.
    Example
 Index usage statistics mysql>select * from schema_index_statistics limit 5G
 *************************** 1. row ***************************
 table_schema: world
 table_name: city
 index_name: Name
 rows_selected: 31
 select_latency: 37.45 s
 rows_inserted: 0
 insert_latency: 0 ps
 rows_updated: 0
 update_latency: 0 ps
 rows_deleted: 0
 delete_latency: 0 ps
 *************************** 2. row ***************************
 table_schema: world
 table_name: city
 index_name: PRIMARY
 rows_selected: 1
 select_latency: 21.05 us
 rows_inserted: 0
 insert_latency: 0 ps
 rows_updated: 4
 update_latency: 262.35 us
 rows_deleted: 0
 delete_latency: 0 ps
  • 43.
    Example
 Statement Analysis mysql> select* from statement_analysis limit 5G
 *************************** 1. row ***************************
 query: UPDATE `City` SET NAME = ? WHERE NAME = ?
 full_scan:
 exec_count: 3
 err_count: 0
 warn_count: 0
 total_latency: 37.44 s
 max_latency: 37.44 s
 avg_latency: 12.48 s
 rows_sent: 0
 rows_sent_avg: 0
 rows_scanned: 6
 digest: 0ab3971f852eb3541dc88efc2c278be0
 *************************** 2. row ***************************
 query: INSERT INTO `City` VALUES (...)
 full_scan:
 exec_count: 4079
 err_count: 0
 warn_count: 0
 total_latency: 994.07 ms
 max_latency: 866.28 us
 avg_latency: 243.70 us
 rows_sent: 0!
  • 44.
    More context, examples: •Mark Leith
 http://www.markleith.co.uk/ps_helper/ • Todd Farmer
 http://mysqlblog.fivefarmers.com
  • 45.
    Information  Schema Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 46.
    Concept • Similar toPerformance Schema, but designed for static, slower changing data. • Introduced in MySQL 5.0, but dramatically more meta data in MySQL 5.6.
  • 47.
    New I_S Tables(5.6) INNODB_LOCKS INNODB_TRX INNODB_SYS_DATAFILES INNODB_LOCK_WAITS INNODB_SYS_TABLESTATS INNODB_CMP INNODB_METRICS INNODB_CMP_RESET INNODB_CMP_PER_INDEX INNODB_CMPMEM_RESET INNODB_FT_DELETED INNODB_BUFFER_PAGE_LRU INNODB_SYS_FOREIGN INNODB_SYS_COLUMNS INNODB_SYS_INDEXES INNODB_FT_DEFAULT_STOPWORD INNODB_SYS_FIELDS INNODB_CMP_PER_INDEX_RESET INNODB_BUFFER_PAGE INNODB_CMPMEM INNODB_FT_INDEX_TABLE INNODB_FT_BEING_DELETED INNODB_SYS_TABLESPACES INNODB_FT_INDEX_CACHE INNODB_SYS_FOREIGN_COLS INNODB_SYS_TABLES INNODB_BUFFER_POOL_STATS INNODB_FT_CONFIG
  • 48.
    5.6 Example -Estimate Working Set • The five minute rule:
 If you load a page into memory, then require it again within 5 minutes it should stay in memory.
 
 
 • Pages in memory is in information_schema!
  • 49.
    Estimate Working Set(cont.) • I have only a 128MB innodb_buffer_pool_size. • Example shows I need 100679 pages (16K each)
 = 1573MB of memory to be efficient. mysql> call test.estimate_working_set(10, 30);! .. lines omitted for brevity ..
 +----------------------+
 | pages_in_working_set |
 +----------------------+
 | 100679 |
 +----------------------+
 1 row in set (5 min 55.61 sec)
  • 50.
    The  Future Copyright  ©  2012  Oracle  and/or  its  affiliates.  All  rights  reserved.
  • 51.
    MySQL 5.7 • MorePerformance Schema • Instrumentation for transactions and memory allocation. • More Online DDL • Extend VARCHAR • More EXPLAIN enhancements • EXPLAIN a running query. • Show cost information.
  • 52.
    MySQL 5.7 (cont.) •More online usability • Online replication filter reconfiguration. • Improved error log configuration and verbosity. • More annoying bugs fixed. • Warning when creating duplicate indexes. • Control-C support in the client
  • 53.
    PS Example
 Memory usageper user mysql> select * from memory_by_user_by_current_bytes;
 +------+--------------------+-------------------+------------------+-------------------+-----------------+
 | user | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
 +------+--------------------+-------------------+-------------------+-------------------+-----------------+
 | NULL | 2085 | 10.64 MiB | 5.23 KiB | 8.00 MiB | 30.16 MiB |
 | root | 1401 | 1.09 MiB | 815 bytes | 334.97 KiB | 42.73 MiB |
 | mark | 201 | 496.08 KiB | 2.47 KiB | 334.97 KiB | 5.50 MiB |
 +------+--------------------+-------------------+-------------------+-------------------+-----------------+
  • 54.
  • 56.
    By Actual Features/ Enhancements Scalable Read Only Transactions Improvement to Buffer Pool Flushing Precise spatial operations in GIS Concurrent Innodb data file extension Subquery Optimizations Password hashes instead of plain passwords in Query Logs Non-Recursive Deadlock Detection More efficient Optimizer SHA256 hashing with Salt for Authentication Faster Locking Primitives Optimized ROW Based Replication Use obfuscated password storage for command line tools Improved Innodb Thread Concurrency Multi-Threaded Slave Policy Based password validation Multiple background Purge Threads Global Transaction Identifiers Plugin authentication support for Replication Improved Purge lag control (now works) Crash Safe Slave and Binlog INNODB_METRICS (I_S) Split of “Kernel Mutex” Replication Event Checksums Meta Data Information Tables (I_S) Data Dictionary Cache Time Delayed Replication Buffer Pool Information Tables (I_S) Improved Adaptive Flushing Server UUID Reduced Overhead (PS) Page Cleaner/Separate Flush Thread Improved Logging for Row based Replication Simplified Configuration (PS) Group Commit for Binary Log Replication Utilities for Failover and Admin Table Access instrumentation (PS) Fight Cache Coherence and False Sharing issues Separate Tablespaces for Innodb Undo Logs Statements instrumentation (PS) Reduced Innodb Memory Fragmentation Fast Restart – Preloading Innodb Buffer Pool Stages Instrumentation (PS) Reduced Locking for Partitioned tables Online DDL Aggregations by User, Host etc (PS) Reduced Contention for LOCK_open Import/Export for Partitioned Tables Network IO Instrumentation (PS) Support for multiple table_open_cache instances Remote Binlog Backup Show Host Cache Contents (PS) Large (over 4GB) redo logs support Innodb Transportable Tablespaces Improved File I/O Instrumentation (PS) Index Condition pushdown (ICP) New configuration files defaults Explain for UPDATE/DELETE queries Multi-Range-Read (MRR) User Defined DATA DIRECTORY for Innodb Tables JSON output with more information Faster ORDER BY nidxcol LIMIT N Connection Attributes Optimizer Tracing Persistent Statistics for Innodb MemcacheD API in Innodb Deadlock Logging Improvements to Innodb Compression Explicit Partition Selection in queries GET DIAGNOSTICS Fast Page Checksums (CRC32) Full Text Search index for Innodb 4K and 8K Page sizes for Innodb Microsecond TIME precision