MySQL 5.7: Core Server Changes

MySQL Core Server
Changes
Morgan Tocker 
MySQL Community Manager

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!2
The following is intended to outline our general product direction. It is intended  
for information purposes only, and may not be incorporated into any contract.  
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.

MySQL 5.5
MySQL Enterprise Monitor 2.3 & 3.0
MySQL Enterprise Backup
Security
Scalability
HA
Audit
MySQL 5.6
MySQL Workbench 6.1
MySQL Utilities
MySQL Applier for
Hadoop
MySQL Workbench 5.2 & 6.0
MySQL Enterprise Oracle
Certifications
MySQL@Oracle: 4 Years of MySQL Innovation
MySQL Cluster Manager
Windows installer & Tools
MySQL Cluster 7.2
MySQL Cluster 7.1
MySQL Migration
Wizard
MySQL 5.7
MySQL Cluster 7.4
MySQL Fabric
MySQL Cluster 7.3

Discussion Topics
▪ Let’s talk about MySQL 5.7!
▪ Less discussion about InnoDB, Performance Schema,
Benchmarks, Replication.
▪ Well covered in other talks.

Discussion Topics (cont.)
▪ New Features
▪ Changes to Existing Features
▪ Our Ongoing Efforts to Refactor and Improve MySQL.

Quick Recap of the MySQL
Development Process

The DMR Model
▪ Up to 4 Development Milestone Releases per year (DMRs).
▪ Features are merged into main branch only when stable.
▪ High level review complete.
▪ Low level review complete.
▪ 80%+ code coverage testing goal.
▪ DMRs are intended to be of “Release Candidate” Quality.

MySQL 5.7 DMR Schedule
April 2013
DMR1
221 Bugs Fixed
29 Changes
21 Worklogs
Sep 2013
DMR2
384 Bugs Fixed
41 Changes
52 Worklogs
Dec 2013
DMR3
155 Bugs Fixed
23 Changes
27 Worklogs
April 2014
DMR4
256 Bugs Fixed
38 Changes
56 Worklogs
5.6 GA
Feb 2013

Labs Releases
▪ For functionality that doesn’t meet the DMR requirements.
▪ Intended to be an earlier preview.
▪ Recent Labs Releases:
▪ Hadoop Applier for MySQL
▪ MySQL Fabric
▪ Multi-Source Replication
▪ NEW - InnoDB GIS+Optimizer GIS+Parser Refactoring!
▪ NEW - InnoDB Page/IO Compression!
▪ NEW - Multi-threaded Slaves!
labs.mysql.com

InnoDB GIS+Optimizer GIS+Parser  
Refactoring
▪ InnoDB R-Tree Index Support
▪ Based on Boost.Geometry
▪ Previously only Available in MyISAM
labs.mysql.com

InnoDB GIS+Optimizer GIS+Parser  
Refactoring (cont.)
▪ Improves readability, maintainability, and stability
– Cleanly separate the parsing, optimizing, and execution stages
– Allows for easier feature additions, with lessened risk
!
▪ Transforms SQL Parser into a true bottom-up parser
– Better support for larger syntax rules in the future
– Fewer intermediate nodes in the parse tree
▪ Fewer node allocations
▪ Faster grammar
labs.mysql.com

InnoDB Page/IO Compression
▪ Transparent Page Level Compression
▪ Builds on new multiple page cleaner threads from DMR4.
▪ Compress/decompress transparently in background threads
▪ Applies to all tables, including the system tablespace and UNDO
logs
▪ Thank you to Fusion-IO!
labs.mysql.com

MySQL 5.7 DMRs To Date

Common Themes
▪ Improved support for high connection
creation rate
▪ Lock Thread count impact reduced
▪ Multiple Metadata Locking (MDL)
improvements
▪ InnoDB Improvements for:
▪ Index lock contention
▪ Read Only Transactions
▪ Temporary Tables
▪ Redo Logs
▪ Multiple page cleaner threads.
▪ Scalability
▪ Parallel Replication (Intra-schema)
Performance & Scalability
▪ Overhead has been reduced.
▪ Improved to do lazy initialization on new
connections.
▪ Now instruments:
▪ Memory usage
▪ Stored programs
▪ Replication
▪ Metadata Locking (MDL)
▪ Transactions
▪ Prepared Statements
From DMR1 to DMR4
Performance Schema

Common Themes (cont.)
▪ Internal logging API
▪ Timestamps are now written with
log entries in YYYY-MM-
DDThh:mm:ss.uuuuuu format.
▪ Configurable Verbosity
▪ InnoDB Refactored:
▪ Locking Code
▪ Redo Log Code (for
performance)
▪ Tablespace Class
▪ Atomics refactored
▪ Optimizer refactoring (multiple
improvements)
Deep Refactoring
▪ Password expiration support +
timestamp of password last changed.
▪ mysql_secure_installation changed
to C++ program.
▪ Improvements to SSL Support.
▪ Support to change AES key length
and block encryption mode
From DMR1 to DMR4
Security

DMR0
▪ Error issued when creating duplicate index.
▪ Warning when strict SQL Mode removed.
▪ Ctl-C fixed in MySQL CLI!
▪ Non-blocking SHOW SLAVE STATUS
▪ Idempotent mysqlbinlog mode.
▪ Stacked Diagnostics Area
Internal Release Only

DMR1
▪ Online DDL Now supports RENAME INDEX and VARCHAR
extensions.
▪ InnoDB support for Spatial Data Types.
▪ Rewrite-db option for mysqlbinlog with RBR.
Released April 2013

DMR2
▪ Semi sync replication improvements.
▪ Multiple Triggers per event per table.
▪ EXPLAIN FOR CONNECTION
▪ EXPLAIN showing cost information.
Released September 2013

Example: Cost Information in EXPLAIN
*************************** 1. row *************************** 
EXPLAIN: { 
"query_block": { 
"select_id": 1, 
"cost_info": { 
"query_cost": "4786.00" 
}, 
"ordering_operation": {
"using_temporary_table": true, 
"using_filesort": true, 
"cost_info": { 
"sort_cost": "2151.00" 
}, 
"nested_loop": [ 
{ 
"table": { 
.. 
], 
"key_length": "3", 
"rows_examined_per_scan": 239, 
"rows_produced_per_join": 239, 
"filtered": 100, 
"using_index": true, 
"cost_info": { 
"read_cost": "6.00", 
"eval_cost": "47.80", 
"prefix_cost": "53.80", 
"data_read_per_join": "61K" 
},
..
EXPLAIN FORMAT=JSON FOR CONNECTION 2;

Example: EXPLAIN in MySQL Workbench
Visual EXPLAIN Including Cost Information!

DMR3
▪ UNION ALL no longer requires temporary table.
▪ Filesort now packs VARCHAR and CHAR keys in temporary files.
▪ Optimizer Support for row value constructors.
▪ i.e. WHERE (a,b) in ((‘a’, ‘b’), (‘c’, ‘d’));
Released December 2013

DMR3 (cont.)
▪ InnoDB Fulltext now supports external parsers.
▪ Dynamic replication filters.
▪ Support ICP optimization for partitioned tables.
Released December 2013

Example: UNION ALL Optimization
***************** 1. row *****************
id: 1
select_type: PRIMARY
table: table_a
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3
Extra: NULL
***************** 2. row *****************
id: 2
select_type: UNION
table: table_b
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3
Extra: NULL
***************** 3. row *****************
id: NULL
select_type: UNION RESULT
table: <union1,2>
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
Extra: Using temporary
***************** 1. row *****************
id: 1
table: table_a
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3
filtered: 100.00
Extra: NULL
***************** 2. row *****************
id: 2
select_type: UNION
table: table_b
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3
filtered: 100.00
Extra: NULL
EXPLAIN SELECT * FROM a UNION ALL SELECT * FROM b
MySQL 5.6: MySQL 5.7:

Example: Row-value Constructors
**************** 1. row ****************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4188
Extra: Using where
1 row in set (0.00 sec)
**************** 1. row ****************
id: 1
select_type: SIMPLE
table: City
partitions: NULL
type: range
possible_keys: CountryCode,CountryCode$Name
key: CountryCode$Name
key_len: 38
ref: NULL
rows: 3
filtered: 100.00
Extra: Using where
EXPLAIN SELECT * FROM City WHERE (CountryCode,Name) IN
(('USA', 'San Francisco'), ('USA', 'Chicago'), ('CAN', ‘Toronto'))
MySQL 5.6: MySQL 5.7:

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
{San Francisco, California, 776733}
{Chicago, Illinois, 2896016}
!25
Filesort Packing
ID Name CountryCode District Population
3805 San Francisco USA California 776733
1812 Toronto CAN Ontario 688275
3795 Chicago USA Illinois 2896016
1818 Vancouver CAN British Columbia 514008
SELECT Name, District, Population FROM City WHERE
CountryCode = ‘USA’ ORDER BY Name;
{San Francisco, California, 776733}
{Chicago, Illinois, 2896016}
Sort Buffer
MySQL 5.7: CHAR, VARCHAR and NULLABLE fixed-size types
are now packed in the sort buffer.
Using Modified Sort Algorithm

DMR4
▪ Statement Timeout.
▪ Change master without stopping SQL thread.
▪ Changed compiler from sun studio to gcc on Solaris.
▪ Transportable tablespaces for partitioned InnoDB tables.
Released April 2014

Example: Server-Side Statement Timeout
▪ SELECT MAX_STATEMENT_TIME = 109 * FROM my_table;
▪ SET SESSION MAX_STATEMENT_TIME = 100;
▪ SET GLOBAL MAX_STATEMENT_TIME = 100;
Incorporates Contributions from Davi Arnaut, the MySQL Community

Changes to Existing Functionality
(Deprecation and Change)

Implementing Change
▪ Not all changes easily support old and new.
▪ Some changes do not easily support deprecation messages.
▪ i.e. GROUP BY with implicit ORDER BY
▪ Always special cases.
▪ i.e. removal of mysqlbug, msql2mysql CLI utilities.

Functionality Changed
▪ Change of Defaults
▪ Replication
▪ SQL Modes
▪ EXPLAIN PARTITIONS and
EXTENDED
▪ Deprecation
▪ NULL synonym N
▪ ENCODE/DECODE functions.
▪ SHOW PROFILES feature
!
!
!
!
▪ Removal
▪ SHOW ENGINE INNODB MUTEX
▪ InnoDB Monitor Tables
▪ ALTER IGNORE TABLE
▪ Unused CLI Utilities
▪ INSERT DELAYED
▪ Specific SQL Modes

Changes to Replication Defaults
▪ MySQL 5.6 improved durable replication performance considerably
(group commit fix)
▪ Proposal is to make replication durable by default starting from 5.7:
▪ sync_binlog = 1
▪ master-info-repository = TABLE
▪ relay-log-info-repository = TABLE

Change SQL_Mode Defaults
mysql> SELECT id, invoice_id, description
FROM invoice_line_items GROUP BY invoice_id;
+----+------------+-------------+
| id | invoice_id | description |
+----+------------+-------------+
| 1 | 1 | New socks |
| 3 | 2 | Shoes |
| 5 | 3 | Tie |
+----+------------+-------------+
3 rows in set (0.00 sec)

SQL Mode: Only Full Group By
▪ Proposal is to produce an error from MySQL 5.7.
▪ Will still be configurable via SQL Mode ONLY_FULL_GROUP_BY.
mysql> SELECT id, invoice_id, description
FROM invoice_line_items GROUP BY invoice_id;
ERROR 1055 (42000): 'test.invoice_line_items.id'
isn't in GROUP BY

EXPLAIN PARTITIONS and EXPLAIN
EXTENDED Always Enabled
mysql> EXPLAIN SELECT * FROM table_aG
********************** 1. row **********************
id: 1
table: table_a
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3
filtered: 100.00
Extra: NULL
2 rows in set, 1 warning (0.00 sec)

EXTENDED (cont.)
▪ Two optional flags that only add value
▪ Automated tooling only likely scenario to break.
▪ May be better suited for FORMAT=JSON

EXTENDED (cont.)
▪ Proposal:
▪ Enables two flags by default, and deprecates the use of the
extended syntax.
▪ Went ahead in 5.7 DMR3.

Deprecation of NULL synonym N
mysql> SELECT NULL is N; 
+------------+ 
| NULL is N | 
+------------+ 
| 1 | 
+------------+ 
 
mysql> INSERT INTO tablea VALUES (3, N); 
Query OK, 1 row affected (0.00 sec)
 
mysql> SELECT * FROM tablea WHERE b IS N; 
+---+------+ 
| a | b | 
+---+------+ 
| 3 | NULL | 
+---+------+ 

Deprecation of NULL synonym N (cont.)
▪ Proposal:
▪ For removal in either 5.7 or 5.8.
▪ Still seeking feedback.

Deprecate ENCODE/DECODE Functions
mysql> SELECT HEX(ENCODE('MyPassword', 'Super Secret!')); 
+--------------------------------------------+ 
| HEX(ENCODE('MyPassword', 'Super Secret!')) |
+--------------------------------------------+ 
| A0E844F97A48FE6E8C1F | 
+--------------------------------------------+
!
mysql> SELECT DECODE(UNHEX('A0E844F97A48FE6E8C1F'), 'Super Secret!'); 
+--------------------------------------------------------+ 
| DECODE(UNHEX('A0E844F97A48FE6E8C1F'), 'Super Secret!') | 
+--------------------------------------------------------+ 
| MyPassword | 
+--------------------------------------------------------+

Deprecate ENCODE/DECODE Functions (cont.)
▪ Functions are cryptographically weak.
▪ For crypto to be upgraded, storage format would become
incompatible.
▪ Overlaps with similar AES functionality already present.

Deprecate SHOW PROFILES
And the INFORMATION_SCHEMA.PROFILING table
mysql> set profiling=1; 
Query OK, 0 rows affected, 1 warning (0.00 sec)
 
mysql> show profiles; 
+----------+------------+-------------------------------------------------------+ 
| Query_ID | Duration | Query | 
+----------+------------+-------------------------------------------------------+ 
| 1 | 0.00039200 | SELECT * FROM table_a UNION ALL SELECT * FROM table_b | 
+----------+------------+-------------------------------------------------------+ 

Deprecate SHOW PROFILES (cont.)
mysql> show profile for query 1; 
+----------------------+----------+ 
| Status | Duration | 
+----------------------+----------+ 
| starting | 0.000059 | 
| checking permissions | 0.000005 | 
| checking permissions | 0.000005 | 
| Opening tables | 0.000091 | 
| System lock | 0.000011 | 
| optimizing | 0.000004 | 
| statistics | 0.000012 | 
| preparing | 0.000009 | 
| optimizing | 0.000002 | 
| statistics | 0.000005 | 
| preparing | 0.000004 | 
| executing | 0.000003 | 
| Sending data | 0.000053 | 
| executing | 0.000002 | 
..
..
| Sending data | 0.000029 | 
| optimizing | 0.000006 | 
| statistics | 0.000006 | 
| preparing | 0.000004 | 
| executing | 0.000002 | 
| Sending data | 0.000019 | 
| removing tmp table | 0.000007 | 
| Sending data | 0.000002 | 
| query end | 0.000007 | 
| closing tables | 0.000010 | 
| freeing items | 0.000019 | 
| cleaning up | 0.000016 | 
+----------------------+----------+ 

Deprecate SHOW PROFILES (cont.)
▪ Overlaps with Performance Schema interface.
▪ Performance Schema has more flexible configuration
▪ Instrumentation level can be increased/lowered to reduce
overhead.
▪ Performance Schema has more data available.

SHOW ENGINE INNODB MUTEX (cont.)
▪ To be removed in 5.7.
▪ Rationale:
▪ Overlaps with PS.
▪ Can not be disabled.
▪ Code refactored so that InnoDB can use different locks internally.

InnoDB Monitor tables
▪ Existed in a time when information_schema did not.
▪ Usage was:
CREATE TABLE innodb_monitor (a INT) ENGINE=INNODB; 
/* view to the error log */ 
DROP TABLE innodb_monitor;

InnoDB Monitor tables (cont.)
▪ To be removed:
▪ innodb_tablespace_monitor
▪ innodb_table_monitor
▪ innodb_mem_validate
▪ To be replaced by config setting:
▪ innodb_monitor
▪ innodb_lock_monitor

InnoDB Monitor tables (cont.)
▪ Notable exception:
▪ In 5.7 DMR4 innodb_tablespace_monitor is removed,
without a replacement.
▪ This is included as part of our release notes.
▪ A resolution will be made before 5.7 GA.

ALTER IGNORE TABLE
mysql> CREATE TABLE a (id INT NOT NULL);
Query OK, 0 rows affected (0.02 sec)
!
mysql> INSERT INTO a VALUES (2), (2), (3);
Records: 3 Duplicates: 0 Warnings: 0
!
mysql> ALTER TABLE a ADD PRIMARY KEY (id);
ERROR 1062 (23000): Duplicate entry '2' for key
'PRIMARY'

ALTER IGNORE TABLE (cont.)
mysql> ALTER IGNORE TABLE a ADD PRIMARY KEY (id);
Records: 3 Duplicates: 1 Warnings: 0
!
mysql> SELECT * FROM a;
+----+
| id |
+----+
| 2 |
| 3 |
+----+

ALTER IGNORE TABLE (cont.)
▪ Rationale:
▪ Will silently drop rows for you!
▪ Has strange semantics for replication and ALTER TABLE.
▪ Proposal went ahead already:
▪ Deprecated in 5.6.17.
▪ Removed in 5.7 DMR4

Removal of unused CLI programs
▪ msql2mysql [Shell Script]
▪ mysqlaccess [Perl program]
▪ mysqlbug [Shell script]
▪ mysql_zap [Shell script]
▪ mysql_waitpid [C program]
▪ mysql_convert_table_format [Perl Program]
▪ mysql_fix_extensions [Perl Program]
▪ mysql_find_rows [Perl Program]
▪ mysql_setpermission [Perl Program]

Removal of unused CLI programs
▪ Aim:
▪ Increase supportability / remove a situation where users use
utilities that have outlived purpose.
▪ Improved cross platform experience in the case of shell scripts.

INSERT DELAYED Removal
mysql> INSERT DELAYED INTO a (id) VALUES (5);
Query OK, 1 row affected, 1 warning (0.00 sec)

INSERT DELAYED Removal (cont.)
▪ Rationale:
▪ INSERT DELAYED is slower than a normal INSERT when
statements do not need to be delayed.
▪ InnoDB has row level locking + MVCC. Reads do not block
writes (usually).
▪ Does not allow server to check for PRIMARY/UNIQUE
constraints.
▪ Errors can not be caught by the application.

Simplify SQL_MODE options
▪ Proposal is to reduce the number of mode options available.
▪ Aim is to increase usage of SQL Mode options, encourage more
users to run MySQL in a stricter way.

Simplify SQL_MODE options (cont.)
▪ Current recommendation:
▪ sql-mode=“STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER,
NO_AUTO_VALUE_ON_ZERO, NO_ENGINE_SUBSTITUTION,
NO_ZERO_DATE, NO_ZERO_IN_DATE,ONLY_FULL_GROUP_BY"

Simplify SQL_MODE options (cont.)
▪ 5.7 Proposal:
▪ Remove the options ERROR_FOR_DIVISION_BY_ZERO,
NO_ZERO_DATE and NO_ZERO_IN_DATE.
▪ These behaviours be enabled by STRICT_TRANS_TABLES or
STRICT_ALL_TABLES.
▪ Improve error reporting when a behaviour is influenced by an SQL
mode.

We Want Your Feedback!
Smooth
Transition
Ideas Feedback
Help make the transition as smooth as possible.

Other Upcoming Oracle Talks
▪ Meet the MySQL Team
▪ Same room directly after this talk!
▪ Tomorrow:
▪ MySQL Fabric
▪ Performance and Benchmarks (5.7)

MySQL 5.7: Core Server Changes

More Related Content

What's hot

Viewers also liked

Similar to MySQL 5.7: Core Server Changes

More from Morgan Tocker

MySQL 5.7: Core Server Changes