MySQL 5.7+ new features !

1. www.vmcd.org
MySQL 5.7 Important Features
InnoDB Read-Only Scalability
Improving the performance for Read-Only and Read-Mostly workloads. Significantly improved how InnoDB handles RO trans-actions
Removed server layer contentions related to Meta Data Locking (MDL) and removed the use of “thread locks” (thd_locks) for InnoDB
In MySQL 5.7.2, there is another optimization where all transactions are treated as a read-only transactions by default, and a transaction is only
assigned an ID and a rollback segment when the transaction attempts to do its first update. This optimization of treating all transactions as
read-only by default has the additional benefit that users do not need to use special syntax or change their applications to take advantage of it.
Reference:
http://dimitrik.free.fr/blog/archives/09-01-2013_09-30-2013.html
http://mysqlserverteam.com/mysql-5-7-over-1m-qps-with-innodb-memcached-plugin/
http://mysqlserverteam.com/transaction-life-cycle-improvements-in-5-7-3/
http://mysqlserverteam.com/mysql-5-7-3-deep-dive-into-1mil-qps-with-innodb-memcached/

2. www.vmcd.org
InnoDB Read-Write Scalability.
Improved the performance of Read-Write (RW) workloads. We have removed the “index lock contention” in InnoDB. The index lock that was
used to protect the entire index tree structure is now replaced by more fine grained “block locks” in the tree.
InnoDB has two "modes" when updating the Btrees. An optimistic mode and a pessimistic mode. They roughly correspond to in-place changes
and tree modification changes. If InnoDB thinks that an insert/delete will result in a tree modification change it will lock the index in X mode.
This blocks readers too and limits concurrency. The fix was to introduce a new lock mode SX. This reduces the need to lock the entire Btree
during tree modification changes.
Reference:
https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements
http://mysqlserverteam.com/mysql-5-7-improves-dml-oriented-workloads/
https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements

3. www.vmcd.org
InnoDB Faster & Parallel Flushing
InnoDB has had some sub-optimal code around traversing the dirty page and LRU list flushing. The problem was that when we release the
buffer pool and associated mutexes during the IO phase we can no longer trust the iterator that is used to scan the aforementioned lists. So, we
restart the scan from the start. The fix was to track whether the iterator was invalidated while the covering mutexes where released. If the
iterator was not invalidated then we can carry on from where we left off. If on the other hand we detect that it was invalidated we go back to
the start and do it the old way. The iterator invalidation should be very rare.
Reducing the number of pages scanned when doing flush list batches, speeding up page flushing
Recently our benchmarks showed that we are scanning excessive number of pages when
doing flush list batches. We fixed it by introducing the concept of Hazard Pointer
which reduced the time complexity of scan from O(n*n) to O(n). This WL builds on
that to extend the same concept to other scans within buffer pool. There are
significant other changes which are made as part of reduced scanning logic.
The time complexity of a scan is reduced from O(n*n) to O(n). We have also implemented parallel flushing by having multiple page_cleaner
threads
Reference:
https://blog.mariadb.org/performance-evaluation-of-mariadb-10-1-and-mysql-5-7-4-labs-tplc/

4. www.vmcd.org
Speeding up Connection Handling
In 5.7 we have offloaded thread initialization and network initialization to a worker thread and more than doubled MySQL’s ability to do high
frequency connect/disconnect cycles, from 26K to 56K connect/disconnect cycles per second
Reference:
http://mysqlserverteam.com/improving-connectdisconnect-performance/
Bulk Data Load Improvements
Bulk Load for Create Index – Faster index creation
This method of index creation is also known as a “sorted index build”. This enhancement, which improves the efficiency of index creation, also
applies to full-text indexes. A new global configuration option, innodb_fill_factor, defines the percentage of space on each page that is filled
with data during a sorted index build, with the remaining space reserved for future index growth. For more information, see Section 14.13.18,
“Bulk Load for CREATE INDEX”.

5. www.vmcd.org
Bulk INSERT operations can be sped up by creating the secondary indexes afterwards. In this way, the records will be merge sorted and then
inserted into the InnoDB index B-trees in order, using sequentially allocated pages. This will create a more optimal layout especially for indexes
whose columns are updated rarely
Resize the InnoDB Buffer Pool Online
Change “ innodb_buffer_pool_size“ dynamically.
Reference:
http://mysqlserverteam.com/resizing-buffer-pool-online/
Automatic Truncation of UNOD Logs
Need separate UNDO tablespaces have been configured.
Reference:
http://mysqllover.com/?p=1051

6. www.vmcd.org
InnoDB Online Alter Table
Onine rename index and enlarge varchar column size
Online Rename Index This work by Marko Mäkelä (WL#6752) and Dmitry Lenev (WL#6555) makes it possible to rename an index as an online operation. Example:
mysql> ALTER TABLE t RENAME INDEX i1 TO i2;
Enlarge VARCHAR column size online (WL#6554) This work by Marko Mäkelä makes it possible to enlarge varchar column sizes as an online operation. This is true as
long as the number of bytes required by the VARCHAR column type (VARCHAR/LONGVARCHAR) remains the same, i.e. from 0 to 255 or from 256 to higher. Example:
mysql> ALTER TABLE t1 ALGORITHM=INPLACE, CHANGE COLUMN c1 c1 VARCHAR(255);
Setting Replication Filters Without Server Restart
Dynamically change Filter behaviors without restart server
Reference:
http://mysqlserverteam.com/mysql-5-7-3-making-mysql-slave-replication-filters-dynamic/

7. www.vmcd.org
CHANGE MASTER Without Stopping the SQL Thread
In order to add/alter an option using the CANGE MASTER TO command, it was previously necessary to issue a STOP SLAVE command before
the CHANGE MASTER TO command. This work relaxes that constraint.
Reference:
http://mysqlserverteam.com/mysql-5-7-4-change-master-without-stopping-slave-altogether/
http://work.shivjijha.com/2014/04/change-master-without-stopping-slave.html
Improved “IN queries” With Row Value Expressions to Be Executed Using Range Scans
As of 5.7.3 this hole in the net is stitched up. The range optimizer gladly opens the door for queries with IN predicates as long as
The predicate is only IN, not NOT IN.
The row on the predicate’s left-hand side is only indexed column references, in the same index.
The rows contain only constants or come from a previously read table in nested-loops join.
Note that ‘constants’ is a pretty broad category. It consists of pre-evaluated expressions, even some sub-queries, SQL variables and similar
beings

8. www.vmcd.org
Improving performance when using in predicates
Reference:
http://mysqlserverteam.com/range-access-now-in-an-in-predicate-near-you/
UNION ALL” No Longer Creates a Temporary Table
In 5.7 the optimizer avoids creating a temporary table for the result of UNION ALL queries when there is no need for it, i.e., when there is no
top-level ORDER BY clause
Reference:
http://mysqlserverteam.com/state-of-the-union/

9. www.vmcd.org
EXPLAIN for Running Queries
This feature is useful if you are running a statement in one session that is taking a long time to complete; using EXPLAIN FOR CONNECTION in
another session may yield useful information about the cause of the delay and thus help you optimize your schema and statements.
JSON EXPLAIN
Make Use of Condition Filtering in the Optimizer
Reference:
http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-1/
http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-2/

10. www.vmcd.org
Improved ONLY_FULL_GROUP_BY SQL Mode
ONLY_FULL_GROUP_BY SQL mode was enabled by default in 5.75+
In MySQL, one can write GROUP BY queries that reference non-aggregated columns in the SELECT list that are not included in the GROUP BY
clause, even if these columns are not functionally dependent upon the GROUP BY clause. This behaviour conforms to none of the SQL standard's
versions. It is possible to avoid this behaviour by including ONLY_FULL_GROUP_BY in the sql_mode server setting, but it might make more sense
to take advantage of the ability to write only partial GROUP BY clauses.
In a nutshell:
It is completely safe to write partial GROUP BY clauses as long as all non-aggregated columns in the SELECT list are functionally dependent upon
the GROUP BY clause.
A partial GROUP BY list can result in better performance, because it keeps the server from evaluating the entire GROUP BY list.
If one does not want to write partial GROUP BY clauses, consider using MIN or MAX to 'aggregate' the functionally dependent columns in the
SELECT list rather than moving the functionally dependent columns to the GROUP BY clause.
Reference:
http://rpbouman.blogspot.co.uk/2007/05/debunking-group-by-myths.html
http://mysqlserverteam.com/mysql-5-7-only_full_group_by-improved-recognizing-functional-dependencies-enabled-by-default/

11. www.vmcd.org
Copying File-Per-Table Tablespaces to Another Server
Prior to MySQL 5.7.4, only non-partitioned InnoDB tables are supported. As of MySQL 5.7.4, partitioned InnoDB tables and individual InnoDB
table partitions and subpartitions are also supported.
Reference:
https://dev.mysql.com/doc/refman/5.7/en/tablespace-copying.html
InnoDB Buffer Pool Dump and Load Enhancements
This work improves both the dump and load scenarios. It is now possible to dump only the hottest N% of the pages from each buffer pool. The load
operation is also made less disruptive to user activity because the load now happens in the background while continuing to serve clients
Reference:

12. www.vmcd.org
http://mysqlserverteam.com/mysql-dumping-and-reloading-the-innodb-buffer-pool/
http://mysqllover.com/?p=1123
Statement Timeouts
MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.
This feature is introduced as part of WL#6936. It is based on a contribution submitted by Davi Arnaut with Bug#68252. Thank you, Davi!
The statement timeouts work by interrupting the execution of the statement when it takes longer than a specified number of milliseconds to
complete. After the specified number of milliseconds has passed, the server aborts the individual query without affecting the larger transaction
or connection contexts. The following error is then reported to the client when the query is aborted:
1907: Query execution was interrupted, max_statement_time exceeded.
To be clear, the execution time limit specified is really a “soft hint”, because the query interruption may not happen precisely at the specified
execution time limit. There can be a minor delay between the timer expiration and the actual query interruption.
A time limit for any SELECT statement run against a MySQL instance can be set by specifying a timeout value in milliseconds for the GLOBAL
system variable max_statement_time.

13. www.vmcd.org
Reference:
http://planet.mysql.com/entry/?id=673840
http://mysqlserverteam.com/server-side-select-statement-timeouts/
Multiple User Level Locks
The difference in lock acquisition behavior as of MySQL 5.7.5 can be seen by the following example. Suppose that you execute these
statements:
SELECT GET_LOCK('lock1',10);
SELECT GET_LOCK('lock2',10);
SELECT RELEASE_LOCK('lock2');
SELECT RELEASE_LOCK('lock1');
In MySQL 5.7.5 or later, the second GET_LOCK() acquires a second lock and both RELEASE_LOCK() calls return 1 (success). Before MySQL 5.7.5,
the second GET_LOCK() releases the first lock ('lock1') and the secondRELEASE_LOCK() returns NULL (failure) because there is no 'lock1' to
release.

14. www.vmcd.org
Waiting for More Transactions to Enter Binlog Group Commit (BGC) Queues
Solve the lag problem in MySQL slave database. We could read Booking’s article for cascade M-S architecture.
binlog-group-commit-sync-delay, binlog-group-commit-sync-no-delay-count were used to control BGC.
Reference:
http://blog.booking.com/evaluating_mysql_parallel_replication_2-slave_group_commit.html
Semi-Sync Replication
Make the Master Wait for More than One Slave to Acknowledge Back
Externalize Transactions Only after ACK is Received
Semi-Sync — Separate ACKs Collector
Reference:

15. www.vmcd.org
http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html
http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html
http://my-replication-life.blogspot.sg/2014/03/faster-semisync-replication.html
Multi-Threaded Slaves (MTS)
Intra-Schema Parallel Slave Execution
Reference:
http://geek.rohitkalhans.com/2013/09/enhancedMTS-configuration.html
Ordered Commits (Sequential Consistency)
Ensure that the commits by slave applier threads running in parallel will be done in the same order as they were on the master.

16. www.vmcd.org
Support Slave Transaction Retries in Multi-Threaded Slave Mode
Transaction retry feature works for both types of mutli-threaded slave(database based and logical clock based). It also works fine when you
force slave to commit transactions in same order as master by setting system variable “slave_preserve_commit_order” to ON.
By default, transaction retry is on for all types of slave. Multi-threaded slave will not break if it just encounters some temporary errors
mentioned above.
TRUNCATE TABLE Statement Becomes Atomic
This work makes the internal InnoDB TRUNCATE TABLE statement atomic by reinitializing the original table-space header with the same space
id and then physically truncating its.ibd file during the truncation of a single table tablespace.
Reference:
http://mysqllover.com/?p=1053

17. www.vmcd.org
InnoDB Create Tablespace in 5.7
CREATE TABLESPACE `TBS_DADOS_COMPRESS` ADD DATAFILE 'TDS_DC_DATAFILE1.ibd' [FILE_BLOCK_SIZE=4];
CREATE TABLESPACE `TBS_DADOS` ADD DATAFILE 'TDS_D_DATAFILE1.ibd' [FILE_BLOCK_SIZE=8];
CREATE TABLE ITEM_VENDA TABLESPACE=`TBS_DADOS_COMPRESS`;
ALTER TABLE tbl_name TABLESPACE=`TBS_DADOS`
DROP TABLESPACE `TBS_DADOS_COMPRESS`;
Reference:
http://mathiasbrem.com.br/innodb-create-tablespace-mysql-5-7/
InnoDB Native Partitioning
Less handles less use of memory. Easy to access partition tables
Reference:

18. www.vmcd.org
http://mysqlserverteam.com/innodb-native-partitioning-early-access/
InnoDB Temporary Table Performance
Temporary Tables
Faster create/drop
No redo logging
Separate tablespace for all uncompressed temporary tables
Temporary tables don't require recovery therefore there is no point in saving their definitions in the persistent data dictionary. By putting the all
the temporary tables in a special tablespace that is not redo logged we can reduce a lot of the unnecessary overhead. We do UNDO log the
changes to temporary tables, this is required for rollback to savepoint. However, the UNDO logs for temporary tables are not redo logged and
the UNDO logs for temporary tables also reside in the special temporary tablespace.
Temp tables are only visible within the connection/session in which they were created, and they are bound by the lifetime of the server. We
optimized DML for Temp Tables (WL#6470) by removing unnecessary UNDO and REDO logging, change buffering, and locking. We added an
additional type of UNDO log (WL#6915), one that is not REDO logged and resides in a new separate temp tablespace.

19. www.vmcd.org
Multi-Source Replication
Nearly MariaDB. MySQL 5.7 starting to support Multi-Source replication.
Generated Columns
There are two kinds of Generated Columns: virtual (default) and stored. Virtual means that the column will be calculated on the fly when a
record is read from a table. Stored means that the column will be calculated when a new record is written in the table, and after that it will be
treated as a regular field. Both types can have NOT NULL restrictions, but only a stored Generated Column can be be a part of an index.
Reference:
http://mysqlserverteam.com/generated-columns-in-mysql-5-7-5/
http://mablomy.blogspot.com/2015/03/auto-generated-columns-in-mysql-57-two.html
http://mechanics.flite.com/blog/2015/04/13/using-mysql-5-dot-7-generated-columns-to-avoid-error-1070/

20. www.vmcd.org
SYS schema In MySQL
New in MySQL 5.7.7, the MySQL sys schema (originally the ps_helper project) is now included by default within the MySQL server
Reference:
http://mysqlserverteam.com/the-mysql-sys-schema-in-mysql-5-7-7/
Improved alter user syntax support in 5.7
Change Alter user syntax and add more useful commands.
Reference:
http://mysqlblog.fivefarmers.com/2015/04/10/improved-alter-user-syntax-support-in-5-7/

21. www.vmcd.org
SSL/TLS IN MYSQL 5.7
Security is receiving more attention and MySQL 5.7 aims to be the most secure MySQL Server release ever.
Reference:
http://mysqlblog.fivefarmers.com/2015/04/09/ssltls-in-mysql-5-7/
Secondary Indexes on XML BLOBs
We can create secondary index on this Column with generated column feature
Reference:
http://mablomy.blogspot.com/2015/04/secondary-indexes-on-xml-blobs-in-mysql.html

22. www.vmcd.org
Emulating roles with expanded proxy user support in 5.7.7
Explain how to use proxy user in MySQL
Reference:
http://mysqlblog.fivefarmers.com/2015/04/08/emulating-roles-with-expanded-proxy-user-support-in-5-7-7/