Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MySQL 5.7 milestone

1,329 views

Published on

MySQL 5.7+ new features !

Published in: Technology
  • Be the first to comment

MySQL 5.7 milestone

  1. 1. www.vmcd.org MySQL 5.7 Important Features InnoDB Read-Only Scalability Improving the performance for Read-Only and Read-Mostly workloads. Significantly improved how InnoDB handles RO trans-actions Removed server layer contentions related to Meta Data Locking (MDL) and removed the use of “thread locks” (thd_locks) for InnoDB In MySQL 5.7.2, there is another optimization where all transactions are treated as a read-only transactions by default, and a transaction is only assigned an ID and a rollback segment when the transaction attempts to do its first update. This optimization of treating all transactions as read-only by default has the additional benefit that users do not need to use special syntax or change their applications to take advantage of it. Reference: http://dimitrik.free.fr/blog/archives/09-01-2013_09-30-2013.html http://mysqlserverteam.com/mysql-5-7-over-1m-qps-with-innodb-memcached-plugin/ http://mysqlserverteam.com/transaction-life-cycle-improvements-in-5-7-3/ http://mysqlserverteam.com/mysql-5-7-3-deep-dive-into-1mil-qps-with-innodb-memcached/
  2. 2. www.vmcd.org InnoDB Read-Write Scalability. Improved the performance of Read-Write (RW) workloads. We have removed the “index lock contention” in InnoDB. The index lock that was used to protect the entire index tree structure is now replaced by more fine grained “block locks” in the tree. InnoDB has two "modes" when updating the Btrees. An optimistic mode and a pessimistic mode. They roughly correspond to in-place changes and tree modification changes. If InnoDB thinks that an insert/delete will result in a tree modification change it will lock the index in X mode. This blocks readers too and limits concurrency. The fix was to introduce a new lock mode SX. This reduces the need to lock the entire Btree during tree modification changes. Reference: https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements http://mysqlserverteam.com/mysql-5-7-improves-dml-oriented-workloads/ https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements
  3. 3. www.vmcd.org InnoDB Faster & Parallel Flushing InnoDB has had some sub-optimal code around traversing the dirty page and LRU list flushing. The problem was that when we release the buffer pool and associated mutexes during the IO phase we can no longer trust the iterator that is used to scan the aforementioned lists. So, we restart the scan from the start. The fix was to track whether the iterator was invalidated while the covering mutexes where released. If the iterator was not invalidated then we can carry on from where we left off. If on the other hand we detect that it was invalidated we go back to the start and do it the old way. The iterator invalidation should be very rare. Reducing the number of pages scanned when doing flush list batches, speeding up page flushing Recently our benchmarks showed that we are scanning excessive number of pages when doing flush list batches. We fixed it by introducing the concept of Hazard Pointer which reduced the time complexity of scan from O(n*n) to O(n). This WL builds on that to extend the same concept to other scans within buffer pool. There are significant other changes which are made as part of reduced scanning logic. The time complexity of a scan is reduced from O(n*n) to O(n). We have also implemented parallel flushing by having multiple page_cleaner threads Reference: https://blog.mariadb.org/performance-evaluation-of-mariadb-10-1-and-mysql-5-7-4-labs-tplc/
  4. 4. www.vmcd.org Speeding up Connection Handling In 5.7 we have offloaded thread initialization and network initialization to a worker thread and more than doubled MySQL’s ability to do high frequency connect/disconnect cycles, from 26K to 56K connect/disconnect cycles per second Reference: http://mysqlserverteam.com/improving-connectdisconnect-performance/ Bulk Data Load Improvements Bulk Load for Create Index – Faster index creation This method of index creation is also known as a “sorted index build”. This enhancement, which improves the efficiency of index creation, also applies to full-text indexes. A new global configuration option, innodb_fill_factor, defines the percentage of space on each page that is filled with data during a sorted index build, with the remaining space reserved for future index growth. For more information, see Section 14.13.18, “Bulk Load for CREATE INDEX”.
  5. 5. www.vmcd.org Bulk INSERT operations can be sped up by creating the secondary indexes afterwards. In this way, the records will be merge sorted and then inserted into the InnoDB index B-trees in order, using sequentially allocated pages. This will create a more optimal layout especially for indexes whose columns are updated rarely Resize the InnoDB Buffer Pool Online Change “ innodb_buffer_pool_size“ dynamically. Reference: http://mysqlserverteam.com/resizing-buffer-pool-online/ Automatic Truncation of UNOD Logs Need separate UNDO tablespaces have been configured. Reference: http://mysqllover.com/?p=1051
  6. 6. www.vmcd.org InnoDB Online Alter Table Onine rename index and enlarge varchar column size Online Rename Index This work by Marko Mäkelä (WL#6752) and Dmitry Lenev (WL#6555) makes it possible to rename an index as an online operation. Example: mysql> ALTER TABLE t RENAME INDEX i1 TO i2; Enlarge VARCHAR column size online (WL#6554) This work by Marko Mäkelä makes it possible to enlarge varchar column sizes as an online operation. This is true as long as the number of bytes required by the VARCHAR column type (VARCHAR/LONGVARCHAR) remains the same, i.e. from 0 to 255 or from 256 to higher. Example: mysql> ALTER TABLE t1 ALGORITHM=INPLACE, CHANGE COLUMN c1 c1 VARCHAR(255); Setting Replication Filters Without Server Restart Dynamically change Filter behaviors without restart server Reference: http://mysqlserverteam.com/mysql-5-7-3-making-mysql-slave-replication-filters-dynamic/
  7. 7. www.vmcd.org CHANGE MASTER Without Stopping the SQL Thread In order to add/alter an option using the CANGE MASTER TO command, it was previously necessary to issue a STOP SLAVE command before the CHANGE MASTER TO command. This work relaxes that constraint. Reference: http://mysqlserverteam.com/mysql-5-7-4-change-master-without-stopping-slave-altogether/ http://work.shivjijha.com/2014/04/change-master-without-stopping-slave.html Improved “IN queries” With Row Value Expressions to Be Executed Using Range Scans As of 5.7.3 this hole in the net is stitched up. The range optimizer gladly opens the door for queries with IN predicates as long as The predicate is only IN, not NOT IN. The row on the predicate’s left-hand side is only indexed column references, in the same index. The rows contain only constants or come from a previously read table in nested-loops join. Note that ‘constants’ is a pretty broad category. It consists of pre-evaluated expressions, even some sub-queries, SQL variables and similar beings
  8. 8. www.vmcd.org Improving performance when using in predicates Reference: http://mysqlserverteam.com/range-access-now-in-an-in-predicate-near-you/ UNION ALL” No Longer Creates a Temporary Table In 5.7 the optimizer avoids creating a temporary table for the result of UNION ALL queries when there is no need for it, i.e., when there is no top-level ORDER BY clause Reference: http://mysqlserverteam.com/state-of-the-union/
  9. 9. www.vmcd.org EXPLAIN for Running Queries This feature is useful if you are running a statement in one session that is taking a long time to complete; using EXPLAIN FOR CONNECTION in another session may yield useful information about the cause of the delay and thus help you optimize your schema and statements. JSON EXPLAIN Make Use of Condition Filtering in the Optimizer Reference: http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-1/ http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-2/
  10. 10. www.vmcd.org Improved ONLY_FULL_GROUP_BY SQL Mode ONLY_FULL_GROUP_BY SQL mode was enabled by default in 5.75+ In MySQL, one can write GROUP BY queries that reference non-aggregated columns in the SELECT list that are not included in the GROUP BY clause, even if these columns are not functionally dependent upon the GROUP BY clause. This behaviour conforms to none of the SQL standard's versions. It is possible to avoid this behaviour by including ONLY_FULL_GROUP_BY in the sql_mode server setting, but it might make more sense to take advantage of the ability to write only partial GROUP BY clauses. In a nutshell: It is completely safe to write partial GROUP BY clauses as long as all non-aggregated columns in the SELECT list are functionally dependent upon the GROUP BY clause. A partial GROUP BY list can result in better performance, because it keeps the server from evaluating the entire GROUP BY list. If one does not want to write partial GROUP BY clauses, consider using MIN or MAX to 'aggregate' the functionally dependent columns in the SELECT list rather than moving the functionally dependent columns to the GROUP BY clause. Reference: http://rpbouman.blogspot.co.uk/2007/05/debunking-group-by-myths.html http://mysqlserverteam.com/mysql-5-7-only_full_group_by-improved-recognizing-functional-dependencies-enabled-by-default/
  11. 11. www.vmcd.org Copying File-Per-Table Tablespaces to Another Server Prior to MySQL 5.7.4, only non-partitioned InnoDB tables are supported. As of MySQL 5.7.4, partitioned InnoDB tables and individual InnoDB table partitions and subpartitions are also supported. Reference: https://dev.mysql.com/doc/refman/5.7/en/tablespace-copying.html InnoDB Buffer Pool Dump and Load Enhancements This work improves both the dump and load scenarios. It is now possible to dump only the hottest N% of the pages from each buffer pool. The load operation is also made less disruptive to user activity because the load now happens in the background while continuing to serve clients Reference:
  12. 12. www.vmcd.org http://mysqlserverteam.com/mysql-dumping-and-reloading-the-innodb-buffer-pool/ http://mysqllover.com/?p=1123 Statement Timeouts MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements. This feature is introduced as part of WL#6936. It is based on a contribution submitted by Davi Arnaut with Bug#68252. Thank you, Davi! The statement timeouts work by interrupting the execution of the statement when it takes longer than a specified number of milliseconds to complete. After the specified number of milliseconds has passed, the server aborts the individual query without affecting the larger transaction or connection contexts. The following error is then reported to the client when the query is aborted: 1907: Query execution was interrupted, max_statement_time exceeded. To be clear, the execution time limit specified is really a “soft hint”, because the query interruption may not happen precisely at the specified execution time limit. There can be a minor delay between the timer expiration and the actual query interruption. A time limit for any SELECT statement run against a MySQL instance can be set by specifying a timeout value in milliseconds for the GLOBAL system variable max_statement_time.
  13. 13. www.vmcd.org Reference: http://planet.mysql.com/entry/?id=673840 http://mysqlserverteam.com/server-side-select-statement-timeouts/ Multiple User Level Locks The difference in lock acquisition behavior as of MySQL 5.7.5 can be seen by the following example. Suppose that you execute these statements: SELECT GET_LOCK('lock1',10); SELECT GET_LOCK('lock2',10); SELECT RELEASE_LOCK('lock2'); SELECT RELEASE_LOCK('lock1'); In MySQL 5.7.5 or later, the second GET_LOCK() acquires a second lock and both RELEASE_LOCK() calls return 1 (success). Before MySQL 5.7.5, the second GET_LOCK() releases the first lock ('lock1') and the secondRELEASE_LOCK() returns NULL (failure) because there is no 'lock1' to release.
  14. 14. www.vmcd.org Waiting for More Transactions to Enter Binlog Group Commit (BGC) Queues Solve the lag problem in MySQL slave database. We could read Booking’s article for cascade M-S architecture. binlog-group-commit-sync-delay, binlog-group-commit-sync-no-delay-count were used to control BGC. Reference: http://blog.booking.com/evaluating_mysql_parallel_replication_2-slave_group_commit.html Semi-Sync Replication Make the Master Wait for More than One Slave to Acknowledge Back Externalize Transactions Only after ACK is Received Semi-Sync — Separate ACKs Collector Reference:
  15. 15. www.vmcd.org http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html http://my-replication-life.blogspot.sg/2014/03/faster-semisync-replication.html Multi-Threaded Slaves (MTS) Intra-Schema Parallel Slave Execution Reference: http://geek.rohitkalhans.com/2013/09/enhancedMTS-configuration.html Ordered Commits (Sequential Consistency) Ensure that the commits by slave applier threads running in parallel will be done in the same order as they were on the master.
  16. 16. www.vmcd.org Support Slave Transaction Retries in Multi-Threaded Slave Mode Transaction retry feature works for both types of mutli-threaded slave(database based and logical clock based). It also works fine when you force slave to commit transactions in same order as master by setting system variable “slave_preserve_commit_order” to ON. By default, transaction retry is on for all types of slave. Multi-threaded slave will not break if it just encounters some temporary errors mentioned above. TRUNCATE TABLE Statement Becomes Atomic This work makes the internal InnoDB TRUNCATE TABLE statement atomic by reinitializing the original table-space header with the same space id and then physically truncating its.ibd file during the truncation of a single table tablespace. Reference: http://mysqllover.com/?p=1053
  17. 17. www.vmcd.org InnoDB Create Tablespace in 5.7 CREATE TABLESPACE `TBS_DADOS_COMPRESS` ADD DATAFILE 'TDS_DC_DATAFILE1.ibd' [FILE_BLOCK_SIZE=4]; CREATE TABLESPACE `TBS_DADOS` ADD DATAFILE 'TDS_D_DATAFILE1.ibd' [FILE_BLOCK_SIZE=8]; CREATE TABLE ITEM_VENDA TABLESPACE=`TBS_DADOS_COMPRESS`; ALTER TABLE tbl_name TABLESPACE=`TBS_DADOS` DROP TABLESPACE `TBS_DADOS_COMPRESS`; Reference: http://mathiasbrem.com.br/innodb-create-tablespace-mysql-5-7/ InnoDB Native Partitioning Less handles less use of memory. Easy to access partition tables Reference:
  18. 18. www.vmcd.org http://mysqlserverteam.com/innodb-native-partitioning-early-access/ InnoDB Temporary Table Performance Temporary Tables Faster create/drop No redo logging Separate tablespace for all uncompressed temporary tables Temporary tables don't require recovery therefore there is no point in saving their definitions in the persistent data dictionary. By putting the all the temporary tables in a special tablespace that is not redo logged we can reduce a lot of the unnecessary overhead. We do UNDO log the changes to temporary tables, this is required for rollback to savepoint. However, the UNDO logs for temporary tables are not redo logged and the UNDO logs for temporary tables also reside in the special temporary tablespace. Temp tables are only visible within the connection/session in which they were created, and they are bound by the lifetime of the server. We optimized DML for Temp Tables (WL#6470) by removing unnecessary UNDO and REDO logging, change buffering, and locking. We added an additional type of UNDO log (WL#6915), one that is not REDO logged and resides in a new separate temp tablespace.
  19. 19. www.vmcd.org Multi-Source Replication Nearly MariaDB. MySQL 5.7 starting to support Multi-Source replication. Generated Columns There are two kinds of Generated Columns: virtual (default) and stored. Virtual means that the column will be calculated on the fly when a record is read from a table. Stored means that the column will be calculated when a new record is written in the table, and after that it will be treated as a regular field. Both types can have NOT NULL restrictions, but only a stored Generated Column can be be a part of an index. Reference: http://mysqlserverteam.com/generated-columns-in-mysql-5-7-5/ http://mablomy.blogspot.com/2015/03/auto-generated-columns-in-mysql-57-two.html http://mechanics.flite.com/blog/2015/04/13/using-mysql-5-dot-7-generated-columns-to-avoid-error-1070/
  20. 20. www.vmcd.org SYS schema In MySQL New in MySQL 5.7.7, the MySQL sys schema (originally the ps_helper project) is now included by default within the MySQL server Reference: http://mysqlserverteam.com/the-mysql-sys-schema-in-mysql-5-7-7/ Improved alter user syntax support in 5.7 Change Alter user syntax and add more useful commands. Reference: http://mysqlblog.fivefarmers.com/2015/04/10/improved-alter-user-syntax-support-in-5-7/
  21. 21. www.vmcd.org SSL/TLS IN MYSQL 5.7 Security is receiving more attention and MySQL 5.7 aims to be the most secure MySQL Server release ever. Reference: http://mysqlblog.fivefarmers.com/2015/04/09/ssltls-in-mysql-5-7/ Secondary Indexes on XML BLOBs We can create secondary index on this Column with generated column feature Reference: http://mablomy.blogspot.com/2015/04/secondary-indexes-on-xml-blobs-in-mysql.html
  22. 22. www.vmcd.org Emulating roles with expanded proxy user support in 5.7.7 Explain how to use proxy user in MySQL Reference: http://mysqlblog.fivefarmers.com/2015/04/08/emulating-roles-with-expanded-proxy-user-support-in-5-7-7/

×