Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Whats New in MySQL 5.7

4,759 views

Published on

Geir Høydalsvik (Oracle) and Simon Mudd (Booking.com)

Published in: Software
  • Be the first to comment

Whats New in MySQL 5.7

  1. 1. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Whats New in MySQL 5.7 Geir Høydalsvik, Sr. Director, MySQL Engineering Simon Mudd, Senior DBA, Booking.com
  2. 2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  3. 3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Agenda First Geir will talk about “Features” • Highlights of 5.7 Improvements and Additions Next Simon will provide Booking’s Perspective • Upgrading and Putting 5.7 into production Oracle Confidential – Internal/Restricted/Highly Restricted 3
  4. 4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Benefit Highlights of MySQL 5.7 • Faster – Server – quicker queries, more IO, more connections • InnoDB engine, Faster Connect/Disconnect, More Optimized Queries • No need to use MyISAM – can have speed and consistency both now – Replication – scales higher, faster slave processing • Parallel replications (parallel replication) – order of magnitude faster(?) – GIS searching – quickly search geographic data • Native InnoDB Spatial indexes – Compression • Leverages new advanced Operating File System capabilities • Improves performance, saves space Oracle Confidential – Internal/Restricted/Highly Restricted 4
  5. 5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Benefit Highlights of MySQL 5.7 (con’t) Easier • Easier Configurability – dynamic “online” changes • More Flexible – Native JSON support – datatype, binary storage format – Virtual Columns/Functional indexes – index any data and find quickly • Simpler to Manage – More instruments, Easy access to data using SYS schema • More Secure – By default and more options to secure the server Oracle Confidential – Internal/Restricted/Highly Restricted 5
  6. 6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7 Sysbench Benchmark: SQL Point Selects / sec 3x Faster than MySQL 5.6 4x Faster than MySQL 5.5 1,600,000 QPS 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 8 16 32 64 128 256 512 1,024 QueriesperSecond Connections MySQL 5.7: Sysbench OLTP Read Only (SQL Point Selects) MySQL 5.7 MySQL 5.6 MySQL 5.5 Intel(R) Xeon(R) CPU E7-8890 v3 4 sockets x 18 cores-HT (144 CPU threads) 2.5 Ghz, 512GB RAM Linux kernel 3.16 6
  7. 7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7 Sysbench Benchmark: Mixed OLTP Read Only 3x Faster than MySQL 5.6 6x Faster than MySQL 5.5 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 8 16 32 64 128 256 512 1,024 QueriesperSecond Connections MySQL 5.7: Sysbench OLTP Read Only (Mixed) MySQL 5.7 MySQL 5.6 MySQL 5.5 Intel(R) Xeon(R) CPU E7-8890 v3 4 sockets x 18 cores-HT (144 CPU threads) 2.5 Ghz, 512GB RAM Linux kernel 3.16 Near 1M QPS 7
  8. 8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 82% Faster than MySQL 5.6 0 20,000 40,000 60,000 80,000 100,000 120,000 8 16 32 64 128 256 512 1,024 Connect/sec Connections MySQL 5.7: Sysbench OLTP Read Only (Connect) MySQL 5.7 MySQL 5.6 MySQL 5.5 Intel(R) Xeon(R) CPU E7-8890 v3 4 sockets x 18 cores-HT (144 CPU threads) 2.5 Ghz, 512GB RAM Linux kernel 3.16 100K Connect / Sec MySQL 5.7 Sysbench Benchmark: Connect / sec 8
  9. 9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7 Sysbench Benchmark: Complex OLTP Read Write 1.5x Faster than MySQL 5.6 3x Faster than MySQL 5.5 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 8 16 32 64 128 256 512 1,024 QueriesperSecond Connections MySQL 5.7: Sysbench OLTP Read Write (Complex) MySQL 5.7 MySQL 5.6 MySQL 5.5 Intel(R) Xeon(R) CPU E7-8890 v3 4 sockets x 18 cores-HT (144 CPU threads) 2.5 Ghz, 512GB RAM Linux kernel 3.16 9
  10. 10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7: JSON Overview • Native JSON data type – Native internal binary format for efficient processing & storage • Built-in JSON functions – Allowing you to efficiently store, search, update, and manipulate Documents • JSON Comparator – Allows for easy integration of Document data within your SQL queries • Indexing of Documents using Generated Columns – InnoDB supports indexes on both stored and virtual Generated Columns – New expression analyzer automatically uses the best “functional” index available 10
  11. 11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • Optimizer and Parser refactoring – Cleanly separate the parsing, optimizing, and execution stages • New hint framework – Easier to manage – With support for additional new hints • Improved JSON EXPLAIN • EXPLAIN for running thread • Generated Columns • New Cost based Optimizer – Configurable and tunable • mysql.server_cost and mysql.engine_cost tables • API for determining where data resides: on disk or in cache • Support for InnoDB based internal temp tables • SQL Standard compliant ONLY_FULL_GROUP_BY mode is ON by default • Many specific new optimizations MySQL 5.7: Optimizer Improvements 11 Queries execute faster, while using less CPU and disk space!
  12. 12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7: Query Rewrite Plugin • New pre and post parse query rewrite APIs – Users can write their own plug-ins • Provides a post-parse query plugin – Rewrite problematic queries without the need to make application changes – Add hints – Modify join order – Many more … • Improve problematic queries from ORMs, third party apps, etc • Eliminates many legacy use cases for proxies 12
  13. 13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Memory Instrumentation • Aggregates statistics by – Type of memory used (caches, internal buffers, …) – Thread/account/user/host indirectly performing the memory operation • Attributes include – Memory used (bytes) – Operation counts – High/Low Water Marks Statement Instrumentation • Stored Procedures • Stored Functions • Prepared Statements • Transactions Additional Information • Replication slave status • MDL lock instrumentation • Status and variables per thread • Server stage tracking • Track long running SQL • Improved configuration and ease-of-use • All while reducing total footprint and overhead MySQL 5.7: Performance Schema 13
  14. 14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL 5.7: SYS Schema SYS views make it easier to - Monitor server health, user, host statistics - Spot, diagnose, and tune performance issues • Provides insights into - IO hot spots, Locking, Costly SQL statements - Schema, table and index statistics • SYS is similar to - Oracle V$ catalog views - Microsoft SQL DMVs (Dynamic Mgmnt Views) 14
  15. 15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • Replaced custom code with Boost.Geometry – For spatial calculations – For spatial analysis – Enabling full OGC compliance – We’re also Boost.Geometry contributors! • InnoDB R-tree based spatial indexes – Full ACID, MVCC, & transactional support – Index records contain minimum bounding box • GeoHash • GeoJSON • Helper functions such as ST_Distance_Sphere() and ST_MakeEnvelope() MySQL 5.7: GIS Improvements 15
  16. 16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • Native Partitioning – Eliminates previous limitations – Eliminates resource usage problems – Transportable tablespace support • Native Full-Text Search – Including full CJK support! • Native Spatial Indexes • Transparent page compression • Support for 32K and 64K pages – Use with transparent page compression for very high compression ratios • General TABLESPACE support – Store multiple tables in user defined shared tablespaces • Support for MySQL Group Replication – High priority transactions • Improved support for cache preloading – Load your hottest data loaded at startup • Configurable fill-factor – Allows for improvements in storage footprint • Improved bulk-data load performance • Resize the InnoDB Buffer Pool online MySQL 5.7: InnoDB Improvements 16
  17. 17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • AES 256 Encryption now the default • Password rotation policies – Can be set globally, and at the user level • Deployment: enable secure unattended install by default – Random password set on install – Remove anonymous accounts – Deployment without test account, schema, demo files • Easier instance initialization and setup: mysqld –initialize • New detection and support for systemd • SSL – Enabled by default – Auto-detection of existing keys and certs – Auto generation of keys and certs when needed – New helper utility: mysql_ssl_rsa_setup – New --require_secure_transport option to prevent insecure communications – Added SSL support to binary log clients • Extended Proxy User Support – Added Built-in Authentication Plugins support for Proxy Users – Allows multiple users to share a single set of managed privileges MySQL 5.7: Security Improvements 17
  18. 18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • GTID enhancements – On-line, phased deployment of GTIDs – Binary logging on slave now optional • Enhanced Semi-synchronous replication – Write guaranteed to be received by slave before being observed by clients of the master – Option to wait on Acks from multiple slaves • Multi-Source Replication – Consolidate updates from multiple Masters into one Slave • Dynamic slave filters • 8-10x Faster slave throughput – Often removes slave as a bottleneck; keep pace with master with 8+ slave threads – Option to preserve Commit order – Automatic slave transaction retries MySQL 5.7: Replication Improvements 18 0% 50% 100% 150% 200% 250% 1 8 24 48 Slave Threads Slave throughput vs. 96 Thread Master
  19. 19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Perspectives on MySQL 5.7 - Upgrading to MySQL 5.7 Simon Mudd, Senior DBA 19
  20. 20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | About Booking.com Booking.com is part of the Priceline Group • World leader in booking accommodation online • Established in 1996 and based in Amsterdam – Over 801,000 properties in 221 countries – 42 languages (website and customer service) – 170 offices worldwide • We Use MySQL: – Thousands of servers: ~85% replicating – Over 110 masters: ~25 have 50+ slaves, ~8 have 100+ slaves 20
  21. 21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | What we look for when upgrading to a new major version? Basically motivation for upgrades: • Longer uptime and more dynamic configuration (avoid mysqld restart) • Better performance related to – Replication – Concurrency – Better stability under load • Seamless upgrade path from existing 5.6 setup to using 5.7 • More features and flexible configuration • Incorporation of features requested while using 5.6 • Better visibility of what MySQL is doing and where it is busy 21
  22. 22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | What does 5.7 provide? • Specific Features of Interest: – Parallel Replication – GTID migration possible on live system and no binlogs needed on slaves – Dynamic InnoDB buffer pool size reconfiguration – Dynamic replication configuration – Improved cache preloading on startup (configurable percentage to load) – InnoDB Fill-size – should be good for statistics tables which are write-once – Loss-less semi-sync • Plus others mentioned previously 22
  23. 23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Why booking.com tests MySQL 5.7 DMR releases Testing of DMR releases has allowed us to: • Preview potential new features and give feedback • Check performance meets expectations • Catch unexpected behaviour early Resources required to do this are not significant 23
  24. 24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | How to upgrade - in pictures +---+ | M | +---+ | +----+---+---------+ | | | +----+ +----+ +----+ | s1 | | s2 | ... | sn | +----+ +----+ +----+ Everyone starts this way All same version of MySQL (say 5.6.27) Master Slave 1 Slave 2 Slave N … 24
  25. 25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | How to upgrade - in pictures – start with slaves Master Slave 1 Slave 2 Slave N MySQL 5.7 … Upgrade slaves 25
  26. 26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Upgrading MySQL - upgrade steps If more than 1 master and slave the procedure is basically the same. 1. Make a new slave or upgrade an existing one 2. Configure /etc/my.cnf for 5.7 3. Change binaries 4. Start mysqld with replication stopped 5. Run: mysql_upgrade 6. Restart mysqld normally 7. Test… 26
  27. 27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | How to upgrade - in pictures – Final Steps 1. Move slaves under 5.7 intermediate master 2. Move application writes to 5.7 master Master Slave 1 MySQL 5.7 Slave 2 MySQL 5.7 Slave 3 MySQL 5.7 Slave N MySQL 5.7 Intermediate Master MySQL 5.7 … 27
  28. 28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | What are the Risks? For initial testing risks are minimal. • If something breaks: – throw it away, fix internal settings or – report a bug • With Cloud infrastructure it is easy to build a spare & test for a few hours As you continue testing MySQL 5.7 • You control the rate of progress of deploying 5.7 slaves • Easy to back out until right at the end (with 5.7 intermediate master) • No application downtime except for the final move when you write to the new 5.7 master. 28
  29. 29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Moving MySQL 5.7 into production – observations • The setup of a new environment is simpler – the new defaults require less cleanup • Configuration changes in /etc/my.cnf – Few required changes • log_syslog • innodb_buffer_pool_dump_pct • show_compatibility_56 – Pay attention to changes in default values between 5.6 and 5.7 – We have had to add some extra settings • Upgrading to 5.7 vs previous upgrades – Slight changes in procedures but generally they are simpler – Just run mysql_upgrade, no need for --skip-grant-tables 29
  30. 30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Moving MySQL 5.7 into production – observations (cont.) • Grant management – Procedures need to be MySQL version aware (new disable accounts vs old change password) – New ALTER USER is replication safe, but can’t use on master until all servers run 5.7 • Improve monitoring • Add more checks • Collect 5.7 specific metrics from performance_schema / sys – Replication – Memory usage – Locks 30
  31. 31. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Results of our MySQL 5.7 DMR testing • 2 years following the evolution of MySQL 5.7 DMR – First installed in 06/2013 (5.7.1_m11) • We provided feedback on several issues, e.g. – SQL_MODE changes caused problems with replication (5.6 to 5.7) – ALTER, CREATE, DROP USER were not replication safe (5.6 to 5.7) – Unexpected crashes on startup in some early releases of 5.7 – Parallel Replication • MySQL engineering responded to these issues – No longer an issue in the recently announced GA version – Avoided potential breakage in a GA version where functional changes would have been impossible/undesirable • Now 5.7 is GA we are in a good state to start to roll out more aggressively 31
  32. 32. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Community Reception @ Oracle OpenWorld • Tuesday, October 27: 7 pm • Jillian’s at Metreon: 175 Fourth Street, San Francisco At the corner of Howard and 4th st.; only 2-min walk from Moscone Center (same place as last year) Join us! Celebrate, Have Fun and Mingle with Oracle’s MySQL Engineers & Your Peers 32
  33. 33. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Sessions • CON3716 - InnoDB: What's new in 5.7 – Sunny Bains | Room 254 at 4.00 pm • CON2353 - MySQL Server Performance Tuning 101 – Ligaya Isler-turmelle | Room 262 – CON2257 - MySQL Sys Schema Deep Dive - Mark Leith • CON5097 - An Overview Of The New Replication Features in MySQL 5.7 and Beyond! – Luis Soares | Room 254 at 5:15 pm – CON4043 - MySQL Performance: Demystified Tuning & Best Practices - Dimitri Kravtchuk Today, Tues Oct 27 33
  34. 34. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Sessions • CON3379 - MySQL 5.7: What is new in optimizer? – Manyi Lu | Room 262 at 12.15 pm • CON2131 - JSON support in MySQL 5.7 – Evgeny Potemkin | Room 262 at 1.30 pm • CON2174 - MySQL: Support Virtual Column and Effective Functional Index in InnoDB – Jimmy Yang 245 | Room 262 at 2.45 pm • CON4975 - Scalable High-Availability Systems using MySQL Fabric – Mats Kindahl | Room 262 at 4.00 pm Tomorrow, Weds 0ct 28 34
  35. 35. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Sessions • CON5349 - Update Everywhere With MySQL Group Replication – Nuno Carvalho | Room 250 at 9.30 am • CON4507 - Oracle Enterprise Manager for MySQL Database Latest Features – Carlos Proal Aguilar | Room 262 at 10.45 am • CON4062 - Securing your MySQL Environment with MySQL Enterprise Monitor – Mark Matthews | Room 262 at 12.00 pm • CON1559 - What's New in MySQL 5.7 Security – Georgi Kodinov | Room 262 at 1.15 pm • CON2625 - OpenStack and MySQL – Matt Lord | Room 262 at 1.15 pm Thurs Oct 29 35
  36. 36. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Booking.com Sessions • CON4098 - Binlog Servers at Booking.com – Jean-François Gagne | Room 250 Tuesday at 4.00 pm • BOF5528 - Riding the Binary Logs: Forthcoming Evolution in the Replication Stream – Jean-François Gagne | Moscone South—262 Tuesday at 6.15 pm • CON4073 - The Scalability Axes of MySQL Architectures – Nicolai Plum (Booking) | Room 250 Wednesday at 2.45 pm 36
  37. 37. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Central @ OpenWorld • Keynotes • Conferences Sessions • Birds-of-a-feather sessions • Tutorials • Hands-on Labs • Demos • Receptions & Customer Appreciation Event • Access to Oracle OpenWorld Extensive Content October 25-29, 2015 | San Francisco 37

×