TLV - Whats new in MySQL 8

1. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0 RC1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
Mark Swarbrick
Mark.swarbrick@oracle.com

2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
A Modern Database for the Digital Age
Digital Disruptors & Large Enterprises Rely on MySQL to Innovate
3

4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
They Scale
with MySQL
Mobile Network
Supporting Over 800
Million Subscribers
1.7 Billion Active Users
100 TB of User Data for PayPal
IDs Processed for 1
Billion Citizens
850 Million Candy
Crush Game Plays/Day
2 Billion Events/Day for
Booking.com
4

5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Innovation: 5.7 -> 8.0
- 3x Better Performance
- Replication Enhancements
- Optimizer Cost Model
- JSON Support
- Improved Security
- Sys & Performance Schema
- GIS
MySQL 5.7 (GA)
MySQL InnoDB Cluster (GA)
- MySQL Group Replication
- MySQL Router
- MySQL Shell
MySQL 8.0
- Document Store
- Data Dictionary
- Roles
- Unicode
- CTEs
- Window Functions
- Security
- Replication
- SysSchema
- GIS
2 Years in Development
400+ Worklogs
5000+ Bugs Fixed
500 New Tests

6. 24x7 at Scale
Mobile First
Developer First
Data Driven
8.0
GA

7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Transactional (InnoDB) Data Dictionary
• Persistent runtime configuration changes (SET PERSIST)
• Auto-increment counter value will now persist across server restarts!
• Indexing of PS tables
• SQL roles (collection of privileges)
• MySQL privilege tables are now InnoDB
• INVISIBLE indexes
• Common Table Expressions
• Windows Functions
What’s new
7

8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• New default: utf8mb4 (UTF 9)
• Expanded GIS Support
• Expanded Query Hints Support
• Improved Scan Query Performance
• Improved BLOB Storage
• Improved InnoDB Memcached Interface
• Descending indexes
• Scalability Improvements
• Parser Refactoring
• New! Document Store
• Improvements to Temporary Tables
• C++11 and Toolchain Improvements
• Replication Applier Thread Progress
Reports
• GTID_PURGED always settable
• …
8
MySQL 8.0: All this and more….

9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 9
The character set for the Web
MySQL 8.0: UTF-8
• New! Support for the latest Unicode 9.0
• Per-country collation rules
– Accent Sensitive
– Case Sensitive
• UTF8MB4 as the default character set
– Project started in MySQL 5.7
– Many improvements to reduce performance impact

10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
UTF-8
• UTF-8 is the dominating character
set in today’s applications
• Requires 1-4 bytes for storing
characters
• Historically a performance problem
The character set for the Web
10
https://en.wikipedia.org/wiki/UTF-8

11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Increased Reliability
• Built on InnoDB
– No FRM files
– No DB.OPT files
– No TRG files
– No TRN files
– No PAR files
– MyISAM not required
• Atomic DDL
– Better Replication
– Simplifies server edge cases
• MDL for Foreign Keys
• Flexible Metadata API
– Easier path to adding new features
• Better cross-platform experience
– No dependencies on filesystem
semantics
11
MySQL 8.0: Native Data Dictionary

12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 12
MySQL 8.0: Performance Schema Indexes
• Allows for more efficient access to Performance
Schema tables
• A total of 115 indexes across 93 tables
• Adds zero overhead
– A physical index is not maintained internally
– Implementation of indexes tricks the optimizer into better execution plan
Feature Request
from DBAs

13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 13
MySQL 8.0: Performance of SYS Schema
0 5 10 15 20 25 30 35 40
Query Time MySQL 8.0
MySQL 5.7
SELECT * FROM sys.session
1000 active sessions
Time in Seconds (Lower is better)
Up to 30x Faster!

14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Cloud Friendly
• New variable innodb_dedicated_server
– according to the amount of memory detected on the server, automatically configure
innodb_buffer_pool_size, innodb_log_file_size, innodb_flush_method
• Persistent runtime configuration changes
– SET PERSIST variable_name = value;
– Any GLOBAL DYNAMIC configuration variable can be modified
– the settings will be persisted to a file named mysqld-auto.cnf (JSON format)
– New table performance_schema.variables_info
14

15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 15
MySQL 8.0: Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
• Two use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
Feature Request
from DBAs

16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Security Roles
• Fully Function, Flexible, Properly Architected Roles
• Create and Drop Roles, Grant to Roles
• Grant Roles to Roles, Grant Roles to Users
• Limit Hosts that can use roles, Define Default Roles
• Decide what roles are applicable during a session
• And even visualize Roles with SQL function ROLES_GRAPHML()

17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 17
MySQL 8.0: Atomicity in Privileges
• Privilege Tables now 100% InnoDB
• User Management DDLs Atomic
– CREATE USER
– ALTER USER
– RENAME USER
– DROP USER
– GRANT
– REVOKE

18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Better Handing of Hot Rows
• Common problem:
– Hot row contention, multiple worker threads accessing the same rows
• Solution 1:
– Only read rows that are not locked
– InnoDB skips a locked row, and the next one goes to the result set
• Example:
– Booking system: Skip orders that are pending
SELECT ... FOR UPDATE SKIP LOCKED
18
START TRANSACTION;
SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO‘
FOR UPDATE SKIP LOCKED;
Feature Request
from Developers

19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Better Handing of Hot Rows (cont.)
• Common problem:
– Hot row contention, multiple worker threads accessing the same rows
• Solution 2:
– If any of the rows are already locked, the statement should fail immediately
– Without NOWAIT, have to wait for innodb_lock_wait_timeout (default: 50 sec) while
trying to acquire lock
SELECT… FOR UPDATE NOWAIT
19
START TRANSACTION;
SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO‘
FOR UPDATE NOWAIT;
ERROR 3572 (HY000): Statement aborted because lock(s) could not be acquired …
Feature Request
from Developers

20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
GIS
• Geography Support
• Spatial Reference Systems (SRS) Support
• SQL/MM Information Schema views
• Standard compliant axis ordering in import/export functions
• Helper functions to manipulate and convert data:
• st_y(geom, y)
• st_srid(geom, srid)
20

21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Windows Functions
• Window functions gather results into a set of rows,
returning a single aggregated value for each of the rows
in the result set. Gives you a very practical tool for
estimations and rankings.
• Perform some calculation on a set of rows
• Over(partition by ...)
• Simplifies writing complex SQL:
• OVER(), OTHER(partition)
21
Feature Request
from Developers

22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Windows Functions
22
Feature Request
from Developers

23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Windows Functions
23
Feature Request
from Developers
• Use the usual aggregate functions or...
• RANK(), DENSE_RANK(), PERCENT_RANK()
• CUME_DIST()
• NTILE()
• ROW_NUMBER()
• FIRST_VALUE(), LAST_VALUE(), NTH_VALUE()
• LEAD(), LAG()

24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
New! UUID and Bit-wise Improvements
• Functions to convert UUID to and from binary:
– UUID_TO_BIN()
– BIN_TO_UUID()
– plus IS_UUID()
• Bit-wise operations on binary data types
• Bit-wise operations on binary data types
– Designed with IPv6 in mind:
– INET6_ATON() & INET6_NTOA()
– No longer truncation beyond 64 bits
24
Feature Request
from Developers

25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 25
• Evolution from MySQL 5.5 to 8.0
Performance Schema
MySQL 8.0
Histograms
Indexes
Data Locks instrumentation
SQL Errors instrumentation
Variables Instrumentation
Table plugin
Improved Defaults
MySQL 5.7
Memory Instrumentation
Prepared Statements Instrumentation
Transactions Instrumentation
Scalable Memory Allocation
Bundled SYS schema
Lower Overhead
MySQL 5.6
Statement Instrumentation
Lower Overhead
MySQL 5.5
Event Waits
Mutexes
Files
Threads
Feature Request
from DBAs

26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Improved JSON Support
– Better perfomances, new functions (aggregate, formatting...), support for ranges
• Better Handling of Hot Rows
• Unicode 9.0
• Improvements to Query Consistency (Optimizer Histograms)
• Extended GIS
What’s new
26

27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Cloud Friendly
– innodb_dedicated_server
– SET PERSIST
• variables_info Performance Schema Table
• Native Data Dictionary
• Invisible Indexes
• Improvements to Defaults
– innodb_undo_log_truncate, innodb_undo_tablespaces ,
innodb_autoinc_lock_mode...
• Refactoring and Modernization
What’s new
27

28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
JSON Developer Experience
• Ranges in JSON path expressions:
• Optimizer Support for in-place update
• Performance Improvements
28
MySQL 8.0
SELECT doc->>"$[last]" AS last from t1;
+------+
| last |
+------+
| 5 |
+------+
1 row in set (0.00 sec)
SELECT JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[1 to 3]');
+----------------------------------------------+
| JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[1 to 3]') |
+----------------------------------------------+
| [2, 3, 4] |
+----------------------------------------------+
1 row in set (0.00 sec)

29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
JSON Functions
29
MySQL 5.7 and 8.0
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
JSON_PRETTY()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_ARRAYAGG()
JSON_OBJECTAGG()

30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 30
• Improved to consider buffer pool fit
Optimizer Cost Model
Feature Request
from DBAs
SELECT * FROM Country
WHERE population > 20000000;
Model for a table scan:
# pages in table *
(IO_BLOCK_READ_COST |
MEMORY_BLOCK_READ_COST)!
# records * ROW_EVALUATE_COST
= 25.4 100% in memory
= 29.9 100% on disk
Model for a range scan:
# records_in_range *
(IO_BLOCK_READ_COST |
MEMORY_BLOCK_READ_COST)!
# records_in_range *
ROW_EVALUATE_COST + #
records_in_range *
ROW_EVALUATE_COST !
= 22.5 100% in memory
= 60 100% on disk
Model accounts for
memory fit. For data on
disk an IO block read
defaults to 1.0. In
memory defaults to 0.25.
Much larger
performance
difference for range
scan not in memory
(good)

31. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
query: INSERT INTO `t1` VALUES (...)
db: mysqlslap
total_latency: 54.43 s
exec_count: 58377
lock_latency: 1.70 s
..
digest: 4e0c5b796c4052b0da4548fd7cb694be
first_seen: 2017-04-16 20:59:16
last_seen: 2017-04-16 21:00:34
latency_distribution:
0us+
10us+ #############################
100us+ ####################
1ms+ #
10ms+
100ms+
1s+
10s+
New! Performance Schema Histograms
31
+---------+--------------------------------+-------+
| bucket | visualization | count |
+---------+--------------------------------+-------+
| 0us+ | # | 1253 |
| 10us+ | ############################## | 43102 |
| 100us+ | ################# | 25013 |
| 1ms+ | # | 2003 |
| 10ms+ | | 325 |
| 100ms+ | | 17 |
| 1s+ | | 0 |
| 10s+ | | 0 |
+---------+--------------------------------+-------+
8 rows in set (0.08 sec)
Showing distribution of query time from a run of mysqlslap
Generated with a quick CTE
over
events_statements_histogram
_global
Feature Request
from DBAs
Available on a per statement
digest level. Can quickly
aggregate top-N statements
with latency distribution.

32. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
New! Better Handing of Hot Rows
32
SELECT seat_no
FROM seats
JOIN seat_rows USING ( row_no )
WHERE seat_no IN (3,4)
AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seat_rows NOWAIT;
Non
deterministically
skip over locked
rows
Error immediately if
a row is already
locked
Feature Request
from Developers

33. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What is the MySQL Document Store?
"An easy, straight forward way to work with JSON
documents in MySQL"
33

34. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Document Oriented Databases
• Schemaless: No centralized database schema
– Data model enforcement and validation (if any) at application layer
– Simpler schema updates (no ALTER TABLE penalty)
• NoSQL APIs: Simpler programming interfaces
– No specialized language for queries and data manipulation
– Complex queries handled at application layer (no complex SELECTs, JOINs)
– Document in, document out, manipulations at client side
• Scalability, but some drawbacks:
– Limited database features (no foreign keys, no transactions, etc.)
– Weak consistency guarantees
34
Usability & Scalability

35. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Why not…
• Have both schema-less and schema in the same technology stack?
• One that checks all the boxes of all stakeholders:
35
Developers:
[ x ] Schemaless or/and Schema
[ x ] Rapid Prototyping/Simpler APIs
[ x ] Document Model
[ x ] Transactions
Operations:
[ x ] Performance Management/Visibility
[ x ] Robust Replication, Backup, Restore
[ x ] Comprehensive Tooling Ecosystem
[ x ] Simpler application schema upgrades
Business Owner:
[ x ] Don’t lose my data = ACID transactions
[ x ] Capture all my data = Extensible/Schemaless
[ x ] Products On Schedule/Time to Market = Rapid Development

36. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL as a Document Store – a FULL Stack
• Native JSON Datatype, Indexes on JSON Documents
• Native Collections with Key Value Semantics
• Interactive Shell “MySQL Shell” – Javascript, Python, SQL modes
• Connectors include NoSQL CRUD APIs
– Java, New NodeJS, NET, C++/C, PHP, Python
– Method Chaining and Pipelining
– Supports Combined Document and Relational
Store, Retrieve, Search and Managing JSON documents
36

37. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Document Store: Components
• MySQL X Plugin
• Introduces X Protocol for relational- and
document operations
• Maps CRUD operations to standard SQL
(relational tables, JSON datatype and
functions)
• X Protocol
• New MySQL client protocol based on top of
industry standard (Protobuf)
• Works for both, CRUD and SQL operations
• InnoDB Cluster
• Read-Scaling, Write-Scaling, HA
• X DevAPI
• New, modern, async developer API for CRUD
and SQL operations on top of X Protocol
• Introduces Collections as new Schema obj.
• MySQL Shell
• Offers interactive X DevAPI mode for app
prototyping
• MySQL Connectors
• Support for X DevAPI for
• JavaScript, Python, PHP, Java, C#, C++
37

38. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How does the Document Store work?
Architecture from the Application’s POV
38
Frontend
CRUD requests + JSON
Backend MySQL
JSON
Application
Document Store

39. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
New! Descending Indexes
CREATE TABLE t1 (
a INT,
b INT,
INDEX a_desc_b_asc (a DESC, b ASC)
);
39
For B+tree indexes
mysql> EXPLAIN SELECT * FROM t1 ORDER BY a DESC, b ASC;
.. +--------------+---------+------+------+----------+-------------+
.. | key | key_len | ref | rows | filtered | Extra |
.. +--------------+---------+------+------+----------+-------------+
.. | a_desc_b_asc | 10 | NULL | 10 | 100.00 | Using index |
.. +--------------+---------+------+------+----------+-------------+
mysql 8.0> EXPLAIN SELECT * FROM t1 ORDER BY a ASC, b ASC;
.. +--------------+---------+------+------+----------+-----------------------------+
.. | key | key_len | ref | rows | filtered | Extra |
.. +--------------+---------+------+------+----------+-----------------------------+
.. | a_desc_b_asc | 10 | NULL | 10 | 100.00 | Using index; Using filesort |
.. +--------------+---------+------+------+----------+-----------------------------+

40. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
References
• Manual
– https://dev.mysql.com/doc/refman/8.0/en/
• Release Candidate main features
– http://mysqlserverteam.com/the-mysql-8-0-3-release-candidate-is-available/
• Common Table Expressions
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-
in-mysql-ctes/
• Windows Funtions
– http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
40