MySQL 8.0, what's new ? - Forum PHP 2018

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blog : http://dasini.net/blog
What’s New in MySQL 8.0 ?

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.

Me, Myself & I

MySQL Geek
 Addicted to MySQL for 15+ years!
 Playing with databases for 20+ years

MySQL Writer, Blogger and Speaker
 Also former : DBA, Consultant, Architect, Trainer, ...

MySQL Principal Solutions Architect EMEA at Oracle

Stay tuned! :
 Twitter: @freshdaz
 Blog: http://dasini.net/blog
3
Olivier DASINI

DaDay Pay Parisris
4
Migration to MySQL 8.0
https://bit.ly/2PkjP6Phttps://bit.ly/2PkjP6P
MySQL Document Store
High Availability
MySQL Security
JSON
InnoDB Cluster

Agenda
Database Overview
MySQL 8.0 Features for Developers
MySQL 8.0 Features for DBAs
1
2
3
5

The world's most popular open source database
7

Stack Overflow Developer Survey 2018
Developer Survey Results
MySQL is the most popular database
8

JetBrains Survey 2018
The State of Developer Ecosystem
MySQL is the most popular database
9

Powers Social
10

Powers eCommerce
11

Powers SaaS
12

Powers Finance
13

Powers the Cloud
14

Our Server Usability Guidelines
15
« As well adding new features, our goal in the server
team is to make MySQL easier to use.
We try to design features with the goal of making a
net improvement on usability. »

Continuous Innovation: 5.7 -> 8.0
16
- 3x Better Performance
- Replication Enhancements
- Optimizer Cost Model
- JSON Support
- Improved Security
- Sys & Performance Schema
- GIS
MySQL 5.7 (GA)
MySQL InnoDB Cluster (GA)
- MySQL Group Replication
- MySQL Router
- MySQL Shell
MySQL 8.0 (GA)
- NoSQL Document Store
- JSON
- New Data Dictionary
- Roles
- Unicode
- CTEs
- Window Functions
- Improved Security
- Improved Replication
- Improved GIS
- Histograms
- Resource Group
- ...
2 Years in Development
400+ Worklogs
5000+ Bugs Fixed
500 New Tests
GA

Scalable & Stable
Better handling of high
contention, improved security,
and minimizing downtime
Data Driven
Optimizing services with real
time data analysis
Developer First
Hybrid data model and data
access APIs for flexibility for
developers
Mobile Friendly
Ready for location based
services. Handling Emoji
and Unicode characters
17
24x7at Scale
8.0 Enables Modern Web Applications

Developers Requests
18

19
8.0 Features for Devs
• MySQL JSON Document Store
• New JSON functions
• Utf8mb4 as default character
• Common Table Expressions
• Window functions
• SKIP LOCKED & NOWAIT
• Functional indexes
• Function or expression as default value

Document Store
An easy, straight forward way to work with
JSON documents in MySQL
https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-mysql-document-store/
20
NoSQL + SQL = MySQL

Document Oriented Databases
• Schemaless: No centralized database schema
– Data model enforcement and validation (if any) at application layer
– Simpler schema updates (no ALTER TABLE penalty)
• NoSQL APIs: Simpler programming interfaces
– No specialized language for queries and data manipulation
– Complex queries handled at application layer (no complex SELECTs, JOINs)
– Document in, document out, manipulations at client side
• Scalability, but some drawbacks:
– Limited database features (no foreign keys, no transactions, etc.)
– Weak consistency guarantees
21
Usability & Scalability

- A One Stop Shop
• Expensive to manage many data stores
• Better few databases – more flexibility
– More developers
• Can harness both NoSQL and SQL savvy
– More DBAs
• No shortage of highly experienced MySQL DBAs
– Less training – don’t need to learn many products
– Cross data store exchange – easier to move from from docs to tables etc.
– One connector/driver needed for apps
22
Combining Relational and Document Stores

NoSQL, MySQL, Why not…
• Have both schema-less AND schema in the same technology stack?
• One that checks all the boxes of all stakeholders:
23
Developers:
[✔] Schemaless or/and Schema
[✔] Rapid Prototyping/Simpler APIs
[✔] Document Model
[✔] Transactions
Operations:
[✔] Performance Management/Visibility
[✔] Robust Replication, Backup, Restore
[✔] Comprehensive Tooling Ecosystem
[✔] Simpler application schema upgrades
Business Owner:
[✔] Don’t lose my data = ACID transactions
[✔] Capture all my data = Extensible/Schemaless
[✔] Products On Schedule/Time to Market = Rapid Development

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 24
NoSQL
JSON Documents
Schemaless JSON Collections
MySQL
Relational Tables
Foreign Keys
X Dev API
SQL
CRUD
MySQL
Document
Store

• MySQL X Plugin
• Introduces X Protocol for relational- and
document operations
• Maps CRUD operations to standard SQL
(relational tables, JSON datatype and
functions)
• X Protocol
• New MySQL client protocol based on top of
industry standard (Protobuf)
• Works for both, CRUD and SQL operations
• InnoDB Cluster
• Read-Scaling, Write-Scaling, HA
• X DevAPI
• New, modern, async developer API for CRUD
and SQL operations on top of X Protocol
• Introduces Collections as new Schema obj.
• MySQL Shell
• Offers interactive X DevAPI mode for app
prototyping
• MySQL Connectors
• Support for X DevAPI for
• JavaScript, Python, PHP, Java, C#, C++
25
Document Store: Components

Document Store: Architecture
26

Document Store
SQL is now optional!
Write applications using X DevAPI
27
https://insidemysql.com/mysql-document-store-crud-quick-start/

Welcome to the X DevAPI!
https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/
28
Motivation
• We are doing something that has not been done before
• Document databases exist! Relational databases exist!
• We even see databases that support relational and document querying over the same data set
• However we have yet to see a relational database include a document model so that a user can use
document objects alongside their existing relational data.
MySQL X devAPI connector for:
• Java
– https://insidemysql.com/connector-j-8-0-11-the-face-for-your-brand-new-document-oriented-database/
• .NET
– https://insidemysql.com/introducing-connector-net-with-full-support-for-mysql-8-0/
• Node.JS
– https://insidemysql.com/introducing-connector-node-js-for-mysql-8-0/
• C++
– https://insidemysql.com/what-is-new-in-connector-c-8-0/
• Python
– https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
• PHP
– https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
• ODBC
– https://insidemysql.com/what-is-new-in-connector-odbc-8-0/

Welcome to the X DevAPI! - PHP Example
29
Writing a program using MySQL 8.0 Document Store
$user = 'my_user';
$passwd = 's0S3kR*T';
$host = 'localhost';
$port = '33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
$session = mysql_xdevapigetSession($connection_uri);
$schema = $session->getSchema("test");
$collection = $schema->getCollection("my_collection");
$result = $collection->find("Name like :param")->bind(["param" => "Olivier"])->execute();
$docs = $result->fetchAll();
//… print results …
$session->close();
Requirements / Installation
https://dev.mysql.com/doc/apis-php/en/apis-php-mysql-xdevapi.setup.html
https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/

Welcome to the X DevAPI! - Python Example
https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
30
Writing a program using MySQL 8.0 Document Store
import mysqlx
session = mysqlx.get_session({
"host": "localhost",
"port": 33060,
"user": "my_user",
"password": "s0S3kR*T"
})
schema = session.get_schema("test")
collection = schema.get_collection("my_collection")
result = collection.find("name like :param").bind("param", "Olivier").limit(1).execute()
docs = result.fetch_all()
print("Name: {0}".format(docs[0]["name"]))
session.close()
Connector/Python 8.0 installation
shell> pip install mysql-connector-python

js> session.createSchema('demo')
js> use demo
js> db.getCollections()
js> db.createCollection('myCollection')
js> db.getCollections()
js> db.myCollection.add({"param1":"value1", "param2":"value2"})
js> db.myCollection.find()
js> db.myCollection.find().limit(1)
js> db.myCollection.find("_id = '00005af018430000000000000002'")
js> db.myCollection.modify("_id = '1234'").set("param","value")
js> db.myCollection.remove("_id = '1234'")
js> session.startTransaction()
js> …
js> session.rollback()
js> db.myCollection.createIndex("ageIdx",{fields:[{"field":"$.age","type":"INT",required:false}]})
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
MySQL Document Store “cheat sheet”
CCreate
RRead
UUpdate
DDelete
Index
TTransaction

Shell
• DevOps Tool
– Designed for DevOps operations
• Scripting for Javascript, Python, and SQL mode
– Rapid prototyping capabilities
• InnoDB Cluster Support
– Setup your HA solution within minutes
• Supports MySQL Standard and X Protocols
• Document and Relational Models
– CRUD Document and Relational APIs via scripting
• Traditional Table, JSON, Tab Separated output results formats
• Both Interactive and Batch operations
Interface for Development and Administration of MySQL
32

• Customizable prompt
– Include context and session information
• Custom font and color support
• Persistent command line history
• Auto-complete / Content Assistance
• Full Unicode support
33
Shell: What’s New
Get Started in Minutes

JSON Support
• Native File Format
– Standardized as ECMA-404 (http://json.org)
– Binary Storage
• Virtual Columns
• 20+ Functions
 Search Functions
 Aggregations Functions
• Query structured data and semi-structured JSON data
34

8.0: JSON datatype & Document Store API
Hybrid APISQL FunctionData Type
MySQL X DevAPIJSON FunctionsJSON Datatype
Hybrid CRUD API of both
SQL and NoSQL provides
more flexibility for
development
Various SQL functions
to search and modify
JSON. Analysing JSON with
SQL by converting into
table with JSON_TABLE()
Seamlessly managing
“unstructured” data in
RDBMS tables with
efficient update
performance
{ } ();
35

MySQL 5.7 and 8.0
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE[_PRESERVE]()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
JSON_PRETTY()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_ARRAYAGG()
JSON_OBJECTAGG()
JSON_MERGE_PATCH()
JSON_TABLE()
https://mysqlserverteam.com/mysql-8-0-from-sql-tables-to-json-documents-and-back-again/
JSON Functions

JSON_PRETTY – Improve readability
SELECT doc FROM countryinfo WHERE _id='FRA'G
********************* 1. row *********************
doc: {"GNP": 1424285, "_id": "FRA", "Name": "France",
"IndepYear": 843, "geography": {"Region": "Western Europe",
"Continent": "Europe", "SurfaceArea": 551500}, "government":
{"HeadOfState": "Jacques Chirac", "GovernmentForm":
"Republic"}, "demographics": {"Population": 59225700,
"LifeExpectancy": 78.80000305175781}}
JSON Functions examples
http://dasini.net/blog/2018/07/23/30-mins-with-mysql-json-functions/
SELECT JSON_PRETTY(doc) FROM countryinfo WHERE _id='FRA'G
**************************** 1. row ****************************
JSON_PRETTY(doc): {
"GNP": 1424285,
"_id": "FRA",
"Name": "France",
"IndepYear": 843,
"geography": {
"Region": "Western Europe",
"Continent": "Europe",
"SurfaceArea": 551500
},
"government": {
"HeadOfState": "Jacques Chirac",
"GovernmentForm": "Republic"
...

• JSON is NOT limited to CRUD, can also be used in complex queries
• JSON_TABLE convert JSON into SQL table
• Leverage existing SQL framework for aggregation
38
JSON Functions examples - JSON_TABLE

JSON_TABLE - Extract data from a JSON document &
returns it as a relational table having the specified columns
SELECT GNP, Name, LifeExpectancy
FROM countryinfo,
JSON_TABLE(doc, "$" COLUMNS (GNP int PATH "$.GNP",
Name char(255) PATH "$.Name",
LifeExpectancy int PATH "$.demographics.LifeExpectancy")) AS jst
WHERE _id IN ('FRA', 'USA');
+---------+---------------+----------------+
| GNP | Name | LifeExpectancy |
+---------+---------------+----------------+
| 1424285 | France | 79 |
| 8510700 | United States | 77 |
+---------+---------------+----------------+
http://dasini.net/blog/2018/07/23/30-mins-with-mysql-json-functions/
JSON Functions examples - JSON_TABLE

• Emoji characters used as input
• MySQL 8.0 defaults to utf8mb4
• Latest Unicode 9.0 Support
• New collations based on DUCET,
accent and case sensitive collations,
Japanese, Russian
8.0: UTF8MB4 as default character set
40

1 2 4 8 16 32 64 128 256 512 1,024
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1,000,000
MySQL 8.0
MySQL 5.7
MySQL 5.6
Users
QueriesperSecond
SysBench OLTP Read Only (Mixed - utf8mb4)
MySQL 8.0 is 40% faster than MySQL 5.7
41
OS : Oracle Linux 7.4
CPU : 48cores-HT Intel Skylake 2.7Ghz
(2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU)
RAM: 256GB
Storage : x2 Intel Optane flash devices
(Intel (R) Optane (TM) SSD P4800X Series)

• « WITH queries » : named temporary result set that exists within the scope of a single
statement
– can be referred to later within that statement
– possibly multiple times
• Both Recursive and Non-Recursive Forms
• Simplifies writing complex SQL
43
8.0 Common Table Expressions (CTE)
WITH [RECURSIVE] cte_name [( <list of column names> )] AS (
Subquery <SELECT ...>
)
[, <any number of other CTE definitions> ]
<SELECT/UPDATE/DELETE statement>
https://dev.mysql.com/doc/refman/8.0/en/with.html
Boost Developer / Data Analyst Productivity

• Fibonacci number (10 firsts)
46
WITH RECURSIVE Fibonacci (num, PrevN, N) AS (
SELECT 1, 0, 1 /* Anchor member */
UNION ALL
SELECT num+1, N, PrevN + N /* Recursive member */
FROM Fibonacci
WHERE N < 50 /* Stop condition */
)
SELECT num, PrevN as Fibonacci_Numbers FROM Fibonacci;
+------+-------------------+
| num | Fibonacci_Numbers |
+------+-------------------+
| 1 | 0 |
| 2 | 1 |
| 3 | 1 |
| 4 | 2 |
| 5 | 3 |
| 6 | 5 |
| 7 | 8 |
| 8 | 13 |
| 9 | 21 |
| 10 | 34 |
+------+-------------------+ https://en.wikipedia.org/wiki/Fibonacci_number

• Hierarchy Traversal with CTE
49
SELECT * FROM cte.employees;
+------+---------+------------+
| id | name | manager_id |
+------+---------+------------+
| 29 | Pedro | 198 |
| 72 | Pierre | 29 |
| 123 | Adil | 692 |
| 198 | John | 333 |
| 333 | Yasmina | NULL |
| 692 | Tarek | 333 |
| 4610 | Sarah | 29 |
+------+---------+------------+
WITH RECURSIVE emp_ext (ID, NAME, PATH) AS (
SELECT id, name, name
FROM cte.employees
WHERE manager_id IS NULL
UNION ALL
SELECT s.id, s.name, CONCAT(m.path, " <- ", s.name)
FROM emp_ext m JOIN cte.employees s ON m.id=s.manager_id
)
SELECT * FROM emp_ext ORDER BY path;
+------+---------+------------------------------------+
| ID | NAME | PATH |
+------+---------+------------------------------------+
| 333 | Yasmina | Yasmina |
| 198 | John | Yasmina <- John |
| 29 | Pedro | Yasmina <- John <- Pedro |
| 72 | Pierre | Yasmina <- John <- Pedro <- Pierre |
| 4610 | Sarah | Yasmina <- John <- Pedro <- Sarah |
| 692 | Tarek | Yasmina <- Tarek |
| 123 | Adil | Yasmina <- Tarek <- Adil |
+------+---------+------------------------------------+

• Hierarchy Traversal with LEFT JOIN
51
SELECT * FROM cte.employees;
+------+---------+------------+
| id | name | manager_id |
+------+---------+------------+
| 29 | Pedro | 198 |
| 72 | Pierre | 29 |
| 123 | Adil | 692 |
| 198 | John | 333 |
| 333 | Yasmina | NULL |
| 692 | Tarek | 333 |
| 4610 | Sarah | 29 |
+------+---------+------------+
SELECT t1.id AS ID, t1.name AS NAME, CONCAT(t1.name, IFNULL(CONCAT(" ->
",t2.name), ""), IFNULL(CONCAT(" -> ",t3.name), ""), IFNULL(CONCAT(" ->
",t4.name), "")) AS PATH
FROM cte.employees AS t1
LEFT JOIN cte.employees AS t2 ON t2.id = t1.manager_id
WHERE t1.name IN (SELECT name FROM cte.employees)
ORDER BY t4.name, t3.name, t2.name, t1.name;
+------+---------+------------------------------------+
| ID | NAME | PATH |
+------+---------+------------------------------------+
| 333 | Yasmina | Yasmina |
| 198 | John | John -> Yasmina |
| 692 | Tarek | Tarek -> Yasmina |
| 29 | Pedro | Pedro -> John -> Yasmina |
| 123 | Adil | Adil -> Tarek -> Yasmina |
| 72 | Pierre | Pierre -> Pedro -> John -> Yasmina |
| 4610 | Sarah | Sarah -> Pedro -> John -> Yasmina |
+------+---------+------------------------------------+

CTE : query cost = 6.95 Left joins : query cost = 11.65
Graphs generated with

53
8.0 Window Functions
https://dev.mysql.com/doc/refman/8.0/en/window-functions.html
• Frequently requested feature for data analysis like ranking of data
• For each row from a query, perform a calculation using rows related to that row
• Non-aggregate window functions that, for each row from a query, perform a
calculation using rows related to that row
– FIRST_VALUE(), LAST_VALUE(), RANK(), ROW_NUMBER(), ...
– https://dev.mysql.com/doc/refman/8.0/en/window-function-descriptions.html
• Group (aggregate) functions that operate on sets of values.
– MAX(), MIN(), SUM(), AVG(), COUNT(), JSON_ARRAYAGG(), JSON_OBJECTAGG(), ...
– https://dev.mysql.com/doc/refman/8.0/en/group-by-functions.html
Improving Support for Analytics & Reporting

55
SELECT
ROW_NUMBER() OVER() AS `#`,
year, country, product, profit
FROM sales
ORDER BY country, year, product, profit;
• A window function performs an aggregate-like operation on a set of query rows. However, whereas an aggregate
operation groups query rows into a single result row, a window function produces a result for each query row:
– The row for which function evaluation occurs is called the current row
– The query rows related to the current row over which function evaluation occurs comprise the window for the current row
+----+------+---------+------------+--------+
| # | year | country | product | profit |
+----+------+---------+------------+--------+
| 1 | 2000 | Finland | Computer | 1500 |
| 2 | 2000 | Finland | Phone | 100 |
| 3 | 2001 | Finland | Phone | 10 |
| 4 | 2000 | India | Calculator | 75 |
| 5 | 2000 | India | Calculator | 75 |
| 6 | 2000 | India | Computer | 1200 |
| 7 | 2000 | USA | Calculator | 75 |
| 8 | 2000 | USA | Computer | 1500 |
| 9 | 2001 | USA | Calculator | 50 |
| 10 | 2001 | USA | Computer | 1200 |
| 11 | 2001 | USA | Computer | 1500 |
| 12 | 2001 | USA | TV | 100 |
| 13 | 2001 | USA | TV | 150 |
+----+------+---------+------------+--------+

56
SELECT
year, country, product, profit,
SUM(profit) OVER() AS total_profit,
SUM(profit) OVER(PARTITION BY country) AS country_profit
FROM sales
– The row for which function evaluation occurs is called the current row
– The query rows related to the current row over which function evaluation occurs comprise the window for the current row
+------+---------+------------+--------+--------------+----------------+
| year | country | product | profit | total_profit | country_profit |
+------+---------+------------+--------+--------------+----------------+
| 2000 | Finland | Computer | 1500 | 7535 | 1610 |
| 2000 | Finland | Phone | 100 | 7535 | 1610 |
| 2001 | Finland | Phone | 10 | 7535 | 1610 |
| 2000 | India | Calculator | 75 | 7535 | 1350 |
| 2000 | India | Computer | 1200 | 7535 | 1350 |
| 2000 | USA | Calculator | 75 | 7535 | 4575 |
| 2000 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | Calculator | 50 | 7535 | 4575 |
| 2001 | USA | Computer | 1200 | 7535 | 4575 |
| 2001 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | TV | 100 | 7535 | 4575 |
| 2001 | USA | TV | 150 | 7535 | 4575 |
+------+---------+------------+--------+--------------+----------------+

57
SELECT
SUM(profit) OVER(PARTITION BY country) AS country_profit
FROM sales
+------+---------+------------+--------+--------------+----------------+
| year | country | product | profit | total_profit | country_profit |
+------+---------+------------+--------+--------------+----------------+
| 2000 | Finland | Computer | 1500 | 7535 | 1610 |
| 2000 | Finland | Phone | 100 | 7535 | 1610 |
| 2001 | Finland | Phone | 10 | 7535 | 1610 |
| 2000 | India | Computer | 1200 | 7535 | 1350 |
| 2000 | USA | Calculator | 75 | 7535 | 4575 |
| 2000 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | Calculator | 50 | 7535 | 4575 |
| 2001 | USA | Computer | 1200 | 7535 | 4575 |
| 2001 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | TV | 100 | 7535 | 4575 |
| 2001 | USA | TV | 150 | 7535 | 4575 |
+------+---------+------------+--------+--------------+----------------+
– The row for which function evaluation occurs is called the current row.
– The query rows related to the current row over which function evaluation occurs comprise the window for the current row.

58
SELECT
SUM(profit) OVER(PARTITION BY country) AS country_profit,
SUM(profit) OVER(ORDER BY country, year ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS cumulativ_sum
FROM sales
• Cumulative sum
+------+---------+------------+--------+--------------+----------------+---------------+
| year | country | product | profit | total_profit | country_profit | cumulativ_sum |
+------+---------+------------+--------+--------------+----------------+---------------+
| 2000 | Finland | Computer | 1500 | 7535 | 1610 | 1500 |
| 2000 | Finland | Phone | 100 | 7535 | 1610 | 1600 |
| 2001 | Finland | Phone | 10 | 7535 | 1610 | 1610 |
| 2000 | India | Calculator | 75 | 7535 | 1350 | 1685 |
| 2000 | India | Calculator | 75 | 7535 | 1350 | 1760 |
| 2000 | India | Computer | 1200 | 7535 | 1350 | 2960 |
| 2000 | USA | Calculator | 75 | 7535 | 4575 | 3035 |
| 2000 | USA | Computer | 1500 | 7535 | 4575 | 4535 |
| 2001 | USA | Calculator | 50 | 7535 | 4575 | 4585 |
| 2001 | USA | Computer | 1200 | 7535 | 4575 | 5785 |
| 2001 | USA | Computer | 1500 | 7535 | 4575 | 7285 |
| 2001 | USA | TV | 100 | 7535 | 4575 | 7385 |
| 2001 | USA | TV | 150 | 7535 | 4575 | 7535 |
+------+---------+------------+--------+--------------+----------------+---------------+
https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777

60
SELECT DISTINCT
FIRST_VALUE(value) OVER w AS First,
LAST_VALUE(value) OVER w AS Last,
SUM(value) OVER w AS Sum,
AVG(value) OVER w AS Avg,
CAST(ts AS DATE) AS date
FROM ts
WINDOW w AS (
PARTITION BY CAST(ts AS DATE)
ORDER BY ts
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
);
+-------+------+------+----------+------------+
| first | last | sum | avg | date |
+-------+------+------+----------+------------+
| 142 | 154 | 609 | 152.2500 | 2018-08-10 |
| 158 | 170 | 655 | 163.7500 | 2018-08-11 |
| 172 | 249 | 771 | 192.7500 | 2018-08-12 |
+-------+------+------+----------+------------+
• Handling Time Series
SELECT * FROM ts;
+----+-------+---------------------+
| id | value | ts |
+----+-------+---------------------+
| 1 | 142 | 2018-08-10 00:00:00 |
| 2 | 163 | 2018-08-10 06:00:00 |
| 3 | 150 | 2018-08-10 12:00:00 |
| 4 | 154 | 2018-08-10 18:00:00 |
| 5 | 158 | 2018-08-11 00:00:00 |
| 6 | 161 | 2018-08-11 06:00:00 |
| 7 | 166 | 2018-08-11 12:00:00 |
| 8 | 170 | 2018-08-11 18:00:00 |
| 9 | 172 | 2018-08-12 00:00:00 |
| 10 | 190 | 2018-08-12 06:00:00 |
| 11 | 160 | 2018-08-12 12:00:00 |
| 12 | 249 | 2018-08-12 18:00:00 |
+----+-------+---------------------+

61
8.0 Window Functions & JSON
SELECT doc FROM cs;
+----------------------------------------------------------------------------------------------+
| doc |
+----------------------------------------------------------------------------------------------+
| {"id": 1, "ts": "2018-08-10 00:00:00", "_id": "00005bd1ea94000000000000000d", "value": 142} |
| {"id": 2, "ts": "2018-08-10 06:00:00", "_id": "00005bd1ea94000000000000000e", "value": 163} |
| {"id": 3, "ts": "2018-08-10 12:00:00", "_id": "00005bd1ea94000000000000000f", "value": 150} |
| {"id": 4, "ts": "2018-08-10 18:00:00", "_id": "00005bd1ea940000000000000010", "value": 154} |
+----------------------------------------------------------------------------------------------+
• Handling Time Series data in a JSON column or from a collection

62
8.0 Window Functions & JSON
SELECT DISTINCT
FIRST_VALUE(value) OVER w AS First,
LAST_VALUE(value) OVER w AS Last,
SUM(value) OVER w AS Sum,
AVG(value) OVER w AS Avg,
CAST(ts AS DATE) AS date
FROM cs, JSON_TABLE(doc, "$" COLUMNS (id int PATH "$.id",
value int PATH "$.value",
ts timestamp PATH "$.ts")) AS tcs
WINDOW w AS (
PARTITION BY CAST(ts AS DATE)
ORDER BY ts
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
);
+-------+------+------+----------+------------+
| first | last | sum | avg | date |
+-------+------+------+----------+------------+
| 142 | 154 | 609 | 152.2500 | 2018-08-10 |
| 158 | 170 | 655 | 163.7500 | 2018-08-11 |
| 172 | 249 | 771 | 192.7500 | 2018-08-12 |
+-------+------+------+----------+------------+
• Handling Time Series data in a JSON column or from a collection

63
https://dev.mysql.com/doc/refman/8.0/en/innodb-locking-reads.html#innodb-locking-reads-nowait-skip-locked
SELECT seat_no
FROM seats
JOIN seat_rows USING ( row_no )
WHERE seat_no IN (3,4)
AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seat_rows NOWAIT;
• Locking Read Concurrency with NOWAIT and SKIP LOCKED
8.0 Better Handling Hot Row Contention 1/2
SKIP LOCKED / NOWAIT
Non deterministically
skip over
locked rows
Error immediately
if a row is
already locked

64
-- Session 1
START TRANSACTION;
SELECT * FROM t WHERE i = 2 FOR UPDATE;
+---+
| i |
+---+
| 2 |
+---+
8.0 Better Handling Hot Row Contention 2/2
SKIP LOCKED / NOWAIT
-- Session 2
START TRANSACTION;
SELECT * FROM t WHERE i = 2 FOR UPDATE NOWAIT;
ERROR 3572 (HY000): Statement aborted because lock(s) could not be
acquired immediately and NOWAIT is set.
-- Session 3
START TRANSACTION;
SELECT * FROM t FOR UPDATE SKIP LOCKED;
+---+
| i |
+---+
| 1 |
| 3 |
+---+

8.0 UUID and Bit-wise Improvements
• Functions to convert UUID to and from binary:
– UUID_TO_BIN()
– BIN_TO_UUID()
– plus IS_UUID()
• Binary format is now smaller and insert-order efficient:
11e678fe53303f87a4778c89a52c4f3b
53303f87-78fe-11e6-a477-8c89a52c4f3bFrom VARCHAR(36)
To VARBINARY(16)
• Bit-wise operations on binary data types.
• Bit-wise operations on binary data types.
– Designed with IPv6 in mind:
●
INET6_ATON(address) & INET6_ATON(network)
65

8.0 SQL Functional Indexes
• Add the possibility to create functional indexes in MySQL
– Implemented as indexed hidden generated columns
66
CREATE TABLE t1 (
col1 INT,
col2 INT,
INDEX func_index ((ABS(col1)))
);
CREATE INDEX idx1 ON t1 ((col1 + col2));
CREATE INDEX idx2 ON t1 ((col1 + col2), (col1 - col2), col1);
ALTER TABLE t1 ADD INDEX ((col1 * 40));
https://dev.mysql.com/worklog/task/?id=1075
Support for functional index (8.0.13+)

8.0 SQL Default Value
• Add the capability to use functions as default values as well as allowing
default values for GIS and JSON datatypes
– The most common use case would be to emulate a UUID-like auto_increment
67
Permit default value to be a function or expression (8.0.13+)
CREATE TABLE t0
( id BINARY(16) DEFAULT (uuid_to_bin(uuid())), doc JSON DEFAULT ('[]') );
INSERT INTO t0 () VALUES ();
Query OK, 1 row affected (0,11 sec)
SELECT bin_to_uuid(id), doc FROM t0;
+--------------------------------------+------+
| bin_to_uuid(id) | doc |
+--------------------------------------+------+
| f6d56292-d8a6-11e8-a42f-dc53609b8b8d | [] |
+--------------------------------------+------+

DBAs/OPS Requests
68

69
8.0 Features for DBAs
• MySQL 8.0 InnoDB Cluster
• Instant Add Column
• Invisible indexes
• Persist Configuration
• Upgrade Checker utility
• GIS Improvement
• Prohibits table w/o primary Key
• Histograms
• Improved Security

Virtually all organizations require their
most critical systems to be highly available
70
100%
High Availability

InnoDB Cluster
71
App Servers with
MySQL Router
MySQL Group Replication
MySQL Shell
Setup, Manage,
Orchestrate
“High Availability becomes a core
first class feature of MySQL!”
https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/

Shell: DBA Admin API
• The global variable 'dba' is used to access the MySQL AdminAPI
• JS> dba.help()
• Perform DBA operations
– Manage MySQL InnoDB clusters
• Create clusters
• Validate MySQL instances
• Configure MySQL instances
• Get cluster info
• Modify clusters
• and much more ...
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
72

Group Replication: Database HA
• Group Replication library
– Implementation of Replicated Database State Machine
• MySQL GCS is based on our home-grown Paxos implementation
– Provides virtually synchronous replication for MySQL 5.7+
• Guarantees eventual consistency
– Automates operations
• Conflict detection and resolution
• Failure detection, fail-over, recovery
• Group membership management and reconfiguration
“Multi-master update anywhere replication plugin for MySQL with built-in conflict detection
and resolution, automatic distributed recovery, and group membership.”
73
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

8.0: Alter Table – Instant Add Column
• Contribution from Tencent Games
– Only a metadata change
– No copying of data
– Smaller final data size
– Forward compatibility with old data file
• ALTER TABLE ... ADD COLUMN c, ALGORITHM = INSTANT
• Supports DYNAMIC/COMPACT/REDUNDANT row formats
74

8.0 Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
• Two use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
75
Feature Request
from DBAs
• I don’t think this index is used any more:
ALTER TABLE Country ALTER INDEX c INVISIBLE;
• I need to revert:
ALTER TABLE Country ALTER INDEX c VISIBLE;
• It is now safe to drop:
ALTER TABLE Country DROP INDEX c;
e.g. Soft Delete

8.0: Persist Configuration
76
Cloud Friendly
• Persist Global Server Variables
 SET PERSIST max_connections = 500 ;
• Examples Inlude :
 Offline_mode
 Read_Only
• Requires no filesystem access
• Inludes timestamp and change user

8.0: Upgrade Checker 1/2
77
• Quick and Easy MySQL Shell Utility
– JavaScript
– Python
• Identifies Issues Based on Severity
– No Issues
– Potential Errors
– Errors that must be fixed before Upgrading
• Recommends Fixes
– Schema, Configuration
– Data on Server, etc.
https://mysqlserverteam.com/mysql-shell-8-0-4-introducing-upgrade-checker-utility/
Feature Request
from DBAs

8.0: Upgrade Checker 2/2
78

8.0: InnoDB Improvements
• Document Store/JSON
– More efficient BLOBs, up to 4x improvement on update
– Better performance for both small and large BLOBs
• New Scalable WAL
– Lock free design
• Atomic DDL
• Persistent Auto Increment
• Contention Aware Transaction Scheduling (CATS)
– Contribution from University of Michigan
• Removed IO bottlenecks
– More scalable design
79

8.0: Spatial Reference Systems
81
https://mysqlserverteam.com/spatial-reference-systems-in-mysql-8-0/

MySQL 8.0MySQL 5.7
8.0: Improved GIS
82
• The world is flat
• The world is infinite
• Axes are unitless
• Axes are orthogonal
• Axis order is irrelevant
• Axis direction is irrelevant
• The world can be flat or ellipsoidal
• Geographic coordinate systems wrap around
• Axes have units
• Geographic axes are not orthogonal
• Geogrphic axis order matters
• Axis direction may be relevant

• Highly Accurate
 Now better than competitor databases
• High Performance
• Simple – No extra configuration or installation
 Completely built in and ready to use
• Powerful
 Full Geography Support

Projected – Flat/Across 2 dimensions

Geographic – Spheroid
 Details

5107 predefined SRSs from the EPSG Dataset 9.2
 4628 projected
 479 gegraphic
8.0: GIS Improvements
83

8.0 Prohibits Tables w/o Primary Key
• New system variable : sql_require_primary_key
• Cause CREATE or ALTER statements that would yield a table without
primary key, to fail
• Tables without primary keys may cause performance problems
• https://dev.mysql.com/worklog/task/?id=11807
85
Option to Disallow Tables without a Primary Key (8.0.13+)

Performance Improvements
86

New! Performance Improvements
• Improved Query Consistency
– Histograms
– Improved Cost Model
• Faster Table/Range Scans
87
• Parallel Replication
• UTF8MB4
• Information Schema
• Performance Schema Indexes
http://dimitrik.free.fr/blog/

New! Transactional Data Dictionary
• Crash-safe Database
– Common data dictionary for the server and
InnoDB
• Crash-safe & Atomic DDL
– CREATE USER <userlist>, DROP DATABASE with
all-or-none semantic
– Simplifies replication failure cases
88
• Meta-data locking for FK constraints
– FK moved from InnoDB to server layer
• Scalable Information Schema
– Queries are now executed as set of SQL views
on tables
– Large performance improvements
Atomic and crash-safe DDL

Column statistics
8.0: Histograms
Feature Request
by DBAs
• Provides the optimizer with information about column value distribution
• To create/recalculate histogram for a column:
– ANALYZE TABLE table UPDATE HISTOGRAM ON column WITH n BUCKETS;
• May use sampling
– Sample size is based on available memory ( histogram_generation_max_mem_size )
• Automatically chooses between two histogram types:
– Singleton: One value per bucket
– Equi-height: Multiple value per bucket
89

Better query plans
8.0: Histograms
• More consistent query execution for cases when
data is skewed
• Lower cost to maintain than an index
Feature Request
by DBAs

1 2 4 8 16 32 64 128 256 512
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
MySQL 8.0
MySQL 5.7
MySQL 5.6
Users
QueriesperSecond
(2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU)
RAM: 256GB
2x Faster than MySQL 5.7
8.0: SysBench IO Bound Read Only (Point Selects)

(2CPU sockets, Intel(R) Xeon(R) Platinum 8168
RAM: 256GB
2x Faster than MySQL 5.7
8.0: SysBench Read/Write (Update NoKey)
92

Improved to consider buffer pool fit
8.0: Optimizer Cost Model
Feature Request
by DBAs• Storage engines :
– Estimate for how much of data and
indexes are in memory buffer
– Estimate for hit rate for memory buffer
• Optimizer cost model :
– Take into account wether data is already
in memory or need to be read from disk
93

Improved to consider buffer pool fit
8.0: Optimizer Cost Model
Feature Request
by DBAs
In Memory
innodb_buffer_pool=32G
Disk Bond
Plan A 5.8 secs 9 min 47 secs
Plan B 77.5 secs 3 min 49 secs
In Memory
Disk Bond
MySQL 5.6 Plan B
MySQL 5.7 Plan A
MySQL 8.0 Plan A Plan B
94

Security
https://dev.mysql.com/doc/mysql-security-excerpt/8.0/en/
95

• Improving MySQL Access Controls
• Easier to manage user and applications rights
• As standards compliant as practically possible
• Multiple default roles
• Export the role graph in GraphML
– Visualize with ROLES_GRPHML()
96
Directly
Indirectly
Set Role(s)
Default Role(s)
Set of
ACLS
Set of
ACLS
New! SQL Roles
Feature Request
from DBAs

Password Features
• Password history - provides DBAs more password management
– Require new passwords not reuse old ones - By number of changes and/or time.
– Establish password-reuse policy globally as well as on a per-account basis.
• SHA2 with Caching
– Strong and Fast
– Strong - SHA-256 password hashing (many rounds, seeds, …)
– Fast - Caching
• Greatly reduces latency
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-default-authentication-plugin-considerations/
• Supports for more connection protocols
• Seamless RSA password-exchange capabilities (No linking OpenSSL)
97

Password Features
• New! Current password required for SET PASSWORD (8.0.13+)
– Enhances the security by authorizing the users to change their password only if they could provide the
current password
– The new behavior is controlled by password_require_current column in the mysql.user table for
every user
– https://dev.mysql.com/worklog/task/?id=11544
98

8.0: File and Tablespace Encryption
• AES 256 encryption of UNDO and REDO Logs
• Super Simple to manage - Set
– innodb_undo_log_encrypt=ON/OFF
– innodb_redo_log_encrypt=ON/OFF
• And
– ON - Pages written after setting are encrypted
– OFF - Pages written after setting are not
99
• New! Support Transparent Data Encryption for Shared Tablespaces (8.0.13+)
– Extends tablespace encryption to general/shared tablespsces
– Encryption ON/OFF can be specified either at create time or later with the SQL Syntax
●
CREATE/ALTER TABLESPACE my_tablespace ENCRYPTION='Y'/'N'; (default is unencrypted)
• Encryption is handled at the tablespace level, so either all or non of the tables in the tablespace is
encrypted
• https://dev.mysql.com/worklog/task/?id=9286

8.0: All these features plus…
• Source code now documented with Doxygen
• Plugin Infrastructure!
• Expanded GIS Support
• Expanded Query Hints Support
• Improved Scan Query Performance
• Improved BLOB Storage
• Improved Memcached Interface
• Cost Model Improvements
• Scalability Improvements
• Atomicity in Privileges
• Parser Refactoring
• Improvements to Temporary Tables
• C++11 and Toolchain Improvements
• GTID_PURGED always settable
• Persistent Auto Increment
• Native InnoDB Data dictionary
• Improved Information Schema performance
• SQL Grouping Function
• Optimizer Trace detailed sort statistics
• Descending Indexes
• Smaller Package Downloads
• JSON Aggregate, Pretty print functions
• JSON performance improvements
• Expanded Query Hints
• Improved usability of cost constant
configuration
• Group Replication in 8.0
• Transaction Save Point support in Group
Replication
• Improved Replication Applier Lag Statistics
in Performance Schema
• Per Multi-source Channel Replication Filters
• Atomic DDL extended to the Binary Log
• Performance Improvements on the
Replication Applier
• Parallel Replication Applier Policy
• Binary Log Management Enhancements
• Additional Metadata Into the Binary Log
• Improve SELECT COUNT(*) performance
• Multiple addresses for --bind-address
• Support for BLOBs in temp table engine
• ...
101

• DocStore
– SQL + NoSQL = MySQL
• InnoDB Cluster
– HA made easy
• SQL
• JSON
• GIS
8.0 – Innovating and Evolving
• Replication
• Reliability
• Observability
• Manageability
• Security
• Performance
• Scalability
102

Resources – MySQL Blogs 1/3
103
• MySQL Server Team’s blog
– http://mysqlserverteam.com/
• MySQL High Availability Team’s blog
– http://mysqlhighavailability.com/
• MySQL Release Engineering Blog
– https://mysqlrelease.com/
• MySQL Dev Team Blog
– https://insidemysql.com/
• MySQL Enterprise Team Blogs
– https://blogs.oracle.com/mysql/
• Lefred’s blog (MySQL Community Manager)
– http://lefred.be/
• Olivier Dasini’s blog
– http://dasini.net/blog/

104
• MySQL Document Store
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://github.com/mysql/mysql-connector-nodejs
– https://www.npmjs.com/package/@mysql/xdevapi
– https://dev.mysql.com/doc/refman/8.0/en/document-store.html
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://dev.mysql.com/doc/dev/connector-nodejs/8.0/
– https://www.mysql.com/news-and-events/web-seminars/mysql-document-store-and-node-js/
• MySQL Performance by Dimitri
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-over-18m-qps-with-80-ga-on-2s-skylake.html
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-80-and-utf8-impact.html
• MySQL InnoDB Cluster
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/
– https://mysqlserverteam.com/mysql-innodb-cluster-whats-new-in-the-8-0-ga-release/
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-a-hands-on-tutorial/

105
• Common Table Expressions (CTE)
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-two-how-to-generate-series/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-three-hierarchies/
– http://mysqlserverteam.com/mysql-8-0-1-recursive-common-table-expressions-in-mysql-ctes-part-four-depth-first-or-breadth-first-traversal-
transitive-closure-cycle-avoidance/
• Window Functions
– http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
– https://mysqlserverteam.com/row-numbering-ranking-how-to-use-less-user-variables-in-mysql-queries/
– https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777
• Security
– https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/
– https://mysqlserverteam.com/mysql-8-0-4-openssl-and-mysql-community-edition/
– http://lefred.be/content/mysql-8-0-listing-roles/
– https://dev.mysql.com/doc/refman/8.0/en/password-management.html
• Upgrade from 5.7 to 8.0
– https://mysqlserverteam.com/inplace-upgrade-from-mysql-5-7-to-mysql-8-0/
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/

@ Facebook
106

@ Twitter
107

@ Booking.com
108

Thanks for using MySQL!

MySQL 8.0, what's new ? - Forum PHP 2018

MySQL 8.0, what's new ? - Forum PHP 2018

More Related Content

What's hot

Similar to MySQL 8.0, what's new ? - Forum PHP 2018

More from Olivier DASINI

Recently uploaded

MySQL 8.0, what's new ? - Forum PHP 2018