More Related Content Similar to MySQL 8.0 - What's New ? (20) More from Olivier DASINI (11) MySQL 8.0 - What's New ?1. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blog : http://dasini.net/blog
What’s New in MySQL 8.0 ?
2. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
3. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Me, Myself & I
MySQL Geek
Addicted to MySQL for 15+ years!
Playing with databases for 20+ years
MySQL Writer, Blogger and Speaker
Also former : DBA, Consultant, Architect, Trainer, ...
MySQL Principal Solutions Architect EMEA at Oracle
Stay tuned! :
Twitter: @freshdaz
Blog: http://dasini.net/blog
3
Olivier DASINI
4. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
The world's most popular open source database
6
5. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Powers Social
7
6. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Powers eCommerce
8
7. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Powers FinTech
9
8. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Powers SaaS
10
9. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Powers the Cloud
11
10. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Continuous Innovation: 5.7 -> 8.0
14
- 3x Better Performance
- Replication Enhancements
- Optimizer Cost Model
- JSON Support
- Improved Security
- Sys & Performance Schema
- GIS
MySQL 5.7 (GA)
MySQL InnoDB Cluster (GA)
- MySQL Group Replication
- MySQL Router
- MySQL Shell
MySQL 8.0 (GA)
- NoSQL Document Store
- JSON
- New Data Dictionary
- Roles
- Unicode
- CTEs
- Window Functions
- Improved Security
- Improved Replication
- Improved GIS
- Histograms
- Resource Group
- ...
2 Years in Development
400+ Worklogs
5000+ Bugs Fixed
500 New Tests
GA
11. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Our Server Usability Guidelines
15
« As well adding new features, our goal in the server
team is to make MySQL easier to use.
We try to design features with the goal of making a
net improvement on usability. »
12. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Scalable & Stable
Better handling of high
contention, improved security,
and minimizing downtime
Data Driven
Optimizing services with real
time data analysis
Developer First
Hybrid data model and data
access APIs for flexibility for
developers
Mobile Friendly
Ready for location based
services. Handling Emoji
and Unicode characters
17
24x7at Scale
8.0 Enables Modern Web Applications
13. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Developers Requests
18
14. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Document Store
An easy, straight forward way to work with
JSON documents in MySQL
https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-mysql-document-store/
19
15. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Document Oriented Databases
• Schemaless: No centralized database schema
– Data model enforcement and validation (if any) at application layer
– Simpler schema updates (no ALTER TABLE penalty)
• NoSQL APIs: Simpler programming interfaces
– No specialized language for queries and data manipulation
– Complex queries handled at application layer (no complex SELECTs, JOINs)
– Document in, document out, manipulations at client side
• Scalability, but some drawbacks:
– Limited database features (no foreign keys, no transactions, etc.)
– Weak consistency guarantees
20
Usability & Scalability
16. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
- A One Stop Shop
• Expensive to manage many data stores
• Better few databases – more flexibility
– More developers
• Can harness both NoSQL and SQL savvy
– More DBAs
• No shortage of highly experienced MySQL DBAs
– Less training – don’t need to learn many products
– Cross data store exchange – easier to move from from docs to tables etc.
– One connector/driver needed for apps
21
Combining Relational and Document Stores
17. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
NoSQL, MySQL, Why not…
• Have both schema-less AND schema in the same technology stack?
• One that checks all the boxes of all stakeholders:
22
Developers:
[✔] Schemaless or/and Schema
[✔] Rapid Prototyping/Simpler APIs
[✔] Document Model
[✔] Transactions
Operations:
[✔] Performance Management/Visibility
[✔] Robust Replication, Backup, Restore
[✔] Comprehensive Tooling Ecosystem
[✔] Simpler application schema upgrades
Business Owner:
[✔] Don’t lose my data = ACID transactions
[✔] Capture all my data = Extensible/Schemaless
[✔] Products On Schedule/Time to Market = Rapid Development
18. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 23
NoSQL
JSON Documents
Schemaless JSON Collections
MySQL
Relational Tables
Foreign Keys
X Dev API
SQL
CRUD
MySQL
Document
Store
19. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• MySQL X Plugin
• Introduces X Protocol for relational- and
document operations
• Maps CRUD operations to standard SQL
(relational tables, JSON datatype and
functions)
• X Protocol
• New MySQL client protocol based on top of
industry standard (Protobuf)
• Works for both, CRUD and SQL operations
• InnoDB Cluster
• Read-Scaling, Write-Scaling, HA
• X DevAPI
• New, modern, async developer API for CRUD
and SQL operations on top of X Protocol
• Introduces Collections as new Schema obj.
• MySQL Shell
• Offers interactive X DevAPI mode for app
prototyping
• MySQL Connectors
• Support for X DevAPI for
• JavaScript, Python, PHP, Java, C#, C++
24
Document Store: Components
20. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Document Store: Architecture
25
21. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Document Store
SQL is now optional!
Write applications using X DevAPI
26
https://insidemysql.com/mysql-document-store-crud-quick-start/
22. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 27
Welcome to the X DevAPI!
https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/
27
Motivation
• We are doing something that has not been done before
• Document databases exist! Relational databases exist!
• We even see databases that support relational and document querying over the same data set
• However we have yet to see a relational database include a document model so that a user can use
document objects alongside their existing relational data.
MySQL X devAPI connector for:
• Java
– https://insidemysql.com/connector-j-8-0-11-the-face-for-your-brand-new-document-oriented-database/
• .NET
– https://insidemysql.com/introducing-connector-net-with-full-support-for-mysql-8-0/
• Node.JS
– https://insidemysql.com/introducing-connector-node-js-for-mysql-8-0/
• C++
– https://insidemysql.com/what-is-new-in-connector-c-8-0/
• Python
– https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
• PHP
– https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
• ODBC
– https://insidemysql.com/what-is-new-in-connector-odbc-8-0/
23. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 28
Welcome to the X DevAPI! - Python Example
https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
28
Writing a program using MySQL 8.0 Document Store
import mysqlx
session = mysqlx.get_session({
"host": "localhost",
"port": 33060,
"user": "my_user",
"password": "s0S3kR*T"
})
schema = session.get_schema("test")
collection = schema.get_collection("my_collection")
result = collection.find("name like :param").bind("param", "Olivier").limit(1).execute()
docs = result.fetch_all()
print("Name: {0}".format(docs[0]["name"]))
session.close()
Connector/Python 8.0 installation
shell> pip install mysql-connector-python
24. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 29
Welcome to the X DevAPI! - PHP Example
29
Writing a program using MySQL 8.0 Document Store
$user = 'my_user';
$passwd = 's0S3kR*T';
$host = 'localhost';
$port = '33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
$session = mysql_xdevapigetSession($connection_uri);
$schema = $session->getSchema("test");
$collection = $schema->getCollection("my_collection");
$result = $collection->find("Name like :param")->bind(["param" => "Olivier"])->execute();
$docs = $result->fetchAll();
//… print results …
$session->close();
Requirements / Installation
https://dev.mysql.com/doc/apis-php/en/apis-php-mysql-xdevapi.setup.html
https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
25. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 30
js> session.createSchema('demo')
js> use demo
js> db.getCollections()
js> db.createCollection('myCollection')
js> db.getCollections()
js> db.myCollection.add({"param1":"value1", "param2":"value2"})
js> db.myCollection.find()
js> db.myCollection.find().limit(1)
js> db.myCollection.find("_id = '00005af018430000000000000002'")
js> db.myCollection.modify("_id = '1234'").set("param","value")
js> db.myCollection.remove("_id = '1234'")
js> session.startTransaction()
js> …
js> session.rollback()
js> db.myCollection.createIndex("ageIdx",{fields:[{"field":"$.age","type":"INT",required:false}]})
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
MySQL Document Store “cheat sheet”
CCreate
RRead
UUpdate
DDelete
Index
TTransaction
26. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Shell
• DevOps Tool
– Designed for DevOps operations
• Scripting for Javascript, Python, and SQL mode
– Rapid prototyping capabilities
• InnoDB Cluster Support
– Setup your HA solution within minutes
• Supports MySQL Standard and X Protocols
• Document and Relational Models
– CRUD Document and Relational APIs via scripting
• Traditional Table, JSON, Tab Separated output results formats
• Both Interactive and Batch operations
Interface for Development and Administration of MySQL
33
27. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Customizable prompt
– Include context and session information
• Custom font and color support
• Persistent command line history
• Auto-complete / Content Assistance
• Full Unicode support
34
Shell: What’s New
Get Started in Minutes
28. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
JSON Support
• Native File Format
– Standardized as ECMA-404 (http://json.org)
– Binary Storage
• Virtual Columns
• 20+ Functions
Search Functions
Aggregations Functions
• Query structured data and semi-structured JSON data
35
29. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: JSON datatype & Document Store API
3
6
Hybrid APISQL FunctionData Type
MySQL X DevAPIJSON FunctionsJSON Datatype
Hybrid CRUD API of both
SQL and NoSQL provides
more flexibility for
development
Various SQL functions
to search and modify
JSON. Analysing JSON with
SQL by converting into
table with JSON_TABLE()
Seamlessly managing
“unstructured” data in
RDBMS tables with
efficient update
performance
{ } ();
30. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 37
MySQL 5.7 and 8.0
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE[_PRESERVE]()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
JSON_PRETTY()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_ARRAYAGG()
JSON_OBJECTAGG()
JSON_MERGE_PATCH()
JSON_TABLE()
https://mysqlserverteam.com/mysql-8-0-from-sql-tables-to-json-documents-and-back-again/
JSON Functions
31. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 38
JSON_PRETTY – Improve readability
SELECT doc FROM countryinfo WHERE _id='FRA'G
********************* 1. row *********************
doc: {"GNP": 1424285, "_id": "FRA", "Name": "France",
"IndepYear": 843, "geography": {"Region": "Western Europe",
"Continent": "Europe", "SurfaceArea": 551500}, "government":
{"HeadOfState": "Jacques Chirac", "GovernmentForm":
"Republic"}, "demographics": {"Population": 59225700,
"LifeExpectancy": 78.80000305175781}}
JSON Functions examples
http://dasini.net/blog/2018/07/23/30-mins-with-mysql-json-functions/
SELECT JSON_PRETTY(doc) FROM countryinfo WHERE _id='FRA'G
**************************** 1. row ****************************
JSON_PRETTY(doc): {
"GNP": 1424285,
"_id": "FRA",
"Name": "France",
"IndepYear": 843,
"geography": {
"Region": "Western Europe",
"Continent": "Europe",
"SurfaceArea": 551500
},
"government": {
"HeadOfState": "Jacques Chirac",
"GovernmentForm": "Republic"
...
32. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 39
JSON_TABLE - Extract data from a JSON document & returns
it as a relational table having the specified columns
SELECT GNP, Name, LifeExpectancy
FROM countryinfo,
JSON_TABLE(doc, "$" COLUMNS (GNP int PATH "$.GNP",
Name char(255) PATH "$.Name",
LifeExpectancy int PATH "$.demographics.LifeExpectancy")) AS jst
WHERE _id IN ('FRA', 'USA');
+---------+---------------+----------------+
| GNP | Name | LifeExpectancy |
+---------+---------------+----------------+
| 1424285 | France | 79 |
| 8510700 | United States | 77 |
+---------+---------------+----------------+
JSON Functions examples
http://dasini.net/blog/2018/07/23/30-mins-with-mysql-json-functions/
33. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Emoji characters used as input
• MySQL 8.0 defaults to utf8mb4
• Latest Unicode 9.0 Support
• New collations based on DUCET,
accent and case sensitive collations,
Japanese, Russian
8.0: UTF8MB4 as default character set
40
34. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Boosts Developer Productivity / Data Analyst Productivity
• New! CTEs and Recursive CTEs
– CTE = Common Table Expressions
– Used for hierachy traversal
• New! Window Functions
– Aggregation, Ranking, Analytics
– Used for analytics and Reporting
• New! SKIP LOCKED and NOWAIT
– Better handling of hot rows
• New! UUIDs and Bitwise Functions
42
Feature Request
from Developers
35. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• « WITH queries » : named temporary result set that exists within the scope of a single
statement
– can be referred to later within that statement
– possibly multiple times
• Both Recursive and Non-Recursive Forms
• Simplifies writing complex SQL
43
8.0 Common Table Expressions (CTE) 1/5
WITH [RECURSIVE] cte_name [( <list of column names> )] AS (
Subquery <SELECT ...>
)
[, <any number of other CTE definitions> ]
<SELECT/UPDATE/DELETE statement>
https://dev.mysql.com/doc/refman/8.0/en/with.html
36. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Chained CTEs - Neat, but not very useful example :)
44
8.0 Common Table Expressions (CTE) 2/5
WITH
cte1(txt) AS (SELECT "This "),
cte2(txt) AS (SELECT CONCAT(cte1.txt,"is a ") FROM cte1),
cte3(txt) AS (SELECT "nice query"
UNION SELECT "query that rocks"
UNION SELECT "query"),
cte4(txt) AS (SELECT concat(cte2.txt, cte3.txt) FROM cte2, cte3)
SELECT MAX(txt), MIN(txt) FROM cte4;
+----------------------------+----------------------+
| MAX(txt) | MIN(txt) |
+----------------------------+----------------------+
| This is a query that rocks | This is a nice query |
+----------------------------+----------------------+
37. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Print 1 to 10
45
8.0 Common Table Expressions (CTE) 3/5
WITH RECURSIVE cte (n) AS (
SELECT 1
UNION ALL
SELECT n + 1 FROM cte WHERE n < 10
)
SELECT * FROM cte;
+------+
| n |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+------+
38. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Fibonacci number (10 firsts)
46
8.0 Common Table Expressions (CTE) 4/5
WITH RECURSIVE Fibonacci (num, PrevN, N) AS (
SELECT 1, 0, 1 /* Anchor member */
UNION ALL
SELECT num+1, N, PrevN + N /* Recursive member */
FROM Fibonacci
WHERE N < 50 /* Stop condition */
)
SELECT num, PrevN as Fibonacci_Numbers FROM Fibonacci;
+------+-------------------+
| num | Fibonacci_Numbers |
+------+-------------------+
| 1 | 0 |
| 2 | 1 |
| 3 | 1 |
| 4 | 2 |
| 5 | 3 |
| 6 | 5 |
| 7 | 8 |
| 8 | 13 |
| 9 | 21 |
| 10 | 34 |
+------+-------------------+ https://en.wikipedia.org/wiki/Fibonacci_number
39. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Hierarchy Traversal
48
8.0 Common Table Expressions (CTE) 5/5
SELECT * FROM cte.employees;
+------+---------+------------+
| id | name | manager_id |
+------+---------+------------+
| 29 | Pedro | 198 |
| 72 | Pierre | 29 |
| 123 | Adil | 692 |
| 198 | John | 333 |
| 333 | Yasmina | NULL |
| 692 | Tarek | 333 |
| 4610 | Sarah | 29 |
+------+---------+------------+
WITH RECURSIVE emp_ext (id, name, path) AS (
SELECT id, name, name
FROM cte.employees
WHERE manager_id IS NULL
UNION ALL
SELECT s.id, s.name, CONCAT(m.path, ",", s.name)
FROM emp_ext m JOIN cte.employees s ON m.id=s.manager_id
)
SELECT * FROM emp_ext ORDER BY path;
+------+---------+---------------------------+
| id | name | path |
+------+---------+---------------------------+
| 333 | Yasmina | Yasmina |
| 198 | John | Yasmina,John |
| 29 | Pedro | Yasmina,John,Pedro |
| 72 | Pierre | Yasmina,John,Pedro,Pierre |
| 4610 | Sarah | Yasmina,John,Pedro,Sarah |
| 692 | Tarek | Yasmina,Tarek |
| 123 | Adil | Yasmina,Tarek,Adil |
+------+---------+---------------------------+
40. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Hierarchy Traversal with LEFT JOIN
49
8.0 Common Table Expressions (CTE) 5/5
SELECT * FROM cte.employees;
+------+---------+------------+
| id | name | manager_id |
+------+---------+------------+
| 29 | Pedro | 198 |
| 72 | Pierre | 29 |
| 123 | Adil | 692 |
| 198 | John | 333 |
| 333 | Yasmina | NULL |
| 692 | Tarek | 333 |
| 4610 | Sarah | 29 |
+------+---------+------------+
SELECT t1.id, t1.name AS lev1, t2.name as lev2, t3.name as lev3,
t4.name as lev4
FROM cte.employees AS t1
LEFT JOIN cte.employees AS t2 ON t2.id = t1.manager_id
LEFT JOIN cte.employees AS t3 ON t3.id = t2.manager_id
LEFT JOIN cte.employees AS t4 ON t4.id = t3.manager_id
WHERE t1.name IN (SELECT name FROM cte.employees)
ORDER BY lev4, lev3,lev2,lev1;
+------+---------+---------+---------+---------+
| id | lev1 | lev2 | lev3 | lev4 |
+------+---------+---------+---------+---------+
| 333 | Yasmina | NULL | NULL | NULL |
| 198 | John | Yasmina | NULL | NULL |
| 692 | Tarek | Yasmina | NULL | NULL |
| 29 | Pedro | John | Yasmina | NULL |
| 123 | Adil | Tarek | Yasmina | NULL |
| 72 | Pierre | Pedro | John | Yasmina |
| 4610 | Sarah | Pedro | John | Yasmina |
+------+---------+---------+---------+---------+
41. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
52
8.0 Window Functions 1/4
https://dev.mysql.com/doc/refman/8.0/en/window-functions.html
SELECT
name, dept_id, salary,
RANK() OVER w AS `rank`
FROM employee
WINDOW w AS
(PARTITION BY dept_id
ORDER BY salary DESC);
• Frequently requested feature for data analysis like ranking of data
• For each row from a query, perform a calculation using rows related to that row
42. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
53
8.0 Window Functions 2/4
SELECT
year, country, product, profit,
SUM(profit) OVER() AS total_profit,
SUM(profit) OVER(PARTITION BY country) AS country_profit
FROM sales
ORDER BY country, year, product, profit;
• A window function performs an aggregate-like operation on a set of query rows. However, whereas an aggregate
operation groups query rows into a single result row, a window function produces a result for each query row:
– The row for which function evaluation occurs is called the current row
– The query rows related to the current row over which function evaluation occurs comprise the window for the current row
+------+---------+------------+--------+--------------+----------------+
| year | country | product | profit | total_profit | country_profit |
+------+---------+------------+--------+--------------+----------------+
| 2000 | Finland | Computer | 1500 | 7535 | 1610 |
| 2000 | Finland | Phone | 100 | 7535 | 1610 |
| 2001 | Finland | Phone | 10 | 7535 | 1610 |
| 2000 | India | Calculator | 75 | 7535 | 1350 |
| 2000 | India | Calculator | 75 | 7535 | 1350 |
| 2000 | India | Computer | 1200 | 7535 | 1350 |
| 2000 | USA | Calculator | 75 | 7535 | 4575 |
| 2000 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | Calculator | 50 | 7535 | 4575 |
| 2001 | USA | Computer | 1200 | 7535 | 4575 |
| 2001 | USA | Computer | 1500 | 7535 | 4575 |
| 2001 | USA | TV | 100 | 7535 | 4575 |
| 2001 | USA | TV | 150 | 7535 | 4575 |
+------+---------+------------+--------+--------------+----------------+
43. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
55
8.0 Window Functions 3/4
SELECT
year, country, product, profit,
SUM(profit) OVER() AS total_profit,
SUM(profit) OVER(PARTITION BY country) AS country_profit,
SUM(profit) OVER(ORDER BY country, year ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS cumulativ_sum
FROM sales
ORDER BY country, year, product, profit;
• Cumulative sum
+------+---------+------------+--------+--------------+----------------+---------------+
| year | country | product | profit | total_profit | country_profit | cumulativ_sum |
+------+---------+------------+--------+--------------+----------------+---------------+
| 2000 | Finland | Computer | 1500 | 7535 | 1610 | 1500 |
| 2000 | Finland | Phone | 100 | 7535 | 1610 | 1600 |
| 2001 | Finland | Phone | 10 | 7535 | 1610 | 1610 |
| 2000 | India | Calculator | 75 | 7535 | 1350 | 1685 |
| 2000 | India | Calculator | 75 | 7535 | 1350 | 1760 |
| 2000 | India | Computer | 1200 | 7535 | 1350 | 2960 |
| 2000 | USA | Calculator | 75 | 7535 | 4575 | 3035 |
| 2000 | USA | Computer | 1500 | 7535 | 4575 | 4535 |
| 2001 | USA | Calculator | 50 | 7535 | 4575 | 4585 |
| 2001 | USA | Computer | 1200 | 7535 | 4575 | 5785 |
| 2001 | USA | Computer | 1500 | 7535 | 4575 | 7285 |
| 2001 | USA | TV | 100 | 7535 | 4575 | 7385 |
| 2001 | USA | TV | 150 | 7535 | 4575 | 7535 |
+------+---------+------------+--------+--------------+----------------+---------------+
https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777
44. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
56
8.0 Window Functions 4/4
SELECT DISTINCT
FIRST_VALUE(value) OVER w AS First,
LAST_VALUE(value) OVER w AS Last,
SUM(value) OVER w AS Sum,
AVG(value) OVER w AS Avg,
CAST(ts AS DATE) AS date
FROM ts
WINDOW w AS (
PARTITION BY CAST(ts AS DATE)
ORDER BY ts
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
);
+-------+------+------+----------+------------+
| first | last | sum | avg | date |
+-------+------+------+----------+------------+
| 142 | 154 | 609 | 152.2500 | 2018-08-10 |
| 158 | 170 | 655 | 163.7500 | 2018-08-11 |
| 172 | 249 | 771 | 192.7500 | 2018-08-12 |
+-------+------+------+----------+------------+
• Handling Time Series
SELECT * FROM ts;
+----+-------+---------------------+
| id | value | ts |
+----+-------+---------------------+
| 1 | 142 | 2018-08-10 00:00:00 |
| 2 | 163 | 2018-08-10 06:00:00 |
| 3 | 150 | 2018-08-10 12:00:00 |
| 4 | 154 | 2018-08-10 18:00:00 |
| 5 | 158 | 2018-08-11 00:00:00 |
| 6 | 161 | 2018-08-11 06:00:00 |
| 7 | 166 | 2018-08-11 12:00:00 |
| 8 | 170 | 2018-08-11 18:00:00 |
| 9 | 172 | 2018-08-12 00:00:00 |
| 10 | 190 | 2018-08-12 06:00:00 |
| 11 | 160 | 2018-08-12 12:00:00 |
| 12 | 249 | 2018-08-12 18:00:00 |
+----+-------+---------------------+
45. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
59
https://dev.mysql.com/doc/refman/8.0/en/innodb-locking-reads.html#innodb-locking-reads-nowait-skip-locked
SELECT seat_no
FROM seats
JOIN seat_rows USING ( row_no )
WHERE seat_no IN (3,4)
AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seat_rows NOWAIT;
• Locking Read Concurrency with NOWAIT and SKIP LOCKED
8.0 Better Handling Hot Row Contention 1/2
SKIP LOCKED / NOWAIT
Non deterministically
skip over
locked rows
Error immediately
if a row is
already locked
46. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
60
-- Session 1
START TRANSACTION;
SELECT * FROM t WHERE i = 2 FOR UPDATE;
+---+
| i |
+---+
| 2 |
+---+
8.0 Better Handling Hot Row Contention 2/2
SKIP LOCKED / NOWAIT
-- Session 2
START TRANSACTION;
SELECT * FROM t WHERE i = 2 FOR UPDATE NOWAIT;
ERROR 3572 (HY000): Statement aborted because lock(s) could not be
acquired immediately and NOWAIT is set.
-- Session 3
START TRANSACTION;
SELECT * FROM t FOR UPDATE SKIP LOCKED;
+---+
| i |
+---+
| 1 |
| 3 |
+---+
47. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
DBAs/OPS Requests
63
48. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Virtually all organizations require their
most critical systems to be highly available
64
100%
High Availability
49. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
InnoDB Cluster
65
App Servers with
MySQL Router
MySQL Group Replication
MySQL Shell
Setup, Manage,
Orchestrate
“High Availability becomes a core
first class feature of MySQL!”
https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/
50. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Shell: DBA Admin API
• The global variable 'dba' is used to access the MySQL AdminAPI
• JS> dba.help()
• Perform DBA operations
– Manage MySQL InnoDB clusters
• Create clusters
• Validate MySQL instances
• Configure MySQL instances
• Get cluster info
• Modify clusters
• and much more ...
App Servers with
MySQL Router
MySQL Group Replication
MySQL Shell
Setup, Manage,
Orchestrate
66
51. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Group Replication: Database HA
• Group Replication library
– Implementation of Replicated Database State Machine
• MySQL GCS is based on our home-grown Paxos implementation
– Provides virtually synchronous replication for MySQL 5.7+
• Guarantees eventual consistency
– Automates operations
• Conflict detection and resolution
• Failure detection, fail-over, recovery
• Group membership management and reconfiguration
“Multi-master update anywhere replication plugin for MySQL with built-in conflict detection
and resolution, automatic distributed recovery, and group membership.”
67
App Servers with
MySQL Router
MySQL Group Replication
MySQL Shell
Setup, Manage,
Orchestrate
52. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Alter Table – Instant Add Column
• Contribution from Tencent Games
– Only a metadata change
– No copying of data
– Smaller final data size
– Forward compatibility with old data file
• ALTER TABLE ... ADD COLUMN c, ALGORITHM = INSTANT
• Supports DYNAMIC/COMPACT/REDUNDANT row formats
68
53. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0 Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
• Two use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
69
Feature Request
from DBAs
• I don’t think this index is used any more:
ALTER TABLE Country ALTER INDEX c INVISIBLE;
• I need to revert:
ALTER TABLE Country ALTER INDEX c VISIBLE;
• It is now safe to drop:
ALTER TABLE Country DROP INDEX c;
e.g. Soft Delete
54. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Persist Configuration
70
Cloud Friendly
• Persist Global Server Variables
SET PERSIST max_connections = 500 ;
• Examples Inlude :
Offline_mode
Read_Only
• Requires no filesystem access
• Inludes timestamp and change user
55. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Upgrade Checker 1/2
71
• Quick and Easy MySQL Shell Utility
– JavaScript
– Python
• Identifies Issues Based on Severity
– No Issues
– Potential Errors
– Errors that must be fixed before Upgrading
• Recommends Fixes
– Schema, Configuration
– Data on Server, etc.
https://mysqlserverteam.com/mysql-shell-8-0-4-introducing-upgrade-checker-utility/
Feature Request
from DBAs
56. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Upgrade Checker 2/2
72
57. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: InnoDB Improvements
• Document Store/JSON
– More efficient BLOBs, up to 4x improvement on update
– Better performance for both small and large BLOBs
• New Scalable WAL
– Lock free design
• Atomic DDL
• Persistent Auto Increment
• Contention Aware Transaction Scheduling (CATS)
– Contribution from University of Michigan
• Removed IO bottlenecks
– More scalable design
74
58. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: Spatial Reference Systems
76
https://mysqlserverteam.com/spatial-reference-systems-in-mysql-8-0/
59. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0MySQL 5.7
8.0: Improved GIS
77
• The world is flat
• The world is infinite
• Axes are unitless
• Axes are orthogonal
• Axis order is irrelevant
• Axis direction is irrelevant
• The world can be flat or ellipsoidal
• Geographic coordinate systems wrap around
• Axes have units
• Geographic axes are not orthogonal
• Geogrphic axis order matters
• Axis direction may be relevant
60. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Highly Accurate
Now better than competitor databases
• High Performance
• Simple – No extra configuration or installation
Completely built in and ready to use
• Powerful
Full Geography Support
Projected – Flat/Across 2 dimensions
Geographic – Spheroid
Details
5107 predefined SRSs from the EPSG Dataset 9.2
4628 projected
479 gegraphic
8.0: GIS Improvements
78
61. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Performance Improvements
80
62. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
New! Performance Improvements
• Improved Query Consistency
– Histograms
– Improved Cost Model
• Faster Table/Range Scans
81
• Parallel Replication
• UTF8MB4
• Information Schema
• Performance Schema Indexes
http://dimitrik.free.fr/blog/
63. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
New! Transactional Data Dictionary
• Crash-safe Database
– Common data dictionary for the server and
InnoDB
• Crash-safe & Atomic DDL
– CREATE USER <userlist>, DROP DATABASE with
all-or-none semantic
– Simplifies replication failure cases
82
• Meta-data locking for FK constraints
– FK moved from InnoDB to server layer
• Scalable Information Schema
– Queries are now executed as set of SQL views
on tables
– Large performance improvements
Atomic and crash-safe DDL
64. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Column statistics
8.0: Histograms
Feature Request
by DBAs
• Provides the optimizer with information about column value distribution
• To create/recalculate histogram for a column:
– ANALYZE TABLE table UPDATE HISTOGRAM ON column WITH n BUCKETS;
• May use sampling
– Sample size is based on available memory ( histogram_generation_max_mem_size )
• Automatically chooses between two histogram types:
– Singleton: One value per bucket
– Equi-height: Multiple value per bucket
83
65. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 85
1 2 4 8 16 32 64 128 256 512
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
MySQL 8.0
MySQL 5.7
MySQL 5.6
Users
QueriesperSecond
OS : Oracle Linux 7.4
CPU : 48cores-HT Intel Skylake 2.7Ghz
(2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU)
RAM: 256GB
Storage : x2 Intel Optane flash devices
(Intel (R) Optane (TM) SSD P4800X Series)
2x Faster than MySQL 5.7
8.0: SysBench IO Bound Read Only (Point Selects)
66. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 86
OS : Oracle Linux 7.4
CPU : 48cores-HT Intel Skylake 2.7Ghz
(2CPU sockets, Intel(R) Xeon(R) Platinum 8168
RAM: 256GB
Storage : x2 Intel Optane flash devices
(Intel (R) Optane (TM) SSD P4800X Series)
2x Faster than MySQL 5.7
8.0: SysBench Read/Write (Update NoKey)
67. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Improved to consider buffer pool fit
8.0: Optimizer Cost Model
Feature Request
by DBAs• Storage engines :
– Estimate for how much of data and
indexes are in memory buffer
– Estimate for hit rate for memory buffer
• Optimizer cost model :
– Take into account wether data is already
in memory or need to be read from disk
88
68. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Security
https://dev.mysql.com/doc/mysql-security-excerpt/8.0/en/
90
69. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
• Improving MySQL Access Controls
• Easier to manage user and applications rights
• As standards compliant as practically possible
• Multiple default roles
• Export the role graph in GraphML
– Visualize with ROLES_GRPHML()
91
Directly
Indirectly
Set Role(s)
Default Role(s)
Set of
ACLS
Set of
ACLS
New! SQL Roles
Feature Request
from DBAs
70. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Password Features
• New! Password history - provides DBAs more password management
– Require new passwords not reuse old ones - By number of changes and/or time.
– Establish password-reuse policy globally as well as on a per-account basis.
• New! SHA2 with Caching
– Strong and Fast
– Strong - SHA-256 password hashing (many rounds, seeds, …)
– Fast - Caching
• Greatly reduces latency
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-default-authentication-plugin-considerations/
• New! Supports for more connection protocols
• New! Seamless RSA password-exchange capabilities (No linking OpenSSL)
92
71. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
8.0: File Encryption
• New! AES 256 encryption of UNDO and REDO Logs
• Super Simple to manage - Set
– innodb_undo_log_encrypt=ON/OFF
– innodb_redo_log_encrypt=ON/OFF
• And
– ON - Pages written after setting are encrypted
– OFF - Pages written after setting are not.
93
72. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 9
4
8.0: All these features plus…
• Source code now documented with Doxygen
• Plugin Infrastructure!
• Expanded GIS Support
• Expanded Query Hints Support
• Improved Scan Query Performance
• Improved BLOB Storage
• Improved Memcached Interface
• Cost Model Improvements
• Scalability Improvements
• Atomicity in Privileges
• Parser Refactoring
• Improvements to Temporary Tables
• C++11 and Toolchain Improvements
• GTID_PURGED always settable
• Persistent Auto Increment
• Native InnoDB Data dictionary
• Improved Information Schema
performance
• SQL Grouping Function
• Optimizer Trace detailed sort statistics
• Descending Indexes
• Smaller Package Downloads
• JSON Aggregate, Pretty print functions
• JSON performance improvements
• Expanded Query Hints
• Improved usability of cost constant
configuration
• Group Replication in 8.0
• Transaction Save Point support in Group
Replication
• Improved Replication Applier Lag
Statistics
in Performance Schema
• Per Multi-source Channel Replication
Filters
• Atomic DDL extended to the Binary Log
• Performance Improvements on the
Replication Applier
• Parallel Replication Applier Policy
• Binary Log Management Enhancements
• Additional Metadata Into the Binary Log
• ...
73. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 95
• DocStore
– SQL + NoSQL = MySQL
• InnoDB Cluster
– HA made easy
• SQL
• JSON
• GIS
8.0 – Innovating and Evolving
• Replication
• Reliability
• Observability
• Manageability
• Security
• Performance
• Scalability
74. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Resources – MySQL Blogs 1/3
96
• MySQL Server Team’s blog
– http://mysqlserverteam.com/
• MySQL High Availability Team’s blog
– http://mysqlhighavailability.com/
• MySQL Release Engineering Blog
– https://mysqlrelease.com/
• MySQL Dev Team Blog
– https://insidemysql.com/
• MySQL Enterprise Team Blogs
– https://blogs.oracle.com/mysql/
• Lefred’s blog (MySQL Community Manager)
– http://lefred.be/
• Olivier Dasini’s blog
– http://dasini.net/blog/
75. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Resources – MySQL Blogs 2/3
97
• MySQL Document Store
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://github.com/mysql/mysql-connector-nodejs
– https://www.npmjs.com/package/@mysql/xdevapi
– https://dev.mysql.com/doc/refman/8.0/en/document-store.html
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://dev.mysql.com/doc/dev/connector-nodejs/8.0/
– https://www.mysql.com/news-and-events/web-seminars/mysql-document-store-and-node-js/
• MySQL Performance by Dimitri
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-over-18m-qps-with-80-ga-on-2s-skylake.html
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-80-and-utf8-impact.html
• MySQL InnoDB Cluster
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/
– https://mysqlserverteam.com/mysql-innodb-cluster-whats-new-in-the-8-0-ga-release/
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-a-hands-on-tutorial/
76. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Resources – MySQL Blogs 3/3
98
• Common Table Expressions (CTE)
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-two-how-to-generate-series/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-three-hierarchies/
– http://mysqlserverteam.com/mysql-8-0-1-recursive-common-table-expressions-in-mysql-ctes-part-four-depth-first-or-breadth-first-traversal-
transitive-closure-cycle-avoidance/
• Window Functions
– http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
– https://mysqlserverteam.com/row-numbering-ranking-how-to-use-less-user-variables-in-mysql-queries/
– https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777
• Security
– https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/
– https://mysqlserverteam.com/mysql-8-0-4-openssl-and-mysql-community-edition/
– http://lefred.be/content/mysql-8-0-listing-roles/
– https://dev.mysql.com/doc/refman/8.0/en/password-management.html
• Upgrade from 5.7 to 8.0
– https://mysqlserverteam.com/inplace-upgrade-from-mysql-5-7-to-mysql-8-0/
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/
77. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Thanks for using MySQL!