SlideShare a Scribd company logo
1 of 67
Download to read offline
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Optimizer: What’s New in 8.0
Manyi Lu
Director
Øystein Grøvlen
Senior Principal Software Engineer
MySQL Optimizer Team, Oracle
October, 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
4
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
5
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Common Table Expression
• A derived table is a subquery in the FROM clause
SELECT … FROM (subquery) AS derived, t1 ...
• Common Table Expression (CTE) is just like a derived table, but its
declaration is put before the query block instead of in FROM clause
WITH derived AS (subquery)
SELECT … FROM derived, t1 ...
• A CTE may precede SELECT/UPDATE/DELETE including sub-queries
WITH derived AS (subquery)
DELETE FROM t1 WHERE t1.a IN (SELECT b FROM derived);
6
Alternative to derived table
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Common Table Expression vs Derived Table
Better readability
Can be referenced multiple times
Can refer to other CTEs
Improved performance
7
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Better Readability
• Derived table:
SELECT …
FROM t1 LEFT JOIN ((SELECT … FROM …) AS dt JOIN t2 ON …) ON …
• CTE:
WITH dt AS (SELECT ... FROM ...)
SELECT ...
FROM t1 LEFT JOIN (dt JOIN t2 ON ...) ON ...
8
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Can Be Referenced Multiple Times
• Derived table can not be referenced twice:
SELECT ...
FROM (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b) AS d1
JOIN (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b) AS d2 ON d1.b = d2.a;
• CTE can:
WITH d AS (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b)
SELECT ... FROM d AS d1 JOIN d AS d2 ON d1.b = d2.a;
• Better performance with materialization:
– Multiple references only materialized once
– Derived tables and views will be materialized once per reference.
9
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Can Refer to Other CTEs
• Derived tables can not refer to other derived tables:
SELECT …
FROM (SELECT … FROM …) AS d1, (SELECT … FROM d1 …) AS d2 …
ERROR: 1146 (42S02): Table ‘db.d1’ doesn’t exist
• CTEs can refer other CTEs:
WITH d1 AS (SELECT … FROM …),
d2 AS (SELECT … FROM d1 …)
SELECT
FROM d1, d2 …
10
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Recursive CTE
• A recursive CTE refers to itself in a subquery
• The “seed” SELECT is executed once to create the initial data subset, the
recursive SELECT is repeatedly executed to return subsets of.
• Recursion stops when an iteration does not generate any new rows
– To limit recursion, set cte_max_recursion_depth
• Useful to dig in hierarchies (parent/child, part/subpart)
11
WITH RECURSIVE cte AS
( SELECT ... FROM table_name /* "seed" SELECT */
UNION [DISTINCT|ALL]
SELECT ... FROM cte, table_name ) /* "recursive" SELECT */
SELECT ... FROM cte;
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Recursive CTE
12
A simple example
Print 1 to 10 :
WITH RECURSIVE qn AS
( SELECT 1 AS a
UNION ALL
SELECT 1+a FROM qn WHERE a<10
)
SELECT * FROM qn;
a
1
2
3
4
5
6
7
8
9
10
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hierarchy Traversal
13
Employee database
CREATE TABLE employees (
id INT PRIMARY KEY,
name VARCHAR(100),
manager_id INT,
FOREIGN KEY (manager_id)
REFERENCES employees(id) );
INSERT INTO employees VALUES
(333, "Yasmina", NULL), # CEO
(198, "John", 333), # John reports to 333
(692, "Tarek", 333),
(29, "Pedro", 198),
(4610, "Sarah", 29),
(72, "Pierre", 29),
(123, "Adil", 692);
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hierarchy Traversal
14
List reporting chain
WITH RECURSIVE
emp_ext (id, name, path) AS (
SELECT id, name, CAST(id AS CHAR(200))
FROM employees
WHERE manager_id IS NULL
UNION ALL
SELECT s.id, s.name,
CONCAT(m.path, ",", s.id)
FROM emp_ext m JOIN employees s
ON m.id=s.manager_id )
SELECT * FROM emp_ext ORDER BY path;
id name path
333 Yasmina 333
198 John 333,198
692 Tarek 333,692
29 Pedro 333,198,29
123 Adil 333,692,123
4610 Sarah 333,198,29,4610
72 Pierre 333,198,29,72
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hierarchy Traversal
15
List reporting chain
WITH RECURSIVE
emp_ext (id, name, path) AS (
SELECT id, name, CAST(id AS CHAR(200))
FROM employees
WHERE manager_id IS NULL
UNION ALL
SELECT s.id, s.name,
CONCAT(m.path, ",", s.id)
FROM emp_ext m JOIN employees s
ON m.id=s.manager_id )
SELECT * FROM emp_ext ORDER BY path;
id name path
333 Yasmina 333
198 John 333,198
29 Pedro 333,198,29
4610 Sarah 333,198,29,4610
72 Pierre 333,198,29,72
692 Tarek 333,692
123 Adil 333,692,123
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
16
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Window Functions: What Are They?
• A window function performs a calculation across a set of rows that are
related to the current row, similar to an aggregate function.
• But unlike aggregate functions, a window function does not cause rows to
become grouped into a single output row.
• Window functions can access values of other rows “in the vicinity” of the
current row
17
Aggregate function Window function
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Window Function Example
18
PARTITION == disjoint
set of rows in result set
name dept_id salary dept_total
Newt NULL 75000 75000
Dag 10 NULL 370000
Ed 10 100000 370000
Fred 10 60000 370000
Jon 10 60000 370000
Michael 10 70000 370000
Newt 10 80000 370000
Lebedev 20 65000 130000
Pete 20 65000 130000
Jeff 30 300000 370000
Will 30 70000 370000
Sum up total salary for each department:
SELECT name, dept_id, salary,
SUM(salary) OVER (PARTITION BY
dept_id) AS dept_total
FROM employee
ORDER BY dept_id, name;
The OVER keyword
signals a window function
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
name dept_id salary total
Newt NULL 75000 75000
Dag 10 NULL NULL
Ed 10 100000 100000
Fred 10 60000 160000
Jon 10 60000 220000
Michael 10 70000 190000
Newt 10 80000 210000
Lebedev 20 65000 65000
Pete 20 65000 130000
Jeff 30 300000 300000
Will 30 70000 370000
19
Window Function Example: Frames
ORDER BY name
within each partition
moving window frame:
SUM(salary) ...
ROWS 2 PRECEDING
A frame is a subset of a
partition
SELECT name, dept_id, salary,
SUM(salary)
OVER (PARTITION BY dept_id
ORDER BY name
ROWS 2 PRECEDING) total
FROM employee
ORDER BY dept_id, name;
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20
Window Function Example: Frames
SELECT name, dept_id, salary,
SUM(salary)
OVER (PARTITION BY dept_id
ORDER BY name
ROWS 2 PRECEDING) total
FROM employee
ORDER BY dept_id, name;
name dept_id salary total
Newt NULL 75000 75000
Dag 10 NULL NULL
Ed 10 100000 100000
Fred 10 60000 160000
Jon 10 60000 220000
Michael 10 70000 190000
Newt 10 80000 210000
Lebedev 20 65000 65000
Pete 20 65000 130000
Jeff 30 300000 300000
Will 30 70000 370000
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
21
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Improved UTF-8 Support in MySQL 8.0
• Support for the latest Unicode 9.0
• utf8mb4 made default character set!
–utf8mb4_0900_ai_ci default collation
• Accent and case sensitive collations
–Including 20+ language specific collations
–Now also Japanese and Russian
• Significantly improved performance
22
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What Is in MySQL 5.7 and Earlier Versions?
• Default charset is “latin1” and default collation is “latin1_swedish_ci”
• utf8 = utf8mb3: support BMP only
• utf8mb4 character set:
–Only accent and case insensitive collations
–Default collation is utf8mb4_general_ci, compares all characters beyond
BMP, e.g. emojis, to be equal
–20+ language specific collations
–Recommend to use: utf8mb4_unicode_520_ci
23
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
New Default Character Set
• No changes to existing tables
• Only has effect on new tables/schemas where character set is not
explicitly defined.
• Separating character set/collation change from server upgrade
– Upgrade first, change charset/collation afterwards
• Recommend users to not mixing collations
– Error “Illegal mix of collations”
– Slower query because index can no longer be used
24
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
25
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
GIS
• Geography Support
– longitude/latitude
• Spatial Reference Systems (SRS) Support
– SRID 4326 = WGS 84 (“GPS coordinates”)
• Information Schema views
– ST_GEOMETRY_COLUMNS
• Standard compliant axis ordering : longitude-latitude
• Example functions :
– ST_Distance( g1, g2)
– ST_SwapXY(g)
– ST_SRID(g [, new_srid_val])
26
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
27
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
SELECT ... FOR UPDATE SKIP LOCKED
• Common problem:
– Hot row contention, multiple worker threads accessing the same rows
• Solution 1:
– Only read rows that are not locked
– InnoDB skips a locked row, and the next one goes to the result set
• Example:
– Booking system: Skip orders that are pending
28
START TRANSACTION;
SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO'
FOR UPDATE SKIP LOCKED;
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
SELECT… FOR UPDATE NOWAIT
• Common problem:
– Hot row contention, multiple worker threads accessing the same rows
• Solution 2:
– If any of the rows are already locked, the statement should fail immediately
– Without NOWAIT, have to wait for innodb_lock_wait_timeout (default: 50 sec) while
trying to acquire lock
• Usage:
29
START TRANSACTION;
SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO'
FOR UPDATE NOWAIT;
ERROR 3572 (HY000): Statement aborted because lock(s) could not be acquired …
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
30
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JSON Aggregation
31
Combine JSON documents in multiple rows into a JSON array
CREATE TABLE t1 (id INT, grp INT,
jsoncol JSON);
INSERT INTO t1 VALUES (1, 1,
'{"key1":"value1","key2":"value2"}');
INSERT INTO t1 VALUES (2, 1,
'{"keyA":"valueA","keyB":"valueB"}');
INSERT INTO t1 VALUES (3, 2,
'{"keyX":"valueX","keyY":"valueY"}');
SELECT JSON_ARRAYAGG(jsoncol)
FROM t1;
[{"key1":"value1","key2":"value2"},
{"keyA":"valueA","keyB":"valueB"},
{"keyX":"valueX","keyY":"valueY"}]
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JSON Aggregation
32
Combine JSON documents in multiple rows into a JSON object
CREATE TABLE t1 (id INT, grp INT,
jsoncol JSON);
INSERT INTO t1 VALUES (1, 1,
'{"key1":"value1","key2":"value2"}');
INSERT INTO t1 VALUES (2, 1,
'{"keyA":"valueA","keyB":"valueB"}');
INSERT INTO t1 VALUES (3, 2,
'{"keyX":"valueX","keyY":"valueY"}');
SELECT grp, JSON_OBJECTAGG(id, jsoncol)
FROM t1
GROUP BY grp;
1 | {"1":{"key1":"value1","key2":"value2"},
"2":{"keyA":"valueA","keyB":"valueB"}}
2 | {"3":{"keyX":"valueX","keyY":"valueY"}}
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JSON_TABLE
• Table t1 has a column json_col with content like this:
{ "people": [
{ "name":"John Smith", "address":"780 Mission St, San Francisco, CA 94103"},
{ "name":"Sally Brown", "address":"75 37th Ave S, St Cloud, MN 94103"},
{ "name":"Paul Johnson", "address":"1262 Roosevelt Trail, Raymond, ME 04071"},
… ] }
• Convert JSON column into a table with 2 columns:
SELECT people.* FROM t1, JSON_TABLE(json_col, '$.people[*]' COLUMNS (
name VARCHAR(40) PATH '$.name',
address VARCHAR(100) PATH '$.address')) people
WHERE people.address LIKE '%San Francisco%';
33
Convert JSON documents to relational tables Labs release
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 34
JSON_TABLE – Nested Arrays
id father married child_id child age
1 John 1 1 Eric 12
1 John 1 2 Beth 10
2 Paul 0 1 Sarah 9
2 Paul 0 2 Noah 3
2 Paul 0 3 Peter 1
[
{ "father":"John", "mother":"Mary",
"marriage_date":"2003-12-05",
"children": [
{ "name":"Eric", "age":12 },
{ "name":"Beth", "age":10 } ] },
{ "father":"Paul", "mother":"Laura",
"children": [
{ "name":"Sarah", "age":9},
{ "name":"Noah", "age":3} ,
{ "name":"Peter", "age":1} ] }
]
Labs release
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 35
JSON_TABLE – Nested Arrays
id father married child_id child age
1 John 1 1 Eric 12
1 John 1 2 Beth 10
2 Paul 0 1 Sarah 9
2 Paul 0 2 Noah 3
2 Paul 0 3 Peter 1
JSON_TABLE (families, '$[*]' COLUMNS (
id FOR ORDINALITY,
father VARCHAR(30) PATH '$.father',
married INTEGER EXISTS PATH
'$.marriage_date',
NESTED PATH '$.children[*]' COLUMNS (
child_id FOR ORDINALITY,
child VARCHAR(30) PATH '$.name',
age INTEGER PATH '$.age‘ ) )
Labs release
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JSON_TABLE
SELECT father, COUNT(*) "#children", AVG(age) "age average"
FROM t, JSON_TABLE (families, '$[*]' COLUMNS (
id FOR ORDINALITY,
father VARCHAR(30) PATH '$.father',
NESTED PATH '$.children[*]' COLUMNS (age INTEGER PATH '$.age‘ ) ) AS fam
GROUP BY id, father;
36
SQL aggregation on JSON data
father #children age average
John 2 11.0000
Paul 3 4.3333
Labs release
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JSON Utility Functions
• JSON_PRETTY(json_value)
– Pretty-print the JSON value
• JSON_STORAGE_SIZE(json_value)
– The number of bytes used to store the binary representation of a JSON document
• JSON_STORAGE_FREE(json_value)
– The number of bytes in its binary representation that is current not used.
– The binary representation may have unused space after a JSON column was updated
in place using JSON_SET()
37
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
38
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Invisible Index
• Index is maintained by the SE, but ignored by the Optimizer
• Primary key cannot be INVISIBLE
• Use case: Check for performance drop BEFORE dropping an index
• To see an invisible index: set optimizer_switch='use_invisible_indexes=on';
39
ALTER TABLE t1 ALTER INDEX idx INVISIBLE;
mysql> SHOW INDEXES FROM t1;
+---------+------------------+----------------------+---------------+
| Table | Key_name | Column_name | Visible |
+---------+------------------+----------------------+---------------+
| t1 | idx | a | NO |
+---------+------------------+----------------------+---------------+
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Descending Index
• In 5.7: Index in ascending order is created, server scans it backwards
• In 8.0: Index in descending order is created, server scans it forwards
• Works on B-tree indexes only
• Benefits:
– Use indexes instead of filesort for ORDER BY clause with ASC/DESC sort key
– Forward index scan is slightly faster than backward index scan
40
CREATE TABLE t1 (
a INT,
b INT,
INDEX a_b (a DESC, b ASC)
);
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
41
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Motivation for Improving the MySQL Cost Model
• Produce more correct cost estimates
– Better decisions by the optimizer should improve performance
• Adapt to new hardware architectures
– SSD, larger memories, caches
• More maintainable cost model implementation
– Avoid hard coded “cost constants”
– Refactoring of existing cost model code
• Configurable and tunable
• Make more of the optimizer cost-based
Faster
queries
42
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
• Time to do a table scan of 10
million records:
• Adjust cost model to support
different storage technologies
• Provide configurable cost constants
for different storage technologies
New Storage Technologies
Memory 5 s
SSD 20 - 146 s
Hard disk 32 - 1465 s
Provide a program that could
measure performance and
suggest good cost constant
configuration for a running
MySQL server?
43
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
• Storage engines:
– Estimate for how much of data and
indexes are in a memory buffer
– Estimate for hit rate for memory buffer
• Optimizer cost model:
– Take into account whether data is
already in memory or need to be read
from disk
Memory Buffer Aware Cost Estimates
Server
Storage
engine
Disk data
Query
executor
Database
buffer
44
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0: Disk vs Memory Access
• New defaults for const constants:
• InnoDB reports for each table/index percentage of data cached in buffer
pool
• Note: Query plan may change between executions
Confidential – Oracle Internal/Restricted/Highly Restricted 45
Cost MySQL 5.7 MySQL 8.0
Read a random disk page 1.0 1.0
Read a data page from memory buffer 1.0 0.25
Evaluate query condition 0.2 0.1
Compare keys/rows 0.1 0.05
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
DBT-3 Query 8
SELECT o_year, SUM(CASE WHEN nation = 'FRANCE' THEN volume ELSE 0 END) / SUM(volume) AS
mkt_share
FROM (
SELECT EXTRACT(YEAR FROM o_orderdate) AS o_year,
l_extendedprice * (1 - l_discount) AS volume, n2.n_name AS nation
FROM part
JOIN lineitem ON p_partkey = l_partkey
JOIN supplier ON s_suppkey = l_suppkey
JOIN orders ON l_orderkey = o_orderkey
JOIN customer ON o_custkey = c_custkey
JOIN nation n1 ON c_nationkey = n1.n_nationkey
JOIN region ON n1.n_regionkey = r_regionkey
JOIN nation n2 ON s_nationkey = n2.n_nationkey
WHERE r_name = 'EUROPE' AND o_orderdate BETWEEN '1995-01-01' AND '1996-12-31'
AND p_type = 'PROMO BRUSHED STEEL'
) AS all_nations GROUP BY o_year ORDER BY o_year;
National Market Share Query
High selectivity
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 47
Alternative query plans
Plan A
Plan B
DBT-3 Query 8
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Execution time (MySQL 8.0.3)
In-memory Disk-bound
Plan A 5.8 secs 9 min 47 secs
Plan B 77.5 secs 3 min 49 secs
Selected plan
In-memory Disk-bound
MySQL 5.6 Plan B
MySQL 5.7 Plan A
MySQL 8.0 Plan A Plan B
48
DBT-3 Query 8
DBT-3 Scale factor 10
In-Memory: innodb_buffer_pool_size = 32 GB
Disk-bound: innodb_buffer_pool_size = 1 GB
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Histograms
• Provides the optimizer with information about column value distribution
• To create/recalculate histogram for a column:
ANALYZE TABLE table UPDATE HISTOGRAM ON column WITH n BUCKETS;
• May use sampling
– Sample size is based on available memory (histogram_generation_max_mem_size)
• Automatically chooses between two histogram types:
– Singleton: One value per bucket
– Equi-height: Multiple value per bucket
49
Column statistics
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Histograms
EXPLAIN SELECT *
FROM customer JOIN orders ON c_custkey = o_custkey
WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01';
50
Example query
id
select
type
table type possible keys key
key
len
ref rows filtered extra
1 SIMPLE orders ALL
i_o_orderdate,
i_o_custkey
NULL NULL NULL 15000000 31.19
Using
where
1 SIMPLE customer
eq_
ref
PRIMARY PRIMARY 4
dbt3.orders.
o_custkey
1 33.33
Using
where
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Histograms
ANALYZE TABLE customer UPDATE HISTOGRAM ON c_acctbal WITH 1024 buckets;
EXPLAIN SELECT *
FROM customer JOIN orders ON c_custkey = o_custkey
WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01';
51
Create histogram to get a better plan
id
select
type
table type possible keys key
key
len
ref rows filtered extra
1 SIMPLE customer ALL PRIMARY NULL NULL NULL 1500000 0.00
Using
where
1 SIMPLE orders ref
i_o_orderdate,
i_o_custkey
i_o_custkey 5
dbt3.
customer.
c_custkey
15 31.19
Using
where
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Comparing Join Order
0
2
4
6
8
10
12
14
16
QueryExecutionTime(seconds)
orders → customer customer → orders
Performance
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
53
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hints: Join Order
• Hints to control table order for join execution
• 5.7: STRAIGHT_JOIN to force the listed order in FROM clause
• 8.0:
– JOIN_FIXED_ORDER /* replacement for STRAIGHT_JOIN*/
– JOIN_ORDER /* use specified order */
– JOIN_PREFIX /* use specified order for first tables */
– JOIN_SUFFIX /* use specified order for last tables */
– No need to reorganize the FROM clause to add join order hints like you will for
STRAIGHT_JOIN
54
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Join Order Hints - Example
EXPLAIN SELECT /*+ JOIN_ORDER(customer, orders) */ *
FROM customer JOIN orders ON c_custkey = o_custkey
WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01';
55
Change join order with hint
id
select
type
table type possible keys key
key
len
ref rows filtered extra
1 SIMPLE customer ALL PRIMARY NULL NULL NULL 1500000 33.33
Using
where
1 SIMPLE orders ref
i_o_orderdate,
i_o_custkey
i_o_custkey 5
dbt3.
customer.
c_custkey
15 31.19
Using
where
Alternatives with same effect for this query:
JOIN_PREFIX(customer) JOIN_SUFFIX(orders) JOIN_FIXED_ORDER()
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hints: Index Merge
• Index merge: Merge rows from multiple range scans on a single table
• Algorithms: union, intersection, sort union
• Users can specify which indexes to use for index merge
– /*+ INDEX_MERGE() */
– /*+ NO_INDEX_MERGE() */
56
10INDEX(a)
10INDEX(b)
a=10 AND b=10Result:
Intersection
SELECT * FROM t1 WHERE a=10 AND b=10
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Index Merge Hint - Example
EXPLAIN SELECT count(*) FROM users
WHERE user_type=2 AND status=0 AND parent_id=0;
57
id
select
type
table type
possible
keys
key
key
len
ref rows Extra
1 SIMPLE users
index_
merge
parent_id,
status,
user_type
user_type,
status,
parent_id
1,1,4 NULL 2511
Using intersect (user_type,
status, parent_id);
Using where; Using index
mysql> SELECT count(*) FROM users WHERE user_type=2 AND status=0 AND parent_id=0;
...
1 row in set (1.37 sec)
mysql> SELECT /*+ INDEX_MERGE(users user_type, status) */ count(*)
-> FROM users WHERE user_type=2 AND status=0 AND parent_id=0;
...
1 row in set (0.18 sec)
Low selectivity
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Hints: Set session variables
• Set a session variable for the duration of a single statement
• Examples:
SELECT /* SET_VAR(sort_buffer_size = 16M) */ name FROM people ORDER BY name;
INSERT /* SET_VAR(foreign_key_checks = OFF) */ INTO t2 VALUES (1, 1), (2, 2), (3, 3);
SELECT /* SET_VAR(optimizer_switch = 'condition_fanout_filter = off') */ *
FROM customer JOIN orders ON c_custkey = o_custkey
WHERE c_acctbal < 0 AND o_orderdate < '1993-01-01';
• NB! Not all session variables are settable through hints:
mysql> SELECT /*+ SET_VAR(max_allowed_packet=128M) */ * FROM t1;
Empty set, 1 warning (0,01 sec)
Warning (Code 4537): Variable 'max_allowed_packet' cannot be set using
SET_VAR hint.
58
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
59
 Common table expressions
 Window functions
 UTF8 support
 GIS
 SKIP LOCKED, NOWAIT
 JSON functions
 Index extensions
 Cost model
 Hints
 Better IPv6 and UUID support
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Improved Support for UUID
• Five “-” separated hexadecimal numbers
• MySQL uses version 1, the first three numbers are generated from the low,
middle, and high parts of a timestamp.
• 36 characters, inefficient for storage
Convert to BINARY(16) datatype, only 16 bytes
60
mysql> select uuid();
+---------------------------------------------------------+
| uuid() |
+---------------------------------------------------------+
| aab5d5fd-70c1-11e5-a4fb-b026b977eb28 |
+---------------------------------------------------------+
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Improved Support for UUID
• UUID_TO_BIN(string_uuid, swap_flag)
• BIN_TO_UUID(binary_uuid, swap_flag)
• IS_UUID(string_uuid)
61
Functions to convert UUID to and from binary
Feature Request
from Developers
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
UUID_TO_BIN Optimization
• Binary format is now smaller
• Shuffling low part with the high part improves index performance
62
25 26 27 28 29
Insert Performance Optimized
Original
11e678fe53303f87a4778c89a52c4f3b
53303f87-78fe-11e6-a477-8c89a52c4f3bFrom VARCHAR(36)
To BINARY(16)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
IPv6
• New! Bit-wise operations on binary data types
– Designed with IPv6 in mind:
– INET6_ATON(address) & INET6_ATON(network)
– No longer truncation beyond 64 bits
63
Feature Request
from Developers
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
• Improved Query Scan Performance
• GROUPING()
• Hint: Merge/materialize derived
table/view
• JSON:
– Partial update
– Improved performance of sorting and
grouping of JSON values
– Path expressions: Array index ranges
– JSON_MERGE_PATCH()
• Skip index dives for FORCE INDEX
• Parser Refactoring
64
All These Features and More…
Try it out!
Give us feedback!
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
65
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 66
MySQL Optimizer: What’s New in 8.0

More Related Content

What's hot

How to analyze and tune sql queries for better performance webinar
How to analyze and tune sql queries for better performance webinarHow to analyze and tune sql queries for better performance webinar
How to analyze and tune sql queries for better performance webinar
oysteing
 

What's hot (20)

How to analyze and tune sql queries for better performance webinar
How to analyze and tune sql queries for better performance webinarHow to analyze and tune sql queries for better performance webinar
How to analyze and tune sql queries for better performance webinar
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
 
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZEMySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
MySQL 8.0.18 latest updates: Hash join and EXPLAIN ANALYZE
 
How to analyze and tune sql queries for better performance percona15
How to analyze and tune sql queries for better performance percona15How to analyze and tune sql queries for better performance percona15
How to analyze and tune sql queries for better performance percona15
 
Ctes percona live_2017
Ctes percona live_2017Ctes percona live_2017
Ctes percona live_2017
 
MySQL Optimizer Cost Model
MySQL Optimizer Cost ModelMySQL Optimizer Cost Model
MySQL Optimizer Cost Model
 
How to Take Advantage of Optimizer Improvements in MySQL 8.0
How to Take Advantage of Optimizer Improvements in MySQL 8.0How to Take Advantage of Optimizer Improvements in MySQL 8.0
How to Take Advantage of Optimizer Improvements in MySQL 8.0
 
SQL window functions for MySQL
SQL window functions for MySQLSQL window functions for MySQL
SQL window functions for MySQL
 
MySQL Optimizer Overview
MySQL Optimizer OverviewMySQL Optimizer Overview
MySQL Optimizer Overview
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
 
Optimizer overviewoow2014
Optimizer overviewoow2014Optimizer overviewoow2014
Optimizer overviewoow2014
 
LATERAL Derived Tables in MySQL 8.0
LATERAL Derived Tables in MySQL 8.0LATERAL Derived Tables in MySQL 8.0
LATERAL Derived Tables in MySQL 8.0
 
Optimizing MySQL Queries
Optimizing MySQL QueriesOptimizing MySQL Queries
Optimizing MySQL Queries
 
Agile Database Development with JSON
Agile Database Development with JSONAgile Database Development with JSON
Agile Database Development with JSON
 
Optimizing queries MySQL
Optimizing queries MySQLOptimizing queries MySQL
Optimizing queries MySQL
 
Explain that explain
Explain that explainExplain that explain
Explain that explain
 
Query optimization techniques for partitioned tables.
Query optimization techniques for partitioned tables.Query optimization techniques for partitioned tables.
Query optimization techniques for partitioned tables.
 
New SQL features in latest MySQL releases
New SQL features in latest MySQL releasesNew SQL features in latest MySQL releases
New SQL features in latest MySQL releases
 
Partition and conquer large data in PostgreSQL 10
Partition and conquer large data in PostgreSQL 10Partition and conquer large data in PostgreSQL 10
Partition and conquer large data in PostgreSQL 10
 

Similar to MySQL Optimizer: What’s New in 8.0

Developers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman OracleDevelopers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman Oracle
mCloud
 
Prog1 chap1 and chap 2
Prog1 chap1 and chap 2Prog1 chap1 and chap 2
Prog1 chap1 and chap 2
rowensCap
 
Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007
paulguerin
 

Similar to MySQL Optimizer: What’s New in 8.0 (20)

MySQL Optimizer: What's New in 8.0
MySQL Optimizer: What's New in 8.0MySQL Optimizer: What's New in 8.0
MySQL Optimizer: What's New in 8.0
 
20180420 hk-the powerofmysql8
20180420 hk-the powerofmysql820180420 hk-the powerofmysql8
20180420 hk-the powerofmysql8
 
MySQL 8.0 Optimizer Guide
MySQL 8.0 Optimizer GuideMySQL 8.0 Optimizer Guide
MySQL 8.0 Optimizer Guide
 
MySQL 8.0.1 DMR
MySQL 8.0.1 DMRMySQL 8.0.1 DMR
MySQL 8.0.1 DMR
 
MySQL 8.0: What Is New in Optimizer and Executor?
MySQL 8.0: What Is New in Optimizer and Executor?MySQL 8.0: What Is New in Optimizer and Executor?
MySQL 8.0: What Is New in Optimizer and Executor?
 
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
 
Developers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman OracleDevelopers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman Oracle
 
What's New MySQL 8.0?
What's New MySQL 8.0?What's New MySQL 8.0?
What's New MySQL 8.0?
 
MySQL Optimizer Overview
MySQL Optimizer OverviewMySQL Optimizer Overview
MySQL Optimizer Overview
 
Inno db 5_7_features
Inno db 5_7_featuresInno db 5_7_features
Inno db 5_7_features
 
ClickHouse Query Performance Tips and Tricks, by Robert Hodges, Altinity CEO
ClickHouse Query Performance Tips and Tricks, by Robert Hodges, Altinity CEOClickHouse Query Performance Tips and Tricks, by Robert Hodges, Altinity CEO
ClickHouse Query Performance Tips and Tricks, by Robert Hodges, Altinity CEO
 
MySQL8.0 in COSCUP2017
MySQL8.0 in COSCUP2017MySQL8.0 in COSCUP2017
MySQL8.0 in COSCUP2017
 
15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performance15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performance
 
Prog1 chap1 and chap 2
Prog1 chap1 and chap 2Prog1 chap1 and chap 2
Prog1 chap1 and chap 2
 
Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007Myth busters - performance tuning 101 2007
Myth busters - performance tuning 101 2007
 
Melbourne Groundbreakers Tour - Upgrading without risk
Melbourne Groundbreakers Tour - Upgrading without riskMelbourne Groundbreakers Tour - Upgrading without risk
Melbourne Groundbreakers Tour - Upgrading without risk
 
Sangam 18 - The New Optimizer in Oracle 12c
Sangam 18 - The New Optimizer in Oracle 12cSangam 18 - The New Optimizer in Oracle 12c
Sangam 18 - The New Optimizer in Oracle 12c
 
JDBC Next: A New Asynchronous API for Connecting to a Database
JDBC Next: A New Asynchronous API for Connecting to a Database JDBC Next: A New Asynchronous API for Connecting to a Database
JDBC Next: A New Asynchronous API for Connecting to a Database
 
Wellington APAC Groundbreakers tour - Upgrading to the 12c Optimizer
Wellington APAC Groundbreakers tour - Upgrading to the 12c OptimizerWellington APAC Groundbreakers tour - Upgrading to the 12c Optimizer
Wellington APAC Groundbreakers tour - Upgrading to the 12c Optimizer
 
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
 

Recently uploaded

CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 

Recently uploaded (20)

The Guide to Integrating Generative AI into Unified Continuous Testing Platfo...
The Guide to Integrating Generative AI into Unified Continuous Testing Platfo...The Guide to Integrating Generative AI into Unified Continuous Testing Platfo...
The Guide to Integrating Generative AI into Unified Continuous Testing Platfo...
 
BUS PASS MANGEMENT SYSTEM USING PHP.pptx
BUS PASS MANGEMENT SYSTEM USING PHP.pptxBUS PASS MANGEMENT SYSTEM USING PHP.pptx
BUS PASS MANGEMENT SYSTEM USING PHP.pptx
 
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
ManageIQ - Sprint 236 Review - Slide Deck
ManageIQ - Sprint 236 Review - Slide DeckManageIQ - Sprint 236 Review - Slide Deck
ManageIQ - Sprint 236 Review - Slide Deck
 
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comHR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
 
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
 
LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456
LEVEL 5   - SESSION 1 2023 (1).pptx - PDF 123456LEVEL 5   - SESSION 1 2023 (1).pptx - PDF 123456
LEVEL 5 - SESSION 1 2023 (1).pptx - PDF 123456
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
 
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
 
Pharm-D Biostatistics and Research methodology
Pharm-D Biostatistics and Research methodologyPharm-D Biostatistics and Research methodology
Pharm-D Biostatistics and Research methodology
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
 
Azure_Native_Qumulo_High_Performance_Compute_Benchmarks.pdf
Azure_Native_Qumulo_High_Performance_Compute_Benchmarks.pdfAzure_Native_Qumulo_High_Performance_Compute_Benchmarks.pdf
Azure_Native_Qumulo_High_Performance_Compute_Benchmarks.pdf
 
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdfThe Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
 

MySQL Optimizer: What’s New in 8.0

  • 1.
  • 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  • 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Optimizer: What’s New in 8.0 Manyi Lu Director Øystein Grøvlen Senior Principal Software Engineer MySQL Optimizer Team, Oracle October, 2017
  • 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 4  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 5  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Common Table Expression • A derived table is a subquery in the FROM clause SELECT … FROM (subquery) AS derived, t1 ... • Common Table Expression (CTE) is just like a derived table, but its declaration is put before the query block instead of in FROM clause WITH derived AS (subquery) SELECT … FROM derived, t1 ... • A CTE may precede SELECT/UPDATE/DELETE including sub-queries WITH derived AS (subquery) DELETE FROM t1 WHERE t1.a IN (SELECT b FROM derived); 6 Alternative to derived table
  • 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Common Table Expression vs Derived Table Better readability Can be referenced multiple times Can refer to other CTEs Improved performance 7
  • 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Better Readability • Derived table: SELECT … FROM t1 LEFT JOIN ((SELECT … FROM …) AS dt JOIN t2 ON …) ON … • CTE: WITH dt AS (SELECT ... FROM ...) SELECT ... FROM t1 LEFT JOIN (dt JOIN t2 ON ...) ON ... 8
  • 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Can Be Referenced Multiple Times • Derived table can not be referenced twice: SELECT ... FROM (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b) AS d1 JOIN (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b) AS d2 ON d1.b = d2.a; • CTE can: WITH d AS (SELECT a, b, SUM(c) s FROM t1 GROUP BY a, b) SELECT ... FROM d AS d1 JOIN d AS d2 ON d1.b = d2.a; • Better performance with materialization: – Multiple references only materialized once – Derived tables and views will be materialized once per reference. 9
  • 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Can Refer to Other CTEs • Derived tables can not refer to other derived tables: SELECT … FROM (SELECT … FROM …) AS d1, (SELECT … FROM d1 …) AS d2 … ERROR: 1146 (42S02): Table ‘db.d1’ doesn’t exist • CTEs can refer other CTEs: WITH d1 AS (SELECT … FROM …), d2 AS (SELECT … FROM d1 …) SELECT FROM d1, d2 … 10
  • 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Recursive CTE • A recursive CTE refers to itself in a subquery • The “seed” SELECT is executed once to create the initial data subset, the recursive SELECT is repeatedly executed to return subsets of. • Recursion stops when an iteration does not generate any new rows – To limit recursion, set cte_max_recursion_depth • Useful to dig in hierarchies (parent/child, part/subpart) 11 WITH RECURSIVE cte AS ( SELECT ... FROM table_name /* "seed" SELECT */ UNION [DISTINCT|ALL] SELECT ... FROM cte, table_name ) /* "recursive" SELECT */ SELECT ... FROM cte;
  • 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Recursive CTE 12 A simple example Print 1 to 10 : WITH RECURSIVE qn AS ( SELECT 1 AS a UNION ALL SELECT 1+a FROM qn WHERE a<10 ) SELECT * FROM qn; a 1 2 3 4 5 6 7 8 9 10
  • 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hierarchy Traversal 13 Employee database CREATE TABLE employees ( id INT PRIMARY KEY, name VARCHAR(100), manager_id INT, FOREIGN KEY (manager_id) REFERENCES employees(id) ); INSERT INTO employees VALUES (333, "Yasmina", NULL), # CEO (198, "John", 333), # John reports to 333 (692, "Tarek", 333), (29, "Pedro", 198), (4610, "Sarah", 29), (72, "Pierre", 29), (123, "Adil", 692);
  • 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hierarchy Traversal 14 List reporting chain WITH RECURSIVE emp_ext (id, name, path) AS ( SELECT id, name, CAST(id AS CHAR(200)) FROM employees WHERE manager_id IS NULL UNION ALL SELECT s.id, s.name, CONCAT(m.path, ",", s.id) FROM emp_ext m JOIN employees s ON m.id=s.manager_id ) SELECT * FROM emp_ext ORDER BY path; id name path 333 Yasmina 333 198 John 333,198 692 Tarek 333,692 29 Pedro 333,198,29 123 Adil 333,692,123 4610 Sarah 333,198,29,4610 72 Pierre 333,198,29,72
  • 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hierarchy Traversal 15 List reporting chain WITH RECURSIVE emp_ext (id, name, path) AS ( SELECT id, name, CAST(id AS CHAR(200)) FROM employees WHERE manager_id IS NULL UNION ALL SELECT s.id, s.name, CONCAT(m.path, ",", s.id) FROM emp_ext m JOIN employees s ON m.id=s.manager_id ) SELECT * FROM emp_ext ORDER BY path; id name path 333 Yasmina 333 198 John 333,198 29 Pedro 333,198,29 4610 Sarah 333,198,29,4610 72 Pierre 333,198,29,72 692 Tarek 333,692 123 Adil 333,692,123
  • 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 16  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Window Functions: What Are They? • A window function performs a calculation across a set of rows that are related to the current row, similar to an aggregate function. • But unlike aggregate functions, a window function does not cause rows to become grouped into a single output row. • Window functions can access values of other rows “in the vicinity” of the current row 17 Aggregate function Window function
  • 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Window Function Example 18 PARTITION == disjoint set of rows in result set name dept_id salary dept_total Newt NULL 75000 75000 Dag 10 NULL 370000 Ed 10 100000 370000 Fred 10 60000 370000 Jon 10 60000 370000 Michael 10 70000 370000 Newt 10 80000 370000 Lebedev 20 65000 130000 Pete 20 65000 130000 Jeff 30 300000 370000 Will 30 70000 370000 Sum up total salary for each department: SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_total FROM employee ORDER BY dept_id, name; The OVER keyword signals a window function
  • 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | name dept_id salary total Newt NULL 75000 75000 Dag 10 NULL NULL Ed 10 100000 100000 Fred 10 60000 160000 Jon 10 60000 220000 Michael 10 70000 190000 Newt 10 80000 210000 Lebedev 20 65000 65000 Pete 20 65000 130000 Jeff 30 300000 300000 Will 30 70000 370000 19 Window Function Example: Frames ORDER BY name within each partition moving window frame: SUM(salary) ... ROWS 2 PRECEDING A frame is a subset of a partition SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id ORDER BY name ROWS 2 PRECEDING) total FROM employee ORDER BY dept_id, name;
  • 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20 Window Function Example: Frames SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id ORDER BY name ROWS 2 PRECEDING) total FROM employee ORDER BY dept_id, name; name dept_id salary total Newt NULL 75000 75000 Dag 10 NULL NULL Ed 10 100000 100000 Fred 10 60000 160000 Jon 10 60000 220000 Michael 10 70000 190000 Newt 10 80000 210000 Lebedev 20 65000 65000 Pete 20 65000 130000 Jeff 30 300000 300000 Will 30 70000 370000
  • 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 21  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Improved UTF-8 Support in MySQL 8.0 • Support for the latest Unicode 9.0 • utf8mb4 made default character set! –utf8mb4_0900_ai_ci default collation • Accent and case sensitive collations –Including 20+ language specific collations –Now also Japanese and Russian • Significantly improved performance 22
  • 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | What Is in MySQL 5.7 and Earlier Versions? • Default charset is “latin1” and default collation is “latin1_swedish_ci” • utf8 = utf8mb3: support BMP only • utf8mb4 character set: –Only accent and case insensitive collations –Default collation is utf8mb4_general_ci, compares all characters beyond BMP, e.g. emojis, to be equal –20+ language specific collations –Recommend to use: utf8mb4_unicode_520_ci 23
  • 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | New Default Character Set • No changes to existing tables • Only has effect on new tables/schemas where character set is not explicitly defined. • Separating character set/collation change from server upgrade – Upgrade first, change charset/collation afterwards • Recommend users to not mixing collations – Error “Illegal mix of collations” – Slower query because index can no longer be used 24
  • 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 25  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | GIS • Geography Support – longitude/latitude • Spatial Reference Systems (SRS) Support – SRID 4326 = WGS 84 (“GPS coordinates”) • Information Schema views – ST_GEOMETRY_COLUMNS • Standard compliant axis ordering : longitude-latitude • Example functions : – ST_Distance( g1, g2) – ST_SwapXY(g) – ST_SRID(g [, new_srid_val]) 26
  • 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 27  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | SELECT ... FOR UPDATE SKIP LOCKED • Common problem: – Hot row contention, multiple worker threads accessing the same rows • Solution 1: – Only read rows that are not locked – InnoDB skips a locked row, and the next one goes to the result set • Example: – Booking system: Skip orders that are pending 28 START TRANSACTION; SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO' FOR UPDATE SKIP LOCKED;
  • 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | SELECT… FOR UPDATE NOWAIT • Common problem: – Hot row contention, multiple worker threads accessing the same rows • Solution 2: – If any of the rows are already locked, the statement should fail immediately – Without NOWAIT, have to wait for innodb_lock_wait_timeout (default: 50 sec) while trying to acquire lock • Usage: 29 START TRANSACTION; SELECT * FROM seats WHERE seat_no BETWEEN 2 AND 3 AND booked = 'NO' FOR UPDATE NOWAIT; ERROR 3572 (HY000): Statement aborted because lock(s) could not be acquired …
  • 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 30  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | JSON Aggregation 31 Combine JSON documents in multiple rows into a JSON array CREATE TABLE t1 (id INT, grp INT, jsoncol JSON); INSERT INTO t1 VALUES (1, 1, '{"key1":"value1","key2":"value2"}'); INSERT INTO t1 VALUES (2, 1, '{"keyA":"valueA","keyB":"valueB"}'); INSERT INTO t1 VALUES (3, 2, '{"keyX":"valueX","keyY":"valueY"}'); SELECT JSON_ARRAYAGG(jsoncol) FROM t1; [{"key1":"value1","key2":"value2"}, {"keyA":"valueA","keyB":"valueB"}, {"keyX":"valueX","keyY":"valueY"}]
  • 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | JSON Aggregation 32 Combine JSON documents in multiple rows into a JSON object CREATE TABLE t1 (id INT, grp INT, jsoncol JSON); INSERT INTO t1 VALUES (1, 1, '{"key1":"value1","key2":"value2"}'); INSERT INTO t1 VALUES (2, 1, '{"keyA":"valueA","keyB":"valueB"}'); INSERT INTO t1 VALUES (3, 2, '{"keyX":"valueX","keyY":"valueY"}'); SELECT grp, JSON_OBJECTAGG(id, jsoncol) FROM t1 GROUP BY grp; 1 | {"1":{"key1":"value1","key2":"value2"}, "2":{"keyA":"valueA","keyB":"valueB"}} 2 | {"3":{"keyX":"valueX","keyY":"valueY"}}
  • 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | JSON_TABLE • Table t1 has a column json_col with content like this: { "people": [ { "name":"John Smith", "address":"780 Mission St, San Francisco, CA 94103"}, { "name":"Sally Brown", "address":"75 37th Ave S, St Cloud, MN 94103"}, { "name":"Paul Johnson", "address":"1262 Roosevelt Trail, Raymond, ME 04071"}, … ] } • Convert JSON column into a table with 2 columns: SELECT people.* FROM t1, JSON_TABLE(json_col, '$.people[*]' COLUMNS ( name VARCHAR(40) PATH '$.name', address VARCHAR(100) PATH '$.address')) people WHERE people.address LIKE '%San Francisco%'; 33 Convert JSON documents to relational tables Labs release
  • 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 34 JSON_TABLE – Nested Arrays id father married child_id child age 1 John 1 1 Eric 12 1 John 1 2 Beth 10 2 Paul 0 1 Sarah 9 2 Paul 0 2 Noah 3 2 Paul 0 3 Peter 1 [ { "father":"John", "mother":"Mary", "marriage_date":"2003-12-05", "children": [ { "name":"Eric", "age":12 }, { "name":"Beth", "age":10 } ] }, { "father":"Paul", "mother":"Laura", "children": [ { "name":"Sarah", "age":9}, { "name":"Noah", "age":3} , { "name":"Peter", "age":1} ] } ] Labs release
  • 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 35 JSON_TABLE – Nested Arrays id father married child_id child age 1 John 1 1 Eric 12 1 John 1 2 Beth 10 2 Paul 0 1 Sarah 9 2 Paul 0 2 Noah 3 2 Paul 0 3 Peter 1 JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', married INTEGER EXISTS PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age‘ ) ) Labs release
  • 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | JSON_TABLE SELECT father, COUNT(*) "#children", AVG(age) "age average" FROM t, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', NESTED PATH '$.children[*]' COLUMNS (age INTEGER PATH '$.age‘ ) ) AS fam GROUP BY id, father; 36 SQL aggregation on JSON data father #children age average John 2 11.0000 Paul 3 4.3333 Labs release
  • 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | JSON Utility Functions • JSON_PRETTY(json_value) – Pretty-print the JSON value • JSON_STORAGE_SIZE(json_value) – The number of bytes used to store the binary representation of a JSON document • JSON_STORAGE_FREE(json_value) – The number of bytes in its binary representation that is current not used. – The binary representation may have unused space after a JSON column was updated in place using JSON_SET() 37
  • 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 38  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Invisible Index • Index is maintained by the SE, but ignored by the Optimizer • Primary key cannot be INVISIBLE • Use case: Check for performance drop BEFORE dropping an index • To see an invisible index: set optimizer_switch='use_invisible_indexes=on'; 39 ALTER TABLE t1 ALTER INDEX idx INVISIBLE; mysql> SHOW INDEXES FROM t1; +---------+------------------+----------------------+---------------+ | Table | Key_name | Column_name | Visible | +---------+------------------+----------------------+---------------+ | t1 | idx | a | NO | +---------+------------------+----------------------+---------------+
  • 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Descending Index • In 5.7: Index in ascending order is created, server scans it backwards • In 8.0: Index in descending order is created, server scans it forwards • Works on B-tree indexes only • Benefits: – Use indexes instead of filesort for ORDER BY clause with ASC/DESC sort key – Forward index scan is slightly faster than backward index scan 40 CREATE TABLE t1 ( a INT, b INT, INDEX a_b (a DESC, b ASC) );
  • 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 41  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 42. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Motivation for Improving the MySQL Cost Model • Produce more correct cost estimates – Better decisions by the optimizer should improve performance • Adapt to new hardware architectures – SSD, larger memories, caches • More maintainable cost model implementation – Avoid hard coded “cost constants” – Refactoring of existing cost model code • Configurable and tunable • Make more of the optimizer cost-based Faster queries 42
  • 43. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | • Time to do a table scan of 10 million records: • Adjust cost model to support different storage technologies • Provide configurable cost constants for different storage technologies New Storage Technologies Memory 5 s SSD 20 - 146 s Hard disk 32 - 1465 s Provide a program that could measure performance and suggest good cost constant configuration for a running MySQL server? 43
  • 44. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | • Storage engines: – Estimate for how much of data and indexes are in a memory buffer – Estimate for hit rate for memory buffer • Optimizer cost model: – Take into account whether data is already in memory or need to be read from disk Memory Buffer Aware Cost Estimates Server Storage engine Disk data Query executor Database buffer 44
  • 45. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL 8.0: Disk vs Memory Access • New defaults for const constants: • InnoDB reports for each table/index percentage of data cached in buffer pool • Note: Query plan may change between executions Confidential – Oracle Internal/Restricted/Highly Restricted 45 Cost MySQL 5.7 MySQL 8.0 Read a random disk page 1.0 1.0 Read a data page from memory buffer 1.0 0.25 Evaluate query condition 0.2 0.1 Compare keys/rows 0.1 0.05
  • 46. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | DBT-3 Query 8 SELECT o_year, SUM(CASE WHEN nation = 'FRANCE' THEN volume ELSE 0 END) / SUM(volume) AS mkt_share FROM ( SELECT EXTRACT(YEAR FROM o_orderdate) AS o_year, l_extendedprice * (1 - l_discount) AS volume, n2.n_name AS nation FROM part JOIN lineitem ON p_partkey = l_partkey JOIN supplier ON s_suppkey = l_suppkey JOIN orders ON l_orderkey = o_orderkey JOIN customer ON o_custkey = c_custkey JOIN nation n1 ON c_nationkey = n1.n_nationkey JOIN region ON n1.n_regionkey = r_regionkey JOIN nation n2 ON s_nationkey = n2.n_nationkey WHERE r_name = 'EUROPE' AND o_orderdate BETWEEN '1995-01-01' AND '1996-12-31' AND p_type = 'PROMO BRUSHED STEEL' ) AS all_nations GROUP BY o_year ORDER BY o_year; National Market Share Query High selectivity
  • 47. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 47 Alternative query plans Plan A Plan B DBT-3 Query 8
  • 48. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Execution time (MySQL 8.0.3) In-memory Disk-bound Plan A 5.8 secs 9 min 47 secs Plan B 77.5 secs 3 min 49 secs Selected plan In-memory Disk-bound MySQL 5.6 Plan B MySQL 5.7 Plan A MySQL 8.0 Plan A Plan B 48 DBT-3 Query 8 DBT-3 Scale factor 10 In-Memory: innodb_buffer_pool_size = 32 GB Disk-bound: innodb_buffer_pool_size = 1 GB
  • 49. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Histograms • Provides the optimizer with information about column value distribution • To create/recalculate histogram for a column: ANALYZE TABLE table UPDATE HISTOGRAM ON column WITH n BUCKETS; • May use sampling – Sample size is based on available memory (histogram_generation_max_mem_size) • Automatically chooses between two histogram types: – Singleton: One value per bucket – Equi-height: Multiple value per bucket 49 Column statistics
  • 50. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Histograms EXPLAIN SELECT * FROM customer JOIN orders ON c_custkey = o_custkey WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01'; 50 Example query id select type table type possible keys key key len ref rows filtered extra 1 SIMPLE orders ALL i_o_orderdate, i_o_custkey NULL NULL NULL 15000000 31.19 Using where 1 SIMPLE customer eq_ ref PRIMARY PRIMARY 4 dbt3.orders. o_custkey 1 33.33 Using where
  • 51. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Histograms ANALYZE TABLE customer UPDATE HISTOGRAM ON c_acctbal WITH 1024 buckets; EXPLAIN SELECT * FROM customer JOIN orders ON c_custkey = o_custkey WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01'; 51 Create histogram to get a better plan id select type table type possible keys key key len ref rows filtered extra 1 SIMPLE customer ALL PRIMARY NULL NULL NULL 1500000 0.00 Using where 1 SIMPLE orders ref i_o_orderdate, i_o_custkey i_o_custkey 5 dbt3. customer. c_custkey 15 31.19 Using where
  • 52. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Comparing Join Order 0 2 4 6 8 10 12 14 16 QueryExecutionTime(seconds) orders → customer customer → orders Performance
  • 53. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 53  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 54. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hints: Join Order • Hints to control table order for join execution • 5.7: STRAIGHT_JOIN to force the listed order in FROM clause • 8.0: – JOIN_FIXED_ORDER /* replacement for STRAIGHT_JOIN*/ – JOIN_ORDER /* use specified order */ – JOIN_PREFIX /* use specified order for first tables */ – JOIN_SUFFIX /* use specified order for last tables */ – No need to reorganize the FROM clause to add join order hints like you will for STRAIGHT_JOIN 54
  • 55. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Join Order Hints - Example EXPLAIN SELECT /*+ JOIN_ORDER(customer, orders) */ * FROM customer JOIN orders ON c_custkey = o_custkey WHERE c_acctbal < -1000 AND o_orderdate < '1993-01-01'; 55 Change join order with hint id select type table type possible keys key key len ref rows filtered extra 1 SIMPLE customer ALL PRIMARY NULL NULL NULL 1500000 33.33 Using where 1 SIMPLE orders ref i_o_orderdate, i_o_custkey i_o_custkey 5 dbt3. customer. c_custkey 15 31.19 Using where Alternatives with same effect for this query: JOIN_PREFIX(customer) JOIN_SUFFIX(orders) JOIN_FIXED_ORDER()
  • 56. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hints: Index Merge • Index merge: Merge rows from multiple range scans on a single table • Algorithms: union, intersection, sort union • Users can specify which indexes to use for index merge – /*+ INDEX_MERGE() */ – /*+ NO_INDEX_MERGE() */ 56 10INDEX(a) 10INDEX(b) a=10 AND b=10Result: Intersection SELECT * FROM t1 WHERE a=10 AND b=10
  • 57. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Index Merge Hint - Example EXPLAIN SELECT count(*) FROM users WHERE user_type=2 AND status=0 AND parent_id=0; 57 id select type table type possible keys key key len ref rows Extra 1 SIMPLE users index_ merge parent_id, status, user_type user_type, status, parent_id 1,1,4 NULL 2511 Using intersect (user_type, status, parent_id); Using where; Using index mysql> SELECT count(*) FROM users WHERE user_type=2 AND status=0 AND parent_id=0; ... 1 row in set (1.37 sec) mysql> SELECT /*+ INDEX_MERGE(users user_type, status) */ count(*) -> FROM users WHERE user_type=2 AND status=0 AND parent_id=0; ... 1 row in set (0.18 sec) Low selectivity
  • 58. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Hints: Set session variables • Set a session variable for the duration of a single statement • Examples: SELECT /* SET_VAR(sort_buffer_size = 16M) */ name FROM people ORDER BY name; INSERT /* SET_VAR(foreign_key_checks = OFF) */ INTO t2 VALUES (1, 1), (2, 2), (3, 3); SELECT /* SET_VAR(optimizer_switch = 'condition_fanout_filter = off') */ * FROM customer JOIN orders ON c_custkey = o_custkey WHERE c_acctbal < 0 AND o_orderdate < '1993-01-01'; • NB! Not all session variables are settable through hints: mysql> SELECT /*+ SET_VAR(max_allowed_packet=128M) */ * FROM t1; Empty set, 1 warning (0,01 sec) Warning (Code 4537): Variable 'max_allowed_packet' cannot be set using SET_VAR hint. 58
  • 59. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 59  Common table expressions  Window functions  UTF8 support  GIS  SKIP LOCKED, NOWAIT  JSON functions  Index extensions  Cost model  Hints  Better IPv6 and UUID support
  • 60. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Improved Support for UUID • Five “-” separated hexadecimal numbers • MySQL uses version 1, the first three numbers are generated from the low, middle, and high parts of a timestamp. • 36 characters, inefficient for storage Convert to BINARY(16) datatype, only 16 bytes 60 mysql> select uuid(); +---------------------------------------------------------+ | uuid() | +---------------------------------------------------------+ | aab5d5fd-70c1-11e5-a4fb-b026b977eb28 | +---------------------------------------------------------+
  • 61. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Improved Support for UUID • UUID_TO_BIN(string_uuid, swap_flag) • BIN_TO_UUID(binary_uuid, swap_flag) • IS_UUID(string_uuid) 61 Functions to convert UUID to and from binary Feature Request from Developers
  • 62. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | UUID_TO_BIN Optimization • Binary format is now smaller • Shuffling low part with the high part improves index performance 62 25 26 27 28 29 Insert Performance Optimized Original 11e678fe53303f87a4778c89a52c4f3b 53303f87-78fe-11e6-a477-8c89a52c4f3bFrom VARCHAR(36) To BINARY(16)
  • 63. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | IPv6 • New! Bit-wise operations on binary data types – Designed with IPv6 in mind: – INET6_ATON(address) & INET6_ATON(network) – No longer truncation beyond 64 bits 63 Feature Request from Developers
  • 64. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | • Improved Query Scan Performance • GROUPING() • Hint: Merge/materialize derived table/view • JSON: – Partial update – Improved performance of sorting and grouping of JSON values – Path expressions: Array index ranges – JSON_MERGE_PATCH() • Skip index dives for FORCE INDEX • Parser Refactoring 64 All These Features and More… Try it out! Give us feedback!
  • 65. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 65
  • 66. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 66