SlideShare a Scribd company logo
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0:
An Internal Server ComponentThat Matters
Ståle Deraas, Senior Development Manager
Oracle, MySQL
27 Apr 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
What is a Data Dictionary?
Data Dictionary before MySQL 8.0
Transactional Data Dictionary in MySQL 8.0
What’s in it for you?
1
2
3
4
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is a Data Dictionary?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
• Metadata is information about data in an RDBMS
– Column definitions, Index definitions, Foreign key definitions
...
• Data Dictionary is a collection of metadata for all data in an RDBMS
ID NAME WEIGHT HEIGHT GENDER
3 Bob 80 185 M
5 Liz 55 165 F
Metadata
Data
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Server
Data Dictionary
6
Role in Server
Query Executor
Optimizer
Storage
Engine
SQL
statement
Client
Parser
Result Data
Dictionary
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
CREATE PROCEDURE p1(v INT)
SQL SECURITY INVOKER
BEGIN
...
END
Data Dictionary
Types of Metadata
Data Dictionary
Table Definitions SP Definitions
View Definitions Schemas
ACL
CREATE TABLE customers(
id INT AUTO_INCREMENT
...
PRIMARY KEY (id),
INDEX ...
FOREIGN KEY ...
)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary before MySQL 8.0
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
• Metadata stored in a mix of files and tables :
– File based
• FRM, TRN, TRG, OPT, PAR
...
– Table based: non-transactional
• mysql.proc
...
– Table based: transactional
• Innodb.SYS
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
10
Data Dictionary
Files
FRM TRG OPT
System Tables (mysql.)
user procevents
InnoDB System Tables
MyISAM
File system
InnoDB
SQL
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
• INFORMATION_SCHEMA queries are slow
• Inconsistencies due to non-transactional storage of metadata
• Inconsistencies between InnoDB metadata and Server metadata
• Showstopper for crash-safe / transactional DDL
• Replication is challenging, as DDLs are not atomic
• Difficult to extend
• No uniform API
11
Problems
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary
MySQL 8.0
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
13
Data Dictionary
InnoDB
SQL
DD TableDD TableDD Table
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
• All metadata stored in tables
• Single metadata repository for all MySQL server subsystems
• Reliable, crash-safe InnoDB tables
• INFORMATION_SCHEMA implemented as views over DD tables
– Queries can be optimized
– Improved performance
– Simpler, uniform implementation, easier to maintain
Main features
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
15
Overview
InnoDB
Data Dictionary
DD Table User Table
INFORMATION
SCHEMA
Views
Archive
User Table
CSV
User Table
Optimizer
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
• The data dictionary schema is based on SQL standard definitions
• Designed for automated upgrade of metadata
• The data dictionary is designed to be easily extended for new requirements
• Designed to allow plugins/components to add INFORMATION_SCHEMA
views and PERFORMANCE_SCHEMA tables
Main features, cont’ed
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
InnoDB
Transactional Data Dictionary in MySQL 8.0
17
Architecture
Query Executor
Parser Optimizer
Data
Dictionary
Tablespace
Data Dictionary API
Storage Engine
PluginUser TableUser Table
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary API in MySQL 8.0
• A single way to deal with Data Dictionary
– For the server core
– For Storage Engines
• A uniform API for all types of metadata
• Provide a way to handle Storage Engine private data
– Storage Engines no longer have to store their own metadata
– The API enables SEs to store key-value pairs
18
Design goals
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
SDI: Serialized Dictionary Information
- Copy of metadata from the Data Dictionary in JSON format
- Stored in data tablespaces(InnoDB) or .sdi file (MyISAM)
- Used for data migration and redundancy
The InnoDB Data Dictionary tablespace is the metadata storage
19
Reliability and Redundancy
InnoDB User Tablespaces
User Table
InnoDB Data Dictionary tablespace
mysql.tables
SDI
ID Name
1 User Table 1
2 User Table 2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
What’s in it for you?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary Benefits
• INFORMATION SCHEMA
– Improved Performance and Scalability
– Solves longstanding issues
• Reliability
– Atomic DDL
– Serialized Dictionary Information
– Automated Data Dictionary upgrade
• +++
21
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22
MySQLClient
I_S Query Results
MySQL Server
Create temporary table.
Heuristic optimization
Read metadata from File system or
from MyISAM/InnoDB engine.
.
TEMP TABLE
Return rows to user. File system / MyISAM / InnoDB engine
INFORMATION SCHEMA MySQL 5.7: Multiple sources, heuristics, creation of temp tables
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23
MySQL Client
I_S Query Results
MySQL Server
Optimizer prepares
execution plan
Executor reads metadata from
data dictionary tables
InnoDB storage engine
Return rows to user
INFORMATION SCHEMA MySQL 8.0: Uniform, simple and using server standard features
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA Performance
Confidential – Oracle Internal/Restricted/Highly Restricted 24
Uniform, simpler implemention makes it a lot faster
MySQL Client
I_S Query Results
MySQL Server
Optimizer prepares
execution plan.
Executor reads metadata from
data dictionary tables.
InnoDB storage engine
Return rows to user.
INFORMATION_SCHEMA in 8.0
MySQL Client
I_S Query Results
MySQL Server
Create temporary table.
Heuristic optimization.
Read metadata from File system or
from MyISAM/InnoDB engine.
.
TEMP TABLE
Return rows to user.
INFORMATION_SCHEMA in 5.7
File system / MyISAM
/ InnoDB engine
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA Performance and Scalability
• Typically 30X performance improvements over MySQL 5.7
• More than 100X for some queries like: List all InnoDB table columns
25
I_S queries scale, both with database size and query load
0 20 40 60 80 100 120 140 160
List all InnoDB tables columns 5k tables
List all InnoDB tables columns 10k tables
MySQL 8.0
MySQL 5.7
Time in Seconds (Lower is better)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 26
100 schemas times 50 tables (5000 tables)
Information Schema Performance
0 0.5 1 1.5 2 2.5 3 3.5 4
Count All Schemas
Schema aggregate size stats
All Dynamic Table Info
All Static Table Info
Auto Increments Near Limit
Count All Columns
Count All Indexes
MySQL 8.0
MySQL 5.7
Time in Seconds (Lower is better)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA in MySQL 8.0: Dynamic values
• TABLES.TABLE_ROWS, TABLES.DATA_FREE, …
– Handled by opening table and retrieving data from SE in MySQL 5.7 – expensive!!!
• Two approaches in MySQL 8.0
– information_schema_stats=cached
• Default
• Stored in auxiliary tables
• ANALYZE TABLE
– information_schema_stats=latest
• Native functions in the I_S views
• Uses special SE API (InnoDB) or falls back to opening tables (MyISAM)
27
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA longstanding issues solved in 8.0
– Bug#34921 comparison with information schema tables don’t honor collation
– Bug#48445 Inconsistency with SHOW and SELECT FROM I_S
– Bug#61846 SHOW FULL COLUMNS displays incorrect privileges for table
– Bug#65121 Inconsistent result for select on INFORMATION_SCHEMA.STATISTICS
– Bug#75532 Join between I_S schema tables is case insensitive/returns wrong value
– Bug#81347 unnecessary scanned all databases for information schema
– .....
28
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Atomic DDL
• All metadata stored in InnoDB (transactional SE)
• DDL code changed to avoid intermediate commits
– At server level
– At InnoDB level
– At replication level
29
Transactional behavior
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Atomic DDL : Use Case - DROP DATABASE
30
MySQL 5.7
• Delete tables
– Metadata, TRN/TRG/FRM files
– Data, InnoDB tables
• Delete stored programs
– Metadata, rows in MyISAM (non-
transactional)
• Delete schema
– Metadata, DB.OPT file
Mix of filesystem, non-
transactional/transactional
storage and multiple commits
MySQL 8.0
• Delete tables
– Metadata, rows in InnoDB
– Data, InnoDB tables
• Delete stored programs
– Metadata, rows in InnoDB
• Delete schema
– Metadata, rows in InnoDB
Updates to transactional storage,
one commit
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Disaster Recovery
• MySQL 8.0, metadata and data stored in InnoDB (transactional SE)
– Metadata copy stored in SDI embedded in .ibd
– Data stored in .ibd
• MySQL before 8.0, metadata in files, data in InnoDB
– Metadata stored in .FRM
– Data stored in .ibd
31
Use case – IMPORT using Serialized Dictionary Information, InnoDB
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL before 8.0
• Moving .FRM and data files around (MyISAM)
• Create tables and do import using SQL (InnoDB)
CREATE TABLE t1 (...)
ALTER TABLE t1 DISCARD TABLESPACE
ALTER TABLE t1 IMPORT TABLESPACE ...
• Easy to do mistakes
MySQL 8.0
• Self-descriptive tablespaces (SDI)
• New IMPORT statement will import
the tablespace:
– Read metadata from the SDI, and
create the tables
– Import the data into these tables
• Worst case, if SDI is corrupt, you
can extract it (JSON format) and
edit
32
Reliability – Disaster Recovery
• Use case – IMPORT using Serialized Dictionary Information
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability and Ease of Use – Automated Dictionary Upgrade
• The Data Dictionary is versioned
• The MySQL Server will support upgrading dictionary tables automatically
– Done directly by the executable and not in a script
– Reduces user and privilege issues for the process doing the upgrade
• The Data Dictionary will be upgraded atomically
33
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Extending Dictionary Tables from Plugins
• API for plugins to extend INFORMATION_SCHEMA &
PERFORMANCE_SCHEMA
– Add new virtual tables/views
• API for plugins to store their specific data in the Data Dictionary
34
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Try for yourself!
• Downloadable 8.0.1 DMR
– dev.mysql.com
• A «hot» labs release right off the press: 8.0.1 + Unified Data Dictionary
• Enjoy and give us your feedback!
• Thank you for listening
• http://mysqlserverteam.com
35

More Related Content

What's hot

MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바
NeoClova
 
How to upgrade like a boss to my sql 8.0?
How to upgrade like a boss to my sql 8.0?How to upgrade like a boss to my sql 8.0?
How to upgrade like a boss to my sql 8.0?
Alkin Tezuysal
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0
Mayank Prasad
 
How to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleHow to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScale
MariaDB plc
 
Redo log improvements MYSQL 8.0
Redo log improvements MYSQL 8.0Redo log improvements MYSQL 8.0
Redo log improvements MYSQL 8.0
Mydbops
 
Demystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash SafetyDemystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash Safety
Jean-François Gagné
 
Using all of the high availability options in MariaDB
Using all of the high availability options in MariaDBUsing all of the high availability options in MariaDB
Using all of the high availability options in MariaDB
MariaDB plc
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
Sveta Smirnova
 
Parallel Replication in MySQL and MariaDB
Parallel Replication in MySQL and MariaDBParallel Replication in MySQL and MariaDB
Parallel Replication in MySQL and MariaDB
Mydbops
 
[2018] MySQL 이중화 진화기
[2018] MySQL 이중화 진화기[2018] MySQL 이중화 진화기
[2018] MySQL 이중화 진화기
NHN FORWARD
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
Olivier DASINI
 
MySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & OperationsMySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & Operations
Frederic Descamps
 
How to set up orchestrator to manage thousands of MySQL servers
How to set up orchestrator to manage thousands of MySQL serversHow to set up orchestrator to manage thousands of MySQL servers
How to set up orchestrator to manage thousands of MySQL servers
Simon J Mudd
 
ProxySQL for MySQL
ProxySQL for MySQLProxySQL for MySQL
ProxySQL for MySQL
Mydbops
 
MariaDB Optimization
MariaDB OptimizationMariaDB Optimization
MariaDB Optimization
JongJin Lee
 
MySQL GTID 시작하기
MySQL GTID 시작하기MySQL GTID 시작하기
MySQL GTID 시작하기
I Goo Lee
 
MySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete Tutorial
Sveta Smirnova
 
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
Severalnines
 
Introduction to Galera Cluster
Introduction to Galera ClusterIntroduction to Galera Cluster
Introduction to Galera Cluster
Codership Oy - Creators of Galera Cluster
 
Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기
NeoClova
 

What's hot (20)

MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바
 
How to upgrade like a boss to my sql 8.0?
How to upgrade like a boss to my sql 8.0?How to upgrade like a boss to my sql 8.0?
How to upgrade like a boss to my sql 8.0?
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0
 
How to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleHow to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScale
 
Redo log improvements MYSQL 8.0
Redo log improvements MYSQL 8.0Redo log improvements MYSQL 8.0
Redo log improvements MYSQL 8.0
 
Demystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash SafetyDemystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash Safety
 
Using all of the high availability options in MariaDB
Using all of the high availability options in MariaDBUsing all of the high availability options in MariaDB
Using all of the high availability options in MariaDB
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
 
Parallel Replication in MySQL and MariaDB
Parallel Replication in MySQL and MariaDBParallel Replication in MySQL and MariaDB
Parallel Replication in MySQL and MariaDB
 
[2018] MySQL 이중화 진화기
[2018] MySQL 이중화 진화기[2018] MySQL 이중화 진화기
[2018] MySQL 이중화 진화기
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
 
MySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & OperationsMySQL InnoDB Cluster - Advanced Configuration & Operations
MySQL InnoDB Cluster - Advanced Configuration & Operations
 
How to set up orchestrator to manage thousands of MySQL servers
How to set up orchestrator to manage thousands of MySQL serversHow to set up orchestrator to manage thousands of MySQL servers
How to set up orchestrator to manage thousands of MySQL servers
 
ProxySQL for MySQL
ProxySQL for MySQLProxySQL for MySQL
ProxySQL for MySQL
 
MariaDB Optimization
MariaDB OptimizationMariaDB Optimization
MariaDB Optimization
 
MySQL GTID 시작하기
MySQL GTID 시작하기MySQL GTID 시작하기
MySQL GTID 시작하기
 
MySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete Tutorial
 
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
 
Introduction to Galera Cluster
Introduction to Galera ClusterIntroduction to Galera Cluster
Introduction to Galera Cluster
 
Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기Maria db 이중화구성_고민하기
Maria db 이중화구성_고민하기
 

Similar to Data dictionary pl17

MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshellMySQL 8.0 in a nutshell
MySQL 8.0 in a nutshell
OracleMySQL
 
State ofdolphin short
State ofdolphin shortState ofdolphin short
State ofdolphin short
Mandy Ang
 
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
OracleMySQL
 
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017
Ivan Ma
 
MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?
Olivier DASINI
 
Novinky v Oracle Database 18c
Novinky v Oracle Database 18cNovinky v Oracle Database 18c
Novinky v Oracle Database 18c
MarketingArrowECS_CZ
 
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
OracleMySQL
 
New data dictionary an internal server api that matters
New data dictionary an internal server api that mattersNew data dictionary an internal server api that matters
New data dictionary an internal server api that matters
Alexander Nozdrin
 
MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015
Mario Beck
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
Geir Høydalsvik
 
How to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performanceHow to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performance
oysteing
 
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
Cloud Native Day Tel Aviv
 
What's New in MySQL 5.7
What's New in MySQL 5.7What's New in MySQL 5.7
What's New in MySQL 5.7
Olivier DASINI
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New Features
Tarique Saleem
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp
 
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
Cedar Consulting
 
MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018
Olivier DASINI
 
MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017
MySQL Brasil
 
Unlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQLUnlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQL
Ricky Setyawan
 
MySQL :What's New #GIDS16
MySQL :What's New #GIDS16MySQL :What's New #GIDS16
MySQL :What's New #GIDS16
Sanjay Manwani
 

Similar to Data dictionary pl17 (20)

MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshellMySQL 8.0 in a nutshell
MySQL 8.0 in a nutshell
 
State ofdolphin short
State ofdolphin shortState ofdolphin short
State ofdolphin short
 
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
 
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017
 
MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?
 
Novinky v Oracle Database 18c
Novinky v Oracle Database 18cNovinky v Oracle Database 18c
Novinky v Oracle Database 18c
 
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
 
New data dictionary an internal server api that matters
New data dictionary an internal server api that mattersNew data dictionary an internal server api that matters
New data dictionary an internal server api that matters
 
MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
 
How to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performanceHow to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performance
 
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
 
What's New in MySQL 5.7
What's New in MySQL 5.7What's New in MySQL 5.7
What's New in MySQL 5.7
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New Features
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
 
MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018
 
MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017
 
Unlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQLUnlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQL
 
MySQL :What's New #GIDS16
MySQL :What's New #GIDS16MySQL :What's New #GIDS16
MySQL :What's New #GIDS16
 

Recently uploaded

Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Yara Milbes
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
devvsandy
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
Ayan Halder
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 

Recently uploaded (20)

Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 

Data dictionary pl17

  • 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0: An Internal Server ComponentThat Matters Ståle Deraas, Senior Development Manager Oracle, MySQL 27 Apr 2017 Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
  • 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  • 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Agenda What is a Data Dictionary? Data Dictionary before MySQL 8.0 Transactional Data Dictionary in MySQL 8.0 What’s in it for you? 1 2 3 4
  • 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | What is a Data Dictionary?
  • 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Dictionary • Metadata is information about data in an RDBMS – Column definitions, Index definitions, Foreign key definitions ... • Data Dictionary is a collection of metadata for all data in an RDBMS ID NAME WEIGHT HEIGHT GENDER 3 Bob 80 185 M 5 Liz 55 165 F Metadata Data
  • 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Server Data Dictionary 6 Role in Server Query Executor Optimizer Storage Engine SQL statement Client Parser Result Data Dictionary
  • 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | CREATE PROCEDURE p1(v INT) SQL SECURITY INVOKER BEGIN ... END Data Dictionary Types of Metadata Data Dictionary Table Definitions SP Definitions View Definitions Schemas ACL CREATE TABLE customers( id INT AUTO_INCREMENT ... PRIMARY KEY (id), INDEX ... FOREIGN KEY ... )
  • 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Dictionary before MySQL 8.0
  • 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 • Metadata stored in a mix of files and tables : – File based • FRM, TRN, TRG, OPT, PAR ... – Table based: non-transactional • mysql.proc ... – Table based: transactional • Innodb.SYS
  • 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 10 Data Dictionary Files FRM TRG OPT System Tables (mysql.) user procevents InnoDB System Tables MyISAM File system InnoDB SQL
  • 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 • INFORMATION_SCHEMA queries are slow • Inconsistencies due to non-transactional storage of metadata • Inconsistencies between InnoDB metadata and Server metadata • Showstopper for crash-safe / transactional DDL • Replication is challenging, as DDLs are not atomic • Difficult to extend • No uniform API 11 Problems
  • 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary MySQL 8.0
  • 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 13 Data Dictionary InnoDB SQL DD TableDD TableDD Table
  • 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 • All metadata stored in tables • Single metadata repository for all MySQL server subsystems • Reliable, crash-safe InnoDB tables • INFORMATION_SCHEMA implemented as views over DD tables – Queries can be optimized – Improved performance – Simpler, uniform implementation, easier to maintain Main features
  • 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 15 Overview InnoDB Data Dictionary DD Table User Table INFORMATION SCHEMA Views Archive User Table CSV User Table Optimizer
  • 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 • The data dictionary schema is based on SQL standard definitions • Designed for automated upgrade of metadata • The data dictionary is designed to be easily extended for new requirements • Designed to allow plugins/components to add INFORMATION_SCHEMA views and PERFORMANCE_SCHEMA tables Main features, cont’ed
  • 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Transactional Data Dictionary in MySQL 8.0 17 Architecture Query Executor Parser Optimizer Data Dictionary Tablespace Data Dictionary API Storage Engine PluginUser TableUser Table
  • 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary API in MySQL 8.0 • A single way to deal with Data Dictionary – For the server core – For Storage Engines • A uniform API for all types of metadata • Provide a way to handle Storage Engine private data – Storage Engines no longer have to store their own metadata – The API enables SEs to store key-value pairs 18 Design goals
  • 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 SDI: Serialized Dictionary Information - Copy of metadata from the Data Dictionary in JSON format - Stored in data tablespaces(InnoDB) or .sdi file (MyISAM) - Used for data migration and redundancy The InnoDB Data Dictionary tablespace is the metadata storage 19 Reliability and Redundancy InnoDB User Tablespaces User Table InnoDB Data Dictionary tablespace mysql.tables SDI ID Name 1 User Table 1 2 User Table 2
  • 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 What’s in it for you?
  • 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary Benefits • INFORMATION SCHEMA – Improved Performance and Scalability – Solves longstanding issues • Reliability – Atomic DDL – Serialized Dictionary Information – Automated Data Dictionary upgrade • +++ 21
  • 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22 MySQLClient I_S Query Results MySQL Server Create temporary table. Heuristic optimization Read metadata from File system or from MyISAM/InnoDB engine. . TEMP TABLE Return rows to user. File system / MyISAM / InnoDB engine INFORMATION SCHEMA MySQL 5.7: Multiple sources, heuristics, creation of temp tables
  • 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23 MySQL Client I_S Query Results MySQL Server Optimizer prepares execution plan Executor reads metadata from data dictionary tables InnoDB storage engine Return rows to user INFORMATION SCHEMA MySQL 8.0: Uniform, simple and using server standard features
  • 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA Performance Confidential – Oracle Internal/Restricted/Highly Restricted 24 Uniform, simpler implemention makes it a lot faster MySQL Client I_S Query Results MySQL Server Optimizer prepares execution plan. Executor reads metadata from data dictionary tables. InnoDB storage engine Return rows to user. INFORMATION_SCHEMA in 8.0 MySQL Client I_S Query Results MySQL Server Create temporary table. Heuristic optimization. Read metadata from File system or from MyISAM/InnoDB engine. . TEMP TABLE Return rows to user. INFORMATION_SCHEMA in 5.7 File system / MyISAM / InnoDB engine
  • 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA Performance and Scalability • Typically 30X performance improvements over MySQL 5.7 • More than 100X for some queries like: List all InnoDB table columns 25 I_S queries scale, both with database size and query load 0 20 40 60 80 100 120 140 160 List all InnoDB tables columns 5k tables List all InnoDB tables columns 10k tables MySQL 8.0 MySQL 5.7 Time in Seconds (Lower is better)
  • 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 26 100 schemas times 50 tables (5000 tables) Information Schema Performance 0 0.5 1 1.5 2 2.5 3 3.5 4 Count All Schemas Schema aggregate size stats All Dynamic Table Info All Static Table Info Auto Increments Near Limit Count All Columns Count All Indexes MySQL 8.0 MySQL 5.7 Time in Seconds (Lower is better)
  • 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA in MySQL 8.0: Dynamic values • TABLES.TABLE_ROWS, TABLES.DATA_FREE, … – Handled by opening table and retrieving data from SE in MySQL 5.7 – expensive!!! • Two approaches in MySQL 8.0 – information_schema_stats=cached • Default • Stored in auxiliary tables • ANALYZE TABLE – information_schema_stats=latest • Native functions in the I_S views • Uses special SE API (InnoDB) or falls back to opening tables (MyISAM) 27
  • 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA longstanding issues solved in 8.0 – Bug#34921 comparison with information schema tables don’t honor collation – Bug#48445 Inconsistency with SHOW and SELECT FROM I_S – Bug#61846 SHOW FULL COLUMNS displays incorrect privileges for table – Bug#65121 Inconsistent result for select on INFORMATION_SCHEMA.STATISTICS – Bug#75532 Join between I_S schema tables is case insensitive/returns wrong value – Bug#81347 unnecessary scanned all databases for information schema – ..... 28
  • 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Atomic DDL • All metadata stored in InnoDB (transactional SE) • DDL code changed to avoid intermediate commits – At server level – At InnoDB level – At replication level 29 Transactional behavior
  • 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Atomic DDL : Use Case - DROP DATABASE 30 MySQL 5.7 • Delete tables – Metadata, TRN/TRG/FRM files – Data, InnoDB tables • Delete stored programs – Metadata, rows in MyISAM (non- transactional) • Delete schema – Metadata, DB.OPT file Mix of filesystem, non- transactional/transactional storage and multiple commits MySQL 8.0 • Delete tables – Metadata, rows in InnoDB – Data, InnoDB tables • Delete stored programs – Metadata, rows in InnoDB • Delete schema – Metadata, rows in InnoDB Updates to transactional storage, one commit
  • 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Disaster Recovery • MySQL 8.0, metadata and data stored in InnoDB (transactional SE) – Metadata copy stored in SDI embedded in .ibd – Data stored in .ibd • MySQL before 8.0, metadata in files, data in InnoDB – Metadata stored in .FRM – Data stored in .ibd 31 Use case – IMPORT using Serialized Dictionary Information, InnoDB
  • 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL before 8.0 • Moving .FRM and data files around (MyISAM) • Create tables and do import using SQL (InnoDB) CREATE TABLE t1 (...) ALTER TABLE t1 DISCARD TABLESPACE ALTER TABLE t1 IMPORT TABLESPACE ... • Easy to do mistakes MySQL 8.0 • Self-descriptive tablespaces (SDI) • New IMPORT statement will import the tablespace: – Read metadata from the SDI, and create the tables – Import the data into these tables • Worst case, if SDI is corrupt, you can extract it (JSON format) and edit 32 Reliability – Disaster Recovery • Use case – IMPORT using Serialized Dictionary Information
  • 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability and Ease of Use – Automated Dictionary Upgrade • The Data Dictionary is versioned • The MySQL Server will support upgrading dictionary tables automatically – Done directly by the executable and not in a script – Reduces user and privilege issues for the process doing the upgrade • The Data Dictionary will be upgraded atomically 33
  • 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Extending Dictionary Tables from Plugins • API for plugins to extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA – Add new virtual tables/views • API for plugins to store their specific data in the Data Dictionary 34
  • 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Try for yourself! • Downloadable 8.0.1 DMR – dev.mysql.com • A «hot» labs release right off the press: 8.0.1 + Unified Data Dictionary • Enjoy and give us your feedback! • Thank you for listening • http://mysqlserverteam.com 35