A new Data Dictionary based on transactional tables is being developed for the MySQL server. That project is a huge step forward improving many aspects of the server. The new Data Dictionary provides API which is intended to be used by all the participants of the MySQL Server Ecosystem. The slides make a brief introduction about what general Data Dictionary is, provide overview of the MySQL traditional Data Dictionary and its limitations. Then, the presentation shows the design goals of the new Data Dictionary and sketch the main architectural decisions. It also provides the description of a few visible advantages for the MySQL users. These are the slides for my session on OOW 2014.

1. New Data Dictionary:
An Internal Server API
That Matters
Alexander Nozdrin, Principle Software Developer
Copyright Copyright © © 2014, 2014, Oracle Oracle and/and/or or its its affiliates. affiliates. All All rights rights reserved. reserved.
|

2. Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
2

3. MySQL Community Reception @ Oracle OpenWorld
Mingle with the MySQL community and the MySQL team from Oracle for
a fun and informative evening!
• Time: September 30 (Tue) @ 7pm
• Jillian’s at Metreon
175 Fourth Street, San Francisco, CA
At the corner of Howard and 4th st.; only 2-min walk from Moscone Center
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Agenda
What is a Data Dictionary?
The MySQL Traditional Data Dictionary
New Data Dictionary
Benefits for users
Q & A
1
2
3
4
5

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
What is it

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
Definition
• Metadata is information about user data
– User table structure
– Column definitions
– Index definitions
– Foreign key definitions
– Stored program definitions
...
• Data Dictionary collects all metadata in RDBMS

7. CREATE PROCEDURE p1(v INT)
SQL SECURITY INVOKER
BEGIN
...
END
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
Overview
Data Dictionary
Table Definitions SP Definitions
View Definitions Plugins
Time zones
Privileges
CREATE TABLE customers(
id INT AUTO_INCREMENT
...
PRIMARY KEY (id),
INDEX ...
FOREIGN KEY ...
)

8. SE InnoDB
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Server
Data Dictionary Definition
8
The ecosystem
Query Executor
Optimizer
Performance
Schema
SQL
statement
Client
Parser
Result
Information
Schema
Data
Dictionary

9. The MySQL Traditional Data Dictionary
MySQL 5.6 and earlier
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

10. The MySQL Traditional Data Dictionary
• A mix of files and tables:
– File based
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Tables: FRM
• Triggers: TRN, TRG
...
– Table based
• mysql.time_zone
...
• InnoDB has a separate data dictionary

11. The MySQL Traditional Data Dictionary
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
11
Data Dictionary
Files
FRM TRG OPT
System tables (mysql.)
user time_zone proc
InnoDB internal data dictionary
Archive
CSV
InnoDB
INFORMATION_SCHEMA
File
Scan
Table
Scan
Intern.
Access

12. The MySQL Traditional Data Dictionary
• Poor INFORMATION_SCHEMA performance
• Makes crash-safe / transactional DDL impossible
• Inconsistencies between files and tables
• Inconsistencies between DD in InnoDB and the server
• File-system dependency (lower-case-table-names)
• Makes replication of DDL statements difficult
• Too difficult to extend
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
12
Problems

13. New Data Dictionary
A great leap forward
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

14. New Data Dictionary : Main Features
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Stored in InnoDB tables
• Reliable & crash-safe
• Single repository of metadata
– for the MySQL server
– for Storage Engines
– for Plugins
• Redundancy
• Data Dictionary API
• INFORMATION_SCHEMA SQL VIEWs
– Queries can be optimized
– Improved performance
• Metadata versioning
• Extendable
– Simplify metadata upgrades
– Designed with plugins in mind

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
15
Transition
Data Dictionary
Files
FRM TRG OPT
System tables (mysql.)
user time_zone proc
InnoDB internal dictionary
Archive
CSV
InnoDB
INFORMATION_SCHEMA
File
Scan
SQL DD Table
VIEW
Table
Scan
Intern.
Access

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
16
Overview
InnoDB
Data Dictionary
DD Table User Table
INFORMATION
SCHEMA
Views
Archive
User Table
CSV
User Table

17. Plugin Plugin Plugin
Tablespace User Table Storage
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
InnoDB
17
Architecture
Query Executor
Parser Optimizer
Data
Dictionary
Tablespace
Data Dictionary Internal API
Data Dictionary External API
Plugin Plugin
Archive
Engine
Storage
Engine

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
• WL#6379: Schema definitions for new DD
• InnoDB Data Dictionary Tablespace
• Designed with INFORMATION_SCHEMA in mind
• Ability to store SE-specific data
• Use PK / FK to ensure consistency
18
Data Dictionary tables

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary API
• The only way to access Data Dictionary
– For the server core
– For Storage Engines
– For plugins
• Hard to misuse
• Internal API (non-stable) and external API (stable)
• Provide a way to handle SE specific data
19
Design goals

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
SDI : Serialized Dictionary Information
20
Redundancy
InnoDB
Single User TS
User Table
SDI
General TS
User Table
SDI
System TS
User Table
SDI
Data Dictionary
Privileges
User Table
Definition
Stored
Program

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
New Data Dictionary
21
FRM shipping for MySQL Cluster?
SELECT ... FROM t1
FSRDMI

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• No FRM files
• New INFORMATION_SCHEMA
• Migrated to InnoDB:
– time zone tables
– help tables
–mysql.plugins
–mysql.servers
• Draft of Data Dictionary API
• http://labs.mysql.com
– Do NOT use it in production
– Install on a spare server
• MTR can be run
22
New Data Dictionary : Labs Release

23. New Data Dictionary
Why does it matter for YOU?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

24. INFORMATION_SCHEMA performance improvements
Get per table size
Blog post by Shlomi Noach: http://tinyurl.com/y8cnj7o
SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE,
SUM(DATA_LENGTH+INDEX_LENGTH) AS size,
SUM(INDEX_LENGTH) AS index_size
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA') AND ENGINE IS NOT NULL
GROUP BY TABLE_SCHEMA, TABLE_NAME;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
24
Version Time
5.7.5-m15 0.38 sec
Labs Release 0.08 sec

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Case: server crash
• There are some data files
• No / outdated backup
• FRM files lost
• How to use those data files?
25
The problem

26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Traditional Data Dictionary
• “Move FRM files around”
• CREATE TABLE t1 (...)
• ALTER TABLE t1
DISCARD TABLESPACE
• ALTER TABLE t1
IMPORT TABLESPACE ...
• Easy to make mistakes
New Data Dictionary
• Self-descriptive tablespaces (SDI)
• Dedicated IMPORT statement
• Goal: error-proof procedure
26
Case: server crash

27. For Plugin Developers
• A way to access Data Dictionary
• Persistent Storage for plugins
– Store/restore custom data
• Plugins can extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA
– Add new tables
– Add new columns to existing tables
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
27

28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
Summary

29. Data Dictionary : Takeaways
• Fundamental component in RDBMS
• Critical for performance
• Critical for reliability
• Critical for scalability
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
29

30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
The MySQL Traditional
Data Dictionary
• Mix of files and tables
• Server DD and InnoDB DD
• Inefficient INFORMATION_SCHEMA
• Difficult to extend
New Data Dictionary
• Crash-safe InnoDB tables
• Single repository
• INFORMATION_SCHEMA as VIEWs
• Designed to be extendable
• Aims for backward compatibility
• Huge reengineering
Data Dictionary : Takeaways

31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Questions?

32. MySQL Community Reception @ Oracle OpenWorld
Mingle with the MySQL community and the MySQL team from Oracle for
a fun and informative evening!
• Time: September 30 (Tue) @ 7pm
• Jillian’s at Metreon
175 Fourth Street, San Francisco, CA
At the corner of Howard and 4th st.; only 2-min walk from Moscone Center
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

33. Oracle University MySQL Training Services
Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications
Top Courses for Administrators and Developers
Top Certifications
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“Training and team skill have the most significant impact
on overall performance of technology and success of
technology projects.” - IDC, 2013
Premier Support customers eligible to
save 20% on learning credits.
Benefits
 Expert-led training to support your MySQL learning needs
 Flexibility to train in the classroom or online
 Hands-on experience to gain real world experience
 Key skills needed for database administrators and developers
• MySQL for Beginners
MySQL for Database Administrators
MySQL Performance Tuning
MySQL Cluster – NEW - Register Your Interest!
MySQL and PHP - Developing Dynamic Web Applications
MySQL for Developers
MySQL Developer Techniques
MySQL 5.6 Database Administrator
MySQL 5.6 Developer
To find out more about available MySQL Training & Certification
offerings, go to: education.oracle.com/mysql
RECENTLY RELEASED
ALL NEW! MySQL Cluster Training
To Register your interest to influence the
schedule on this newly released course – go to
education.oracle.com/mysql and click on the
MySQL Cluster Course

34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Thank You!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.