Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

New data dictionary an internal server api that matters

1,265 views

Published on

A new Data Dictionary based on transactional tables is being developed for the MySQL server. That project is a huge step forward improving many aspects of the server. The new Data Dictionary provides API which is intended to be used by all the participants of the MySQL Server Ecosystem. The slides make a brief introduction about what general Data Dictionary is, provide overview of the MySQL traditional Data Dictionary and its limitations. Then, the presentation shows the design goals of the new Data Dictionary and sketch the main architectural decisions. It also provides the description of a few visible advantages for the MySQL users.

These are the slides for my session on OOW 2014.

Published in: Software
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

New data dictionary an internal server api that matters

  1. 1. New Data Dictionary: An Internal Server API That Matters Alexander Nozdrin, Principle Software Developer Copyright Copyright © © 2014, 2014, Oracle Oracle and/and/or or its its affiliates. affiliates. All All rights rights reserved. reserved. |
  2. 2. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 2
  3. 3. MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! • Time: September 30 (Tue) @ 7pm • Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4th st.; only 2-min walk from Moscone Center Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  4. 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Agenda What is a Data Dictionary? The MySQL Traditional Data Dictionary New Data Dictionary Benefits for users Q & A 1 2 3 4 5
  5. 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary What is it
  6. 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Definition • Metadata is information about user data – User table structure – Column definitions – Index definitions – Foreign key definitions – Stored program definitions ... • Data Dictionary collects all metadata in RDBMS
  7. 7. CREATE PROCEDURE p1(v INT) SQL SECURITY INVOKER BEGIN ... END Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Overview Data Dictionary Table Definitions SP Definitions View Definitions Plugins Time zones Privileges CREATE TABLE customers( id INT AUTO_INCREMENT ... PRIMARY KEY (id), INDEX ... FOREIGN KEY ... )
  8. 8. SE InnoDB Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Server Data Dictionary Definition 8 The ecosystem Query Executor Optimizer Performance Schema SQL statement Client Parser Result Information Schema Data Dictionary
  9. 9. The MySQL Traditional Data Dictionary MySQL 5.6 and earlier Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  10. 10. The MySQL Traditional Data Dictionary • A mix of files and tables: – File based Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Tables: FRM • Triggers: TRN, TRG ... – Table based • mysql.time_zone ... • InnoDB has a separate data dictionary
  11. 11. The MySQL Traditional Data Dictionary Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11 Data Dictionary Files FRM TRG OPT System tables (mysql.) user time_zone proc InnoDB internal data dictionary Archive CSV InnoDB INFORMATION_SCHEMA File Scan Table Scan Intern. Access
  12. 12. The MySQL Traditional Data Dictionary • Poor INFORMATION_SCHEMA performance • Makes crash-safe / transactional DDL impossible • Inconsistencies between files and tables • Inconsistencies between DD in InnoDB and the server • File-system dependency (lower-case-table-names) • Makes replication of DDL statements difficult • Too difficult to extend Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 12 Problems
  13. 13. New Data Dictionary A great leap forward Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  14. 14. New Data Dictionary : Main Features Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Stored in InnoDB tables • Reliable & crash-safe • Single repository of metadata – for the MySQL server – for Storage Engines – for Plugins • Redundancy • Data Dictionary API • INFORMATION_SCHEMA SQL VIEWs – Queries can be optimized – Improved performance • Metadata versioning • Extendable – Simplify metadata upgrades – Designed with plugins in mind
  15. 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 15 Transition Data Dictionary Files FRM TRG OPT System tables (mysql.) user time_zone proc InnoDB internal dictionary Archive CSV InnoDB INFORMATION_SCHEMA File Scan SQL DD Table VIEW Table Scan Intern. Access
  16. 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 16 Overview InnoDB Data Dictionary DD Table User Table INFORMATION SCHEMA Views Archive User Table CSV User Table
  17. 17. Plugin Plugin Plugin Tablespace User Table Storage Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary InnoDB 17 Architecture Query Executor Parser Optimizer Data Dictionary Tablespace Data Dictionary Internal API Data Dictionary External API Plugin Plugin Archive Engine Storage Engine
  18. 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary • WL#6379: Schema definitions for new DD • InnoDB Data Dictionary Tablespace • Designed with INFORMATION_SCHEMA in mind • Ability to store SE-specific data • Use PK / FK to ensure consistency 18 Data Dictionary tables
  19. 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary API • The only way to access Data Dictionary – For the server core – For Storage Engines – For plugins • Hard to misuse • Internal API (non-stable) and external API (stable) • Provide a way to handle SE specific data 19 Design goals
  20. 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary SDI : Serialized Dictionary Information 20 Redundancy InnoDB Single User TS User Table SDI General TS User Table SDI System TS User Table SDI Data Dictionary Privileges User Table Definition Stored Program
  21. 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 21 FRM shipping for MySQL Cluster? SELECT ... FROM t1 FSRDMI
  22. 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • No FRM files • New INFORMATION_SCHEMA • Migrated to InnoDB: – time zone tables – help tables –mysql.plugins –mysql.servers • Draft of Data Dictionary API • http://labs.mysql.com – Do NOT use it in production – Install on a spare server • MTR can be run 22 New Data Dictionary : Labs Release
  23. 23. New Data Dictionary Why does it matter for YOU? Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  24. 24. INFORMATION_SCHEMA performance improvements Get per table size Blog post by Shlomi Noach: http://tinyurl.com/y8cnj7o SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE, SUM(DATA_LENGTH+INDEX_LENGTH) AS size, SUM(INDEX_LENGTH) AS index_size FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA') AND ENGINE IS NOT NULL GROUP BY TABLE_SCHEMA, TABLE_NAME; Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 24 Version Time 5.7.5-m15 0.38 sec Labs Release 0.08 sec
  25. 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Case: server crash • There are some data files • No / outdated backup • FRM files lost • How to use those data files? 25 The problem
  26. 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Traditional Data Dictionary • “Move FRM files around” • CREATE TABLE t1 (...) • ALTER TABLE t1 DISCARD TABLESPACE • ALTER TABLE t1 IMPORT TABLESPACE ... • Easy to make mistakes New Data Dictionary • Self-descriptive tablespaces (SDI) • Dedicated IMPORT statement • Goal: error-proof procedure 26 Case: server crash
  27. 27. For Plugin Developers • A way to access Data Dictionary • Persistent Storage for plugins – Store/restore custom data • Plugins can extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA – Add new tables – Add new columns to existing tables Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 27
  28. 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Summary
  29. 29. Data Dictionary : Takeaways • Fundamental component in RDBMS • Critical for performance • Critical for reliability • Critical for scalability Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 29
  30. 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary • Mix of files and tables • Server DD and InnoDB DD • Inefficient INFORMATION_SCHEMA • Difficult to extend New Data Dictionary • Crash-safe InnoDB tables • Single repository • INFORMATION_SCHEMA as VIEWs • Designed to be extendable • Aims for backward compatibility • Huge reengineering Data Dictionary : Takeaways
  31. 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Questions?
  32. 32. MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! • Time: September 30 (Tue) @ 7pm • Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4th st.; only 2-min walk from Moscone Center Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  33. 33. Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications Top Courses for Administrators and Developers Top Certifications Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | “Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013 Premier Support customers eligible to save 20% on learning credits. Benefits  Expert-led training to support your MySQL learning needs  Flexibility to train in the classroom or online  Hands-on experience to gain real world experience  Key skills needed for database administrators and developers • MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster – NEW - Register Your Interest! MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques MySQL 5.6 Database Administrator MySQL 5.6 Developer To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql RECENTLY RELEASED ALL NEW! MySQL Cluster Training To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the MySQL Cluster Course
  34. 34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Thank You! Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

×