Schemadoc
  Bill Karwin, Percona Inc.
Schema Can Be Unclear
CREATE TABLE AccountActivity (
    Id        INT PRIMARY KEY,
                                           whose name?
    Name      VARCHAR(100),
    Status    VARCHAR(20),           what are the values?

    EntityNum INT,                 what’s an entity?
    Order     INT                  what’s a number?
 );                          commerce, or ordinal?




                                          www.percona.com
Comments
CREATE TABLE AccountActivity (
   Id        INT PRIMARY KEY,
   Name      VARCHAR(100) COMMENT ‘person who placed the order’,
   Status    VARCHAR(20) COMMENT ‘new, open, or closed’,
   EntityNum INT            COMMENT ‘how many items in the order’,
   Order     INT            COMMENT ‘reference to the Orders table’
) COMMENT ‘any change to e-commerce orders’;




                                                    www.percona.com
Length
• Limits increased in MySQL 5.5.3:
  • Per column: 1024 characters
  • Per index: 1024 characters
  • Per table: 2048 characters
  • Per partition: 80 characters
     (MySQL 5.6 increases this to 1024)




                                          www.percona.com
Why don’t we use comments?
SELECT TABLE_NAME, TABLE_COMMENT
FROM INFORMATION_SCHEMA.TABLES
WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’);
SELECT COLUMN_NAME, COLUMN_COMMENT
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’)
ORDER BY ORDINAL_POSITION;
SELECT INDEX_NAME, CONCAT(‘(’, GROUP_CONCAT(COLUMN_NAME ORDER
BY SEQ_IN_INDEX), ‘)’) AS INDEX_COLUMNS, COMMENT AS INDEX_COMMENT
FROM INFORMATION_SCHEMA.STATISTICS
WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’)
GROUP BY INDEX_NAME;
SELECT PARTITION_NAME, PARTITION_COMMENT
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’)
ORDER BY PARTITION_ORDINAL_POSITION;



                                                         www.percona.com
“Is there a tool for MySQL
 like javadoc for Java code?”




                         www.percona.com
development
              dba

   documentation



                    www.percona.com
schemadoc


            www.percona.com
Usage
$ schemadoc -v h=dbserver,D=EcommerceDB,u=root,p=xxxx
Loading metadata...
Running online analysis...
done.
Outputting to html.
Rendering pages for tables. done.
Rendering pages for views done.
Rendering pages for procedures done.
Rendering pages for triggers done.
Rendering page for schema summary... done.
Rendering page for schema link... done.
Rendering page for deprecated-list... done.
Rendering page for all-index... done.
Rendering page for help-doc... done.
Rendering page for allschema-frame... done.
Rendering page for index... done.


                                              www.percona.com
hooray.




          www.percona.com
But Wait, There’s More
• Parses SQL dump file
• Reads from a live MySQL instance
• Document many schemas
• Update one schema at a time




                                     www.percona.com
Schema Analysis
• Number of indexes and columns per data type
• Columns with same name but different type
• ENUM columns that aren’t NOT NULL
• Use of FLOAT or DOUBLE
• Tables with one index per column
• Tables with one index over all columns
• Tables with no indexes
• INT(N) other than default N
• Lack of INT UNSIGNED columns
                                      www.percona.com
Schema Analysis
• Tables that look like Entity-Attribute-Value
• Tables that look like Polymorphic Associations
• Superfluous primary key
• Report table with least / most indexes
• IP addresses stored in VARCHAR
• Excessive use of VARCHAR(255),
   report actual max string length
...And supports plugins for other checks!


                                            www.percona.com
yes, we get it.




                  www.percona.com
Future of Schemadoc
• Integrate with Percona Toolkit library
• Read extra metadata (foreign keys)
• Show table sizes and growth rates
• Show SQL privileges (who has access?)
• CLI for adding / updating comments
• Batch-mode output

• Redesign HTML output with modern look & feel...

                                       www.percona.com
Future of Schemadoc




                  www.percona.com
launchpad.net/
  schemadoc

             www.percona.com
Copyright 2012 Bill Karwin
           www.slideshare.net/billkarwin

          Released under a Creative Commons 3.0 License:
           http://creativecommons.org/licenses/by-nc-nd/3.0/
              You are free to share - to copy, distribute and
             transmit this work, under the following conditions:




   Attribution.               Noncommercial.          No Derivative Works.
You must attribute this   You may not use this work       You may not alter,
 work to Bill Karwin.      for commercial purposes.      transform, or build
                                                           upon this work.

                                                               www.percona.com

Schemadoc

  • 1.
    Schemadoc BillKarwin, Percona Inc.
  • 2.
    Schema Can BeUnclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, whose name? Name VARCHAR(100), Status VARCHAR(20), what are the values? EntityNum INT, what’s an entity? Order INT what’s a number? ); commerce, or ordinal? www.percona.com
  • 3.
    Comments CREATE TABLE AccountActivity( Id INT PRIMARY KEY, Name VARCHAR(100) COMMENT ‘person who placed the order’, Status VARCHAR(20) COMMENT ‘new, open, or closed’, EntityNum INT COMMENT ‘how many items in the order’, Order INT COMMENT ‘reference to the Orders table’ ) COMMENT ‘any change to e-commerce orders’; www.percona.com
  • 4.
    Length • Limits increasedin MySQL 5.5.3: • Per column: 1024 characters • Per index: 1024 characters • Per table: 2048 characters • Per partition: 80 characters (MySQL 5.6 increases this to 1024) www.percona.com
  • 5.
    Why don’t weuse comments? SELECT TABLE_NAME, TABLE_COMMENT FROM INFORMATION_SCHEMA.TABLES WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’); SELECT COLUMN_NAME, COLUMN_COMMENT FROM INFORMATION_SCHEMA.COLUMNS WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’) ORDER BY ORDINAL_POSITION; SELECT INDEX_NAME, CONCAT(‘(’, GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX), ‘)’) AS INDEX_COLUMNS, COMMENT AS INDEX_COMMENT FROM INFORMATION_SCHEMA.STATISTICS WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’) GROUP BY INDEX_NAME; SELECT PARTITION_NAME, PARTITION_COMMENT FROM INFORMATION_SCHEMA.PARTITIONS WHERE (TABLE_SCHEMA, TABLE_NAME) = (‘EcommerceDB’, ‘AccountActivity’) ORDER BY PARTITION_ORDINAL_POSITION; www.percona.com
  • 6.
    “Is there atool for MySQL like javadoc for Java code?” www.percona.com
  • 7.
    development dba documentation www.percona.com
  • 8.
    schemadoc www.percona.com
  • 9.
    Usage $ schemadoc -vh=dbserver,D=EcommerceDB,u=root,p=xxxx Loading metadata... Running online analysis... done. Outputting to html. Rendering pages for tables. done. Rendering pages for views done. Rendering pages for procedures done. Rendering pages for triggers done. Rendering page for schema summary... done. Rendering page for schema link... done. Rendering page for deprecated-list... done. Rendering page for all-index... done. Rendering page for help-doc... done. Rendering page for allschema-frame... done. Rendering page for index... done. www.percona.com
  • 11.
    hooray. www.percona.com
  • 12.
    But Wait, There’sMore • Parses SQL dump file • Reads from a live MySQL instance • Document many schemas • Update one schema at a time www.percona.com
  • 13.
    Schema Analysis • Numberof indexes and columns per data type • Columns with same name but different type • ENUM columns that aren’t NOT NULL • Use of FLOAT or DOUBLE • Tables with one index per column • Tables with one index over all columns • Tables with no indexes • INT(N) other than default N • Lack of INT UNSIGNED columns www.percona.com
  • 14.
    Schema Analysis • Tablesthat look like Entity-Attribute-Value • Tables that look like Polymorphic Associations • Superfluous primary key • Report table with least / most indexes • IP addresses stored in VARCHAR • Excessive use of VARCHAR(255), report actual max string length ...And supports plugins for other checks! www.percona.com
  • 15.
    yes, we getit. www.percona.com
  • 16.
    Future of Schemadoc •Integrate with Percona Toolkit library • Read extra metadata (foreign keys) • Show table sizes and growth rates • Show SQL privileges (who has access?) • CLI for adding / updating comments • Batch-mode output • Redesign HTML output with modern look & feel... www.percona.com
  • 18.
    Future of Schemadoc www.percona.com
  • 19.
  • 20.
    Copyright 2012 BillKarwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this You may not use this work You may not alter, work to Bill Karwin. for commercial purposes. transform, or build upon this work. www.percona.com