Virtualizing and Scaling
                          for the Future




Tuesday, March 29, 2011
Who am I?
                          Was Director of Architecture for MySQL


                          Founder/CTO Data Differential




                                            Drizzle
                                           Gearman
                                         Memcached
Tuesday, March 29, 2011
In the beginning...



Tuesday, March 29, 2011
• 2008 Sun buys MySQL
                    • 2008 Drizzle forks from MySQL
                    • 2009 Oracle buys Sun
                    • 2010 Drizzle Developers split from Oracle
                    • 2011 First GA Release, Drizzle7

Tuesday, March 29, 2011
Drizzle

                    • Entirely Open Source RDBMS
                    • C++, with a MicroKernel Design
                    • No Gotchas


Tuesday, March 29, 2011
Drizzle7
                          (Quick Update)




Tuesday, March 29, 2011
Today’s Pluggable
                             Interfaces
                    • Replication      • Table Functions
                    • Storage Engine   • Functions
                    • Logging          • Protocol
                    • Authentication   • Query Rewrite
                    • Authorization    • XA

Tuesday, March 29, 2011
HTTP                       Replication
                                         Client            Client
     Cloud Service            Client



                             HTTP Blob          MySQL         Drizzle
                             Streaming
                                                        SQL



                                 Listener
                                                              Logging
         Query Memcached           Parser
              Cache

                                 Optimizer
                Memcached                               Error Reporting
                                Executioner

                   Gearman     Storage System
                                                        MicroKernel

Tuesday, March 29, 2011
Replication

                    • Google Protocol Buffer Based
                    • Replicates row transformations.
                    • Integrates today with RabbitMQ, Casandra.
                          Memcached, Gearman




Tuesday, March 29, 2011
INSERT


                          INSERT
       Stored
   Transactionally
                          UPDATE


                          INSERT



Tuesday, March 29, 2011
Slave




                          Master   Slave




                                   Slave

Tuesday, March 29, 2011
Slave




                          Master   Slave




                                   Slave

Tuesday, March 29, 2011
Master




                          Master   Master




                                   Master


Tuesday, March 29, 2011
What group commit
                              problem?



Tuesday, March 29, 2011
Group Commit?
                                     Transaction
                                         Log

                          Database
                           Kernel


                                       Binlog


Tuesday, March 29, 2011
No Group Commit
                              Required

                          Database   Transaction
                           Kernel        Log




Tuesday, March 29, 2011
libDrizzle

                    • Supports Drizzle (and MySQL, and SQLite)
                    • Asynchronous
                    • BSD
                    • Lawyer free

Tuesday, March 29, 2011
NoSQL Stream Access
             Amazon S3      Client




                           Instance



                            Storage

Tuesday, March 29, 2011
Is there more?



Tuesday, March 29, 2011
No Gotchas

                    • No NULL when you don’t want NULL
                    • No ALTER TABLES that fake’ it
                    • Real datetime (64bit), with microseconds
                    • IPV6
                    • No updates that “half complete”

Tuesday, March 29, 2011
Authentication/
                          Authorization

                    • No Cost Authentication
                    • PAM, LDAP, HTTP


Tuesday, March 29, 2011
Table Functions

                   • TABLE_CACHE
                   • TABLE_DEFINITION_CACHE
                   • SHOW TEMPORARY TABLES
                   • All new SQL compliant information_schema

Tuesday, March 29, 2011
Data Dictionay

                    • Separate Information Schema for ANSI SQL
                          access.
                    • All based on Table Functions
                    • No Materialization
                    • Single Execution Path

Tuesday, March 29, 2011
Интернет является UTF-8.

                               Web UTF - 8

                            The Web is UTF-8.


Tuesday, March 29, 2011
How about locks?



Tuesday, March 29, 2011
SHOW PROCESSLIST
                          (now based on table functions)




Tuesday, March 29, 2011
--skip-grants
                          (none of the current GRANTs require LOCKS
                                            actually…)




Tuesday, March 29, 2011
store_lock()
                          (all engines are not created equally)




Tuesday, March 29, 2011
120+ Developers Since
                        Day One
                          26/36 on average each month




Tuesday, March 29, 2011
Drizzle 7+



Tuesday, March 29, 2011
Table                       Table

                 Schema   Table              Schema   Table

                          Table                       Table
                                   Users
                          Table                       Table

                 Schema   Table              Schema   Table

                          Table                       Table


                                  Instance
Tuesday, March 29, 2011
Users                         Users


                            Table                       Table

                   Schema   Table              Schema   Table

                            Table                       Table




                                    Instance
Tuesday, March 29, 2011
• Cuts the 30/40% hit from virtualizing
                    • Single instance maintenance
                    • Still allows for IO spread on San/NAS


Tuesday, March 29, 2011
Hot Backup

                    • Supports Partial Backup
                    • No locking
                    • Single solution, point-in-time recovery in a
                          single solution.
                    • Comes with the database.

Tuesday, March 29, 2011
http://drizzle.org
                          http://planetdrizzle.org
                          Freenode #drizzle


Tuesday, March 29, 2011

Drizzle 7.0, Future of Virtualizing

  • 1.
    Virtualizing and Scaling for the Future Tuesday, March 29, 2011
  • 2.
    Who am I? Was Director of Architecture for MySQL Founder/CTO Data Differential Drizzle Gearman Memcached Tuesday, March 29, 2011
  • 3.
  • 4.
    • 2008 Sunbuys MySQL • 2008 Drizzle forks from MySQL • 2009 Oracle buys Sun • 2010 Drizzle Developers split from Oracle • 2011 First GA Release, Drizzle7 Tuesday, March 29, 2011
  • 5.
    Drizzle • Entirely Open Source RDBMS • C++, with a MicroKernel Design • No Gotchas Tuesday, March 29, 2011
  • 6.
    Drizzle7 (Quick Update) Tuesday, March 29, 2011
  • 7.
    Today’s Pluggable Interfaces • Replication • Table Functions • Storage Engine • Functions • Logging • Protocol • Authentication • Query Rewrite • Authorization • XA Tuesday, March 29, 2011
  • 8.
    HTTP Replication Client Client Cloud Service Client HTTP Blob MySQL Drizzle Streaming SQL Listener Logging Query Memcached Parser Cache Optimizer Memcached Error Reporting Executioner Gearman Storage System MicroKernel Tuesday, March 29, 2011
  • 9.
    Replication • Google Protocol Buffer Based • Replicates row transformations. • Integrates today with RabbitMQ, Casandra. Memcached, Gearman Tuesday, March 29, 2011
  • 10.
    INSERT INSERT Stored Transactionally UPDATE INSERT Tuesday, March 29, 2011
  • 11.
    Slave Master Slave Slave Tuesday, March 29, 2011
  • 12.
    Slave Master Slave Slave Tuesday, March 29, 2011
  • 13.
    Master Master Master Master Tuesday, March 29, 2011
  • 14.
    What group commit problem? Tuesday, March 29, 2011
  • 15.
    Group Commit? Transaction Log Database Kernel Binlog Tuesday, March 29, 2011
  • 16.
    No Group Commit Required Database Transaction Kernel Log Tuesday, March 29, 2011
  • 17.
    libDrizzle • Supports Drizzle (and MySQL, and SQLite) • Asynchronous • BSD • Lawyer free Tuesday, March 29, 2011
  • 18.
    NoSQL Stream Access Amazon S3 Client Instance Storage Tuesday, March 29, 2011
  • 19.
  • 20.
    No Gotchas • No NULL when you don’t want NULL • No ALTER TABLES that fake’ it • Real datetime (64bit), with microseconds • IPV6 • No updates that “half complete” Tuesday, March 29, 2011
  • 21.
    Authentication/ Authorization • No Cost Authentication • PAM, LDAP, HTTP Tuesday, March 29, 2011
  • 22.
    Table Functions • TABLE_CACHE • TABLE_DEFINITION_CACHE • SHOW TEMPORARY TABLES • All new SQL compliant information_schema Tuesday, March 29, 2011
  • 23.
    Data Dictionay • Separate Information Schema for ANSI SQL access. • All based on Table Functions • No Materialization • Single Execution Path Tuesday, March 29, 2011
  • 24.
    Интернет является UTF-8. Web UTF - 8 The Web is UTF-8. Tuesday, March 29, 2011
  • 25.
  • 26.
    SHOW PROCESSLIST (now based on table functions) Tuesday, March 29, 2011
  • 27.
    --skip-grants (none of the current GRANTs require LOCKS actually…) Tuesday, March 29, 2011
  • 28.
    store_lock() (all engines are not created equally) Tuesday, March 29, 2011
  • 29.
    120+ Developers Since Day One 26/36 on average each month Tuesday, March 29, 2011
  • 30.
  • 31.
    Table Table Schema Table Schema Table Table Table Users Table Table Schema Table Schema Table Table Table Instance Tuesday, March 29, 2011
  • 32.
    Users Users Table Table Schema Table Schema Table Table Table Instance Tuesday, March 29, 2011
  • 33.
    • Cuts the30/40% hit from virtualizing • Single instance maintenance • Still allows for IO spread on San/NAS Tuesday, March 29, 2011
  • 34.
    Hot Backup • Supports Partial Backup • No locking • Single solution, point-in-time recovery in a single solution. • Comes with the database. Tuesday, March 29, 2011
  • 35.
    http://drizzle.org http://planetdrizzle.org Freenode #drizzle Tuesday, March 29, 2011