MySQL Cluster
Web Scalability, 99.999% Availability



Andrew Morgan
@andrewmorgan
www.clusterdb.com
Safe Harbour Statement

     The following is intended to outline our general product direction. It is
     intended for information purposes only, and may not be incorporated into any
     contract.
     It is not a commitment to deliver any material, code, or functionality, and
     should not be relied upon in making purchasing decisions. The development,
     release, and timing of any features or functionality described for Oracle’s
     products remains at the sole discretion of Oracle.




2   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster: Overview
                 HIGH SCALE,                                      • Auto-Sharding, Multi-Master
                READS + WRITES                                    • ACID Compliant, OLTP + Real-Time Analytics



                      99.999%                                     • Shared nothing, no Single Point of Failure
                    AVAILABILITY                                  • Self Healing + On-Line Operations



                                                                  • In-Memory Optimization + Disk-Data
                       REAL-TIME
                                                                  • Predictable Low-Latency, Bounded Access Time



                                                                  • Key/Value + Complex, Relational Queries
                    SQL + NoSQL
                                                                  • SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++



                                                                  • Open Source + Commercial Editions
                         LOW TCO
                                                                  • Commodity hardware + Management, Monitoring Tools



3   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Who’s Using MySQL Cluster?




4   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster Architecture

                                                                                               Clients




                                                                                                         Application Layer




Management

                                                                                                          Data Layer
                                                                    MySQL Cluster Data Nodes


5   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster Architecture
         On-Line Scaling


                                                                                                   Clients




                                                                                                             Application Layer



Management
                                                                                                             Management

                                                                                                             Data Layer
                                                                        MySQL Cluster Data Nodes


    6   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
http://clusterdb.com/u/pi
 7   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster Architecture
         No Single Point of Failure



                                                                                                   Clients




                                                                                                             Application Layer




Management
                                                                                                                Management

                                                                        MySQL Cluster Data Nodes


    8   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                       Table T1                                                                 Data Node 1

                                                                                               Px   Partition
                                                                                               P1
                                                                                                                Data Node 2
                                                                                               P2

                                                                                               P3               Data Node 3

                                                                                               P4

                                                                                                                Data Node 4




9   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1
                                                                                                Px   Partition
                                                                                                P1
                                                                                                                     Data Node 2
                                                                                                P2

                                                                                                P3                   Data Node 3

                                                                                                P4

                                                                                          Fx    Primary Fragment     Data Node 4

                                                                                          Fx    Secondary Fragment



10   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1
                                                                                                Px   Partition
                                                                                                P1
                                                                                                                     Data Node 2
                                                                                                P2                             F1


                                                                                                P3                   Data Node 3

                                                                                                P4

                                                                                          Fx    Primary Fragment     Data Node 4

                                                                                          Fx    Secondary Fragment



11   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1
                                                                                                Px   Partition
                                                                                                P1
                                                                                                                     Data Node 2
                                                                                                P2                   F3        F1


                                                                                                P3                   Data Node 3

                                                                                                P4

                                                                                          Fx    Primary Fragment     Data Node 4

                                                                                          Fx    Secondary Fragment



12   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1        F3
                                                                                                Px   Partition
                                                                                                P1
                                                                                                                     Data Node 2
                                                                                                P2                   F3        F1


                                                                                                P3                   Data Node 3

                                                                                                P4

                                                                                          Fx    Primary Fragment     Data Node 4

                                                                                          Fx    Secondary Fragment



13   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1        F3
                                                                                                Px   Partition
                                                                                                P1
                                                                                                                     Data Node 2
                                                                                                P2                   F3        F1


                                                                                                P3                   Data Node 3
                                                                                                                     F2        F4
                                                                                                P4

                                                                                          Fx    Primary Fragment     Data Node 4
                                                                                                                     F4        F2
                                                                                          Fx    Secondary Fragment



14   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1        F3
                                                                                                Px   Partition
                                                                                                P1                             Node Group 1
                                                                                                                     Data Node 2
                                                                                                P2                   F3        F1


                                                                                                P3                   Data Node 3
                                                                                                                     F2        F4
                                                                                                P4
                                                                                                                               Node Group 2
                                                                                          Fx    Primary Fragment     Data Node 4
                                                                                                                     F4        F2
                                                                                          Fx    Secondary Fragment



15   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Automatic Data Partitioning
                                        Table T1                                                                     Data Node 1
                                                                                                                     F1        F3
                                                                                                Px   Partition
                                                                                                P1                             Node Group 1
                                                                                                                     Data Node 2
                                                                                                P2                   F3        F1


                                                                                                P3                   Data Node 3
                                                                                                                     F2        F4
                                                                                                P4
                                                                                                                               Node Group 2
                                                                                          Fx    Primary Fragment     Data Node 4
                                                                                                                     F4        F2
                                                                                          Fx    Secondary Fragment



16   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
On-line Scheduled Maintenance




                                                     Scale                                      Backup




                                                      Evolve                                    Upgrade

17   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Multi-Data Center Availability
       Geographic Replication
                                                                                                 DR and Data Locality
                                                                                                 Replicate complete clusters
                                                                                                 across data centers
                                                                                                   – Fully active/active
                                                                                                   – No passive resources
                           Geographic
                           Replication                                                           Split individual clusters across
                                                                                                 data centers
                                                                                                   – Synchronous replication &
                                                                                                     auto-failover between sites




18   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
1.2 Billion UPDATEs per Minute
                               25
     Millions of UPDATEs per

                               20
              Second



                               15


                               10


                                5


                                0
                                    2     4           6           8          10          12      14   16   18   20   22   24   26   28   30
                                                                                MySQL Cluster Data Nodes

                                 NoSQL C++ API, flexaSynch benchmark

                                 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB
                                 ACID Transactions, with Synchronous Replication



19    Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Real-World Test Case
                                                   Query Execution Time Seconds
               100
                90
                80
                70
                60
                                                                                          87.23 seconds                                  70x
                50
                40
                                                                                                                                        Faster
                30
                20
                10
                 0                                                                                                       1.26 seconds
                                                 MySQL Cluster 7.1                                   MySQL Cluster 7.2




 Web-Based Content Management System
      – JOINs 11-tables, 33.5k rows
      – Returns 2k rows, 19 columns per row


 20    Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster 7.3 EA: Node.js NoSQL API
                                                                                           Native JavaScript access to MySQL Cluster
                                                                                                 – End-to-End JavaScript: browser to the app and
                                                                                                   database
                                                                                                 – Storing and retrieving JavaScript objects directly
                                                                                                   in MySQL Cluster
                                                                                                 – Eliminate SQL transformation
                                                                                           Implemented as a module for node.js
                                                                                                 – Integrates full Cluster API library within the web
                                                                                                   app
                                                                                           Couple high performance, distributed apps, with
                                                                                                high performance distributed database


21   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster 7.3 DMR1: Foreign Keys
 Brings MySQL Cluster to a broader range of workloads
       – Packaged apps, custom projects
 Adds powerful functionality while reducing complexity
       – App logic & data model
 Enabled by default
 Enforced for SQL
  & NoSQL APIs
 On-line add and drop


22   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
MySQL Cluster 7.2 and 7.3: Auto-Installer
        Early Access Feature
                                                                                                   Best practices   New!
 Fast configuration
 Workload optimized

      Specify                                         Auto-
     Workload                                        Discover




                                                     Define
     Deploy                                         Topology




23     Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
When to Consider MySQL Cluster
   Scalability demands
             Sharding for write performance?

   Latency demands
             Cost of each second?

   Uptime requirements
             Cost per minute of downtime?
             Failure versus maintenance?

   Application agility
             Developer languages and frameworks?
             SQL or NoSQL?




    24       Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
Next Steps
                            Learn More
                            • www.mysql.com/cluster
                            • Authentic MySQL Curriculum:
                              http://oracle.com/education/mysql

                            Try it Out
                            • dev.mysql.com/cluster
                            • labs.mysql.com
                            • github.com/mysql/mysql-js

                            Let us know what you think
                            • clusterdb.com
                            • @clusterdb
                            • forums.mysql.com/list.php?25


25   Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013

Mysql cluster introduction

  • 1.
    MySQL Cluster Web Scalability,99.999% Availability Andrew Morgan @andrewmorgan www.clusterdb.com
  • 2.
    Safe Harbour Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 3.
    MySQL Cluster: Overview HIGH SCALE, • Auto-Sharding, Multi-Master READS + WRITES • ACID Compliant, OLTP + Real-Time Analytics 99.999% • Shared nothing, no Single Point of Failure AVAILABILITY • Self Healing + On-Line Operations • In-Memory Optimization + Disk-Data REAL-TIME • Predictable Low-Latency, Bounded Access Time • Key/Value + Complex, Relational Queries SQL + NoSQL • SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++ • Open Source + Commercial Editions LOW TCO • Commodity hardware + Management, Monitoring Tools 3 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 4.
    Who’s Using MySQLCluster? 4 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 5.
    MySQL Cluster Architecture Clients Application Layer Management Data Layer MySQL Cluster Data Nodes 5 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 6.
    MySQL Cluster Architecture On-Line Scaling Clients Application Layer Management Management Data Layer MySQL Cluster Data Nodes 6 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 7.
    http://clusterdb.com/u/pi 7 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 8.
    MySQL Cluster Architecture No Single Point of Failure Clients Application Layer Management Management MySQL Cluster Data Nodes 8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 9.
    Automatic Data Partitioning Table T1 Data Node 1 Px Partition P1 Data Node 2 P2 P3 Data Node 3 P4 Data Node 4 9 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 10.
    Automatic Data Partitioning Table T1 Data Node 1 F1 Px Partition P1 Data Node 2 P2 P3 Data Node 3 P4 Fx Primary Fragment Data Node 4 Fx Secondary Fragment 10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 11.
    Automatic Data Partitioning Table T1 Data Node 1 F1 Px Partition P1 Data Node 2 P2 F1 P3 Data Node 3 P4 Fx Primary Fragment Data Node 4 Fx Secondary Fragment 11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 12.
    Automatic Data Partitioning Table T1 Data Node 1 F1 Px Partition P1 Data Node 2 P2 F3 F1 P3 Data Node 3 P4 Fx Primary Fragment Data Node 4 Fx Secondary Fragment 12 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 13.
    Automatic Data Partitioning Table T1 Data Node 1 F1 F3 Px Partition P1 Data Node 2 P2 F3 F1 P3 Data Node 3 P4 Fx Primary Fragment Data Node 4 Fx Secondary Fragment 13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 14.
    Automatic Data Partitioning Table T1 Data Node 1 F1 F3 Px Partition P1 Data Node 2 P2 F3 F1 P3 Data Node 3 F2 F4 P4 Fx Primary Fragment Data Node 4 F4 F2 Fx Secondary Fragment 14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 15.
    Automatic Data Partitioning Table T1 Data Node 1 F1 F3 Px Partition P1 Node Group 1 Data Node 2 P2 F3 F1 P3 Data Node 3 F2 F4 P4 Node Group 2 Fx Primary Fragment Data Node 4 F4 F2 Fx Secondary Fragment 15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 16.
    Automatic Data Partitioning Table T1 Data Node 1 F1 F3 Px Partition P1 Node Group 1 Data Node 2 P2 F3 F1 P3 Data Node 3 F2 F4 P4 Node Group 2 Fx Primary Fragment Data Node 4 F4 F2 Fx Secondary Fragment 16 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 17.
    On-line Scheduled Maintenance Scale Backup Evolve Upgrade 17 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 18.
    Multi-Data Center Availability Geographic Replication  DR and Data Locality  Replicate complete clusters across data centers – Fully active/active – No passive resources Geographic Replication  Split individual clusters across data centers – Synchronous replication & auto-failover between sites 18 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 19.
    1.2 Billion UPDATEsper Minute 25 Millions of UPDATEs per 20 Second 15 10 5 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 MySQL Cluster Data Nodes  NoSQL C++ API, flexaSynch benchmark  30 x Intel E5-2600 Intel Servers, 2 socket, 64GB  ACID Transactions, with Synchronous Replication 19 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 20.
    Real-World Test Case Query Execution Time Seconds 100 90 80 70 60 87.23 seconds 70x 50 40 Faster 30 20 10 0 1.26 seconds MySQL Cluster 7.1 MySQL Cluster 7.2  Web-Based Content Management System – JOINs 11-tables, 33.5k rows – Returns 2k rows, 19 columns per row 20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 21.
    MySQL Cluster 7.3EA: Node.js NoSQL API  Native JavaScript access to MySQL Cluster – End-to-End JavaScript: browser to the app and database – Storing and retrieving JavaScript objects directly in MySQL Cluster – Eliminate SQL transformation  Implemented as a module for node.js – Integrates full Cluster API library within the web app  Couple high performance, distributed apps, with high performance distributed database 21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 22.
    MySQL Cluster 7.3DMR1: Foreign Keys  Brings MySQL Cluster to a broader range of workloads – Packaged apps, custom projects  Adds powerful functionality while reducing complexity – App logic & data model  Enabled by default  Enforced for SQL & NoSQL APIs  On-line add and drop 22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 23.
    MySQL Cluster 7.2and 7.3: Auto-Installer Early Access Feature  Best practices New!  Fast configuration  Workload optimized Specify Auto- Workload Discover Define Deploy Topology 23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 24.
    When to ConsiderMySQL Cluster  Scalability demands  Sharding for write performance?  Latency demands  Cost of each second?  Uptime requirements  Cost per minute of downtime?  Failure versus maintenance?  Application agility  Developer languages and frameworks?  SQL or NoSQL? 24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013
  • 25.
    Next Steps Learn More • www.mysql.com/cluster • Authentic MySQL Curriculum: http://oracle.com/education/mysql Try it Out • dev.mysql.com/cluster • labs.mysql.com • github.com/mysql/mysql-js Let us know what you think • clusterdb.com • @clusterdb • forums.mysql.com/list.php?25 25 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | 3rd February 2013