Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4)

9,471 views

Published on

MySQL Cluster is the distributed, shared-nothing version of MySQL. It’s typically used for applications that need any combination of high availability, real-time performance, and scaling of reads and writes. After a brief introduction to the technology, its uses, and the new features added in MySQL Cluster 7.3, this session focuses on the very latest developments happening in MySQL Cluster 7.4. As you’d expect from a real-time, scalable, distributed, in-memory database, performance continues to be a top priority, as do simplicity of use and robustness. Come hear firsthand what’s being done to make sure MySQL Cluster continues to dominate in mission-critical, high-performance applications.

Published in: Software
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/SqL06 ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/SqL06 ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4)

  1. 1. MySQL Cluster Dive into the Latest Developments Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Andrew Morgan @andrewmorgan clusterdb.com Bernhard Ocklin bernd.ocklin@oracle.com 2nd October 2014
  2. 2. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  3. 3. Today’s Database Requirements Extreme Write Scalability Real-Time User Experience Rock Solid Reliability Rapid Service Innovation Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  4. 4. Today’s Database Requirements Transactional Integrity OLTP + Real-Time Analytics Extreme Write Scalability Real-Time User Experience Standards & Skillsets Rock Solid Reliability Rapid Service Innovation Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  5. 5. Who’s Using MySQL Cluster Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  6. 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Overview • Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations 99.999% AVAILABILITY • In-Memory Optimization + Disk-Data • Predictable Low-Latency, Bounded Access Time REAL-TIME • Key/Value + Complex, Relational Queries • SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL • Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Tools LOW TCO
  7. 7. MySQL Cluster Architecture MySQL Cluster Data Nodes Clients Application Layer Data Layer Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  8. 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Scaling MySQL Cluster Data Nodes Clients Application Layer Data Layer
  9. 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  10. 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster HA MySQL Cluster Data Nodes Clients Application Layer Data Layer
  11. 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  12. 12. Scale On-Line Operations Backup Evolve Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Upgrade
  13. 13. MySQL Cluster 7.3: 1.2 Billion UPDATEs per Minute • NoSQL C++ API, flexaSynch benchmark • 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB • ACID Transactions, with Synchronous Replication Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 25 20 15 10 5 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 Millions of UPDATEs per Second MySQL Cluster Data Nodes
  14. 14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  15. 15. NoSQL Access to MySQL Cluster data Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps JPA Cluster JPA PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached MySQL JNI Node.JS mod_ndb ndb_eng NDB API (C++) MySQL Cluster Data Nodes Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  16. 16. Cluster & Memcached – Schema-Free key value <town:maidenhead,SL6> key value <town:maidenhead,SL6> Key Value town:maidenhead SL6 generic table Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Application view SQL view
  17. 17. Cluster & Memcached - Configured Schema key value <town:maidenhead,SL6> prefix key value <town:maidenhead,SL6> Prefix Table Key-col Val-col policy town: map.zip town code cluster Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Config tables town ... code ... maidenhead ... SL6 ... map.zip Application view SQL view
  18. 18. Real-Time Analytics - Adaptive Query Localization • Scale Complex Queries across Shards • JOINs pushed down to data nodes • Executed in parallel • Returns single result set to MySQL • Opens Up New Use-Cases • Real-time analytics • Recommendations engines • Fraud-detection Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | mysqld Data Nodes mysqld A Q L Data Nodes DON’T COMPROMISE FUNCTIONALITY TO SCALE-OUT !!
  19. 19. Query Execution Time Seconds 87.23 seconds MySQL Cluster 7.1 MySQL Cluster 7.2 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Real-World Test Case 100 90 80 70 60 50 40 30 20 10 0 • Web-Based Content Management System – JOINs 11-tables, 33.5k rows – Returns 2k rows, 19 columns per row 1.26 seconds 70x Faster
  20. 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |  Foreign Key Support  MySQL 5.6  Auto-Installer  NoSQL JavaScript for node.js
  21. 21. MySQL Cluster 7.3: Foreign Keys • Brings MySQL Cluster to a broader range of workloads – Packaged apps, custom projects • Adds powerful functionality while reducing complexity – App logic & data model • Enabled by default • Enforced for SQL & NoSQL APIs • On-line add and drop Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  22. 22. MySQL Cluster 7.3: Auto-Installer Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Fast configuration • Auto-discovery •Workload optimized • Repeatable best practices Specify Workload Auto- Discover Define Deploy Topology
  23. 23. MySQL Cluster 7.3: Connection Thread Scalability  Increases throughput of each connection to the data layer  More client threads can use each connection by splitting mutexes Reduces configuration complexity  Increases cluster scalability  Each connection consumes one of 256 node ids Up to 7.5x higher performance per connection  SQL & NoSQL interfaces Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Data Nodes MySQL Cluster Data Nodes With Connection Thread Scalability
  24. 24. MySQL Cluster 7.3: Connection Thread Scalability  DBT2 Benchmark – Single MySQL Server – Single Data Node – 128 client connections Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 8.5x
  25. 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Node.js NoSQL API • Native JavaScript access to MySQL Cluster – End-to-End JavaScript: browser to the app & DB – Storing and retrieving JavaScript objects directly in MySQL Cluster – Eliminate SQL transformation • Implemented as a module for node.js – Integrates Cluster API library within the web app • Couple high performance, distributed apps, with high performance distributed database • Optionally routes through MySQL Server V8 JavaScript Engine MySQL Cluster Node.js Module MySQL Cluster Data Nodes Clients
  26. 26. MySQL Cluster 7.4.1 DMR Available Now! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  27. 27. MySQL Cluster 7.4.1 DMR Better performance and operational simplicity Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Performance gain over 7.3 – 47% (Read-Only) – 38% (Read-Write) • Faster maintenance operations – Nodal & Rolling restarts – Upgrades
  28. 28. • Asynchronous replication between MySQL Clusters • Active-Active – Update anywhere – Conflict detection • Application notified through exception tables • Can opt to have conflicts resolved automatically – Auto-conflict-resolution • Conflicting transaction and dependent ones are rolled-back • No changes to application schema Active-Active Geo-Replication Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  29. 29. Spend $40 Add $100 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | What is a conflict? John.balance==$100 John.balance-=$40 John.balance==$60 John.balance==$200 John.balance==$100 John.balance+= $100 John.balance==$200 John.balance==$600 $60 $200
  30. 30. MySQL Cluster 7.4.1 DMR • Detects conflicting inserts/updates • Entire transactions (and dependent ones) rolled back • All conflicts are handled before switching primary Later in MySQL Cluster 7.4 • Conflicting deletes • Rolling back of transactions that read conflicted data Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Handling of Conflicts
  31. 31. Detecting Conflicts - Reflected GCI • Primary store logical timestamp (GCI) against updated row – Window for conflict opens • GCI replicated with updated row to Secondary • The same row and GCI is replicated back (reflected) from Secondary to Primary after it has been applied – Closing window for conflict • Primary checks every event originating from the Secondary to ensure it isn’t for a ‘conflictable’ row Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Primary Secondary John.balance==$100 John.balance-=$40 John.balance==$60 John.balance==$200 John.balance==$100 John.balance+= $100 John.balance==$200 John.balance==$600 Spend $40 Add $100 $60 $200
  32. 32. How to Use Conflict Detection/Resolution Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Decide which tables need protecting For each table, specify what to do on conflicts Just record in exception table Application or DBA acts on content Rollback the conflicting row Rollback the conflicting transaction
  33. 33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Restart Times What operations benefit? • Restarting data node with locally checkpointed data – Major improvement • Restarting data node which must recover data from peer – Major improvement – Further speedups to come in 7.4.X (greater parallelization) • Upgrade/rolling restarts – Major improvement • Cluster shutdown and restart – Minor improvement
  34. 34. Enhanced Memory Reporting mysql> CREATE DATABASE clusterdb;USE clusterdb; mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY) ENGINE=NDB; mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes, fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from memory_per_fragment where fq_name like '%simples%'; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1280 | 40 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1280 | 40 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4256 | 133 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4256 | 133 | +------+------+-------------+------------+------------+ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | See how much memory a table is using
  35. 35. Enhanced Memory Reporting See how memory is made available after deleting rows +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1280 | 40 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1280 | 40 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4256 | 133 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4256 | 133 | +------+------+-------------+------------+------------+ mysql> DELETE FROM clusterdb.simples LIMIT 1; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1312 | 41 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1312 | 41 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4288 | 134 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4288 | 134 | +------+------+-------------+------------+------------+ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  36. 36. Enhanced Memory Reporting Check how well partitioned/sharded a table is mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT, species VARCHAR(20) DEFAULT "Human", PRIMARY KEY(id, species)) engine=ndb PARTITION BY KEY(species); Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | // Add some data mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes, fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from ndbinfo.memory_per_fragment where fq_name like '%simples%'; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 0 | 0 | 0 | | 1 | 2 | 196608 | 11732 | 419 | | 2 | 0 | 0 | 0 | 0 | | 2 | 2 | 196608 | 11732 | 419 | | 3 | 1 | 0 | 0 | 0 | | 3 | 3 | 0 | 0 | 0 | | 4 | 1 | 0 | 0 | 0 | | 4 | 3 | 0 | 0 | 0 | +------+------+-------------+------------+------------+
  37. 37. Oracle MySQL HA & Scaling Solutions Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Replication MySQL Fabric Oracle VM Template Oracle Clusterware Solaris Cluster Windows Cluster DRBD MySQL Cluster App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔ Platform Support All All Linux Linux Solaris Windows Linux All Clustering Mode Master + Slaves Master + Slaves Active/Pas sive Active/Passi ve Active/P assive Active/Pas sive Active/P assive Multi- Master Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔ Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔ Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔ Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔ Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔ 37
  38. 38. When to Consider MySQL Cluster • Scalability demands – Sharding for write performance? • Latency demands – Cost of each millisecond? • Uptime requirements – Cost per minute of downtime? – Failure versus maintenance? • Application agility – Developer languages and frameworks? – SQL or NoSQL? Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  39. 39. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  40. 40. MySQL Cluster Auto-Installer Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Fast configuration • Auto-discovery •Workload optimized • Repeatable best practices Specify Workload Auto- Discover Define Deploy Topology
  41. 41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  42. 42. Enhancing DevOps Agility, Reducing Downtime Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Manager Automated Management • Start / Stop node or whole cluster • On-Line Scaling • On-Line Reconfiguration • On-Line Upgrades • On-Line Backup & Restore • Import Running Cluster Self-Healing • Node monitoring • Auto-recovery extended to SQL + mgmt nodes HA Operations • Cluster-wide configuration consistency • Persistent configurations • HA Agents
  43. 43. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Upgrade using MCM MySQL Cluster Data Nodes Application Layer Data Layer upgrade cluster --package=7.3 mycluster;
  44. 44. MySQL Cluster Manager 1.3.2 GA Available Now! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  45. 45. MySQL Cluster Manager 1.3.2 GA “Unmanaged” production Cluster mcm> create cluster --import mcm> import config [--dryrun] mcm> import cluster[--dryrun] Cluster now managed by MCM Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Import a running Cluster into MCM
  46. 46. Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications Top Courses for Administrators and Developers Top Certifications Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | “Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013 Premier Support customers eligible to save 20% on learning credits. Benefits  Expert-led training to support your MySQL learning needs  Flexibility to train in the classroom or online  Hands-on experience to gain real world experience  Key skills needed for database administrators and developers • MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster – NEW - Register Your Interest! MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques MySQL 5.6 Database Administrator MySQL 5.6 Developer To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql RECENTLY RELEASED !!ALL NEW!! MySQL Cluster Training To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the MySQL Cluster Course
  47. 47. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Next Steps Learn More • www.mysql.com/cluster • Authentic MySQL Curriculum: http://oracle.com/education/mysql Try it Out • dev.mysql.com/downloads/cluster/ Let us know what you think • bugs.mysql.com • forums.mysql.com/list.php?25

×