What's new in MySQL Cluster 7.4 webinar charts

21,633 views

Published on

MySQL Cluster powers the subscriber databases of major communication services providers as well as next generation web, cloud, social and mobile applications. It is designed to deliver:
- Real-time, in-memory performance for both OLTP and analytics workloads
- Linear scale-out for both reads and writes
99.999% High Availability
- Transparent, cross-shard transactions and joins
- Update-Anywhere Geographic replication
- SQL or native NoSQL APIs

All that while still providing full ACID transactions.

Published in: Software

What's new in MySQL Cluster 7.4 webinar charts

  1. 1. What’s new in MySQL Cluster 7.4 In-Memory Real-Time Performance, Web Scalability & 99.999% Availability Andrew Morgan Director MySQL Product Management andrew.morgan@oracle.com @andrewmorgan www.clusterdb.com 9th April 2015 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved
  2. 2. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 2
  3. 3. Focus on driving the business rather than on infrastructure Driving new Database Requirements In-Memory Real-Time Performance Extreme Read & Write Scalability Rock Solid Availability Rapid Service Innovation 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 3
  4. 4. MySQL Cluster Overview • Memory optimized tables with durability • Predictable Low-Latency, Bounded Access Time REAL-TIME • Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations 99.999% AVAILABILITY • Key/Value + Complex, Relational Queries • SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL • Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Tools LOW TCO 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 4
  5. 5. Who’s Using MySQL Cluster? 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 5
  6. 6. MySQL Replication MySQL Fabric DRBD Windows/S olaris/Clust erware Clustering or Oracle VM MySQL Cluster MySQL HA Solutions 9th April 2015 9 9 . 9 9 9 % Copyright 2015, oracle and/or its affiliates. All rights reserved 6
  7. 7. Oracle MySQL HA & Scaling Solutions MySQL Replication MySQL Fabric Oracle VM Template Oracle Clusterware Solaris Cluster Windows Cluster DRBD MySQL Cluster App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔ Platform Support All All Linux Linux Solaris Windows Linux All Clustering Mode Master + Slaves Master + Slaves Active/Pas sive Active/Passi ve Active/P assive Active/Pas sive Active/P assive Multi- Master Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔ Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔ Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔ Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔ Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔ 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 7
  8. 8. Company Overview Global telecommunications equipment company, focuses on fixed, mobile, and converged networking hardware, IP technologies, software, and services. Application MySQL Cluster CGE is at the heart of Alcatel-Lucent’s Subscriber Data Manager to deliver converged IMS services to mobile and fixed line users. Why MySQL Cluster CGE? • Delivers the performance, scalability and availability required by this mission critical real-time application • Real-time read & write access for tens of millions of subscribers in a single system • Always-on service: no offline maintenance window for services users depend on (voice, SMS, email, Web, social media…) • Rapid delivery & low TCO Alcatel-Lucent 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 8
  9. 9. Company Overview PayPal allows any business or individual with an email address to securely, conveniently and cost-effectively send and receive payments online. Application PayPal built a cloud-based globally-distributed database with 100 TB of user-related data based on MySQL Cluster. “Must NOT lose data” system, delivering 99,999% availability, transactional, with data available WW anywhere in < 1 Sec. Why MySQL Cluster? “You can achieve high performance and availability without giving up relational models and read consistency.” Daniel Austin, Chief Architect, PayPal PayPal 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 9
  10. 10. Company Overview Playful Play created Latin America’s #1 Facebook game based on "El Chavo del 8“. Application Playful Play selected MySQL Cluster CGE running on a public cloud to power their gaming platform, providing: • 45% improvement in performance • 99.999% uptime • 80% reduction in DBA overhead • Local language support, 24x7 Why MySQL Cluster CGE? “MySQL Cluster has allowed us to keep pace with the rapid success of the game. Being the number one Facebook game in Latin America is a big responsibility, and we rely on Oracle’s products and MySQL support for continued growth.” Ricardo Rocha, CEO, Playful Play Playful Play 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 10
  11. 11. MySQL Cluster Architecture MySQL Cluster Data Nodes Clients Application Layer Data Layer 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 11
  12. 12. MySQL Cluster Node (process) Types MySQL Server • SQL Access to data • Round-robin app access • Geographic replication • Scale out for performance Data Node • Data storage & management • Transparent sharding • Synchronous replication • Scale out for capacity/perf Management Node • Distribute configuration data • Control Cluster membership • Arbitrator in case of n/w partitioning 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 12
  13. 13. Node Group Data Node 1 Data Partitioning • Data transparently sharded between Node Groups • Each fragment active in one Data Node with synchronous replication to 2nd Data Node in same Node Group 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 13 Fragment 1 Fragment 2’ Data Node 2 Fragment 1’ Fragment 2 Node Group Data Node 3 Fragment 3 Fragment 4’ Data Node 4 Fragment 3’ Fragment 4
  14. 14. Data Partitioning • DBA chooses which part of Primary Key to use as shard key • Fragment for each row decided by hashing the sharding key 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 14 User-id (PK) Service (PK) Data 1773467253 chat xxx 6257346892 chat xxx 1773467253 photos xxx 7234782739 photos xxx 8235602099 reminders xxx 8437829249 location xxx Shard Key
  15. 15. MySQL Cluster Architecture MySQL Cluster Data Nodes Clients Application Layer Data Layer 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 15
  16. 16. MySQL Cluster Scaling MySQL Cluster Data Nodes Clients Application Layer Data Layer 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 16
  17. 17. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 17
  18. 18. MySQL Cluster HA MySQL Cluster Data Nodes Clients Application Layer Data Layer 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 18
  19. 19. On-line Operations • Scale the cluster – Add all nodes – Remove MySQL Server nodes • Repartition tables • Upgrade / patch servers & OS • Upgrade / patch MySQL Cluster • Back-Up • Evolve the schema on-line, in real-time 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 19
  20. 20. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 20
  21. 21. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 21
  22. 22. MySQL Cluster 7.2: 1.2 Billion UPDATEs per Minute • NoSQL C++ API, flexaSynch benchmark • 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB • ACID Transactions, with Synchronous Replication http://www.mysql.com/why- mysql/white-papers/mysql-cluster- benchmarks-1-billion-writes-per-minute/0 5 10 15 20 25 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 MillionsofUPDATEsperSecond MySQL Cluster Data Nodes 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 22
  23. 23. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 23
  24. 24. MySQL Cluster Auto-Installer • Fast configuration • Auto-discovery • Workload optimized • Repeatable best practices Specify Workload Auto- Discover Define TopologyDeploy 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 24
  25. 25. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 25
  26. 26. MySQL Cluster Manager 1. Download MCM/Cluster package from edelivery.oracle.com: 2. Unzip 3. Run agent, define, create & start Cluster! $> binmcmd --bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862 Configuring default cluster 'mycluster'... Starting default cluster 'mycluster'... Cluster 'mycluster' started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 26
  27. 27. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 27
  28. 28. NoSQL Access to MySQL Cluster data Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps JPA Cluster JPA PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached MySQL JNI Node.JS mod_ndb ndb_eng NDB API (C++) MySQL Cluster Data Nodes
  29. 29. SQL • Industry standard • Joins & Complex queries • Relational model Memcached • Simple to use API • Key/value • Drivers for many languages Mod-ndb • REST • Html • Plugin for Apache ClusterJ • Simple to Use Java API • Web & telco • Object Relational Mapping • Native & fast access to data ClusterJPA • OpenJPA plugin • Standards defined ORM • Cross table Joins JavaScript/Node.js • Native JavaScript: client to DB • Blazing fast asynchronous throughput Choosing the right application API 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 29
  30. 30. Schema-Free apps • Rapid application evolution – New types of data constantly added – No time to get schema extended – Missing skills to extend schema – Initially roll out to just a few users – Constantly adding to live system 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 30
  31. 31. Cluster & Memcached – Schema-Free <town:maidenhead,SL6> key value <town:maidenhead,SL6> key value Key Value town:maidenhead SL6 generic table Application view SQL view 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 31
  32. 32. Cluster & Memcached - Configured Schema <town:maidenhead,SL6> prefix key value <town:maidenhead,SL6> key value Prefix Table Key-col Val-col policy town: map.zip town code cluster Config tables town ... code ... maidenhead ... SL6 ... map.zip Application view SQL view 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 32
  33. 33. Node.js NoSQL API • Native JavaScript access to MySQL Cluster – End-to-End JavaScript: browser to the app & DB – Storing and retrieving JavaScript objects directly in MySQL Cluster – Eliminate SQL transformation • Implemented as a module for node.js – Integrates Cluster API library within the web app • Couple high performance, distributed apps, with high performance distributed database • Optionally routes through MySQL Server – Use with InnoDB V8 JavaScript Engine MySQL Cluster Node.js Module MySQL Cluster Data Nodes Clients 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 33
  34. 34. NoSQL API for Node.js & FKs FKs enforced on all APIs: { message: 'Error', sqlstate: '23000', ndb_error: null, cause: {message: 'Foreign key constraint violated: No parent row found [255]', sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } } 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 34
  35. 35. MySQL Cluster 7.4 GA 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 35
  36. 36. MySQL Cluster 7.4 GA • 200 Million NoSQL Reads/Sec • 2.5M SQL Ops/Sec • 50% Faster Reads • 40% Faster Mixed Performance • Active-Active Geographic Redundancy • Conflict Detection/Resolution Active-Active • 5X Faster Maintenance Ops • Detailed Reporting Management 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 36
  37. 37. • Memory optimized tables – Durable – Mix with disk-based tables • Massively concurrent OLTP • Distributed Joins for analytics • Parallel table scans for non-indexed searches • MySQL Cluster 7.4 FlexAsych – 200M NoSQL Reads/Second 16/04/2015 37 MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second Copyright 2015, oracle and/or its affiliates. All rights reserved - 50,000,000 100,000,000 150,000,000 200,000,000 250,000,000 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 Readspersecond Data Nodes FlexAsync Reads
  38. 38. • Memory optimized tables – Durable – Mix with disk-based tables • Massively concurrent OLTP • Distributed Joins for analytics • Parallel table scans for non-indexed searches • MySQL Cluster 7.4 DBT2 BM – 2.5M SQL Statements/Second 16/04/2015 38 MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second Copyright 2015, oracle and/or its affiliates. All rights reserved - 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 2 4 6 8 10 12 14 16 SQLStatements/sec Data Nodes DBT2 SQL Statements per Second
  39. 39. Performance Enhancements MySQL Cluster 7.4 50% Read-Only Increase 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 39 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 64 128 192 256 320 384 448 512 Transaconspersecond Threads Sysbench R/W 7.4 7.3 7.2 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 64 128 192 256 320 384 448 512 Transaconpersecond Threads Sysbench RO 7.4 7.3 7.2 40% Read/Write Increase
  40. 40. • Asynchronous replication between MySQL Clusters • Active-Active – Update anywhere – Conflict detection • Application notified through exception tables • Can opt to have conflicts resolved automatically – Auto-conflict-resolution • Conflicting transaction and dependent ones are rolled-back • No changes to application schema Active-Active Geo-Replication 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 40
  41. 41. What is a conflict? John.balance==$100 John.balance-=$40 John.balance==$60 John.balance==$200 John.balance==$100 John.balance+= $100 John.balance==$200 John.balance==$60 Spend $40 Add $100 $60 $200 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 41
  42. 42. • NDB$EPOCH2 and NDB$EPOCH2_TRANS introduced • Detects conflicting inserts/updates/deletes/reads • Entire transactions (and dependent ones) rolled back • Rolling back of transactions that read conflicted data • Improved NDB Exceptions table format – Non-PK columns, operation type, transaction id, before and after values • Online conflict role change 9th April 2015 42 Handling of Conflicts – Extensions in MySQL Cluster 7.4 Copyright 2015, oracle and/or its affiliates. All rights reserved
  43. 43. • Primary stores logical timestamp (GCI) against updated row – Window for conflict opens • GCI replicated with updated row to Secondary • The same row and GCI is replicated back (reflected) from Secondary to Primary after it has been applied – Closing window for conflict • Primary checks every event originating from the Secondary to ensure it isn’t for a ‘conflictable’ row Detecting Conflicts - Reflected GCI John.balance==$100 John.balance-=$40 John.balance==$60 John.balance==$200 John.balance==$100 John.balance+= $100 John.balance==$200 John.balance==$60 Spend $40 Add $100 $60 $200 Primary Secondary 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 43
  44. 44. How to Use Conflict Detection/Resolution Decide which tables need protecting For each table, specify what to do on conflicts Just record in exception table Application or DBA acts on content Rollback the conflicting row Rollback the conflicting transaction 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 44
  45. 45. MySQL Cluster 7.4 Restart Improvements • Duration of long-running maintenance activities dominated by Data Node restart times • MySQL Cluster 7.4 = 5.5X faster restarts • Benefits both nodal and rolling restarts – Upgrades, add-node,… • Benefits both SQL and NoSQL APIs • Benefits both ”manual” and MySQL Cluster Manager operations • Achieve 5x as much during a single maintenance window Make Data Node Restarts Fast! 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 45
  46. 46. MySQL Cluster 7.4 Restart Improvements • Verbose logging – Task start/completion – Data volumes – Parallelism & Wait times • NDBINFO for recent node restarts • More documentation of stages • Goal: Make analysis of a slow restart possible – Determine cause; Detect patterns; Understand the impact of indexes, local checkpoints etc. Observability improvements 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 46
  47. 47. 2014-11-12 16:30:47 [ndbd] INFO -- Start phase 0 completed 2014-11-12 16:30:47 [ndbd] INFO -- Phase 0 has made some file system initialisations 2014-11-12 16:30:47 [ndbd] INFO -- Starting QMGR phase 1 2014-11-12 16:30:47 [ndbd] INFO -- DIH reported initial start, now starting the Node Inclusion Protocol 2014-11-12 16:30:50 [ndbd] INFO -- findNeighbours from: 2220 old (left: 65535 right: 65535) new (3 1) 2014-11-12 16:30:50 [ndbd] INFO -- Include node protocol completed, phase 1 in QMGR completed 2014-11-12 16:30:50 [ndbd] INFO -- Start phase 1 completed 2014-11-12 16:30:50 [ndbd] INFO -- Phase 1 initialised some variables and included node in cluster, locked memory if configured to do so 2014-11-12 16:30:50 [ndbd] INFO -- Asking master node to accept our start (nodeId = 1 is master), GCI = 0 2014-11-12 16:30:50 [ndbd] INFO -- findNeighbours from: 2132 old (left: 3 right: 1) new (3 2) 2014-11-12 16:30:50 [ndbd] INFO -- NDBCNTR master accepted us into cluster, start NDB start phase 1 2014-11-12 16:30:50 [ndbd] INFO -- We are performing initial start of cluster 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Starting REDO log initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Starting REDO log initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Starting REDO log initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Starting REDO log initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Completed REDO log initialisation of logPart = 0 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Completed REDO initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Completed REDO log initialisation of logPart = 0 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Completed REDO initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Completed REDO log initialisation of logPart = 0 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Completed REDO initialisation 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Completed REDO log initialisation of logPart = 0 2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Completed REDO initialisation 2014-11-12 16:30:50 [ndbd] INFO -- Schema file initialisation Starting 2014-11-12 16:30:50 [ndbd] INFO -- Schema file initialisation Completed 2014-11-12 16:30:51 [MgmtSrvr] INFO -- Node 1: Make On-line Database recoverable by waiting for LCP Starting, LCP id = 1 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Local checkpoint 1 started. Keep GCI = 1 oldest restorable GCI = 1 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 1: Completed LCP, num fragments = 4 num records = 528, num bytes = 16976 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 2: Completed LCP, num fragments = 4 num records = 471, num bytes = 15152 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 3: Completed LCP, num fragments = 4 num records = 536, num bytes = 17232 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 4: Completed LCP, num fragments = 4 num records = 513, num bytes = 16496 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 2: Completed LCP, num fragments = 4 num records = 471, num bytes = 15152 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 3: Completed LCP, num fragments = 4 num records = 536, num bytes = 17232 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 1: Completed LCP, num fragments = 4 num records = 528, num bytes = 16976 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 4: Completed LCP, num fragments = 4 num records = 513, num bytes = 16496 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Local checkpoint 1 completed 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Make On-line Database recoverable by waiting for LCP Completed, LCP id = 1 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 5 completed (initial start) … 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 6 completed (initial start) … 2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 7 completed (initial start) 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 47
  48. 48. Restart Times • Restarting data node with locally checkpointed data – Major improvement • Restarting data node which must recover data from peer – Major improvement – Further speedups to come in 7.4.X (greater parallelization) • Upgrade/rolling restarts – Major improvement • Cluster shutdown and restart – Minor improvement What operations benefit? 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 48
  49. 49. MySQL Cluster 7.4 Restart Improvements • Faster Local Checkpoints – Parallel – Tuning & Adaptive throttling • Parallel operations – Adaptive to avoid starving remaining system of resources • Feedback loop – Determine Cluster 7.3 baseline for your environment – Experiment with Cluster 7.4 settings – Consult Oracle with results to establish improvements to be made in configuration Speedups 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 49
  50. 50. MySQL Cluster 7.4 – Enhanced memory reporting • ndbinfo.memory_per_fragment memory usage information for each fragment replica, for each table and index • Allocated memory and how much of that is actually in use. • Exposes – Fragmentation of fixed and var-sized fragment pages – Accurate Data and Index Memory use – Comparison of Primary and Backup fragment usage – Partitioning effectiveness 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 50
  51. 51. mysql> desc ndbinfo.memory_per_fragment; +------------------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------------------+---------------------+------+-----+---------+-------+ | fq_name | varchar(512) | YES | | NULL | | | parent_fq_name | varchar(512) | YES | | NULL | | | type | varchar(512) | YES | | NULL | | | table_id | int(10) unsigned | YES | | NULL | | | node_id | int(10) unsigned | YES | | NULL | | | block_instance | int(10) unsigned | YES | | NULL | | | fragment_num | int(10) unsigned | YES | | NULL | | | fixed_elem_alloc_bytes | bigint(20) unsigned | YES | | NULL | | | fixed_elem_free_bytes | bigint(20) unsigned | YES | | NULL | | | fixed_elem_size_bytes | int(10) unsigned | YES | | NULL | | | fixed_elem_count | bigint(20) unsigned | YES | | NULL | | | fixed_elem_free_count | decimal(16,0) | YES | | NULL | | | var_elem_alloc_bytes | bigint(20) unsigned | YES | | NULL | | | var_elem_free_bytes | bigint(20) unsigned | YES | | NULL | | | var_elem_count | bigint(20) unsigned | YES | | NULL | | | hash_index_alloc_bytes | bigint(20) unsigned | YES | | NULL | | +------------------------+---------------------+------+-----+---------+-------+ 16 rows in set (0.00 sec) 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 51
  52. 52. Enhanced Memory Reporting mysql> CREATE DATABASE clusterdb;USE clusterdb; mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY) ENGINE=NDB; mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes, fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from memory_per_fragment where fq_name like '%simples%'; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1280 | 40 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1280 | 40 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4256 | 133 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4256 | 133 | +------+------+-------------+------------+------------+ See how much memory a table is using 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 52
  53. 53. Enhanced Memory Reporting +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1280 | 40 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1280 | 40 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4256 | 133 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4256 | 133 | +------+------+-------------+------------+------------+ mysql> DELETE FROM clusterdb.simples LIMIT 1; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 131072 | 5504 | 172 | | 1 | 2 | 131072 | 1312 | 41 | | 2 | 0 | 131072 | 5504 | 172 | | 2 | 2 | 131072 | 1312 | 41 | | 3 | 1 | 131072 | 3104 | 97 | | 3 | 3 | 131072 | 4288 | 134 | | 4 | 1 | 131072 | 3104 | 97 | | 4 | 3 | 131072 | 4288 | 134 | +------+------+-------------+------------+------------+ See how memory is made available after deleting rows 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 53
  54. 54. Enhanced Memory Reporting mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT, species VARCHAR(20) DEFAULT "Human", PRIMARY KEY(id, species)) engine=ndb PARTITION BY KEY(species); // Add some data mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes, fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from ndbinfo.memory_per_fragment where fq_name like '%simples%'; +------+------+-------------+------------+------------+ | node | frag | alloc_bytes | free_bytes | spare_rows | +------+------+-------------+------------+------------+ | 1 | 0 | 0 | 0 | 0 | | 1 | 2 | 196608 | 11732 | 419 | | 2 | 0 | 0 | 0 | 0 | | 2 | 2 | 196608 | 11732 | 419 | | 3 | 1 | 0 | 0 | 0 | | 3 | 3 | 0 | 0 | 0 | | 4 | 1 | 0 | 0 | 0 | | 4 | 3 | 0 | 0 | 0 | +------+------+-------------+------------+------------+ Check how well partitioned/sharded a table is 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 54
  55. 55. MySQL Cluster 7.4 – Enhanced activity reporting • ndbinfo.operations_per_fragment activity counters for each fragment replica, for each table and index • PK & scan access – requests, bytes, rows… • Exposes – How traffic maps to tables and indices – Query execution, use of indexes etc. – LDM and node imbalances – Hotspots and scan overloads 16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 55
  56. 56. mysql> desc operations_per_fragment; +-------------------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------------------+---------------------+------+-----+---------+-------+ | fq_name | varchar(512) | YES | | NULL | | | parent_fq_name | varchar(512) | YES | | NULL | | | type | varchar(512) | YES | | NULL | | | table_id | int(10) unsigned | YES | | NULL | | | node_id | int(10) unsigned | YES | | NULL | | | block_instance | int(10) unsigned | YES | | NULL | | | fragment_num | int(10) unsigned | YES | | NULL | | | tot_key_reads | bigint(20) unsigned | YES | | NULL | | | tot_key_inserts | bigint(20) unsigned | YES | | NULL | | | tot_key_updates | bigint(20) unsigned | YES | | NULL | | | tot_key_writes | bigint(20) unsigned | YES | | NULL | | | tot_key_deletes | bigint(20) unsigned | YES | | NULL | | | tot_key_refs | bigint(20) unsigned | YES | | NULL | | | tot_key_attrinfo_bytes | bigint(20) unsigned | YES | | NULL | | | tot_key_keyinfo_bytes | bigint(20) unsigned | YES | | NULL | | | tot_key_prog_bytes | bigint(20) unsigned | YES | | NULL | | | tot_key_inst_exec | bigint(20) unsigned | YES | | NULL | | | tot_key_bytes_returned | bigint(20) unsigned | YES | | NULL | | | tot_frag_scans | bigint(20) unsigned | YES | | NULL | | | tot_scan_rows_examined | bigint(20) unsigned | YES | | NULL | | | tot_scan_rows_returned | bigint(20) unsigned | YES | | NULL | | | tot_scan_bytes_returned | bigint(20) unsigned | YES | | NULL | | | tot_scan_prog_bytes | bigint(20) unsigned | YES | | NULL | | | tot_scan_bound_bytes | bigint(20) unsigned | YES | | NULL | | | tot_scan_inst_exec | bigint(20) unsigned | YES | | NULL | | | tot_qd_frag_scans | bigint(20) unsigned | YES | | NULL | | | conc_frag_scans | int(10) unsigned | YES | | NULL | | | conc_qd_frag_scans | bigint(11) unsigned | YES | | NULL | | | tot_commits | bigint(20) unsigned | YES | | NULL | |16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 56
  57. 57. When to Consider MySQL Cluster  Scalability demands  Sharding for write performance?  Latency demands  Cost of each millisecond?  Uptime requirements  Cost per minute of downtime?  Failure versus maintenance?  Application agility  Developer languages and frameworks?  SQL or NoSQL? 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 57
  58. 58. General Usage Considerations • MySQL Cluster is designed for – Short transactions – Many parallel transactions • Utilize simple access patterns for high running transactions – Use efficient scans and batching interfaces – AQL gives huge performance in JOIN operations • Storage engine configurable for each table…InnoDB or NDB MySQL Cluster Evaluation Guide http://mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 58
  59. 59. MySQL Cluster CGE MySQL Cluster Manager MySQL Enterprise Scalability MySQL Enterprise Audit MySQL Enterprise Security Oracle Premier Lifetime Support Oracle Product Certifications MySQL Enterprise Mornitor MySQL Workbench 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 59
  60. 60. Enhancing DevOps Agility, Reducing Downtime Automated Management • Start / Stop node or whole cluster • On-Line Scaling • On-Line Reconfiguration • On-Line Upgrades • On-Line Backup & Restore • Import Running Cluster Self-Healing • Node monitoring • Auto-recovery extended to SQL + mgmt nodes HA Operations • Cluster-wide configuration consistency • Persistent configurations • HA Agents 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 60
  61. 61. Without MySQL Cluster Manager • 1 x preliminary check of cluster state • 8 x ssh commands per server • 8 x per-process stop commands • 4 x scp of config files (2 x mgmd & 2 x mysqld) • 8 x per-process start commands • 8 x checks for started and re-joined processes • 8 x process completion verifications • 1 x verify completion of the whole cluster. • Excludes manual editing of each configuration file. • Total: 46 commands – 2.5 hours of attended operation With MySQL Cluster Manager mcm> upgrade cluster --package=7.4 mycluster; • Total: 1 Command - – Unattended Operation Software Upgrade – example of benefits of MCM? 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 61
  62. 62. MySQL Cluster Manager 1.3.3 GA “Unmanaged” production Cluster mcm> create cluster --import mcm> import config [--dryrun] mcm> import cluster[--dryrun] Cluster now managed by MCM Import a running Cluster into MCM 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 62
  63. 63. Oracle Premier Support for MySQL • Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support Rely on The Experts - Get Unique Benefits "The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com Only From Oracle 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 63
  64. 64. Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications “Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013 Premier Support customers eligible to save 20% on learning credits. Benefits  Expert-led training to support your MySQL learning needs  Flexibility to train in the classroom or online  Hands-on experience to gain real world experience  Key skills needed for database administrators and developers • MySQL for Beginners  MySQL for Database Administrators  MySQL Performance Tuning  MySQL Cluster – NEW - Register Your Interest!  MySQL and PHP - Developing Dynamic Web Applications  MySQL for Developers  MySQL Developer Techniques  MySQL 5.6 Database Administrator  MySQL 5.6 Developer To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql Top Courses for Administrators and Developers Top Certifications RECENTLY RELEASED !!ALL NEW!! MySQL Cluster Training To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the MySQL Cluster Course 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 64
  65. 65. Next Steps Learn More • www.mysql.com/cluster • Authentic MySQL Curriculum: http://oracle.com/education/mysql Try it Out • dev.mysql.com/downloads/cluster/ Let us know what you think • clusterdb.com • @clusterdb • forums.mysql.com/list.php?25 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 65

×