Download as PDF, PPTX
![Faster Joins – Pushdown Joins
• AcGvated when ndb_join_pushdown is on (default)
• Rules for a join to be pushed down:
1. Joined columns use idenGcal types
2. No reference to BLOB or TEXT columns
3. No explicit lock
4. Child tables in the Join must be accessed using ref, eq_ref, or const
5. Tables not explicitly parGGoned by [LINEAR] HASH, LIST, or RANGE
6. Query execuGon plan doesn’t use mysqld’s internal ‘join buffer'
7. If root of join is of the eq_ref or const type, child tables must be joined by eq_ref
• Run ANALYZE TABLE <table_name> on each table once
• Use EXPLAIN to see what components are being pushed down:
– For example: Extra: Child of 'd' in pushed join@1
Copyright 2016, oracle and/or its affiliates. All rights reserved 43](https://image.slidesharecdn.com/mysqlcluster-whatsnew-and-bestpractices-171025080633/85/MySQL-Cluster-Whats-New-43-320.jpg)
![MySQL Cluster Manager 1.3 GA
“Unmanaged” producGon Cluster
mcm> create cluster --import
mcm> import config [--dryrun]
mcm> import cluster [--dryrun]
Cluster now managed by MCM
Import a running Cluster into MCM
Copyright 2016, oracle and/or its affiliates. All rights reserved 74](https://image.slidesharecdn.com/mysqlcluster-whatsnew-and-bestpractices-171025080633/85/MySQL-Cluster-Whats-New-74-320.jpg)
Uploaded byMark Swarbrick
MySQL Cluster Whats New
This document provides an overview and summary of MySQL Cluster, including: - Key features of MySQL Cluster such as high performance, availability, and scalability - Examples of major companies that use MySQL Cluster such as PayPal, Big Fish, and Alcatel-Lucent - New capabilities in MySQL Cluster 7.4 such as improved performance, active-active replication between clusters, and enhanced conflict detection and resolution for multi-site deployments
More Related Content
![[스마트스터디]MongoDB 의 역습](https://cdn.slidesharecdn.com/ss_thumbnails/mongodb-171107063223-thumbnail.jpg?width=640&height=640&fit=bounds)
MySQL Cluster Whats New
- 1.
- 2.
- 3. MySQL Cluster Overview • Memory opGmized tables with durability • Predictable Low-Latency, Bounded Access Time REAL-TIME • Auto-Sharding, MulG-Master • ACID Compliant, OLTP + Real-Time AnalyGcs HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-Line OperaGons 99.999% AVAILABILITY • Key/Value + Complex, RelaGonal Queries • SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL • Open Source + Commercial EdiGons • Commodity hardware + Management, Monitoring Tools LOW TCO Copyright 2016, oracle and/or its affiliates. All rights reserved 3
- 4. MySQL Replica0on MySQL Fabric DRBD Solaris Cluster, Clusterware or Oracle VM MySQL Cluster MySQL HA SoluGons 9 9 . 9 9 9 % Copyright 2016, oracle and/or its affiliates. All rights reserved 4
- 5. Oracle MySQL HA & Scaling SoluGons Copyright 2016, oracle and/or its affiliates. All rights reserved 5 MySQL Replica0on MySQL Fabric Oracle VM Oracle Clusterware Solaris Cluster MySQL Cluster App Auto-Failover MySQL Router ✔ ✔ ✔ ✔ ✔ Data Layer Auto-Failover MySQL Router ✔ ✔ ✔ ✔ ✔ Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ Plaaorm Support All All All All Solaris All Clustering Mode Master + Slaves Master + Slaves Ac0ve/Passive Ac0ve/Passive Ac0ve/Passive Mul0-Master Failover Time Secs Secs Secs + Secs + Secs + < 1 Sec Scale-out Reads ✔ ✖ ✖ ✖ ✔ Cross-shard operaGons N/A ✖ N/A N/A N/A ✔ Transparent rouGng MySQL Router For HA ✔ ✔ ✔ ✔ Shared Nothing ✔ ✔ ✖ ✖ ✖ ✔ Storage Engine InnoDB InnoDB InnoDB InnoDB InnoDB NDB Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✔
- 6.
- 7.
- 8.
- 9. Company Overview Global telecommunicaGons equipment company, focuses on fixed, mobile, and converged networking hardware, IP technologies, sopware, and services. Applica0on MySQL Cluster CGE is at the heart of Alcatel-Lucent’s Subscriber Data Manager used to deliver converged IMS services to mobile and fixed line users. Why MySQL Cluster CGE? • Delivers the performance, scalability, and availability required by this mission criGcal real-Gme applicaGon. • Real-Gme read & write access for tens of millions of subscribers within a single system. • Always-on service: no offline maintenance window for services users depend on (voice, SMS, email, Web, social media, …) • Rapid delivery & low TCO Alcatel-Lucent Copyright 2016, oracle and/or its affiliates. All rights reserved 9
- 10.
- 11.
- 12.
- 13. NoSQL Access to MySQL Cluster Data Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps JPA Cluster JPA PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached MySQL JNI Node.JS mod_ndb ndb_eng NDB API (C++) MySQL Cluster Data Nodes 13 Copyright 2016, oracle and/or its affiliates. All rights reserved
- 14.
- 15.
- 16. • Memory opGmized tables – Durable – Mix with disk-based tables • Massively concurrent OLTP • Distributed joins for analyGcs • Parallel index scans • Parallel table scans (for non- indexed searches) 16 MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second Copyright 2016, oracle and/or its affiliates. All rights reserved !"!!!! !50,000,000!! !100,000,000!! !150,000,000!! !200,000,000!! !250,000,000!! 2! 4! 6! 8! 10! 12! 14! 16! 18! 20! 22! 24! 26! 28! 30! 32! Reads&per&second& Data&Nodes& FlexAsync&Reads&
- 17. • Memory opGmized tables – Durable – Mix with disk-based tables • Massively concurrent OLTP • Distributed joins for analyGcs • Parallel index scans • Parallel table scans (for non- indexed searches) 17 MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second Copyright 2016, oracle and/or its affiliates. All rights reserved !"!!!! !500,000!! !1,000,000!! !1,500,000!! !2,000,000!! !2,500,000!! !3,000,000!! 2! 4! 6! 8! 10! 12! 14! 16! SQL$Statements/sec$ Data$Nodes$ DBT2$SQL$Statements$per$Second$
- 18. Performance Enhancements MySQL Cluster7.4 50% Read-Only Increase Copyright 2016, oracle and/or its affiliates. All rights reserved 18 0" 1000" 2000" 3000" 4000" 5000" 6000" 7000" 8000" 9000" 64" 128" 192" 256" 320" 384" 448" 512" Transac'ons)per)second) Threads) Sysbench)R/W) 7.4" 7.3" 7.2" 0" 1000" 2000" 3000" 4000" 5000" 6000" 7000" 8000" 9000" 10000" 64" 128" 192" 256" 320" 384" 448" 512" Transac'on)per)second) Threads) Sysbench)RO) 7.4" 7.3" 7.2" 40% Read/Write Increase
- 19. • Asynchronous replicaGon between MySQL Clusters • AcGve-AcGve MulG-Homed – Update anywhere – Conflict detecGon • ApplicaGon noGfied through excepGon tables • Can opt to have conflicts resolved automaGcally – AutomaGc conflict resoluGon • ConflicGng transacGon and dependent ones are rolled-back • No changes to applicaGon / schema AcGve-AcGve Geo-ReplicaGon Copyright 2016, oracle and/or its affiliates. All rights reserved 19
- 20.
- 21. • NDB$EPOCH2 and NDB$EPOCH2_TRANS introduced • Detects conflicGng inserts/updates/ deletes/reads • EnGre transacGons (and dependent ones) rolled back • Rolling back of transacGons that read conflicted data • Improved NDB ExcepGons table format – Non-PK columns, operaGon type, transacGon id, before and aper values • Online conflict role change 21 Handling of Conflicts – Extensions in MySQL Cluster 7.4 Copyright 2016, oracle and/or its affiliates. All rights reserved
- 22. • Primary stores logical Gmestamp (GCI) against updated row(s) – Window for conflict opens • GCI is replicated with updated row(s) to Secondary • The same GCI is replicated back (reflected) from Secondary to Primary aper it has been applied – Closing window for conflict • Primary checks every event originaGng from the Secondary to check for conflicts DetecGng Conflicts - Reflected GCI John.balance == $100 John.balance -= $40 John.balance == $60 John.balance == $200 John.balance == $100 John.balance += $100 John.balance == $200 John.balance == $60 Spend $40 Add $100 $60 $200 Primary Secondary Copyright 2016, oracle and/or its affiliates. All rights reserved 22
- 23.
- 24. MySQL Cluster 7.4 Restart Improvements • DuraGon of long-running maintenance acGviGes dominated by Data Node restart Gmes • MySQL Cluster 7.4 = 5.5x faster restarts! • Benefits any kind of node restart – Upgrades, add-node, … • Benefits both ”manual” and MySQL Cluster Manager operaGons • Accomplish 5x as much work during a single maintenance window Make Data Node Restarts Fast! Copyright 2016, oracle and/or its affiliates. All rights reserved 24
- 25. MySQL Cluster 7.4 Restart Improvements • Verbose logging – Task start and compleGon – Data volume info – Parallelism & wait Gme info • NDBINFO for recent node restarts • More documentaGon of concrete stages • Goal: make proper analysis of a slow restart easy – Determine cause; Detect pa}erns; Understand the impact of indexes, local checkpoints, etc. Observability Improvements Copyright 2016, oracle and/or its affiliates. All rights reserved 25
- 26. MySQL Cluster 7.4 – Enhanced Memory ReporGng • ndbinfo.memory_per_fragment memory usage informaGon for each fragment replica, for each table, and each index • Allocated memory, and how much of that is actually in use • Exposes – FragmentaGon of fixed and variable sized fragment pages – Accurate Data and Index Memory use – Comparison of Primary and Backup fragment usage – ParGGon/Shard distribuGon and effecGveness Copyright 2016, oracle and/or its affiliates. All rights reserved 26
- 27.
- 28.
- 29.
- 30. MySQL Cluster 7.4 – Enhanced AcGvity ReporGng • ndbinfo.operations_per_fragment acGvity counters for each fragment replica, for each table, and each index • PK based and scan accesses – requests, bytes, rows, … • Exposes – How SQL statements map to individual tables and indices – Query execuGon, use of indexes etc. – LDM and node imbalances – Hotspots and scan overloads Copyright 2016, oracle and/or its affiliates. All rights reserved 30
- 31.
- 32.
- 33. MySQL Cluster 7.5 • ConGnue Improving Availability – Convert ndb_binlog_index to ACID storage – Make use of dynamic columns everywhere • Support for online defragmentaGon and space reclamaGon • Support for row transformaGons – encrypGon and compression • ConGnue Improving Quality – Refactor binlog injector in order to address many known issues – Increase the test base – More system correctness and fault injecGon tests Copyright 2016, oracle and/or its affiliates. All rights reserved 33 NEW!
- 34. MySQL Cluster 7.5 • ConGnue Improving Capacity – Enable individual fragments larger than 16GB (now up to 128TB) • Improved Read Scale-Out – The introducGon of “read from backup” for tables • Localized reads on every node in a node group – The introducGon of “global” tables (fully replicated) • Localized reads on every node in the Cluster • Support for Increased Redundancy – Support for up to 4 nodes per node group (1-3 replicas per shard) Copyright 2016, oracle and/or its affiliates. All rights reserved 34 NEW!
- 35.
- 36. When to ConsiderMySQL Cluster l UpGme requirements l Cost per minute of downGme? l Failure versus maintenance? l Scalability demands l Sharding for write scaling? l Latency demands l Cost of each millisecond? l ApplicaGon agility l Developer languages and frameworks? l SQL and NoSQL? Copyright 2016, oracle and/or its affiliates. All rights reserved 36
- 37. General Usage Considerations • MySQL Cluster is designed for – Short transactions – Single key reads and writes – Many parallel transactions • Utilize simple access patterns for high running transactions – Use efficient scans and batching interfaces – Push down joins yield huge performance improvements for JOIN operations – MySQL Cluster 7.4 speeds up table scans • Storage Engine is configurable for each table…InnoDB or NDB MySQL Cluster EvaluaGon Guide h}p://mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php Copyright 2016, oracle and/or its affiliates. All rights reserved 37
- 38. D1 D2 D3D4 API ---------- ---------- ---------- TC TC TC TC 1 2 Accessing Data: PK Lookups • Single TransacGon Coordinator (TC), no addiGonal data distribuGon handling • First statement decides TC • Keep transacGons short! Copyright 2016, oracle and/or its affiliates. All rights reserved 38
- 39. D1 D2 D3D4 API ---------- ---------- ---------- TC TC TC TC 1 2 3 Accessing data: Unique Secondary Key Lookups • Secondary keys implemented as hidden/ system tables • Hidden tables have new secondary key as PK and base table’s PK as value • Data may reside on same node or other node Copyright 2016, oracle and/or its affiliates. All rights reserved 39
- 40. D1 D2 D3D4 API ---------- ---------- ---------- TC TC TC TC 1 3 2 Accessing data: Range Scans and Joins • TC is chosen using round- robin • Data nodes return data directly to API • Flow: – Choose data node – Send request to all LDM – Send data to API Copyright 2016, oracle and/or its affiliates. All rights reserved 40
- 41. SELECT SUM(population) FROMtowns WHERE country=“UK”; SELECT SUM(population) FROM towns WHERE town=“Boston”; • ParGGon selected using hash on specified ParGGon Key – Primary Key by default – User can override in table definiGon • MySQL Server (or NDB API) will a}empt to send transacGon to the correct data node – If all data for the transacGon are in the same parGGon, less messaging -> be}er scalability • Aim to have all rows for high- running queries in same parGGon DistribuGon Aware Apps Copyright 2016, oracle and/or its affiliates. All rights reserved 41
- 42. • Extend parGGon awareness over mulGple tables • Same rule – aim to have all data for instance of high running transacGons in the same parGGon – ALTERTABLE service_ids PARTITION BY KEY(sub_id); DistribuGon Aware – MulGple Tables Copyright 2016, oracle and/or its affiliates. All rights reserved 42
- 43. Faster Joins – Pushdown Joins • AcGvated when ndb_join_pushdown is on (default) • Rules for a join to be pushed down: 1. Joined columns use idenGcal types 2. No reference to BLOB or TEXT columns 3. No explicit lock 4. Child tables in the Join must be accessed using ref, eq_ref, or const 5. Tables not explicitly parGGoned by [LINEAR] HASH, LIST, or RANGE 6. Query execuGon plan doesn’t use mysqld’s internal ‘join buffer' 7. If root of join is of the eq_ref or const type, child tables must be joined by eq_ref • Run ANALYZE TABLE <table_name> on each table once • Use EXPLAIN to see what components are being pushed down: – For example: Extra: Child of 'd' in pushed join@1 Copyright 2016, oracle and/or its affiliates. All rights reserved 43
- 44.
- 45. Local Checkpoints and Logging • Local Checkpoint Protocol (LCP) – Flushes the Data Nodes’ data to disk. Aper 2 LCPs the REDO log is trimmed – Frequency controlled by TimebetweenLocalCheckpoints • Specifies the amount of data that can change before flushing to disk • Not actually Gme! Base-2 logarithm of the number of 4-byte words • Example: Default value of 20 bytes means 4*2^20 = 4MB of data changes Copyright 2016, oracle and/or its affiliates. All rights reserved 45
- 46. Checkpoints and Logging Local & Redo • LCP and REDO Log are used to bring the cluster back online – Following system failure or planned shutdown 1. Data Nodes are restored using the latest LCP 2. REDO logs are applied unGl the latest GCP Copyright 2016, oracle and/or its affiliates. All rights reserved 46
- 47. MySQL Cluster Hardware Requirements • TCP/IP or Infiniband – Minimum 100Mbps, 1Gbps or higher recommended – MySQL Cluster dedicated network: separate Data Nodes from MySQL Servers • Homogeneous environments for each layer: API nodes & Data Nodes • Machines – Linux, Solaris, Windows, OS X • h}ps://www.mysql.com/support/supportedplaaorms/cluster.html – Min. 1Gb RAM – CPU, RAM, DISK: “the more and faster, the be}er” Copyright 2016, oracle and/or its affiliates. All rights reserved 47
- 48. MySQL Cluster Hardware SelecGon • 4 - 24 x86_64 bit CPU threads (vCPUs) • Minimum 4GB of RAM – Memory not as criGcal at this layer – Requirements influenced by connecGons and buffers • 2 Network Interface Cards (NICs) • 2 Power Supply Units (PSUs) • Linux, Solaris, Windows, or OS X operaGng system MySQL Servers Copyright 2016, oracle and/or its affiliates. All rights reserved 48
- 49. MySQL Cluster Hardware SelecGon • Up to 64 x86_64 bit CPU threads (vCPUs) – High frequency: enables faster processing of messages • CalculaGng RAM per Data Node (in-memory database) – Database Size * # Replicas * 1.25 / # data nodes – 50GB database * 2 replicas * 1.25 / 2 data nodes = 64GB of RAM – Non-indexed columns can be stored on disk, reduces RAM requirements • 2 NICs and 2 PSUs • Deep-dive into network best-pracGces in Reference Architecture Guide • Use MaxNoOfExecutionThreads or ThreadConfig to configure how threads should be allocated Data Nodes Copyright 2016, oracle and/or its affiliates. All rights reserved 49
- 50. Entry-Level Mid-Range High-End 1 x SATA 7200 RPM • For read-mostly • No redundancy (but other data node is the mirror) 1 x SAS 10K RPM or SSD • Heavy duty (many MB/s) • No redundancy (but other data node is the mirror) 4 x SAS 10-15K RPM or SSDs • Heavy duty (many MB/s) • Disk redundancy (RAID1+0 / 10), hot swappable LCP / REDOLOG MySQL Cluster Hardware SelecGon • REDO, LCP, BACKUP – wri}en sequenGally in small chunks (256KB) • If possible, use ODirect=1 Copyright 2016, oracle and/or its affiliates. All rights reserved 50 Disk Subsystem for Checkpoin0ng LCP / REDOLOG LCP / REDOLOG
- 51. Recommended Minimum High-End RecommendaGon 2 x SAS 10K RPM or 2 x SSD TABLESPACE LCP REDO LOG UNDO LOG UNDOLOG LCP (REDO LOG / UNDO LOG) TABLESPACE 1 TABLESPACE 2 4 x SAS 10-15K RPM or SSD (REDO LOG) (REDO LOG) MySQL Cluster Hardware SelecGon: • Use High-End RecommendaGon for heavy read/write workloads • Having TABLESPACE on separate disk is good for read performance • Enable WRITE_CACHE on block storage devices Copyright 2016, oracle and/or its affiliates. All rights reserved 51 Disk Based Tables
- 52. Zone B Zone A LocaGon of Nodes • Minority of data nodes lost – Isolated minority will shutdown • Majority pool (> 50% of the data nodes and one from each Node Group) – Remains viable – Exclude failed nodes from cluster operaGons Copyright 2016, oracle and/or its affiliates. All rights reserved 52 Handling Network Par00ons Node Group 1 Node Group 2
- 53. Zone B Zone A LocaGon of Nodes • Network connecGvity lost between zones • Each pool has 50% of data nodes and 1 from each Node Group – Each viable, but Zones will diverge if they both live (split-brain) – So who lives on and who dies? – We need arbitraGon Copyright 2016, oracle and/or its affiliates. All rights reserved 53 Handling Network Par00ons Node Group 1 Node Group 2
- 54. Zone B Zone A LocaGon of Nodes • Arbitrator decides – Contacted by data nodes from each zone – Tells those in one zone to shutdown – Tells those in other zone to stay up • Vital that arbitrator shares no single point of failure with either availability zone • Default arbitrator is an ndb_mgmd node but can also be a mysqld node Copyright 2016, oracle and/or its affiliates. All rights reserved 54 Avoiding Split-Brain During Network Par00ons Node Group 1 Node Group 2 Zone C Arbitrator
- 55. Spli…ng MySQL Cluster Over MulGple Sites • The greater the latency between sites, the higher the impact on performance • Target latency should be <= 10 ms; 20 ms acceptable – Test at the network layer with 1000 byte packets under load • Bandwidth requirements dependent on traffic but aim for 1 Gbps+ (100 Mbps for a low traffic Cluster) • Simplest WAN topology possible (fewer points of failure/failover latency) • Typical WAN failover Gmes should be short enough not to trigger STONITH acGviGes within Cluster Copyright 2016, oracle and/or its affiliates. All rights reserved 55 WAN Engineering
- 56. • Asynchronous replicaGon between MySQL Clusters • AcGve-AcGve MulG-Homed – Update anywhere – Conflict detecGon • ApplicaGon noGfied through excepGon tables • Can opt to have conflicts resolved automaGcally – AutomaGc conflict resoluGon • ConflicGng transacGon and dependent ones are rolled-back • No changes to applicaGon / schema AcGve-AcGve Geo-ReplicaGon (Reminder) Copyright 2016, oracle and/or its affiliates. All rights reserved 56
- 57.
- 58.
- 59. • ParGGon customers across mulGple clusters, distributed by region to opGmize low latency access • Each sub-cluster is replicated for High Availability • AcGve/passive or AcGve/AcGve 59 Geographic ReplicaGon for Low-Latency Local Access Copyright 2016, oracle and/or its affiliates. All rights reserved Cluster 4 Cluster 1 Cluster 1B Cluster 1C Cluster 2 Cluster 2B Cluster 2C Cluster 3 Cluster 3B Cluster 3C Cluster 4C Cluster 4B
- 60. MySQL Cluster Backups • Backup of NDB tables – Online – can have ongoing transacGons – Consistent – only commi}ed data and changes are backed up • ndb_mgm -e "START BACKUP" – Copy backup files from data nodes to safe persistent locaGon – Non-NDB tables must be backed up separately – MySQL system tables are stored in MyISAM or InnoDB • You want to backup (for each MySQL Server) – mysql database/schema – Triggers, rouGnes, events, ... • Use mysqldump – mysqldump mysql > mysql.sql – mysqldump --no-data -A > schemas.sql – mysqldump --no-data --no-create- info -R > routines.sql • Copy my.cnf & config.ini files Copyright 2016, oracle and/or its affiliates. All rights reserved 60
- 61.
- 62. Features to Use With CauGon • Large BLOBs • Disk based tables • Enforced Foreign Keys • Large range scans and joins • Spli…ng a single MySQL Cluster over mulGple data centers • AcGve-AcGve Geographic ReplicaGon – Fine to use but best if conflicts can be avoided on the applicaGon side Copyright 2016, oracle and/or its affiliates. All rights reserved 62 Poten0al Impacts to Performance, Capacity, & Stability
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73. Without MySQL Cluster Manager • 1 x preliminary check of cluster state • 8 x ssh commands per server • 8 x per-process stop commands • 4 x scp of config files (2 x mgmd & 2 x mysqld) • 8 x per-process start commands • 8 x checks for started and re-joined processes • 8 x process compleGon verificaGons • 1 x verify compleGon of the whole cluster • Excludes manual ediGng of each configuraGon file • Total: 46 commands – 2.5 hours of aNended opera8on With MySQL Cluster Manager mcm> upgrade cluster --package=7.4 mycluster; • Total: 1 Command – UnaNended Opera8on Sopware Upgrade – Example of MCM Benefits Copyright 2016, oracle and/or its affiliates. All rights reserved 73
- 74. MySQL Cluster Manager 1.3 GA “Unmanaged” producGon Cluster mcm> create cluster--import mcm> import config [--dryrun] mcm> import cluster [--dryrun] Cluster now managed by MCM Import a running Cluster into MCM Copyright 2016, oracle and/or its affiliates. All rights reserved 74
- 75.
- 76.
- 77.