MySQL Cluster Scaling to a Billion Queries


Published on

My MySQL Cluster presentation given at the NoSQL matters 2012 conference in Cologne.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

MySQL Cluster Scaling to a Billion Queries

  1. 1. MySQL ClusterScaling to Billion Database Queries with MySQL ClusterBernd OcklinMySQL Cluster
  2. 2. Servicing the Most Performance-Intensive Workloads
  3. 3. Program Agenda <Insert Picture Here>• Databases Are Exciting Again!!!• Overview of MySQL Cluster• MySQL Cluster - What’s New• How is it used?
  4. 4. A converging world ... Information Banking Social Messaging Networking Multi- Gaming Media
  7. 7. No Trade-Offs: Cellular Network HLR / HSS Location Updates AuC, Call Routing, Billing Pre & Post Paid • Massive volumes of write traffic • <3ms database response • Downtime & lost transactions = lost $Billing, AuC, MySQL Cluster in Action:
  9. 9. MySQL Cluster – Users & Applications Extreme Scalability, Availability and Affordability• Web • High volume OLTP • eCommerce • User Profile Management • Session Management & Caching • Content Management • On-Line Gaming• Telecoms • Subscriber Databases (HLR / HSS) • Service Delivery Platforms • VAS: VoIP, IPTV & VoD • Mobile Content Delivery • Mobile Payments • LTE Access
  10. 10. Key BenefitsScaling Auto-sharding + Multi-masterReads & Writes Transactional, ACID-compliant relational database99.999% Shared-nothing design, no Single Point of FailureAvailability On-Line operations: Scale, Upgrade Schema, etc.Real-Time High-load, real-time performanceResponsiveness Predictable low latency, bounded access times Complex, relational queries + Key/Value AccessSQL & NoSQLAPIs MySQL, Memcached, C++, Java, JPA, HTTP / RESTLow TCO, GPL & Commercial editionsOpen platform Commodity hardware, management & monitoring tools
  11. 11. Basic architectures 2-tierData accessApplication LogicFront-EndDataIndexes
  12. 12. Basic architectures 3-tier SQL, JDBC,Front-End ADO, ...Application LogicData access(e.g. SQL engine)DataIndexes
  13. 13. Basic architectures 4-tierFront-EndApplicationLogicData access(e.g. SQL engine)DataIndexes
  14. 14. All services share the same data view native ClusterJ REST/JSON LDAP memcached SQL, JDBC, ADO, ... NDB API MySQL Cluster Data Nodes
  15. 15. C++ example     NdbOperation *op = trx­>getNdbOperation(myTable);     op­>insertTuple();     op­>equal("key", i);     op­>setValue("value", &value);     trx­>execute( NdbTransaction::Commit );
  16. 16. Java example Character newCharacter =  session.newInstance(Character.class); newCharacter.setName(„Yoda“); newCharacter.setAttributes(„Force“); Session.persist(newCharacter);
  17. 17. SQL example (requires MySQL Server) Mysql> INSERT INTO Charaters (Name, Attributes)  VALUES („Yoda“, „Force“);
  18. 18. High performance and ScalabilityCluster is• Distributed• Event Driven• Asynchronous• Parallel• Non-locking
  19. 19. Your friends / Your enemies• Disks (life-saver) • Disks (slow fsync)• CPU cache • Network latency• RAM • Heap allocation• Many cores • NUMA • Context switching
  20. 20. Use your friendsDisks (your job saver) – Log your data to disk (asynchrounsly)CPU cache – Align to to itRAM – Preallocate!Many cores – Distribute to cores (have a model that supports this)
  21. 21. Avoid your enemiesDisks – Reduce fsyncs – no swappingNetwork latency – Reduce network round tripsSlow heap allocation – Pre-allocate all memory, avoid using itNUMA – Disable itContext switching – Lock to cores – Get network interrupts out of your way
  22. 22. MySQL Cluster – A distributed hash table 17 Yoda 143 Albert 12 Bernd 42 Ernest md5() % <no of nodes> MySQL Cluster Data Nodes17 Yoda 12 Bernd143 Ernest 143 Albert
  23. 23. Best Practice : Primary Keys• ALWAYS DEFINE A PRIMARY KEY ON THE TABLE! • A hidden PRIMARY KEY is added if no PK is specified. BUT.. • .. NOT recommended • The hidden primary key is for example not replicated (between Clusters)!! • There are problems in this area, so avoid the problems!• So always, at least have id BIGINT AUTO_INCREMENT PRIMARY KEY • Even if you dont “need” it for you applications
  24. 24. Auto-Sharding (distribution) Application [ {id: 12, name: Bernd}, {id: 143, name: Albert}, {id: 42, name: Ernest}, …, {id: 17, name: Yoda}] {id: 17, … } {id: 143, … } {id: 42, … } {id: 12, … } MySQL Cluster Data Nodes
  25. 25. Auto-Sharding (distribution) – Application “knows“ the data location Application find({id: 12}) {id: 12, name: Bernd} MySQL Cluster Data Nodes
  26. 26. Auto-Sharding• Transparent to the application and data access layer • No need for application-layer sharding logic – build into the API & kernel • Partitioning based on hashing all or part of the primary key • Each node stores primary fragment for 1 partition and back-up fragment for another• Transparency maintained during failover, upgrades and scale-out• No need to limit application to single-shard transactions
  27. 27. Adding High Availability – Introducing Node Groups Application [ {id: 12, name: Bernd}, {id: 143, name: Albert}, {id: 42, name: Ernest}, …, {id: 17, name: Yoda}] {id: 17, … } {id: 143, … } {id: 42, … } {id: 12, … } MySQL Cluster Data Nodes
  28. 28. Adding High Availability - Synchronous Replication Application [ {id: 12, name: Bernd}, {id: 143, name: Albert}, {id: 42, name: Ernest}, …, {id: 17, name: Yoda}] {id: 17, … } {id: 143, … } {id: 42, … } {id: 12, … } MySQL Cluster Data Nodes
  29. 29. Adding High Availability – Synchronous Replication 17 Yoda 12 Bernd 42 Ernest 143 Albert 12 Bernd 17 Yoda 143 Albert 42 Ernest
  30. 30. Handling Scheduled Maintenance On-Line Operations• Scale the cluster (add & remove nodes on-line)• Repartition tables• Upgrade / patch servers & OS• Upgrade / patch MySQL Cluster• Back-Up• Evolve the schema on-line, in real-time
  31. 31. Adding disk durability Memory {id: 17, … } In-memory tables Data kept in memory but complemented by logging to disk. Disk Disk based tables Data kept on disk but cached in memory.Logging to disk is decoupled from transaction writing.
  32. 32. Shared Nothing SQL, JDBC, ADO, ... No shared components. Cheap commodity hardware. Proper SAN acceptable but expensive.
  33. 33. Adding High Availability – Extreme resilience Application Service continuing MySQL Cluster Data Nodes
  34. 34. Event driven & asynchronous find(17, {}); {id: 17, name: Albert} {id: 17} {name: Albert} TC LQH ACC TUP
  35. 35. Doing things in parallel find(17, {}); find([12, 17], {}); MySQL Cluster Data Nodes
  36. 36. Doing things in parallel find(17, {}); {id: 17, name: Albert} TC LQH {id: 17} {name: Albert} TC TC LQH LQH TC TC LQH LQH ACC TUP x16 x16
  37. 37. Doing things in parallel• Primary key reads can be directed to the correct shard on the API application level – No waste of resources by doing same operation on all• Each data node can handle up to 16 operations in parallel• One data node can fully utilize up to 51 physical CPU cores
  38. 38. Scaleable & Elastic – Increasing capacity in seconds Application Data Nodes
  39. 39. Basic Deployments 1 Data Node SQL, JDBC,Front-End ADO, ...Application LogicData AccessDataIndexes
  40. 40. Basic Deployments Start scaling with 2 Data Nodes SQL, JDBC,Front-End ADO, ...Application LogicData Access ● Shared Data View ● Automated Load BalancingDataIndexes
  41. 41. Basic Deployments Continue Scaling SQL, JDBC, ADO, ... SQL, JDBC,Front-End ADO, ...Application Logic OS, App or Hardware LoadData access Balancing(e.g. SQL engine) Automated Load BalancingDataIndexes
  42. 42. Basic Deployments Scale on 3 levels SQL, JDBC, ADO, ...Front-End SQL, JDBC,Application Logic ADO, ...Data access(e.g. SQL engine)DataIndexes
  43. 43. Typical Deployments 2 Server Application SQL, JDBC, ADO, ... Web Server MySQL Server Data Nodes Server 1 Server 2
  44. 44. Typical Deployments 4 Server Application SQL, JDBC, ADO, ... Server 3 Server 4 Web Server MySQL Server Data Nodes Server 1 Server 2
  45. 45. Typical Deployments Geo Replication SQL, JDBC, ADO, ...Server 3 Server 4 Server 3 Server 4Server 1 Server 2 Server 1 Server 2
  46. 46. MySQL Cluster 7.2 GA
  47. 47. READSMillion / minute 1.200 1.056 1.000 800 • 8 x Commodity Intel Servers 600 • 2 x 6-core processors 2.93GHz 400 200 • x5670 processors (24 threads 0 per total) 2 node 4 node 8 node • 48GB RAM UPDATE • LinuxMillion / minute 120 109 100 • Infiniband networking 80 60 • flexAsynch benchmark 40 • C++ NoSQL API (NDB API) 20 0 4 node 8 node
  48. 48. Adaptive Query Localization Scaling Distributed Joins 70x More Performance • Perform Complex Queriesmysqld across Shards • JOINs pushed down to data nodes A Data Nodes • Executed in parallel Q • Returns single result set to MySQL L • Opens Up New Use-Cases • Real-time analytics • Recommendations enginesmysqld • Analyze click-streams Data Nodes DON’T COMPROMISE FUNCTIONALITY TO SCALE-OUT !!
  49. 49. MySQL Cluster 7.2 AQL Test Query Web-Based Content Management System MySQL Server Data Data Node1 Node2Copyright 2011 Oracle Corporation 49
  50. 50. Web-Based CMS 70x More Performance 87.23 seconds 1.26 secondsMust Analyze tables for best resultsmysql> ANALYZE TABLE <tab-name>;
  51. 51. Memcached Key-Value API • Persistent, Scalable, HA Back-End to memcached • No application changes: re- uses standard memcached clients & libraries • Consolidate Caching & Database Tiers • Eliminate cache invalidation • Simpler re-use of data across services • Improved service levels New • Flexible DeploymentNoSQL • Schema or Schema-lessAccess storage
  52. 52. Schema-Free apps • Rapid application evolution • New types of data constantly added • No time to get schema extended • Missing skills to extend schema • Initially roll out to just a few users • Constantly adding to live systemCopyright 2011 Oracle Corporation 52
  53. 53. Cluster & Memcached – Schema-Free key value <town:maidenhead,SL6>Application viewSQL view key value <town:maidenhead,SL6> generic table
  54. 54. Cluster & Memcached –Configured Schema key value <town:maidenhead,SL6>Application viewSQL view prefix key value <town:maidenhead,SL6> Config tables
  55. 55. SQL & NoSQL Clients JDBC / ODBC PHP / PERL Native memcached HTTP/REST Python / Ruby NDB API Data Nodes Mix • SQL: Complex, relational queries & • Memcached: Key-Value web servicesMatch • Java: Enterprise Apps • NDB API: Real-time services
  56. 56. MySQL 5.5 Server Integration • Configure storage engine per-table • Choose the right tool for the job • InnoDB: Foreign Keys, XA Transactions, Large Rows • MySQL Cluster: HA, High Write Rates, Real- Time • Reduces Complexity, Simplifies DevOps • Take advantage of MySQL 5.5 • 3x higher performance • Improved partitioning, diagnostics, availability, etc.
  57. 57. Cross Data Center Scalability
  58. 58. Multi-Site Clustering • Split data nodes across data centers • Synchronous replication Node Group 1 and auto-failover betweenData Node 1 Data Node 2 sites Synchronous Synchronous Replication Replication • Improved heartbeating to handle network partitions Node Group 2 • Extends HA OptionsData Node 3 Data Node 4 • Active/Active with no need for conflict handling
  59. 59. Active/Active Geographic Replication •Replicating complete clusters across data centers • DR & data locality • No passive resources Geographic Replication •Simplified Active / Active Replication • Eliminates requirement for application & schema changes • Transaction-level rollback
  60. 60. Ease of Use
  61. 61. Simplified Provisioning & Maintenance User Privilege ConsolidationThe existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion.October 3rd, 2011
  62. 62. MySQL Cluster Manager Reducing TCO and creating a more agile, highly available database environment Automated Management Monitoring & High Recovery Availability OperationCopyright 2011 Oracle Corporation 62
  63. 63. How Does MySQL Cluster Manager Help? Example: Initiating upgrade from MySQL Cluster 7.0 to 7.2 Before MySQL Cluster Manager With MySQL Cluster Manager • 1 x preliminary check of cluster state upgrade cluster --package=7.1 mycluster; • 8 x ssh commands per server • 8 x per-process stop commands • 4 x scp of configuration files (2 x mgmd & 2 x Total: 1 Command - mysqld) • 8 x per-process start commands Unattended Operation • 8 x checks for started and re-joined processes • Results • 8 x process completion verifications • 1 x verify completion of the whole cluster. • Reduces the overhead and complexity • Excludes manual editing of each configuration of managing database clusters file. • Reduces the risk of downtime resulting from administrator error Total: 46 commands - • Automates best practices in database 2.5 hours of attended operation cluster managementCopyright 2011 Oracle Corporation 63
  64. 64. Bootstrap single host Cluster 1. Download MCM from • Package including Cluster 1. Unzip 2. Run agent, define, create & start Cluster! $> binmcmd –bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862 Configuring default cluster mycluster... Starting default cluster mycluster... Cluster mycluster started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root • Connect to Cluster & start using database To bootstrap with Cluster 7.2 replace contents of mcm/cluster directory 2011 Oracle Corporation 64
  65. 65. MySQL Enterprise Monitor 2.3Copyright 2011 Oracle Corporation 65
  66. 66. Evaluate MySQL Cluster CGE30-Day Trial • Navigate to and step through (selecting “MySQL Database” as the Product Pack) • Select MySQL Cluster Manager
  67. 67. Operational Best PracticesCopyright 2011 Oracle Corporation 67
  68. 68. When to Consider MySQL Cluster What are the consequences of downtime or failing to meet performance requirements? How much effort and $ is spent in developing and managing HA in your applications? Are you considering sharding your database to scale write performance? How does that impact your application and developers? Do your services need to be real-time? Will your services have unpredictable scalability demands, especially for writes ? Do you want the flexibility to manage your data with more than just SQL ?
  69. 69. Where would I not Use MySQL Cluster?• “Hot” data sets >3TB • Replicate cold data to InnoDB• Long running transactions• Large rows, without using BLOBs• Foreign Keys • Can use triggers to emulate: •• Full table scans• Savepoints• Geo-Spatial indexes• InnoDB storage engine would be the right choice MySQL Cluster Evaluation Guide
  70. 70. MySQL Cluster in Action Web Reference ArchitecturesSession Management eCommerce Data Content Management Refinery Memcache / Application Servers MySQL Servers MySQL Servers MySQL Master Node Group 1 Node Group 2 Node Group 1 Node Group 2 F1 F2 F1 F2 Slave N F4 Node 3 F3 Node 3 F4 Node 3 F3 Node 3 F1 F2 F1 F2 F3 F4 Node 4 F4 Node 4 F3 Slave 8 Slave 9 Slave 10 Node 4 Node 4 Slave 6 Slave 7 MySQL Cluster Data Nodes MySQL Cluster Data Nodes Slave 1 Slave 2 Slave 3 Slave 4 Slave 5• 4 x Data Nodes: 6k Analyticspage hits per second MySQL Master XOR• Each page hitgenerating 8 – 12database operations Distributed Slave 1 Slave 2 Slave 3 Storage Whitepaper:
  71. 71. World wide use
  72. 72. Case Studies
  73. 73. CUSTOMER PERSPECTIVE “MySQL Cluster won the performance test hands- COMPANY OVERVIEW down, and it fitted our needs perfectly. We • Leading provider of communications evaluated shared-disk clustered databases, but the platforms, solutions & services cost would have been at least 10x more.” • €15.2bn Revenues (2009), 77k employees -- François Leygues, Systems Manager across 130 countries CHALLENGES / OPPORTUNITIES • Converged services driving migration to RESULTS next generation HLR / HSS systems • Scale out on standard ATCA hardware to • New IMS platforms for Unified support 60m+ subscribers on a single Communications platform • Reduce cost per subscriber and accelerate • Low latency, high throughput with time to value 99.999%+ availability • Enabled customers to reduce cost per subscriber and improve margins SOLUTIONS • Delivered data management solution at • MySQL Cluster Carrier Grade Edition 10x less cost than alternatives • MySQL Support & Consulting Services
  74. 74. Shopatron: eCommerce Platform • Applications – Ecommerce back-end, user authentication, order data & fulfilment, payment data & inventory tracking. Supports several thousand queries per second • Key business benefits – Scale quickly and at low cost to meet demand – Self-healing architecture, reducing TCO • Why MySQL? – Low cost scalability – High read and write throughput – Extreme availability“Since deploying MySQL Cluster as our eCommerce database, we have had continuous uptime with linear scalability enabling us to exceed our most stringent SLAs” — Sean Collier, CIO & COO, Shopatron Inc 76
  75. 75. COMPANY OVERVIEW CUSTOMER PERSPECTIVE• Pyro provide comms technology solutions ”MySQL Cluster 7.1 gave us the perfect combination in Core Network, OSS/BSS & VAS of extreme levels of transaction throughput, low• Deployed in 120+ networks worldwide latency & carrier-grade availability. We also reduced• Cell C, one of the largest mobile TCO by being able to scale out on commodity server operators in South Africa blades and eliminate costly shared storage”• 560 roaming partners in 186 countries -- Phani Naik, Head of Technology at Pyro GroupCHALLENGES / OPPORTUNITIES• FIFA 2010 world cup opens up network services to millions of mobile subscribers RESULTS• International roaming SDP to support up • Supported subscriber and traffic volumes to 7m roaming subscribers per day • Delivered continuous availability• Offer local pricing with home network • Implemented in 25% of the time of typical functionality SDP solutions• Minimize cost and time to market • Choice in deployment platforms to eliminate vendor lock-in (migrated from Microsoft)SOLUTIONS• MySQL Cluster 7.1 & Services
  76. 76. CUSTOMER PERSPECTIVE “Telenor has been using MySQL for fixed IPCOMPANY OVERVIEW management since 2003 and are extremely• Leading telecoms provider across Europe satisfied with its speed, availability and and Asia. Largest Nordic provider flexibility. Now we also support mobile• 184m subscribers (Q2, 2010) and LTE IP management with our solution. Telenor has found MySQL Cluster to be the best performing database in the worldCHALLENGES / OPPORTUNITIES for our applications.”• Extend OSS & BSS platforms for new mobile services and evolution to LTE - Peter Eriksson, Manager, Network Provisioning• OSS: IP Management & AAA RESULTS• BSS: Subscriber Data Management & Customer Support • Launch new services with no downtime, due to on-line operations of MySQL Cluster • Consolidated database supportsSOLUTIONS Subscriber Data Management initiatives• MySQL Cluster • MySQL Cluster selected due to 99.999%• MySQL Support Services availability, real time performance and linear scalability on commodity hardware
  77. 77. COMPANY OVERVIEW CUSTOMER PERSPECTIVE• UK-based retail and wholesale ISP & “Since deploying our latest AAA platform, the Hosting Services MySQL environment has delivered continuous• 2010 awards for best home broadband uptime, enabling us to exceed our most stringent and customer service SLAs”• Acquired by BT in 2007 -- Geoff Mitchell Network EngineerCHALLENGES / OPPORTUNITIES• Enter market for wholesale services, demanding more stringent SLAs• Re-architect AAA systems for data RESULTS integrity & continuous availability to • Continuous system availability, exceeding support billing sytems wholesale SLAs• Consolidate data to for ease of reporting • 2x faster time to market for new services and operating efficiency • Agility and scale by separating database• Fast time to market from applications • Improved management & infrastructure efficiency through database consolidationSOLUTIONS• MySQL Cluster• MySQL Server with InnoDB
  78. 78. COMPANY OVERVIEW USER PERSPECTIVE• Division of Docudesk “MySQL Cluster exceeds our requirements for low latency, high throughput performance with• Deliver Document Management SaaS continuous availability, in a single solution that minimizes complexity and overall cost.”CHALLENGES / OPPORTUNITIES -- Casey Brown, Manager of Dev & DBA Services, Docudesk• Provide a single repository for customers to manage, archive, and distribute documents• Implement scalable, fault tolerant, real time data management back-end RESULTS• PHP session state cached for in-service • Successfully deployed document personalization management solution, eliminating paper• Store document meta-data, text (as trails from legal processes BLOBs), ACL, job queues and billing data • Integrate caching and database into one• Data volumes growing at 2% per day layer, reducing complexity & cost • Support workload with 50:50 read/write ratioSOLUTION • Low latency for real-time user experience• MySQL Cluster deployed on EC2 and document time-stamping • Continuous database availability
  79. 79. Getting Started Learn More Scaling Web Databases Guide Evaluate MySQL Cluster 7.2 Bootstrap a Cluster! Download, No Download Today Obligation https://edelivery.oracl downloads/cluster/ 2011 Oracle Corporation 81
  80. 80. Summary Scale Web Services with Carrier-Grade Availability Don’t Trade Functionality for Scale Try it out Today!Copyright 2011 Oracle Corporation 82
  81. 81. Tech Slide Back-up
  82. 82. Multi-threaded Data Node Extensions • Scaling out on commodity hardware is the standard Application Nodes way to increase performance • Add more data nodes and Node 3 API nodes as required • MySQL Cluster 7.2Node 1 increases the ability to also scale-up each data node • Increases maximum number of utilised threads from 8 to 59 Node 2 Node 4 • Can deliver aX single thread performance with bX cores Node Group 1 Node Group 2
  83. 83. Multi-threaded Data Node Extensions • Threads (post GA!): • recv: <= 8 Receive threads Application Nodes • tc: <= 24 Transaction Coordinator threads • ldm: <= 16 Local Query Handler threads • send: <= 8 Send threads • main: 1 Main threadrecv send main • rep: 1 Replication thread • io: 1 I/O thread • Engineering guidelines provided to find the best tc ldm rep io configuration: ZXZX
  84. 84. Multi-threaded Data Node ExtensionsThreadConfig :=<entry> [ ,<entry> ] +entry :=<type>={ [<param> ]+ } • Note that extra send,param := count = N | cpubind = L | recv & tc threads cpuset = L will be part of post-type := ldm | main | recv | rep | GA maintenancemaint | send | tc | io release.Example:ThreadConfig=ldm={count=2,cpubind=1,2},ldm={count=2,cpuset=6-9},main={cpubind=12},rep={cpubind=11}
  85. 85. NoSQL with Memcached• Flexible: set maidenhead 0 0 3 SL6 • Deployment options STORED • Multiple Clusters • Simultaneous SQL Access get maidenhead • Can still cache in VALUE maidenhead 0 3 Memcached server SL6 • Flat key-value store or map END to multiple tables/columns
  86. 86. Multi-Site Clustering – changes to STONITH algorithm• When heartbeat not received, all data nodes will be asked to ping all other data nodes• Each node establishes its list of ‘suspect’ data nodes from whom they don’t receive a ping response within ConnectCheckIntervalDelay msecs• If second period of ConnectCheckIntervalDelay passes without a ping response then each data node will send a Fail report to all data nodes naming its suspected node(s)• On receipt of a Fail message from a suspect node, the receiving node will consider the originating node as failed rather than the requested target• Leaves each side of the temporarily partitioned network with a viable set of data nodes and arbitration is used to select the surviving side if there is no longer a clear majority
  87. 87. Multi-Site Clustering – WAN engineering recommendations based on user experience• (Obviously) the longer the latency between sites, the higher the impact to performance• Target latency should be <= 10 ms; 20 ms acceptable • Test with 1000 byte packet, under load• Bandwidth requirements dependent on traffic but aim for 1 Gbps+ (100 Mbps for low traffic Cluster)• Simplest WAN topology possible (fewer points of failure/failover latency)• Typical WAN failover times should be short enough not to trigger STONITH in Cluster
  88. 88. Active-Active Replication - Detecting Conflicts
  89. 89. Active-Active Replication - Detecting Conflicts
  90. 90. Active-Active Replication - Detecting Conflicts
  91. 91. Active-Active Replication - Detecting Conflicts
  92. 92. Active-Active Replication - Detecting Conflicts
  93. 93. Geographic Replication – what’s changed in conflict resolution• Reflecting GCI (Global Checkpoint Index) removes requirement for applications to maintain timestamp field in each potentially conflicting table • One of the two masters acts as the ‘primary’ and monitors all received replication events from the ‘secondary’ (including its own ‘reflected GCI’) to establish when changes not applied in same order on primary and secondary Clusters • Primary will then overwrite all conflicting transactions (or optionally just the conflicting rows) on the secondary – as well as subsequent transactions influenced by the conflict• To use, set the function in mysql.ndb_replication to NDB$EPOCH() or NDB$EPOCH_TRANS()• Overview & worked example:• Gory details:
  94. 94. How to Push Privilege Data into Data Nodesmysql> SOURCE /usr/local/mysql/share/mysql/ndb_dist_priv.sql;mysql> CALL mysql.mysql_cluster_move_privileges();mysql> SHOW CREATE TABLE mysql.userG*************************** 1. row ***************************Table: userCreate Table: CREATE TABLE `user` (`Host` char(60) COLLATE utf8_bin NOT NULL DEFAULT ,........) ENGINE=ndbcluster DEFAULT CHARSET=utf8 COLLATE=utf8_binCOMMENT=Users and global privileges‘•Fully worked example: (
  95. 95. General Back-up Slides
  96. 96. Scale the ClusterCapacity & Performance Application Nodes Data Node 1 Data Node2Cluster F1 F2Mgmt F2 F1 Data Nodes
  97. 97. Scale the ClusterCapacity & Performance Application Nodes Node Group 1 Node Group 2 Node 1 Node 3Cluster F1 F2 ClusterMgmt F3 F4 Mgmt Node 2 Node 4 F3 F4 F1 F2 Data Nodes
  98. 98. Node Failure Detection & Self-Healing Recovery
  99. 99. On-Line Scaling & Maintenance 1. New node group added 2. Data is re-partitioned 3. Redundant data is deleted 4. Distribution is switched to share load with new node group• Can also update schema on- line• Upgrade hardware & software with no downtime• Perform back-ups on-line
  100. 100. Only MySQL Can…..blend the agility & innovation of the web….….with the trust & capability of the network.
  101. 101. No Trade-Offs: eCommerce • Integrated Service Provider platform • eCommerce • Payment processing • Fulfillment • Supports 1k+ manufacturers & 18k retail partners • Requirements • Scaling, On-Demand • HA: failures & on-line upgrades • High batch & real time loads • Low TCO: capex and opex
  102. 102. No Trade-Offs: Flight Control • US Navy aircraft carriers • Consolidated flight operations management system • Maintenance records • Fuel loads • Weather conditions • Flight deck plans • Requirements • No Single Points of Failure • Complete redundancy • Small footprint, harsh environment • 4 x MySQL Cluster nodes, Linux and Windows MySQL User Conference Session:
  103. 103. Creating & running your first Cluster - the “manual” way (without MCM)• Up & running in 10-15 minutes using Quick Start guides from • Versions for Linux, Windows & Solaris Copyright 2011 Oracle Corporation 106
  104. 104. ACID Compliant Relational Database • SQL & NoSQL interfacesWrite-Scalable & Real-Time • Distributed, multi-master, auto-sharding, optimized in-memory structures & indices99.999% Availability • Shared-nothing, integrated clustering & sub-second recovery, local & geographic replication, on-line operationsLow Barriers to Entry • Open-source, elastic, multiple APIs, management tools, commodity hardware