Oracle OpenWorld - Getting started with MySQL Cluster

492 views

Published on

MySQL Cluster is a proven technology that today is successfully servicing the most performance-intensive workloads. MySQL Cluster is deployed across telecom networks and is powering mission-critical web applications.

Without trading off use of commodity hardware, transactional consistency and use of complex queries, MySQL Cluster provides:
- Web Scalability (web-scale performance on both reads and writes)
- Carrier Grade Availability (99.999%)
- Developer Agility (freedom to use SQL or NoSQL access methods)

MySQL Cluster is recommended in the situations where:
- It is crucial to reduce service downtime, because this produces a heavy impact on business
- Sharding the database to scale write performance highly impacts development of application (in MySQL Cluster the sharding is automatic and transparent to the application)
- There are real-time responses needs
- There are unpredictable scalability demands
- It is important to have data-access flexibility (SQL & NoSQL)

From a practical point of view the HOL's steps were:
- Installation of MySQL Cluster
- Start & Monitoring of MySQL Cluster
- Connecting to MySQL Cluster
- Overview of MySQL Cluster’s Admin Commands & Operations

Published in: Technology
  • Be the first to comment

Oracle OpenWorld - Getting started with MySQL Cluster

  1. 1. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Getting started with MySQL Cluster Hands-on Lab Benedita Paúl Vasconcelos Principal Technical Support Engineer Oracle, MySQL Support October 29, 2015
  2. 2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 3
  3. 3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 6 7 8
  4. 4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 5 6 7 8
  5. 5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Aim of this Hands-on Lab • Objectives 1. Learn the basics of the MySQL Cluster Architecture 2. Learn the basics of MySQL Cluster Configuration and Administration 3. Learn how to start a new Cluster for evaluation purposes and how to connect to it • Benefits – Familiarize with MySQL Cluster – Know where to go next to start (or continue) Cluster evaluation 6
  6. 6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 7 6 7 8
  7. 7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster • Memory optimized tables with durability • Predictable Low-Latency, Bounded Access Time REAL-TIME • Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations 99.999% AVAILABILITY • Key/Value + Complex, Relational Queries • SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL • Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Tools LOW TCO MySQL Cluster Overview – WHAT 8
  8. 8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster MySQL Cluster GA – WHAT 9
  9. 9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster Web High volume OLTP eCommerce User Profile Management Session Management & Caching Content Management On-Line Gaming MySQL Cluster Deployments – WHERE 10 Telecoms Subscriber Databases (HLR / HSS) Service Delivery Platforms VAS: VoIP, IPTV & VoD Mobile Content Delivery Mobile Payments LTE Access Case studies: www.mysql.com/customers/cluster/
  10. 10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | • Scalability demands – Sharding for write performance? • Latency demands – Cost of each millisecond? • Uptime requirements – Cost per minute of downtime? – Failure versus maintenance? • Application agility – Developer languages and frameworks? – SQL or NoSQL? • MySQL Cluster is designed for: – Short transactions – Many parallel transactions Brief Introduction to MySQL Cluster 11 MySQL Cluster Architecture – WHEN
  11. 11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster MySQL Cluster Architecture – HOW 12 MySQL Cluster Data Nodes Clients Application Layer Data Layer Management
  12. 12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster MySQL Cluster Scaling – HOW 13 MySQL Cluster Data Nodes Clients Management Application Layer Data Layer
  13. 13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster MySQL Cluster HA – HOW 14 Clients Management Data Layer Application Layer MySQL Cluster Data Nodes
  14. 14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Brief Introduction to MySQL Cluster 15 MySQL Cluster Customers – WHO
  15. 15. Questions? Brief Introduction to MySQL Cluster 16 WHAT WHERE WHEN HOW WHO
  16. 16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 17 6 7 8
  17. 17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Installation of MySQL Cluster Open the HOL instructions and enter the VM: Open the terminal: • gedit HOL3348_Instructions.txt & • ssh ouser@192.168.56.101 (Password: ouser) It’s your turn 18 Hands-on Lab
  18. 18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 19 Generic Linux package Installation of MySQL Cluster Unpack of the package Creation of the needed directories Installation of SQL nodes Creation of the configuration files 1 2 3 4
  19. 19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Installation of MySQL Cluster Execute the commands at section #1 of the HOL instructions It’s your turn 20 Hands-on Lab
  20. 20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.1 of HOL instructions Commands: • cd $HOME/software/generic • tar -xzfv mysql-cluster-gpl-7.4.7-linux-glibc2.5- i686.tar.gz 21 Unpack of the package
  21. 21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.2 of HOL instructions Commands: • mkdir $HOME/cluster • mkdir $HOME/cluster/7.4.7 • mkdir $HOME/cluster/7.4.7/conf $HOME/cluster/7.4.7/mysqld1_data $HOME/cluster/7.4.7/mysqld2_data $HOME/cluster/7.4.7/ndb_data • ls -l $HOME/cluster/7.4.7/ 22 Creation of the needed directories
  22. 22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.3 of HOL instructions Commands: • cd $HOME/software/generic/mysql-cluster-gpl-7.4.7- linux-glibc2.5-i686 • scripts/mysql_install_db --no-defaults -- datadir=$HOME/cluster/7.4.7/mysqld1_data/ • scripts/mysql_install_db --no-defaults -- datadir=$HOME/cluster/7.4.7/mysqld2_data/ 23 Installation of SQL nodes Initialize MySQL Data Directory Do not read any option files
  23. 23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.4 of HOL instructions Commands: • cd $HOME/cluster/7.4.7/conf/ The files to be created are: –my1.cnf –my2.cnf –config.ini 24 Creation of the configuration files Specifies options for all MySQL Cluster executables Contains the configuration of each node involved in the Cluster
  24. 24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.4 of HOL instructions [mysqld] ndbcluster datadir=/home/ouser/cluster/7.4.7/mysqld1_data basedir=/home/ouser/software/generic/mysql-cluster- gpl-7.4.7-linux-glibc2.5-i686 port=3307 log_error=/home/ouser/cluster/7.4.7/mysqld1_data/mys qld.log 25 my1.cnf Enables NDBCLUSTER storage engine
  25. 25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #1.4 of HOL instructions [ndb_mgmd] hostname=localhost PortNumber=1186 datadir=/home/ouser/cluster/7.4.7/ndb_data NodeId=49 ArbitrationRank=1 [ndbd default] NoOfReplicas=2 datadir=/home/ouser/cluster/7.4.7/ndb_data [ndbd] hostname=localhost NodeId=1 [ndbd] hostname=localhost NodeId=2 [mysqld] NodeId=50 [mysqld] NodeId=51 26 config.ini The [ndb_mgmd] sections are used to configure the behavior of each one of the Cluster’s Management server: • PortNumber: port number (default value) on which the management server listens for configuration requests and management commands • NodeID: each node in the cluster has an unique identity • ArbitrationRank: used to define which nodes can act as arbitrators (1 is the default value) The [ndbd default] section is used to configure the behavior of all the Cluster's Data nodes: • NoOfReplicas: defines the number of replicas for each table stored in the cluster (2 is the default value) The [ndbd] sections are used to configure the behavior of each one of the Cluster's Data nodes: • NodeID: each node in the cluster has an unique identity The [mysqld] sections are used to define the behavior of the MySQL servers (SQL nodes): • NodeID: each node in the cluster has an unique identity
  26. 26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 27 6 7 8
  27. 27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Start of MySQL Cluster 28 Ordered start of the nodes Management node (ndb_mgmd) Data nodes (ndbd/ndbmtd) SQL nodes (mysqld) 1 2 3
  28. 28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Start of MySQL Cluster Execute the commands at section #2 of the HOL instructions It’s your turn 29 Hands-on Lab
  29. 29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #2.1 of HOL instructions Command: • ndb_mgmd -f $HOME/cluster/7.4.7/conf/config.ini -- configdir=$HOME/cluster/7.4.7/conf/ --initial -f : specify the cluster configuration file --configdir : specify the cluster management server's configuration cache directory --initial : causes the management server reload its configuration data from the configuration file, bypassing the configuration cache Start of Management node 30 Suggestion: open now a new terminal to monitor the Cluster log: tail –f ndb_49_cluster.log
  30. 30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Sections #2.2 & #2.3 of HOL instructions Commands: • ndbd -c localhost:1186 --initial --ndb-nodeid=1 • ndbd -c localhost:1186 --initial --ndb-nodeid=2 -c : set connection string for connecting to ndb_mgmd --initial : perform initial start of the Data node, including cleaning the file system --ndb-nodeid : set the respective Data node’s ID 31 Start of Data nodes (ndbd/ndbmtd) Note: an --initial start is to be used only when starting the ndbd process under very special circumstances; this is because this option causes all files to be removed from the MySQL Cluster file system and all redo log files to be re-created.
  31. 31. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Sections #2.4 & #2.5 of HOL instructions Commands: • mysqld --defaults- file=$HOME/cluster/7.4.7/conf/my1.cnf & • mysqld --defaults- file=$HOME/cluster/7.4.7/conf/my2.cnf & 32 Start of SQL nodes (mysqld)
  32. 32. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Monitoring of MySQL Cluster •Cluster log: ndb_<mgm-nodeid>_cluster.log •Data node’s logs: output ; error ; trace Log files •shell> ps aux | grep -i ndb •shell> ps aux | grep -i mysql Cluster processes •SHOW •ALL | <nodeid> STATUS •ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog ndb_mgm commands •memoryusage •nodes •cluster_transactions •restart_info •disk_write_speed_aggregate •etc ndbinfo database •MySQL Cluster Memory •MySQL Cluster Log buffers and files •MySQL Cluster cache •MySQL Cluster node status MySQL Enterprise Monitor (MEM) 33 Manual: ndbinfo Manual: MEM
  33. 33. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Monitoring of MySQL Cluster processes Execute the commands at section #3 of the HOL instructions It’s your turn 34 Hands-on Lab
  34. 34. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #3 of HOL instructions Are the Cluster processes running? • ps aux | grep -i ndb • ps aux | grep –i mysqld Tail of the Management log: • tail –f ndb_49_cluster.log 35 Monitoring of MySQL Cluster processses & logs
  35. 35. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 36 6 7 8
  36. 36. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Connecting to MySQL Cluster 37 Administration + SQL & NoSQL access to Cluster data Management client (administration) MySQL client (SQL access) Connectors & APIs (SQL or NoSQL access) 1 2 3
  37. 37. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Connecting to MySQL Cluster Execute the commands at section #4 of the HOL instructions It’s your turn 38 Hands-on Lab
  38. 38. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #4.1 of HOL instructions Command in shell: • ndb_mgm -c localhost:1186 Command in Management Client (ndb_mgm): • SHOW • EXIT Management Client (administration) 39
  39. 39. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #4.2 of HOL instructions Command in shell: • mysql -uroot -P3307 Commands in MySQL Client (mysql): • USE ndbinfo • SHOW TABLES; • SELECT * FROM memoryusage; • SELECT * FROM nodes; • exit MySQL Client (SQL access) 40
  40. 40. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 41 6 7 8
  41. 41. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Safe Shutdown of MySQL Cluster 42 A two-step process Use of the Cluster’s “shutdown” command Shutdown (or not) all the SQL nodes 1 2
  42. 42. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster shutdown • Cluster's "shutdown" command: ndb_mgm> shutdown Node 1: Cluster shutdown initiated Node 2: Cluster shutdown initiated Node 2: Node shutdown completed. Node 1: Node shutdown completed. 3 NDB Cluster node(s) have shutdown. Disconnecting to allow management server to shutdown. • Tail of the Management log: 2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 1: Cluster shutdown initiated 2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 2: Cluster shutdown initiated 2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 2 Disconnected 2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 2: Node shutdown completed. 2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 1: Node shutdown completed. 2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 1 Disconnected 2015-08-26 12:30:25 [MgmtSrvr] INFO -- Shutting down server... 2015-08-26 12:30:33 [MgmtSrvr] INFO -- Shutdown complete 43 Sample output
  43. 43. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 44 6 7 8
  44. 44. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Overview of main MySQL Cluster’s Admin Commands & Operations • Management Client's (ndb_mgm) Commands: HELP, SHOW, CREATE | DROP NODEGROUP <nodeid> START | STOP | RESTART START BACKUP, ENTER | EXIT SINGLE USER MODE CLUSTER LOG ON | OFF | INFO | FILTER <severity_level> <nodeid> CLUSTERLOG category=<threshold> ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog SHUTDOWN, QUIT | EXIT • Cluster Operations: – On-line Rolling Restart • On-line Scaling (add/remove nodes) • On-line Reconfiguration • On-line Upgrades – On-line Backup & Restore 45
  45. 45. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #6 of HOL instructions Command in shell: • ndb_mgm -c localhost:1186 Command in Management Client (ndb_mgm): • ALL REPORT MEMORY • ALL REPORT BackupStatus 46 MySQL Cluster’s Admin Commands
  46. 46. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Section #6 of HOL instructions Command in Management Client (ndb_mgm): • 1 RESTART • ALL STATUS Note: keep running ALL STATUS during the restart of node 1 so you may see the node passing through some different start phases. 47 MySQL Cluster’s Admin Commands
  47. 47. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Program Agenda with Highlight Aim of this Hands-on Lab Brief Introduction to MySQL Cluster Installation of MySQL Cluster Start & Monitoring of MySQL Cluster Connecting to MySQL Cluster Safe Shutdown of MySQL Cluster Overview of MySQL Cluster’s Admin Commands & Operations MySQL Cluster CGE & Conclusions 1 2 3 4 5 48 6 7 8
  48. 48. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Carrier Grade Edition 49 More: www.mysql.com/buy-mysql No Trade-Offs On the Top
  49. 49. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Manager (MCM) 50 Automated Management • Start / Stop node or whole cluster • On-Line Scaling • On-Line Reconfiguration • On-Line Upgrades • On-Line Backup & Restore • Import Running Cluster Self-Healing • Node monitoring • Auto-recovery extended to SQL + mgmt nodes HA Operations • Cluster-wide configuration consistency • Persistent configurations • HA Agents
  50. 50. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Enterprise Monitor (MEM) Enterprise Monitor, Advisors & Query Analyzer 51 Your virtual MySQL DBA Assistant
  51. 51. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MySQL Cluster Auto-Installer • Fast configuration • Auto-discovery • Workload optimized • Repeatable best practices 52 Specific Workload Auto- Discover Define Topology Deploy
  52. 52. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 53 MySQL Cluster Auto-Installer
  53. 53. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Premier Support for MySQL • Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support 54 Rely on The Experts - Get Unique Benefits Only From Oracle "The analyst quickly answered my questions, and went beyond what I expected, providing resources to get additional information and understanding.“ mysql.com/support/quotes
  54. 54. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Resources mysql.com TCO calculator White papers dev.mysql.com Community downloads Documentation Dev articles Forums PlanetMySQL eDelivery.com Oracle Software Delivery Cloud Download and evaluate all MySQL products support.oracle.com Knowledge Base • Doc ID 1389140.1: Is MySQL Cluster the Right Choice Compared with InnoDB? • Doc ID 1023119.1: How to Add New Nodes To an Existing MySQL Cluster Setup? • Doc ID 1926680.1: Best Practices For MySQL Cluster • Doc ID 2028705.1: How to Upgrade/Downgrade MySQL Cluster When Using MySQL Cluster Manager (MCM) • Doc ID 1941472.1: How to Upgrade MySQL Cluster using RPM Service request (SR) Patches Bugs/Defects Etc 55 MySQL Support: www.mysql.com/support
  55. 55. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Summary • Web-Scale Performance with Carrier-Grade Availability • Scale-Out • Real Time Performance • 99.999% Uptime • SQL & NoSQL Access Methods 56
  56. 56. Q&A Thank you for your attention 57
  57. 57. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 58
  58. 58. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 59

×