This document provides an overview of MySQL Cluster architecture, maintenance, and support. It discusses the core architecture including data distribution across nodes, features like high availability and scalability, and the cluster structure with management, data, and SQL nodes. It also covers configuration files, database partitioning, basic administration functions like starting and stopping nodes, replication demonstrations, logging, backup and restore processes, and monitoring tools.

2. Introduction:
A short presentation with a focus on architecture, maintenance and
support of MySQL Cluster database (Portal database).

3. Agenda:
 MySQL Network Database Architecture
Network Database Overview
Features
Structure
Fault Tolerance
Configuration File
Partitioning
 Maintenance
Basic Admin functions
Monitoring
Backup and restore
 Conclusion

4. MySQL Cluster: Core architecture
 What is it?
 MySQL Cluster; MySQL nodes run mysql, data is stored on data nodes which
run network database daemons (ndbd). Clients connect to any of the MySQL
nodes to run a query. MySQL nodes connect to the data nodes to store and
retrieve the necessary data.

5. MySQL Cluster: Features
 MySQL Cluster!
 A high-availability, high redundancy version of MySQL server for distributed
environments. A tool to partition and synchronize data on multiple servers.
 These network of database resources provide such advantages as redundancy,
zero down time and prevents single-point-of failure.
 How is Clustering achieved?
 Data is split-up among multiple servers so that a single server failure cannot
cause downtime. Data distribution is managed by a network of db’s (ndb).
 Multiple mysql servers connect to an ndb cluster to read/write. If one node
(server) goes down the other nodes (servers) services clients requests.

6. MySQL Cluster: Features
 High Availability
 MySQL Cluster (NDB Cluster) is a storage engine that offers 99.9% availability
 Scalability
 Throughput up to 200M queries per second
 Designed for many parallel and short transactions
 High Performance
 Primary Key based queries, insertions and short index scans
 Low partition latency
 Allows human-unnoticeable delays between an input being processed and the
corresponding output.
 Self-healing (node recovery and cluster recovery)
 Failed nodes automatically restart and resync with other nodes before joining the
cluster. With complete application transparency

7. MySQL Cluster: Features
 Synchronous Replication –
 Data within each data node is synchronously replicated to another data node
 Failed nodes are automatically and transparently updated with current data
 Automatic Failover –
 MySQL cluster heartbeating mechanism instantly detects node failures and
automatically fails over to other nodes within the cluster, without interrupting
service to clients
 Shared Nothing, no single-point-of failure –
 Each node has its own disk and memory, the risk of a failure caused by shared
components such as storage is eliminated.
 If all nodes in partition fails cluster shuts down
 In ndbd partitioning Data Consistency is preferred over Availability (CAPS
theorem)

8. MySQL Cluster: Structure
 NDB is configured with a range of load-balancing and failover options.
 Each part of the cluster is considered a node (process)
 Cluster nodes include;
 Management Nodes (ndb_mgm)
 Manages the other nodes within the cluster (config data, start and stop nodes, running
backups etc). Manages cluster configuration file.
 This node is started first before other nodes
 Data Nodes (ndbd); (two or more nodes)
 Stores and manages data, txn mg’t and query execution. (> ndbd)
 MySQL servers Nodes;
 This node accesses the cluster data, (> mysqld –ndbcluster)
 MySQL Client Node;
 Access for client login
 Management Client;
 Provides admin functions such as cluster status, starting backups etc

9. MySQL Cluster: Model
MGM Node MGM Node
mysql nodemysql node
Application Nodes
LDAP
Server 2Server 1
Node Group
Data Node 1 Data Node 2

10. MySQL Cluster: Fault Tolerance
MGM Node MGM Node
mysql nodemysql node
Application Nodes
LDAP
Server 2Server 1
Node Group
Data Node 1 Data Node 2

11. MySQL Cluster: Fault Tolerance
MGM Node MGM Node
mysql nodemysql node
Application Nodes
LDAP
Server 2Server 1
Node Group
Data Node 1 Data Node 2

12. MySQL Cluster: Fault Tolerance
MGM Node MGM Node
mysql nodemysql node
Application Nodes
LDAP
Server 2Server 1
Node Group
Data Node 1 Data Node 2

13. Portal db Server: Structure
MySQL Server Database IP
PROD Blackhole 192.168.82.83
PROD Wormhole 192.168.82.82
PPRD Cosmicray 192.168.82.80
PPRD Dimension 192.168.82.81
TEST Quantum 192.168.82.68
TEST Supernova 192.168.82.69

14. Portal db Server: Struct-dimension/cosmicray
Node Type Machine Name IP Server
Management - ndb_mgmd(MGM) node id=1 192.168.82.80 Cosmicray
Management - ndb_mgmd(MGM) node id=6 192.168.82.81 Dimensions
Data/Storage - ndbd(NDB) node id=2 192.168.82.80 Cosmicray
Data/Storage - ndbd(NDB) node id=3 192.168.82.81 Dimensions
SQL/API node id=4 192.168.82.80 Cosmicray
SQL/API node id=5 192.168.82.81 Dimensions
SQL/API node id=7 Not connected

15. Configuration Files

16. MySQL Cluster: Configuration files
 To operate nodes in the cluster information about cluster
environment is stored in the cluster configuration file.
 Cluster config file sets all parameters needed
 There are two configuration files
 Local Config file residing on each node (/etc/my.cnf)
 This file provides information on how to nodes connect to the cluster
 Global Config file on management node (/l01/mysql/config.ini)
 Provides information about the cluster as a whole
 Used by management node to start cluster, receive nodes connections

17. MySQL Cluster: Configuration files
 {Server}> /l01/mysql/config.ini
 [ndb_mgmd]
 NodeId=x => node ID usually in numerals
 datadir=/l01/mysql/logs => directory for this mgmt nodes log files
 hostname=192.168.92.80 => hostname or IP address
 [ndbd] => One [ndbd] section per data node
 NodeId=y => node ID usually in numerals
 datadir=/l01/mysql/data => directory for this data nodes data files
 hostname=192.168.92.80 => hostname or IP address
 [mysqld]
 NodeId=z
 Hostname=192.168.92.80
 [tcp default]
 Portnumber=2202 => This is the default however you can use any that is free for all hosts in the cluster
 [ndb default]
 NoOfReplicas=2 => Number of replicas
 DataMemory=500M => Memory allocated for data storage
 IndexMemory=100M => Memory allocated for index storage
 BackupDataDir=/l01/mysql => location of backups

18. Cluster Partitioning

19. MySQL Cluster: Demo Partitioning
 Mysql Cluster data partitioning is a feature that allow data to be
divided and stored on different replication from one node to the
other.
 This involves horizontal partitioning, where rows in a table are divided
horizontally. Cluster uses an internal algorithm to implement this
partitioning so as to have same amount of table roles.
 This ensures evenly balanced memory requirements across all the
nodes.
 Number of cluster partitioning equal number of data nodes in each
cluster group. Each node group has same number of nodes.

20. Data Node 1
Data Node 2
Table T1
MySQL Cluster: Diag-1 Partitioning
P1
P2

21. Data Node 1
Data Node 2
Table T1
F1
MySQL Cluster: Diag-2 Partitioning
P1
P2
FxPrimary Fragment

22. Data Node 1
Data Node 2
F1
Table T1
F1
MySQL Cluster: Diag-3 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment

23. Data Node 1
Data Node 2
F1
Table T1
F1
F2
MySQL Cluster: Diag-4 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment

24. Data Node 1
Data Node 2
F2
F1
Table T1
F1
F2
MySQL Cluster: Diag-5 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment

25. Data Node 1
Data Node 2
F2
F1
Table T1
F1
F2
MySQL Cluster: Diag-6 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment
Node Group
Server 2
Server 1

26. Data Node 1
Data Node 2
F2
F1
Table T1
F1
F2
MySQL Cluster: Diag-7 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment
Node Group
Server 2
Server 1

27. Data Node 1
Data Node 2
F2
F1
Table T1
F1
F2
MySQL Cluster: Diag-8 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment
Node Group
Server 2
Server 1

28. Data Node 1
Data Node 2
F2
F1
Table T1
F1
F2
MySQL Cluster: Diag-9 Partitioning
P1
P2
Fx
FxSecondary Fragment
Primary Fragment
Node Group
Server 2
Server 1

29. Basic Admin Functions

30. MySQL Cluster: Basic Admin Functions
 Using Management Client to maintain nodes
 Login to client;
 Shell > ndb_mgm
 Ndb_mgm> help
 Maintain nodes;
 ndb_mgm> SHOW
 ndb_mgm> <id> STATUS/START/STOP
 ndb_mgm> <id> START
 Cluster Status;
 Shell > ndb_mgm -e "SHOW“

31. MySQL Cluster: Basic Admin Functions
 START & STOP Cluster: 1-[Mg’t Nodes], 2.[Data Nodes], 3.[Mysql Nodes]
 HELP/USAGE;
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh usage
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh help
 START Management Nodes;
 Shell > ndb_mgm –e “SHOW”
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh status
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh start_ndb_mgmd (start mg’mt node)
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh stop_ndb_mgmd (stop mg’mt node)
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh start (start)
 START Data Nodes;
 Shell > ndb_mgm –e “SHOW”
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh status
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh start_ndb (start data node)
 Shell > sudo /l01/mysql/bin/ndb_startup.init.sh stop_ndb (stop data node)
 Shell > ps –ef | grep ndb (verify data node)
 START Mysqld Nodes
 Shell > sudo mysqld_safe &
 Shell > sudo /etc/init.d/mysql.server restart (start/restart/stop)
 Shell > ps –ef | grep mysqld (verify mysqld node)
 STOP Cluster; login to mg’t node and shutdown;
 ndb_mgm > shutdown (shutdown)
 Shell > ndb_mgm –e shutdown

32. MySQL Cluster: Shutdown/Re-Start Cluster
 STOP Cluster:
 shell> ndb_mgm –e shutdown (shutdown Cluster)
 START Cluster:
 shell> sudo /l01/mysql/bin/ndb_startup.init.sh start_ndb_mgmd (start mg’mt)
 shell> ndb_mgm -e "SHOW“
 shell> sudo /l01/mysql/bin/ndb_startup.init.sh start_ndb (start data node)
 shell> sudo /l01/mysql/bin/ndb_startup.init.sh start (start mg’mt & data)
 shell> sudo mysqld_safe & (start mysql nodes)
 shell> ndb_mgm -e "SHOW“ (verify)
 shell> ps –ef | grep ndb (verify ndb)
 shell> ps –ef | grep mysqld (verify mysqld)
 START mysqld:
 shell> sudo /etc/init.d/mysql.server restart (stop/start/restart/)

33. MySQLd: Automatic/Re-Start
 Upon Server Reboot Automatic restart is initiated:
 shell> /etc/rc.d/init.d/mysql.server
 shell> /etc/rc.d/init.d/mysql-cluster
 START MySQLd:
 shell> ps –ef | grep mysqld
 shell> sudo service mysql.server status
 shell> sudo service mysql.server start
 shell> sudo service mysql.server status

34. MySQL Cluster: Demo Cluster Replication
 Demo data replication on Dimension  Cosmicray
 server> mysql –u root –p
 mysql> select @@hostname;
 mysql> show databases;
 mysql> create database ClusterDemo_db;
 mysql> show database;
 mysql> use ClusterDemo_db;
 mysql> create table demotbl (ID int(10), Name char(35), Zip int(10));
 mysql> desc demotbl;
 mysql> insert into demotbl values(001, ‘Kabul’,23508);
 mysql> insert into demotbl values(002, ‘Qandahar’, 20365);
 mysql> insert into demotbl values(003, ‘Herat’, 54231);
 mysql> select * from demotbl;

35. MySQL Cluster: Demo Cluster Replication
 Demo data replication on Dimension  Cosmicray
 mysql> select @@hostname;
 mysql> show databases;
 mysql> use ClusterDemo_db;
 mysql> show tables from ClusterDemo_db;
 mysql> select * from demotbl;
 mysql> alter table demotbl ENGINE=NDBCLUSTER;
 mysql> create table demo (ID int(10), Name char(35), Zip int(10)) ENGINE=NDBCLUSTER;
 mysql> show tables from ClusterDemo_db;
 mysql> select * from demotbl;
 mysql> drop table demotbl;
 mysql> drop database ClusterDemo_db;
 Is data equally replicated/redistributed on all nodes?
 ndb_mgm> all report memory

36. MySQL Cluster: Portal database Logs
 Demo data replication on Dimension  Cosmicray
 shell> cd /l01/mysql/logs
 [shell]> more ndb_3_trace.log.next {}
 [shell]> more ndb_3_error.log {}
 [shell]> more ndb_3_trace.log.5 {}
 [shell]> more ndb_6_out.log {}
 [shell]> more ndb_6_cluster.log {}
 [shell]> more mysqld.log {}
 [shell]> more ndb_3_out.log {}

37. Backup & Restore

38. MySQL Cluster: Backup & Restore
 Backup creates a snapshot of all NDB nodes on the cluster at a given
point in time. This consists of;
 Metadata => Backup-backup_id.node_id.ctl
 table records and => Backup-backup_id.node_id.Data
 transaction logs => Backup-backup_id.node_id.log
 Backup is executed with the “START BACKUP” command and restore
with the command “ndb_restore” via the management client node.
 ndb_mgm > START BACKUP NOWAIT;
 ndb_mgm > ALL REPORT BACKUPSTATUS;
 ndb_mgm > ALL REPORT MEMORY;

39. MySQL Cluster: Backup & Restore
 ndb_restore; the cluster restore program is implemented as a separate
command-line utility with the following options;
 Server> ndb_restore -c 192.168.82.80 -b 1 -n 3 -r /backup-cluster/BACKUP/BACKUP-1
• -c Connecting string
 -b Backup Id = 1
 -n Nodeid = 3
 -r Restore Data
 -m Restore Meta Data
 -Backup location = /backup-cluster/BACKUP/BACKUP-1

40. Monitoring

41. MySQL Cluster: Monitoring
 By continuous monitoring we ensure that databases are up and
servers have enough resources (memory, ram and cpu) to perform
functions efficiently.
 Ensuring that databases are always up and all jobs actively running.
 MySQL monitoring tools include
a. Nagios
b. Crontab/Scripts,
c. OEM
d. Emails

42. MySQL Cluster: Monitoring-Nagios
 Nagios enables distributed monitoring of all Cluster nodes
 disk, load, ndbd, mgmd, telnet and processes

43. MySQL Cluster: Monitoring-Scripts
Scheduled cronjobs enable us to monitor several processes
 Check process script “check_mysql.sh” runs every five (5) mins to ensure that processes
are running. If down “start_mysql.sh” is invoked and email notification sent.
 Check daemon ndbd every 5 mins
 Check daemon ndb_mgm every five mins
 Diskmon process “diskmon.sh” runs every five (5) mins to check the size of disk space.
Notifications are generated when disk space grows beyond the warning and threshold
sizes.
 Backup script “mysql_backup.sh” runs the ff. schedule
 Schedule backups include;
 Monitor if ndb and mgm daemons are up and active
 Monitor if database instance is up
 Monitor diskspace warning (85%) and alert (90%)

44. Conclusion:
 MySQL NDB Cluster is designed to deliver a high availability, fault
tolerant database where no single failure results in loss of service.
 Cluster database provides automatic failover, self healing, shared
nothing architecture and no single point of failure.
 Fault tolerance of cluster databases depends on the following factors;
 Architectural choice of the configuration, placement of nodes on hosts and
resources hosts are dependent on.
 Eliminating single point of failure that could result in either hosts being lost.
 Hosting management node on separate hosts to improve fault-tolerant
solution.

45. QUESTIONS

46. References:
 https://www.digitalocean.com/community/tutorials/how-to-create-a-multi-node-mysql-
cluster-on-ubuntu-16-04
 http://www.clusterdb.com/mysql-cluster/deploying-mysql-cluster-over-multiple-hosts
 https://research.euranova.eu/wp-content/uploads/MySQL-Cluster-Design-and-Architecture-
Principles.compressed.pdf
 http://www.agileload.com/agileload/blog/2013/03/11/optimizing-the-performance-of-
mysql-cluster
 http://messagepassing.blogspot.com/2012/03/cap-theorem-and-mysql-cluster.html
 http://www.clusterdb.com/mysql-cluster/mysql-cluster-fault-tolerance-impact-of-
deployment-decisions
 https://dev.mysql.com/doc/mysql-cluster-excerpt/5.7/en/faqs-mysql-
cluster.html#qandaitem-A-1-40
 https://dev.mysql.com/doc/mysql-cluster-excerpt/5.7/en/mysql-cluster-params-ndbd.html

48. Two node Group Partitioning

49. Data Node 1
Data Node 2
Data Node 3
Data Node 4
Table T1
P2
P3
P4
P1
MySQL Cluster: Demo Partitioning

50. Data Node 1
Data Node 2
F1
Data Node 3
Data Node 4
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

51. Data Node 1
Data Node 2
F1
F1
Data Node 3
Data Node 4
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

52. Data Node 1
Data Node 2
F1
F3 F1
Data Node 3
Data Node 4
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

53. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

54. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

55. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2
F2
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

56. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2
F4 F2
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

57. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Table T1
MySQL Cluster: Demo Partitioning
P2
P3
P4
P1

58. Data Node 1
Data Node 2
F3
F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Table T1
F1
F3
MySQL Cluster: Demo Partitioning
Server 1
P2
P3
P4
P1

59. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
MySQL Cluster: Demo Partitioning
Server 1
Server 2
P2
P3
P4
P1

60. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
MySQL Cluster: Demo Partitioning
Server 1
Server 2
P2
P3
P4
P1

61. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
P
MySQL Cluster: Demo Partitioning
Server 1
Server 2
P2
P3
P4
P1

62. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
MySQL Cluster: Demo Partitioning
Server 1
Server 2
P2
P3
P4
P1

63. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
P2
P3
P4
P1
MySQL Cluster: Demo Partitioning
Server 1
Server 2

64. Data Node 1
Data Node 2
F1 F3
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Table T1
P2
P3
P4
P1
MySQL Cluster: Demo Partitioning
Server 1
Server 2