MySQL Cluster Basics

MySQL Cluster Wagner Bianchi – LAD Senior Principal Consulting [email_address]

Disclaimer The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle ’s products remains at the sole discretion of Oracle.

Technology Specifications ,[object Object],[object Object],NodeType MachineName IP Management node1 192.168.0.101 Management node2 192.168.0.102 Data/Storage node3 192.168.0.103 Data/Storage node4 192.168.0.104 Data/Storage node5 192.168.0.105 Data/Storage node6 192.168.0.106 SQL/API node7 192.168.0.107 SQL/API node8 192.168.0.108 SQL/API node9 192.168.0.109 SQL/API node10 192.168.0.110

Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],- Part I - Part II

Management Node ,[object Object],[object Object],[object Object],[object Object]

Management Node ,[object Object],[object Object],# executing na online native backup shell> ndb_mgm –e “START BACKUP”

Management Node ,[object Object],[object Object],# executing na online native backup shell> ndb_mgm –e “ALL REPORT memoryusage” # checking data node’s status shell> ndb_mgm –e “ALL STATUS”

Management Node ,[object Object],[object Object],# start or restarting data nodes after its execution shell> ndb_mgm –e “4 [ START | RESTART ]”

Management Node ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Management Node ,[object Object],# creating directories -> BASEDIR and DATADIR [ root@node1 ~ ] mkdir -p /usr/local/mysql-cluster [ root@node1 ~ ] mkdir -p /var/lib/mysql-cluster [ root@node1 ~ ] cd /usr/local/mysql-cluster # downloading management node necessary packages [ root@node1 ~ ] wget http://downloads.mysql...management.rpm [ root@node1 ~ ] wget http://downloads.mysql...tools.rpm # installing necessary packages [ root@node1 mysql-cluster ] rpm -ivh MySQL-* Preparing... ############################################ [100%] 1:MySQL-Cluster-gpl-management ########################### [100%] 2:MySQL-Cluster-gpl-tools ################################ [100%]

Management Node ,[object Object],[object Object],# command-line management daemon execution shell> ndb_mgmd --config-file=/path/to/file # using short form to state cluster’s config file shell> ndb_mgmd –f /path/to/file

Management Node ,[object Object],[object Object],[ root@node1 ]# ps aux | grep ndb_mgmd root 103467 17.3 0.7 8398 2564 ? Ssl 3:20 3:55 ndb_mgmd –f /usr/local/mysql-cluster/config.ini # SHOW command retrieves information about cluster [ root@node1 ]# ndb_mgm –e “ SHOW”

Data/Storage Node ,[object Object],[object Object],[object Object]

Data/Storage Node ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data/Storage Node ,[object Object],[object Object],[object Object],[object Object],[object Object],This is the same idea of multi-threaded replication applied on slave server when using MySQL 5.6 (milestone yet!!)

SQL/API Node ,[object Object],[object Object],[object Object],[object Object],[object Object]

A good strategy for packages… ,[object Object]

Configuration File ,[object Object],[object Object],[object Object],[object Object],[object Object]

Local Configuration File ,[object Object],[object Object],[ndbd] # local ndbd’s configuration file - /etc/my.cnf ndb-connectstring=192.168.0.101:1186,192.168.0.102:1186 NoStart # ndbd do not start after be invoked

Global Configuration File ,[object Object],[object Object],[object Object]

Global Configuration File [ndb_mgmd] NodeId=1 HostName=192.168.0.101 [ndbd default] NoOfReplicas=2 DataDir=/var/lib/mysql-cluster StopOnError=false # angel will restart failed nodes [ndbd] NodeId=2 HostName=192.168.0.102 [ndbd] NodeId=3 HostName=192.168.0.103 [mysqld] NodeIde=4 HostName=192.168.0.104 [mysqld] # free reserved slot – e.g. ndb_restore, the native cli for database restore

The Cluster Connectstring ,[object Object],[object Object],[object Object],[object Object],[object Object]

Cluster Partitioning ,[object Object],[object Object],[object Object]

Cluster Partitioning ,[object Object],[object Object],[object Object],[object Object]

Cluster Partitioning ,[object Object],[object Object],#some columns were omitted from the final result to better accommodate it -- Per partition info – Partition Row count Commit count Frag fixed memory Frag varsized memory 2 2 2 65536 327680 1 2 2 65536 327680 3 NDBT_ProgramExit: 0 - OK

Cluster Partitioning ,[object Object],[object Object],# alter partitions in order to adjust and better accommodate data # among all partitions, fragments and data nodes mysql> ALTER ONLINE TABLE world.City -> REORGANIZE PARTITION; Query OK, 0 rows affected (1.52 sec)

Disk-based and In-memory Tables

Disk-based Tables ,[object Object],[object Object],[object Object],[object Object]

Disk-based Tables ,[object Object],[object Object]

Disk-based Tables # creating database world mysql> CREATE DATABASE world; Query OK, 0 rows affected (1.52 sec) # creating logfile group world_log mysql> CREATE LOGFILE GROUP world_log ADD UNDOFILE ‘world_log.dat’ -> UNDO_BUFFER_SIZE=16M INITIAL_SIZE=200M ENGINE=NDB; Query OK, 0 rows affected (2.23 sec) # creating tablespace world_tbs, where will reside table`s data mysql> CREATE TABLESPACE world_tbs ADD DATAFILE ‘world_tbs.dat’ -> USE LOGFILE GROUP world_log INITIAL_SIZE=500M ENGINE=NDB; Query OK, 0 rows affected (1.09 sec) # restoring world database from world.sql file shell> cat world.sql | sed –e ‘s/ENGINE=MyISAM/STORAGE DISK TABLESPACE world_tbs ENGINE = NDB/g’ | mysql world

Disk-based Tables ,[object Object],[object Object],# on data node (node3) I used ndb_show_tables filtering results shell> ndb_show_tables –c 192.168.1.101 –d world | grep Tablespace 9 Tablespace Online - world_tbs

Disk Storage Metadata ,[object Object],[object Object]

Disk Storage Metadata ,[object Object],# creating a simple view to check that main question CREATE VIEW check_space AS SELECT FILE_NAME, ((TOTAL_EXTENTS * EXTENT_SIZE)/(1024 * 1024)) AS ‘Total MB’, ((FREE_EXTENTS * EXTENT_SIZE)/(1024*1024)) AS ‘Free MB’, ( ((FREE_EXTENTS * EXTENT_SIZE)*100) /(TOTAL_EXTENTS * EXTENT_SIZE) ) AS ‘% Free’, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE ENGINE = ‘NDBCLUSTER’ AND FILE_TYPE = ‘DATAFILE’;

Disk Storage Metadata ,[object Object],# results returned after the query’s execution mysql> select * from check_space; +---------------+----------+----------+---------+----------------+ | FILE_NAME | Total MB | Free MB | % Free | EXTRA | +---------------+----------+----------+---------+----------------+ | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=3 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=4 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=5 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=6 | +---------------+----------+----------+---------+----------------+ 4 rows in set (0.11 sec)

Disk Storage Metadata ,[object Object],[object Object],# results returned after the query’s execution mysql> ALTER TABLESPACE world_tbs -> ADD DATAFILE ‘world_tbs-1.dat’ ENGINE=NDB; Query OK, 0 rows affected (4.11 sec) # results returned after the query’s execution mysql> select * from check_space; +-----------------+----------+----------+----------+-----------------+ | FILE_NAME | Total MB | Free MB | % Free | EXTRA | +-----------------+----------+----------+----------+----------------+ | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=3 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=4 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=5 | | world_tbs.dat | 500.0000 | 494.0000 | 98.8000 | CLUSTER_NODE=6 | | world_tbs-1.dat | 100.0000 | 100.0000 | 100.0000 | CLUSTER_NODE=3 | | world_tbs-1.dat | 100.0000 | 100.0000 | 100.0000 | CLUSTER_NODE=4 | | world_tbs-1.dat | 100.0000 | 100.0000 | 100.0000 | CLUSTER_NODE=5 | | world_tbs-1.dat | 100.0000 | 100.0000 | 100.0000 | CLUSTER_NODE=6 | +-----------------+----------+----------+----------+----------------+ 8 rows in set (0.08 sec)

Disk Storage Metadata ,[object Object],# trying to remove a tbs without remove logfile group used mysql> DROP TABLESPACE world_tbs ENGINE=NDB; ERROR 1529 (HY000): Failed to drop TABLESPACE mysql> SHOW WARNINGS; +-----------------------------------------------------------------+ Error | 1296 | Got error 768 'Cant drop filegroup, filegroup is used' from NDB | Error | 1529 | Failed to drop TABLESPACE +-----------------------------------------------------------------+ 2 rows in set (0.01 sec)

Disk Storage Metadata ,[object Object],# checking if logs are reach its max file size mysql> SELECT CONCAT('CLUSTER NODE: ', node_id) AS NodeId, -> FORMAT(total/1024/1024,0) AS 'Total Space (MB)', -> FORMAT(used/1024/1024,0) AS 'Used Space (MB)' -> FROM logspaces -> GROUP BY NodeId; +-----------------+------------------+-----------------+ | NodeId | Total Space (MB) | Used Space (MB) | +-----------------+------------------+-----------------+ | CLUSTER NODE: 3 | 256 | 0 | | CLUSTER NODE: 4 | 256 | 0 | | CLUSTER NODE: 5 | 256 | 0 | | CLUSTER NODE: 6 | 256 | 0 | +-----------------+------------------+-----------------+ 4 rows in set (0.26 sec)

In-memory Tables ,[object Object],[object Object],[object Object]

In-memory Tables ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Good strategy to place’s tables ,[object Object],[object Object],[object Object],[object Object],[object Object]

Calculating DataMemory and IndexMemory ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Install epel repository and use “ yum install ”

Calculating DataMemory and IndexMemory ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Calculating DataMemory and IndexMemory ,[object Object],# executing ndb_size.pl on command-line shell> ndb_size.pl --user=wb --password=“123” --hostname=192.168.1.107 --database=world --format=text # checking how much memory is required to put world database into a “MC” Parameter Minimum Requirements (world database) ----------------------------------------------- * indicates greater than default Parameter Default 4.1 5.0 5.1 DataMemory (KB) 81920 480 480 512 NoOfOrderedIndexes 128 3 3 3 NoOfTables 128 3 3 3 IndexMemory (KB) 18432 192 88 88 NoOfUniqueHashIndexes 64 0 0 0 NoOfAttributes 1000 24 24 24 NoOfTriggers 768 15 15 15

MySQL Cluster Replication ,[object Object],[object Object],[object Object],[object Object]

MySQL Cluster Replication ,[object Object],[object Object]

MySQL Cluster Replication ,[object Object],[object Object],# read only will avoid write query execution from any user # apart from root -- root will continue be able to do writes mysql> SET GLOBAL read_only = 1; Query OK, 0 rows affected (0.00 sec)

MySQL Cluster Replication ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Backup & Restore ,[object Object],[object Object],[object Object]

Backup – SINGLE USER MODE ,[object Object],# entering single user mode – other SQL/API will be ignored ndb_mgm> ENTER SINGLE USER MODE 4 Single user mode entered Access is granted for API node 4 only ndb_mgm> START BACKUP NOWAIT ndb_mgm> ndb_mgm> EXIT SINGLE USER MODE Exiting single user mode in progress Use ALL STATUS or SHOW to see when single user mode has been exited

Backup – START BACKUP ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Backup – START BACKUP ,[object Object],[object Object],[object Object],[object Object],[object Object]

Backup – START BACKUP ,[object Object],# using START BACKUP with NOWAIT and monitoring its process ndb_mgm> START BACKUP NOWAIT ndb_mgm> ALL REPORT BACKUPSTATUS Node 3: Local backup status: backup 3 started from node 1 #Records: 0 #LogRecords: 0 Data: 0 bytes Log: 0 bytes Node 4: Local backup status: backup 3 started from node 1 #Records: 0 #LogRecords: 0 Data: 0 bytes Log: 0 bytes ndb_mgm> Node 3: Backup 3 started from node 1 Node 3: Backup 3 started from node 1 completed StartGCP: 5444 StopGCP: 5447 #Records: 7370 #LogRecords: 0 Data: 497788 bytes Log: 0 bytes

Backup – START BACKUP ,[object Object],[object Object],# listing DataDir/BACKUP subdirectories [root@node3]# ls –l BACKUP-1 # listing files from BACKUP-1 directory [root@node3 BACKUP]# ls -l BACKUP-1/ total 276 -rw-r--r-- 1 root root 253388 Jun 7 17:57 BACKUP-1-0.3.Data -rw-r--r-- 1 root root 17660 Jun 7 17:57 BACKUP-1.3.ctl -rw-r--r-- 1 root root 52 Jun 7 17:57 BACKUP-1.3.log

Restore - ndb_restore ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Restore - ndb_restore ,[object Object]

Engine Condition Pushdown ,[object Object],[object Object]

[object Object],Engine Condition Pushdown Data Nodes Many Rows MySQL Few Rows SQL Query engine_condition_pushdown=0

Engine Condition Pushdown ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Engine Condition Pushdown ,[object Object],engine_condition_pushdown=1 Few Rows MySQL Few Rows SQL Query

Iptraf ,[object Object],[object Object],[object Object],[object Object],[object Object]

MySQL Cluster’s Architecture

Thanks! ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

MySQL Cluster Basics

MySQL Cluster Basics

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(15)

Similar to MySQL Cluster Basics

Similar to MySQL Cluster Basics(20)

More from Wagner Bianchi

More from Wagner Bianchi(20)

Recently uploaded

Recently uploaded(20)

MySQL Cluster Basics

Editor's Notes