OSSCube MySQL Cluster Tutorial By Sonali At Osspac 09

MySQL Cluster Tutorial

Presented by – Sonali Minocha

Who Am I?

MySQL Cluster Tutorial, OSSPAC 09
Singapore, © OSSCube

What is a Cluster?


Single MySQL Server Architecture

SQL Request

MySQL Client MySQL Server

Response

MySQL Cluster Architecture
NDB API Call

SQL Request
MySQL Client
Partitioning
Response Replication
Message

Response
Data Nodes
SQL Node (MySQL server)

Components of MySQL Cluster


Various Nodes

API Node Data
Node

Application

Management Node


NdbCluster NdbCluster NdbCluster
Engine Engine Engine

(Data Nodes)


Organization


Partition


Partitioning


Fragments


Node Groups


Split Brain

I have full
Servers can I have full
responsibility
not see responsibility
of the cluster
each other of the cluster


Arbitration

NG
1

Network
Split
First Node to ask will continue while the other will be shut
down


More Data Nodes

NG Network
1 Split/
Shutdown
N
G

2


Arbitrator Issues


Normal Operation

I a ive
m
Al
Al m
Ia
e
iv

e
liv
A m

Al m
Ia

Ia
e
MySQL Cluster Tutorial, OSSPAC 09 iv

Missed Heartbeats

I a ive
m
Al
Alive
I am

Al m
Ia
e
MySQL Cluster Tutorial, OSSPAC 09 iv

Synchronous


Phase One: Commit-
Request

Commit Commit

Message Message

Response Response

Message Message


Phase Two: Successful
Commit

Transaction Transaction

Successful Successful

Commit Commit



Phase Two: Failure
(Abort Commit)


Failed Successful

Rollback Rollback



Initiate MySQL Cluster
• # file "config.ini" - 2 data nodes and 2 SQL nodes# This file
is placed in the startup directory of ndb_mgmd (the#
management server)
• # The first MySQL Server can be started from any host. The
second# can be started only on the host MySQLd_5.MySQL.com
• [NDBD DEFAULT]
NoOfReplicas= 2
DataDir= /var/lib/MySQL-cluster
• [NDB_MGMD]
Hostname= ndb_mgmd.MySQL.com
DataDir= /var/lib/MySQL-cluster
• [NDBD]
HostName= ndbd_2.MySQL.com
• [NDBD]
HostName=ndbd_3.MySQL.com
• [MySQLD]
• [MySQLD]
HostName= MySQLd_5.MySQL.com

Management Node


Data Nodes
• my.cnf# example additions to
my.cnf for MySQL Cluster
• [ndbd]
connect-
string=ndb_mgmd.MySQL.com


Data Node Configurations
Options


SQL Nodes
• my.cnf
• # example additions to my.cnf for
MySQL Cluster
# (will work on all versions)
• # enable ndbcluster storage engine,
and provide connectstring for
management
• # server host to the default port
1186
• [MySQLd]
Ndbcluster

ndbconnectstring=ndb_mgmd.MySQL.com
:1186 Singapore, © OSSCube

Config.ini Parameters
General Parameters
– Id
– Hostname

File and Directory Location Parameter
– BackupDataDir
– DataDir
– FileSystemPath


•Transaction Handling Parameters
•TransactionInactiveTimeout
•TransactionDeadlockDetectionTimeou
t

•Memory Usage Parameters
•LockPagesInMainMemory

•Utility Programs
•ndb_size.pl

Monitoring the Startup Phases

MySQL Cluster Tutorial, OSSPAC 09 Singapore, © OSSCube.
http://osscube.com/technology/mysql

MySQL Cluster Storage Engine --NDB


NDB Storage Engine


NDB’s Storage Method

• In-Memory Storage

• Disk-Based Storage
– New in MySQL 5.1
– Undo Logging
– Fixed-Width Columns


Tablespace Tablespace

Datafile Datafile Datafile
Datafile

Log File group

Redo log Undo Log

Indexes


Creating Indexes
• Each table in the cluster always has a hash index for a Primary Key
• Ordered indexes can be created with ALTER TABLE ADD INDEX ....


First Node/First Fragment
• Hidden field/table created that will handle the management of the
Unique Index on Name
• Ordered Index (T-Tree) created for Continent field
Code_Hash Code Name Continent
1351 CHE Switzerland Europe
1785 CHN China Asia
Hidden 1943 FIN Finland Europe
Field 2031 GOR United Kingdom Europe
2345 SKO South Korea Asia

Ordered
Index

Hidden
Table


Second Node/Second Fragment
• Same as First Node/Fragment setup
• Contains the remaining data
Code Hash Code Name Continent
1231 BRA Brazil South America
1853 ESP Spain Europe
Hidden
Field 2145 IND India Asia
2197 MOR Morocco Africa

Ordered
Index

Hidden
Table


UniqueKey Lookup network hop
Primary Key Lookup may
Unique Key Lookup
require another

Function() Look up row
PK values Hash Data node

Look up row

Data node

MySQL Server


Parallel Ordered Index
Scan

T-
Data node Tree
Index
rows

MySQL
Server Data node
T-
Tree
Index


Parallel full table scan

Data node

rows

MySQL
Server Data node


Parallel full table scan ( WHERE
condition processed in SQL
node)

SQL Query
Data node
Few Rows
rows Rows
Many

MySQL
Where
Server
Conditi
on Data node


SQL Query
Data node
Where
Few Rows Conditi
Fewrows on
Rows

MySQL
Server

Where Data node
Conditi
on


Storage of Indexes


Characteristics of NDB
storage Engine


Durability


Global Check Point

• A GCP occurs every few seconds, when
transactions for all nodes are synchronized
and the REDO log is flushed to disk


Local Check Point


Memory - RAM

• Memory Allocation
Total Memory = N(data nodes)* Local Memory /
NoOfReplicas
-- or --
Local Memory = Total Memory * NoOfReplicas / N(data
nodes)

• Sizing up the
Memory
– Fixed size in memory
– Variable sized in memory
– Fixed size on disk

Memory Pages


HANDLING DATA IN MySQL
CLUSTER

Disk Data Objects


Creating the Log File Group
• Create Log File Group
CREATE LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_1.dat'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE 2M
ENGINE NDB;
• Adding Undo Files to Log File Group
ALTER LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_2.dat'
INITIAL_SIZE 12M
ENGINE NDB;


Verifying UNDO Files are
Created
• INFORMATION_SCHEMA Tables
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,
EXTRA
FROM INFORMATION_SCHEMA.FILES
WHERE FILE_NAME = 'undo_1.dat';

SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,
EXTRA
WHERE FILE_NAME = 'undo_2.dat';


Creating the Tablespace
• Create a Tablespace
CREATE TABLESPACE ts_1
ADD DATAFILE 'data_1.dat'
USE LOGFILE GROUP lg_1
INITIAL_SIZE 32M
ENGINE NDB;
• Add an Additional Data File
ALTER TABLESPACE ts_1
ADD DATAFILE 'data_2.dat'
INITIAL_SIZE 48M
ENGINE NDB;
• Verifying Data Files are Created
SELECT FILE_NAME, LOGFILE_GROUP_NAME, EXTRA
WHERE TABLESPACE_NAME = 'ts_1' AND FILE_TYPE =
'DATAFILE';


Create a Disk Data Table
• Non-Indexed Columns Stored on Disk in Tablespace
CREATE TABLE dt_1 (
memberId INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
lName VARCHAR(50) NOT NULL, fName VARCHAR(50) NOT NULL,
dob DATE NOT NULL, joined DATE NOT NULL,
INDEX(lName, fName)
) TABLESPACE ts_1 STORAGE DISK ENGINE NDB;

• Alter Existing Non-Cluster Tables
ALTER TABLE city TABLESPACE ts1 STORAGE DISK ENGINE=NDB;

• View Table Status
SHOW TABLE STATUS LIKE 'city'G


Monitoring Free Space


Dropping Cluster Tables


MySQL Cluster’s Native Backup Tool

Backup-1.2.ctl
• Backup Data Data Data Backup-1.2.log
Backup-1.0.2.Data
– .ctl Node Node
Backup-1.1.2.Data
– .log
– .data

• Abort Backup Backup-1.3.ctl
Data Data Backup-1.3.log
ndb_mgm> ABORT BACKUP 6
Node Node Backup-1.0.3.Data
Backup-1.1.3.Data
Abort of backup 6 ordered
Node 2: Backup 10 started from 1 has
been aborted. Error: 1321

MySQLdump


Restoring Backups
• Recovering MySQL Cluster's Native Backup Files (1/3)
– Verify Management Node is Running
shell> ndb_mgm -e "SHOW"
– Start and Empty the Data Nodes
• Start each data node with the --initial option.


Restoring Backups


Restoring MySQLdump Files


EXPLAIN


Full Table Scan
• Without Condition Pushdown
• With Condition Pushdown


Parallel full table scan
( WHERE condition
processed in SQL node)
SQL Query
Data node
Where
Few Rows Condition
rows Rows
Many

MySQL
Server Data node


Parallel full table scan ( WHERE
condition processed in data
node)
Where
SQL Query Condition
Data node
Few Rows
Few Rows
rows

MySQL Where
Server Condition node
Data


Index Statistics


MySQL Cluster Query Cache


Data Size/Usage Issues


Query Design


MySQL Replication

MySQL Server
Application W Slave

MySQL Server Relaylog
Master IO thread
SQL thread

MySQL Server
Binlog
Slave

Relaylog Binlog

MySQLd MySQLd
Update

Data node Update
Data node
Cluster
Data node Data node

Update
Update
Ndb API Ndb API

NDB Injector Thread
•A thread inside MySQL server
•Responsible for injecting rows in
binlog and produces “Single connical
binlog for cluster”
• not just one MySQL Server
•It contains everything written on all
ndbApi program including MySQLd
connected to cluster

MySQL Cluster Replication

MySQLd(slave) Binlog

MySQLd(Master) Replication
NdbClsuter Handler

NdbClsuter Handler Relaylog
IO thread

Binlog Apply thread

Data node Data node
Data node Data node
Cluster
Cluster Data node Data node
Data node Data node

Thank you for your time and attention

www.osscube.com

For more information, please feel free to drop in a line to
sonali@osscube.com or visit http://www.osscube.com


OSSCube MySQL Cluster Tutorial By Sonali At Osspac 09

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to OSSCube MySQL Cluster Tutorial By Sonali At Osspac 09

Similar to OSSCube MySQL Cluster Tutorial By Sonali At Osspac 09 (20)

More from OSSCube

More from OSSCube (20)

Recently uploaded

Recently uploaded (20)

OSSCube MySQL Cluster Tutorial By Sonali At Osspac 09