The OSSCube MySQL High Availability Tutorial

MySQL HA Tutorial Presented by – Sonali Minocha and Rakesh Kumar OSSCube

Who Am I? MySQL Cluster Tutorial © OSSCube

Architecture and Organization

What is a Cluster? MySQL Cluster Tutorial, © OSSCube

Single MySQL Server Architecture MySQL Client SQL Request Response MySQL Cluster Tutorial, © OSSCube MySQL Server

MySQL Cluster Architecture MySQL Client SQL Request Response SQL Node (MySQL server) Data Nodes Response NDB API Call Partitioning Replication Message MySQL Cluster Tutorial, © OSSCube

Components of MySQL Cluster MySQL Cluster Tutorial © OSSCube

Various Nodes Application API Node Data Node Management Node MySQL Cluster Tutorial, © OSSCube

(Data Nodes) MySQL Cluster Tutorial,© OSSCube NdbCluster Engine MySQL Server NdbCluster Engine MySQL Server NdbCluster Engine MySQL Server

Organization MySQL Cluster Tutorial,© OSSCube

Partition MySQL Cluster Tutorial, © OSSCube

Partitioning MySQL Cluster Tutorial © OSSCube

Fragments MySQL Cluster Tutorial© OSSCube

Node Groups MySQL Cluster Tutorial, © OSSCube

Split Brain Servers can not see each other I have full responsibility of the cluster I have full responsibility of the cluster MySQL Cluster Tutorial,© OSSCube

Arbitration Network Split NG 1 MySQL Cluster Tutorial, © OSSCube First Node to ask will continue while the other will be shut down

More Data Nodes NG 1 Network Split/ Shutdown MySQL Cluster Tutorial, © OSSCube NG 2

Arbitrator Issues MySQL Cluster Tutorial, © OSSCube

Normal Operation I am Alive I am Alive I am Alive I am Alive MySQL Cluster Tutorial,© OSSCube

Missed Heartbeats I am Alive I am Alive I am Alive MySQL Cluster Tutorial, © OSSCube

Synchronous MySQL Cluster Tutorial, © OSSCube

Phase One: Commit-Request MySQL Cluster Tutorial, © OSSCube

Phase Two: Successful Commit MySQL Cluster Tutorial, © OSSCube

Phase Two: Failure (Abort Commit) MySQL Cluster Tutorial, © OSSCube

INSTALL MySQL CLUSTER

Initiate MySQL Cluster
# file "config.ini" - 2 data nodes and 2 SQL nodes# This file is placed in the startup directory of ndb_mgmd (the# management server)
# The first MySQL Server can be started from any host. The second# can be started only on the host MySQLd_5.MySQL.com
[NDBD DEFAULT]
NoOfReplicas= 2
DataDir= /var/lib/MySQL-cluster
[NDB_MGMD]
Hostname= ndb_mgmd.MySQL.com
DataDir= /var/lib/MySQL-cluster
[NDBD]
HostName= ndbd_2.MySQL.com
[NDBD]
HostName=ndbd_3.MySQL.com
[MySQLD]
[MySQLD]
HostName= MySQLd_5.MySQL.com
MySQL Cluster Tutorial, © OSSCube

Management Node MySQL Cluster Tutorial, © OSSCube

Data Nodes
my.cnf# example additions to my.cnf for MySQL Cluster
[ndbd]
connect-string=ndb_mgmd.MySQL.com

Data Node Configurations Options MySQL Cluster Tutorial, © OSSCube

SQL Nodes
my.cnf
# example additions to my.cnf for MySQL Cluster # (will work on all versions)
# enable ndbcluster storage engine, and provide connectstring for management
# server host to the default port 1186
[mysqld]
Ndbcluster
ndbconnectstring=ndb_mgmd.MySQL.com:1186

Config.ini Parameters
General Parameters
Id
Hostname
File and Directory Location Parameter
BackupDataDir
DataDir
FileSystemPath

Transaction Handling Parameters
TransactionInactiveTimeout
TransactionDeadlockDetectionTimeout
Memory Usage Parameters
LockPagesInMainMemory
Utility Programs
ndb_size.pl

Monitoring the Startup Phases MySQL Cluster Tutorial, © OSSCube

MySQL Cluster Storage Engine --NDB

NDB Storage Engine MySQL Cluster Tutorial, © OSSCube

NDB’s Storage Method
In-Memory Storage
Disk-Based Storage
New in MySQL 5.1
Undo Logging
Fixed-Width Columns

MySQL Cluster Tutorial, © OSSCube Tablespace Tablespace Log File group Datafile Datafile Datafile Datafile Redo log Undo Log

Indexes MySQL Cluster Tutorial, © OSSCube

Creating Indexes
Each table in the cluster always has a hash index for a Primary Key
Ordered indexes can be created with ALTER TABLE ADD INDEX ....

First Node/First Fragment
Hidden field/table created that will handle the management of the Unique Index on Name
Ordered Index (T-Tree) created for Continent field

Second Node/Second Fragment
Same as First Node/Fragment setup
Contains the remaining data

Primary Key Lookup PK values Function() Hash Data node Look up row Data node MySQL Server Unique Key Lookup Look up row Unique Key Lookup may require another network hop MySQL Cluster Tutorial, © OSSCube

Parallel Ordered Index Scan Data node Data node MySQL Server rows T- Tree Index T- Tree Index MySQL Cluster Tutorial, © OSSCube

Parallel full table scan Data node Data node MySQL Server rows MySQL Cluster Tutorial, OSSPAC 09 Singapore, © OSSCube

Parallel full table scan ( WHERE condition processed in SQL node) Data node Data node MySQL Server rows SQL Query Many Rows Few Rows Where Condition MySQL Cluster Tutorial, © OSSCube

Parallel full table scan ( WHERE condition processed in data node) Data node Data node MySQL Server rows SQL Query Few Rows Few Rows Where Condition Where Condition MySQL Cluster Tutorial, © OSSCube

Storage of Indexes MySQL Cluster Tutorial, © OSSCube

Characteristics of NDB storage Engine MySQL Cluster Tutorial, © OSSCube

Durability MySQL Cluster Tutorial, © OSSCube

Global Check Point
A GCP occurs every few seconds, when transactions for all nodes are synchronized and the REDO log is flushed to disk

Local Check Point MySQL Cluster Tutorial, © OSSCube

Memory - RAM
Memory Allocation
Total Memory = N(data nodes)* Local Memory / NoOfReplicas
-- or --
Local Memory = Total Memory * NoOfReplicas / N(data nodes)
Sizing up the Memory
Fixed size in memory
Variable sized in memory
Fixed size on disk

Memory Pages MySQL Cluster Tutorial, © OSSCube

HANDLING DATA IN MySQL CLUSTER

Disk Data Objects MySQL Cluster Tutorial, © OSSCube

Creating the Log File Group
Create Log File Group
CREATE LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_1.dat'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE 2M
ENGINE NDB;
Adding Undo Files to Log File Group
ALTER LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_2.dat'
INITIAL_SIZE 12M
ENGINE NDB;

Verifying UNDO Files are Created
INFORMATION_SCHEMA Tables
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER, EXTRA
FROM INFORMATION_SCHEMA.FILES
WHERE FILE_NAME = 'undo_1.dat';
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER, EXTRA
WHERE FILE_NAME = 'undo_2.dat';

Creating the Tablespace
Create a Tablespace
CREATE TABLESPACE ts_1
ADD DATAFILE 'data_1.dat'
USE LOGFILE GROUP lg_1
INITIAL_SIZE 32M
ENGINE NDB;
Add an Additional Data File
ALTER TABLESPACE ts_1
ADD DATAFILE 'data_2.dat'
INITIAL_SIZE 48M
ENGINE NDB;
Verifying Data Files are Created
SELECT FILE_NAME, LOGFILE_GROUP_NAME, EXTRA
WHERE TABLESPACE_NAME = 'ts_1' AND FILE_TYPE = 'DATAFILE';

Create a Disk Data Table
Non-Indexed Columns Stored on Disk in Tablespace
CREATE TABLE dt_1 (
memberId INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
lName VARCHAR(50) NOT NULL, fName VARCHAR(50) NOT NULL,
dob DATE NOT NULL, joined DATE NOT NULL,
INDEX(lName, fName)
) TABLESPACE ts_1 STORAGE DISK ENGINE NDB;
Alter Existing Non-Cluster Tables
ALTER TABLE city TABLESPACE ts1 STORAGE DISK ENGINE=NDB;
View Table Status
SHOW TABLE STATUS LIKE 'city'G

MySQL CLUSTER MANAGEMENT

MySQL Cluster’s Native Backup Tool
Backup Data
.ctl
.log
.data
Abort Backup
ndb_mgm> ABORT BACKUP 6
Abort of backup 6 ordered
Node 2: Backup 10 started from 1 has been aborted. Error: 1321

Restoring Backups
Recovering MySQL Cluster's Native Backup Files (1/3)
Verify Management Node is Running
shell> ndb_mgm -e "SHOW"
Start and Empty the Data Nodes
Start each data node with the --initial option.

Optimization

Full Table Scan
Without Condition Pushdown
With Condition Pushdown

Parallel full table scan ( WHERE condition processed in SQL node) Data node Data node MySQL Server rows SQL Query Many Rows Few Rows Where Condition MySQL Cluster Tutorial, © OSSCube

Parallel full table scan ( WHERE condition processed in data node) Data node Data node MySQL Server rows SQL Query Few Rows Where Condition Where Condition Few Rows MySQL Cluster Tutorial, © OSSCube

MySQL Cluster Replication

MySQL Server Master MySQL Server Slave MySQL Server Slave IO thread SQL thread Binlog Relaylog Relaylog Binlog Application W MySQL Replication MySQL Cluster Tutorial, © OSSCube

Cluster Data node Data node Data node Data node mysqld mysqld Update Update Ndb API Ndb API Update Update MySQL Cluster Tutorial, © OSSCube

NDB Injector Thread
A thread inside MySQL server
Responsible for injecting rows in binlog and produces “Single connical binlog for cluster”
not just one MySQL Server
It contains everything written on all ndbApi program including MySQLd connected to cluster

NdbClsuter Handler IO thread Binlog Relaylog Binlog MySQLd(Master) MySQL Cluster Replication Cluster Data node Data node Data node Data node Replication MySQLd(slave) NdbClsuter Handler Apply thread Cluster Data node Data node Data node Data node MySQL Cluster Tutorial, © OSSCube

Thank you for your time and attention
For more information, please feel free to drop in a line to [email_address] or visit http://www.osscube.com Or call us at 1-888-9OSSCube (Toll Free for USA) 1-919-791-5472 (From outside USA) MySQL Cluster Tutorial, © OSSCube

http://www.osscube.com	237
http://staging.osscube.com	67
http://local.osscube.com	8
http://www.osscube.in	5
http://dev3qa.osscube.co.uk	3
http://druposs.osscube.com	3
http://localhost	2
http://dev.osscube.com	2
http://osslocal.com	2
http://dev3qa.osscube.co.in	2
http://osscube.localhost	1
http://stagingcrm.osscube.com	1
http://www.osscube.co.uk	1
http://feeds2.feedburner.com	1

The OSSCube MySQL High Availability Tutorial

by OSSCube LLC A Global Open Source Enterprise for Open Source Solutions on Sep 28, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

14 Embeds 335

Statistics

The OSSCube MySQL High Availability Tutorial — Presentation Transcript