The OSSCube MySQL High Availability Tutorial

4,984
-1

Published on

The OSSCube MySQL High Availability Tutorial presentation given by Rakesh Kumar and Sonali Minocha at OSI Days 2010

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,984
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
59
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

The OSSCube MySQL High Availability Tutorial

  1. 1. MySQL HA Tutorial Presented by – Sonali Minocha and Rakesh Kumar OSSCube
  2. 2. Who Am I? MySQL Cluster Tutorial © OSSCube
  3. 3. Architecture and Organization
  4. 4. What is a Cluster? MySQL Cluster Tutorial, © OSSCube
  5. 5. Single MySQL Server Architecture SQL Request MySQL Server MySQL Client MySQL Client Response MySQL Cluster Tutorial, © OSSCube
  6. 6. MySQL Cluster Architecture NDB API Call SQL Request MySQL Client Partitioning Replication Message Response Response SQL Node (MySQL server) MySQL Cluster Tutorial, © OSSCube Data Nodes
  7. 7. Components of MySQL Cluster MySQL Cluster Tutorial © OSSCube
  8. 8. Various Nodes API Node Data Node Application Management Node MySQL Cluster Tutorial, © OSSCube
  9. 9. NdbCluster Engine NdbCluster Engine (Data Nodes) MySQL Cluster Tutorial,© OSSCube NdbCluster Engine
  10. 10. Organization MySQL Cluster Tutorial,© OSSCube
  11. 11. Partition MySQL Cluster Tutorial, © OSSCube
  12. 12. Partitioning MySQL Cluster Tutorial © OSSCube
  13. 13. Fragments MySQL Cluster Tutorial© OSSCube
  14. 14. Node Groups MySQL Cluster Tutorial, © OSSCube
  15. 15. Split Brain I have full responsibility of the cluster Servers can not see each other I have full responsibility of the cluster MySQL Cluster Tutorial,© OSSCube
  16. 16. Arbitration NG 1 Network Split First Node to ask will continue while the other will be shut down MySQL Cluster Tutorial, © OSSCube
  17. 17. More Data Nodes NG 1 Network Split/ Shutdown NG 2 MySQL Cluster Tutorial, © OSSCube
  18. 18. Arbitrator Issues MySQL Cluster Tutorial, © OSSCube
  19. 19. Arbitrator Issues MySQL Cluster Tutorial, © OSSCube
  20. 20. Normal Operation Ia Al m iv e Ia Al m iv e m I a ive Al Ia A m liv e MySQL Cluster Tutorial,© OSSCube
  21. 21. Missed Heartbeats Ia Al m iv e I am Alive m I a ive Al MySQL Cluster Tutorial, © OSSCube
  22. 22. Synchronous MySQL Cluster Tutorial, © OSSCube
  23. 23. Phase One: CommitRequest Commit Commit Message Message Response Response Message Message MySQL Cluster Tutorial, © OSSCube
  24. 24. Phase Two: Successful Commit Transaction Transaction Successful Successful Commit Commit Transaction Transaction MySQL Cluster Tutorial, © OSSCube
  25. 25. Phase Two: Failure (Abort Commit) Transaction Transaction Failed Successful Rollback Rollback Transaction Transaction MySQL Cluster Tutorial, © OSSCube
  26. 26. INSTALL MySQL CLUSTER
  27. 27. Initiate MySQL Cluster • • • • • • • • # file "config.ini" - 2 data nodes and 2 SQL nodes# This file is placed in the startup directory of ndb_mgmd (the# management server) # The first MySQL Server can be started from any host. The second# can be started only on the host MySQLd_5.MySQL.com [NDBD DEFAULT] NoOfReplicas= 2 DataDir= /var/lib/MySQL-cluster [NDB_MGMD] Hostname= ndb_mgmd.MySQL.com DataDir= /var/lib/MySQL-cluster [NDBD] HostName= ndbd_2.MySQL.com [NDBD] HostName=ndbd_3.MySQL.com [MySQLD] [MySQLD] HostName= MySQLd_5.MySQL.com MySQL Cluster Tutorial, © OSSCube
  28. 28. Management Node MySQL Cluster Tutorial, © OSSCube
  29. 29. Data Nodes • my.cnf# example additions to my.cnf for MySQL Cluster • [ndbd] connectstring=ndb_mgmd.MySQL.com MySQL Cluster Tutorial, © OSSCube
  30. 30. Data Node Configurations Options MySQL Cluster Tutorial, © OSSCube
  31. 31. SQL Nodes • my.cnf • # example additions to my.cnf for MySQL Cluster # (will work on all versions) • # enable ndbcluster storage engine, and provide connectstring for management • # server host to the default port 1186 • [mysqld] Ndbcluster ndbconnectstring=ndb_mgmd.MySQL.com MySQL Cluster Tutorial, © OSSCube :1186
  32. 32. Config.ini Parameters General Parameters – Id – Hostname File and Directory Location Parameter – BackupDataDir – DataDir – FileSystemPath MySQL Cluster Tutorial, © OSSCube
  33. 33. MySQL Cluster Tutorial, © OSSCube
  34. 34. •Transaction Handling Parameters •TransactionInactiveTimeout •TransactionDeadlockDetectionTimeou t •Memory Usage Parameters •LockPagesInMainMemory •Utility Programs •ndb_size.pl Cluster Tutorial, © OSSCube MySQL
  35. 35. Monitoring the Startup Phases MySQL Cluster Tutorial, © OSSCube
  36. 36. MySQL Cluster Storage Engine --NDB
  37. 37. NDB Storage Engine MySQL Cluster Tutorial, © OSSCube
  38. 38. NDB’s Storage Method • In-Memory Storage • Disk-Based Storage – New in MySQL 5.1 – Undo Logging – Fixed-Width Columns MySQL Cluster Tutorial, © OSSCube
  39. 39. Tablespace Datafile Tablespace Datafile Datafile Datafile Log File group Redo log Undo Log MySQL Cluster Tutorial, © OSSCube
  40. 40. Indexes MySQL Cluster Tutorial, © OSSCube
  41. 41. Creating Indexes • Each table in the cluster always has a hash index for a Primary Key • Ordered indexes can be created with ALTER TABLE ADD INDEX .... MySQL Cluster Tutorial, © OSSCube
  42. 42. First Node/First Fragment • • Hidden field/table created that will handle the management of the Unique Index on Name Ordered Index (T-Tree) created for Continent field Hidden Field Code_Hash 1351 1785 1943 2031 2345 Code CHE CHN FIN GOR SKO Name Switzerland China Finland United Kingdom South Korea Continent Europe Asia Europe Europe Asia Ordered Index Hidden Table MySQL Cluster Tutorial, © OSSCube
  43. 43. Second Node/Second Fragment • • Same as First Node/Fragment setup Contains the remaining data Hidden Field Code Hash 1231 1853 2145 2197 Code BRA ESP IND MOR Name Brazil Spain India Morocco Continent South America Europe Asia Africa Ordered Index Hidden Table MySQL Cluster Tutorial, © OSSCube
  44. 44. Unique Key Lookup Primary Key Lookup may UniqueKey Lookup network hop require another PK values Function() Hash Look up row Data node Data node Look up row Data node Data node MySQL Server MySQL Cluster Tutorial, © OSSCube
  45. 45. Parallel Ordered Index Scan Data node Data node TTTree Tree Index Index rows MySQL Server Data node Data node MySQL Cluster Tutorial, © OSSCube TTTree Tree Index Index
  46. 46. Parallel full table scan Data node Data node rows MySQL Server Data node Data node MySQL Cluster Tutorial, OSSPAC 09 Singapore, © OSSCube
  47. 47. Parallel full table scan ( WHERE condition processed in SQL node) SQL Query Data node Data node Few Rows Few Rows Many Rows rows Rows Many MySQL Server Where Conditi on MySQL Cluster Tutorial, © OSSCube Data node Data node
  48. 48. Parallel full table scan ( WHERE condition processed in data node) SQL Query Where Conditi Fewrows on Rows Few Rows Few Rows Few Rows Data node Data node MySQL Server Where Conditi on MySQL Cluster Tutorial, © OSSCube Data node Data node
  49. 49. Storage of Indexes MySQL Cluster Tutorial, © OSSCube
  50. 50. Characteristics of NDB storage Engine MySQL Cluster Tutorial, © OSSCube
  51. 51. Durability MySQL Cluster Tutorial, © OSSCube
  52. 52. Global Check Point • A GCP occurs every few seconds, when transactions for all nodes are synchronized and the REDO log is flushed to disk MySQL Cluster Tutorial, © OSSCube
  53. 53. Local Check Point MySQL Cluster Tutorial, © OSSCube
  54. 54. Memory - RAM • Memory Allocation Total Memory = N(data nodes)* Local Memory / NoOfReplicas -- or -Local Memory = Total Memory * NoOfReplicas / N(data nodes) • Sizing up the Memory – Fixed size in memory – Variable sized in memory – Fixed size on disk MySQL Cluster Tutorial, © OSSCube
  55. 55. Memory Pages MySQL Cluster Tutorial, © OSSCube
  56. 56. MySQL Cluster Tutorial, © OSSCube
  57. 57. HANDLING DATA IN MySQL CLUSTER
  58. 58. Disk Data Objects MySQL Cluster Tutorial, © OSSCube
  59. 59. Creating the Log File Group • Create Log File Group CREATE LOGFILE GROUP lg_1 ADD UNDOFILE 'undo_1.dat' INITIAL_SIZE 16M UNDO_BUFFER_SIZE 2M ENGINE NDB; • Adding Undo Files to Log File Group ALTER LOGFILE GROUP lg_1 ADD UNDOFILE 'undo_2.dat' INITIAL_SIZE 12M ENGINE NDB; MySQL Cluster Tutorial, © OSSCube
  60. 60. Verifying UNDO Files are Created • INFORMATION_SCHEMA Tables SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_NAME = 'undo_1.dat'; SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_NAME = 'undo_2.dat'; MySQL Cluster Tutorial, © OSSCube
  61. 61. Creating the Tablespace • Create a Tablespace CREATE TABLESPACE ts_1 ADD DATAFILE 'data_1.dat' USE LOGFILE GROUP lg_1 INITIAL_SIZE 32M ENGINE NDB; • Add an Additional Data File ALTER TABLESPACE ts_1 ADD DATAFILE 'data_2.dat' INITIAL_SIZE 48M ENGINE NDB; • Verifying Data Files are Created SELECT FILE_NAME, LOGFILE_GROUP_NAME, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE TABLESPACE_NAME = 'ts_1' AND FILE_TYPE = 'DATAFILE'; MySQL Cluster Tutorial, © OSSCube
  62. 62. Create a Disk Data Table • Non-Indexed Columns Stored on Disk in Tablespace CREATE TABLE dt_1 ( memberId INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, lName VARCHAR(50) NOT NULL, fName VARCHAR(50) NOT NULL, dob DATE NOT NULL, joined DATE NOT NULL, INDEX(lName, fName) ) TABLESPACE ts_1 STORAGE DISK ENGINE NDB; • Alter Existing Non-Cluster Tables ALTER TABLE city TABLESPACE ts1 STORAGE DISK ENGINE=NDB; • View Table Status SHOW TABLE STATUS LIKE 'city'G MySQL Cluster Tutorial, © OSSCube
  63. 63. Monitoring Free Space MySQL Cluster Tutorial, © OSSCube
  64. 64. Dropping Cluster Tables MySQL Cluster Tutorial, © OSSCube
  65. 65. MySQL CLUSTER MANAGEMENT
  66. 66. MySQL Cluster’s Native Backup Tool • Backup Data – .ctl – .log – .data Data Node • Abort Backup Data Node Backup-1.2.ctl Backup-1.2.log Backup-1.0.2.Data Backup-1.1.2.Data Backup-1.3.ctl Backup-1.3.log Backup-1.0.3.Data Backup-1.1.3.Data Data Data ndb_mgm> ABORT BACKUP 6 Node Node Abort of backup 6 ordered Node 2: Backup 10 started from 1 has been aborted. Error: 1321 MySQL Cluster Tutorial, © OSSCube
  67. 67. MySQLdump MySQL Cluster Tutorial, © OSSCube
  68. 68. Restoring Backups • Recovering MySQL Cluster's Native Backup Files (1/3) – Verify Management Node is Running shell> ndb_mgm -e "SHOW" – Start and Empty the Data Nodes • Start each data node with the --initial option. MySQL Cluster Tutorial, © OSSCube
  69. 69. Restoring Backups MySQL Cluster Tutorial, © OSSCube
  70. 70. Restoring Backups MySQL Cluster Tutorial, © OSSCube
  71. 71. Restoring MySQLdump Files MySQL Cluster Tutorial, © OSSCube
  72. 72. Optimization
  73. 73. EXPLAIN MySQL Cluster Tutorial, © OSSCube
  74. 74. Full Table Scan • Without Condition Pushdown • With Condition Pushdown MySQL Cluster Tutorial, © OSSCube
  75. 75. Parallel full table scan ( WHERE condition processed in SQL node) SQL Query Few Rows Where Condition Data node Many rows Rows MySQL Server MySQL Cluster Tutorial, © OSSCube Data node
  76. 76. Parallel full table scan ( WHERE condition processed in data node) SQL Query Where Condition Data node Few Rows Few Rows rows MySQL Server MySQL Cluster Tutorial, © OSSCube Where Data Condition node
  77. 77. Index Statistics MySQL Cluster Tutorial, © OSSCube
  78. 78. MySQL Cluster Query Cache MySQL Cluster Tutorial, © OSSCube
  79. 79. Data Size/Usage Issues MySQL Cluster Tutorial, © OSSCube
  80. 80. Query Design MySQL Cluster Tutorial, © OSSCube
  81. 81. MySQL Cluster Replication
  82. 82. MySQL Replication MySQL Server Slave Application W MySQL Server Master Binlog Relaylog IO thread SQL thread MySQL Server Slave Relaylog MySQL Cluster Tutorial, © OSSCube Binlog
  83. 83. mysqld mysqld Update Data node Update Data node Cluster Data node Data node Update Update Ndb API Ndb API MySQL Cluster Tutorial, © OSSCube
  84. 84. NDB Injector Thread • A thread inside MySQL server • Responsible for injecting rows in binlog and produces “Single connical binlog for cluster” • not just one MySQL Server • It contains everything written on all ndbApi program including MySQLd connected to cluster MySQL Cluster Tutorial, © OSSCube
  85. 85. MySQL Cluster Replication MySQLd(slave) MySQLd(Master) Replication NdbClsuter Handler IO thread Binlog Data node Cluster Data node Data node Data node Binlog NdbClsuter Handler Relaylog Apply thread Data node Data node Cluster Data node Data node MySQL Cluster Tutorial, © OSSCube
  86. 86. QnA
  87. 87. Thank you for your time and attention For more information, please feel free to drop in a line to sales@osscube.com or visit http://www.osscube.com Or call us at 1-888-9OSSCube (Toll Free for USA) 1-919-791-5472 (From outside USA) MySQL Cluster Tutorial, © OSSCube

×