Reliable MySQL Using Replication Issac Goldstand Mirimar Networks www.mirimar.net [email_address]
What is replication? <ul><li>Allows 2 or more databases to maintain state between them </li></ul><ul><li>MySQL 3.23 and up...
Replication basics… <ul><li>One master supports multiple slaves </li></ul><ul><li>Each node on the network is assigned a u...
Replication basics… <ul><li>Clients perform data modification on master server </li></ul><ul><li>INSERT, SELECT, DELETE, L...
Replication basics… <ul><li>Immediately following execution of command on master, the command is written to the local bina...
Replication basics… <ul><li>If the slave is online, the command is transmitted to the slave in parallel (well, immediately...
Replication basics… <ul><li>Once the data is received in the slave’s relay log, the slave SQL thread executes the command ...
Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul>Master Slave Slave Slave Client (...
Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular...
Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular...
Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular...
Simple Replication Setup <ul><li>Modify my.cnf to include a unique  server-id  for each node </li></ul><ul><li>On master s...
Configure master my.cnf ------------- [mysqld] server-id = 1 log-bin
Configure slave my.cnf ------------- [mysqld] server-id = 2 master-user = someuser master-password = secret master-host = ...
Initial dataset <ul><li>Binary log provides a record of all modifications to master database starting from a fixed point: ...
Configure log position MASTER mysql> SHOW MASTER STATUS; +---------------------------+----------+--------------+----------...
Using CHANGE MASTER TO <ul><li>MASTER_HOST </li></ul><ul><li>MASTER_USER </li></ul><ul><li>MASTER_PASSWORD </li></ul><ul><...
Advanced CHANGE MASTER TO <ul><li>MASTER_PORT </li></ul><ul><li>RELAY_LOG_FILE </li></ul><ul><li>RELAY_LOG_POS </li></ul><...
Confirming it works SLAVE mysql> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_I...
Permissions <ul><li>Slaves need  REPLICATION SLAVE  permission on master for basic usage </li></ul><ul><li>If  LOAD TABLE ...
Internal Threads <ul><li>Since MySQL 4.0, replication slaves run two threads </li></ul><ul><li>IO thread continuously rece...
IO thread isolation <ul><li>Isolating IO thread means that slave won’t have to wait for long-executing statements to finis...
SQL thread isolation <ul><li>SQL thread isolation allows for replication in an environment without a continuous link betwe...
Master Thread <ul><li>Additionally, the master server runs the Binlog Dump thread </li></ul><ul><li>This thread is simply ...
Status files <ul><li>2 status files for replication’s use </li></ul><ul><li>Their use is to record the state of replicatio...
Information in master.info <ul><li>Master log file </li></ul><ul><li>Read master log pos </li></ul><ul><li>Master Host </l...
Information in relay-log.info <ul><li>Relay log file </li></ul><ul><li>Relay log pos </li></ul><ul><li>Relay master-log po...
Backup master <ul><li>Master backups can be accomplished with mysqldump </li></ul><ul><li>Care must be taken to ensure the...
Backup master files <ul><li>If a file-system level backup is required, care should be taken to manually record binary log ...
Backup slave <ul><li>Same idea as master file system backup </li></ul><ul><li>Instead of recording position, it’s enough t...
Live demo <ul><li>Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup </li></ul>
For more information <ul><li>MySQL documentation </li></ul><ul><li>5.0 documentation  http://mirror.mirimar.net/mysql/doc/...
Thank You! For more information: Issac Goldstand margol@mirimar.net http://www.beamartyr.net/ http://www.mirimar.net/
Upcoming SlideShare
Loading in...5
×

Download presentation

81,103

Published on

19 Comments
19 Likes
Statistics
Notes
No Downloads
Views
Total Views
81,103
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
629
Comments
19
Likes
19
Embeds 0
No embeds

No notes for slide

Download presentation

  1. 1. Reliable MySQL Using Replication Issac Goldstand Mirimar Networks www.mirimar.net [email_address]
  2. 2. What is replication? <ul><li>Allows 2 or more databases to maintain state between them </li></ul><ul><li>MySQL 3.23 and up supports asynchronous replication </li></ul><ul><li>One server acts as a master for one or more slaves </li></ul><ul><li>Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally </li></ul>
  3. 3. Replication basics… <ul><li>One master supports multiple slaves </li></ul><ul><li>Each node on the network is assigned a unique identifying number </li></ul><ul><li>Each slave attempts to connect to the master and fetches data to be replicated </li></ul>Master Slave Slave Slave Slaves
  4. 4. Replication basics… <ul><li>Clients perform data modification on master server </li></ul><ul><li>INSERT, SELECT, DELETE, LOAD DATA </li></ul><ul><li>EXECUTE in MySQL 5.0 and above </li></ul>DATA DATA INSERT INTO … Master Slave Client
  5. 5. Replication basics… <ul><li>Immediately following execution of command on master, the command is written to the local binary log </li></ul><ul><li>Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc. </li></ul>DATA DATA INSERT INTO … Binary Log Master Slave Client
  6. 6. Replication basics… <ul><li>If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log </li></ul><ul><li>Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log </li></ul><ul><li>The slave’s replication IO thread stores the command in the local relay log </li></ul>DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread Master Slave Client
  7. 7. Replication basics… <ul><li>Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master </li></ul><ul><li>In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this was changed to boost performance </li></ul>DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread DATA Master Slave Client
  8. 8. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul>Master Slave Slave Slave Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (INSERT)
  9. 9. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul>MySQL Server MySQL Server MySQL Server Client INSERT INTO… INSERT INTO … Client
  10. 10. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul><ul><li>High availability (hot failover) </li></ul>DATA DATA DATA DATA DATA Client Master Slave
  11. 11. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul><ul><li>High availability (hot failover) </li></ul><ul><li>Snapshot backups </li></ul>Master Slave Backup with LOCK TABLES (or single transaction) Client INSERT / SELECT Client INSERT / SELECT
  12. 12. Simple Replication Setup <ul><li>Modify my.cnf to include a unique server-id for each node </li></ul><ul><li>On master server, ensure that log-bin (binary logging) is enabled in my.cnf </li></ul><ul><li>On slave, configure login credentials on master, either via my.cnf or CHANGE MASTER TO statement </li></ul><ul><li>Copy initial data snapshot from master to slave </li></ul><ul><li>Configure initial binary log position on slave </li></ul><ul><li>Start replication with SLAVE START command </li></ul>
  13. 13. Configure master my.cnf ------------- [mysqld] server-id = 1 log-bin
  14. 14. Configure slave my.cnf ------------- [mysqld] server-id = 2 master-user = someuser master-password = secret master-host = ip.of.master
  15. 15. Initial dataset <ul><li>Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated </li></ul><ul><li>If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date </li></ul><ul><li>Otherwise, a snapshot of the master must be taken, using mysqldump –master-data , to provide an initial dataset for the slave(s) </li></ul><ul><li>If only MyISAM tables are used, the LOAD DATA FROM MASTER statement may be used on the slave(s) </li></ul>
  16. 16. Configure log position MASTER mysql> SHOW MASTER STATUS; +---------------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------------------+----------+--------------+------------------+ | vmware-mirimar-bin.000002 | 79 | | | +---------------------------+----------+--------------+------------------+ SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘ vmware-mirimar-bin.000002 ’, MASTER_LOG_POS= 79 ; SLAVE mysql> START SLAVE;
  17. 17. Using CHANGE MASTER TO <ul><li>MASTER_HOST </li></ul><ul><li>MASTER_USER </li></ul><ul><li>MASTER_PASSWORD </li></ul><ul><li>MASTER_LOG_FILE </li></ul><ul><li>MASTER_LOG_POS </li></ul>
  18. 18. Advanced CHANGE MASTER TO <ul><li>MASTER_PORT </li></ul><ul><li>RELAY_LOG_FILE </li></ul><ul><li>RELAY_LOG_POS </li></ul><ul><li>MASTER_SSL </li></ul><ul><li>SSL: CA/CAPATH/CERT/KEY/CIPHER </li></ul>
  19. 19. Confirming it works SLAVE mysql> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: vmware-mirimar Master_User: someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: vmware-mirimar-bin.000002 Read_Master_Log_Pos: 79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250 Seconds_Behind_Master: 0
  20. 20. Permissions <ul><li>Slaves need REPLICATION SLAVE permission on master for basic usage </li></ul><ul><li>If LOAD TABLE FROM MASTER or LOAD DATA FROM MASTER statements are used, slave will also need SUPER and RELOAD privileges </li></ul>
  21. 21. Internal Threads <ul><li>Since MySQL 4.0, replication slaves run two threads </li></ul><ul><li>IO thread continuously receives updates from master and writes to local relay log </li></ul><ul><li>SQL thread continuously executes statements in relay log </li></ul>
  22. 22. IO thread isolation <ul><li>Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master </li></ul><ul><li>Also, slave will continue reading data from master if a statement creates a data conflict </li></ul>
  23. 23. SQL thread isolation <ul><li>SQL thread isolation allows for replication in an environment without a continuous link between slave and masters </li></ul><ul><li>If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts) </li></ul><ul><li>SQL thread will continue processing relay logs even while IO thread is unable to connect to master </li></ul>
  24. 24. Master Thread <ul><li>Additionally, the master server runs the Binlog Dump thread </li></ul><ul><li>This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave </li></ul><ul><li>If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected </li></ul>
  25. 25. Status files <ul><li>2 status files for replication’s use </li></ul><ul><li>Their use is to record the state of replication between server shutdown and startup </li></ul><ul><li>master.info records information about the slave’s master server </li></ul><ul><li>relay-log.info records information about the local relay logs </li></ul>
  26. 26. Information in master.info <ul><li>Master log file </li></ul><ul><li>Read master log pos </li></ul><ul><li>Master Host </li></ul><ul><li>Master User </li></ul><ul><li>Password (will not be shown in SHOW SLAVE STATUS) </li></ul><ul><li>Master Port </li></ul><ul><li>Connect Retry </li></ul><ul><li>In MySQL 4.1+, SSL options are stored if SSL is used </li></ul>
  27. 27. Information in relay-log.info <ul><li>Relay log file </li></ul><ul><li>Relay log pos </li></ul><ul><li>Relay master-log pos </li></ul><ul><li>Exec master-log pos </li></ul>
  28. 28. Backup master <ul><li>Master backups can be accomplished with mysqldump </li></ul><ul><li>Care must be taken to ensure the following 2 special considerations: </li></ul><ul><ul><li>Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB) </li></ul></ul><ul><ul><li>Recording of binary log information, for use on slaves (master-data) </li></ul></ul>
  29. 29. Backup master files <ul><li>If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement. </li></ul><ul><li>To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS) </li></ul><ul><li>LEAVE THE CLIENT CONNECTED!!! </li></ul><ul><li>After backup finishes, execute UNLOCK TABLES to release the read lock </li></ul>
  30. 30. Backup slave <ul><li>Same idea as master file system backup </li></ul><ul><li>Instead of recording position, it’s enough to backup the master.info and relay-log.info files </li></ul><ul><li>Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes </li></ul>
  31. 31. Live demo <ul><li>Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup </li></ul>
  32. 32. For more information <ul><li>MySQL documentation </li></ul><ul><li>5.0 documentation http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html </li></ul><ul><li>4.1 documentation http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html </li></ul>
  33. 33. Thank You! For more information: Issac Goldstand margol@mirimar.net http://www.beamartyr.net/ http://www.mirimar.net/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×