Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Download presentation

  • Login to see the comments

Download presentation

  1. 1. Reliable MySQL Using Replication Issac Goldstand Mirimar Networks www.mirimar.net [email_address]
  2. 2. What is replication? <ul><li>Allows 2 or more databases to maintain state between them </li></ul><ul><li>MySQL 3.23 and up supports asynchronous replication </li></ul><ul><li>One server acts as a master for one or more slaves </li></ul><ul><li>Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally </li></ul>
  3. 3. Replication basics… <ul><li>One master supports multiple slaves </li></ul><ul><li>Each node on the network is assigned a unique identifying number </li></ul><ul><li>Each slave attempts to connect to the master and fetches data to be replicated </li></ul>Master Slave Slave Slave Slaves
  4. 4. Replication basics… <ul><li>Clients perform data modification on master server </li></ul><ul><li>INSERT, SELECT, DELETE, LOAD DATA </li></ul><ul><li>EXECUTE in MySQL 5.0 and above </li></ul>DATA DATA INSERT INTO … Master Slave Client
  5. 5. Replication basics… <ul><li>Immediately following execution of command on master, the command is written to the local binary log </li></ul><ul><li>Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc. </li></ul>DATA DATA INSERT INTO … Binary Log Master Slave Client
  6. 6. Replication basics… <ul><li>If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log </li></ul><ul><li>Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log </li></ul><ul><li>The slave’s replication IO thread stores the command in the local relay log </li></ul>DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread Master Slave Client
  7. 7. Replication basics… <ul><li>Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master </li></ul><ul><li>In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this was changed to boost performance </li></ul>DATA DATA INSERT INTO … INSERT INTO … Relay Log Replication Thread DATA Master Slave Client
  8. 8. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul>Master Slave Slave Slave Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (INSERT)
  9. 9. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul>MySQL Server MySQL Server MySQL Server Client INSERT INTO… INSERT INTO … Client
  10. 10. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul><ul><li>High availability (hot failover) </li></ul>DATA DATA DATA DATA DATA Client Master Slave
  11. 11. Replication Strategies <ul><li>Load balancing – single write, distributed read </li></ul><ul><li>Load balancing – circular read/write </li></ul><ul><li>High availability (hot failover) </li></ul><ul><li>Snapshot backups </li></ul>Master Slave Backup with LOCK TABLES (or single transaction) Client INSERT / SELECT Client INSERT / SELECT
  12. 12. Simple Replication Setup <ul><li>Modify my.cnf to include a unique server-id for each node </li></ul><ul><li>On master server, ensure that log-bin (binary logging) is enabled in my.cnf </li></ul><ul><li>On slave, configure login credentials on master, either via my.cnf or CHANGE MASTER TO statement </li></ul><ul><li>Copy initial data snapshot from master to slave </li></ul><ul><li>Configure initial binary log position on slave </li></ul><ul><li>Start replication with SLAVE START command </li></ul>
  13. 13. Configure master my.cnf ------------- [mysqld] server-id = 1 log-bin
  14. 14. Configure slave my.cnf ------------- [mysqld] server-id = 2 master-user = someuser master-password = secret master-host = ip.of.master
  15. 15. Initial dataset <ul><li>Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated </li></ul><ul><li>If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date </li></ul><ul><li>Otherwise, a snapshot of the master must be taken, using mysqldump –master-data , to provide an initial dataset for the slave(s) </li></ul><ul><li>If only MyISAM tables are used, the LOAD DATA FROM MASTER statement may be used on the slave(s) </li></ul>
  16. 16. Configure log position MASTER mysql> SHOW MASTER STATUS; +---------------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +---------------------------+----------+--------------+------------------+ | vmware-mirimar-bin.000002 | 79 | | | +---------------------------+----------+--------------+------------------+ SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘ vmware-mirimar-bin.000002 ’, MASTER_LOG_POS= 79 ; SLAVE mysql> START SLAVE;
  17. 17. Using CHANGE MASTER TO <ul><li>MASTER_HOST </li></ul><ul><li>MASTER_USER </li></ul><ul><li>MASTER_PASSWORD </li></ul><ul><li>MASTER_LOG_FILE </li></ul><ul><li>MASTER_LOG_POS </li></ul>
  18. 18. Advanced CHANGE MASTER TO <ul><li>MASTER_PORT </li></ul><ul><li>RELAY_LOG_FILE </li></ul><ul><li>RELAY_LOG_POS </li></ul><ul><li>MASTER_SSL </li></ul><ul><li>SSL: CA/CAPATH/CERT/KEY/CIPHER </li></ul>
  19. 19. Confirming it works SLAVE mysql> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: vmware-mirimar Master_User: someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: vmware-mirimar-bin.000002 Read_Master_Log_Pos: 79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250 Seconds_Behind_Master: 0
  20. 20. Permissions <ul><li>Slaves need REPLICATION SLAVE permission on master for basic usage </li></ul><ul><li>If LOAD TABLE FROM MASTER or LOAD DATA FROM MASTER statements are used, slave will also need SUPER and RELOAD privileges </li></ul>
  21. 21. Internal Threads <ul><li>Since MySQL 4.0, replication slaves run two threads </li></ul><ul><li>IO thread continuously receives updates from master and writes to local relay log </li></ul><ul><li>SQL thread continuously executes statements in relay log </li></ul>
  22. 22. IO thread isolation <ul><li>Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master </li></ul><ul><li>Also, slave will continue reading data from master if a statement creates a data conflict </li></ul>
  23. 23. SQL thread isolation <ul><li>SQL thread isolation allows for replication in an environment without a continuous link between slave and masters </li></ul><ul><li>If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts) </li></ul><ul><li>SQL thread will continue processing relay logs even while IO thread is unable to connect to master </li></ul>
  24. 24. Master Thread <ul><li>Additionally, the master server runs the Binlog Dump thread </li></ul><ul><li>This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave </li></ul><ul><li>If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected </li></ul>
  25. 25. Status files <ul><li>2 status files for replication’s use </li></ul><ul><li>Their use is to record the state of replication between server shutdown and startup </li></ul><ul><li>master.info records information about the slave’s master server </li></ul><ul><li>relay-log.info records information about the local relay logs </li></ul>
  26. 26. Information in master.info <ul><li>Master log file </li></ul><ul><li>Read master log pos </li></ul><ul><li>Master Host </li></ul><ul><li>Master User </li></ul><ul><li>Password (will not be shown in SHOW SLAVE STATUS) </li></ul><ul><li>Master Port </li></ul><ul><li>Connect Retry </li></ul><ul><li>In MySQL 4.1+, SSL options are stored if SSL is used </li></ul>
  27. 27. Information in relay-log.info <ul><li>Relay log file </li></ul><ul><li>Relay log pos </li></ul><ul><li>Relay master-log pos </li></ul><ul><li>Exec master-log pos </li></ul>
  28. 28. Backup master <ul><li>Master backups can be accomplished with mysqldump </li></ul><ul><li>Care must be taken to ensure the following 2 special considerations: </li></ul><ul><ul><li>Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB) </li></ul></ul><ul><ul><li>Recording of binary log information, for use on slaves (master-data) </li></ul></ul>
  29. 29. Backup master files <ul><li>If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement. </li></ul><ul><li>To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS) </li></ul><ul><li>LEAVE THE CLIENT CONNECTED!!! </li></ul><ul><li>After backup finishes, execute UNLOCK TABLES to release the read lock </li></ul>
  30. 30. Backup slave <ul><li>Same idea as master file system backup </li></ul><ul><li>Instead of recording position, it’s enough to backup the master.info and relay-log.info files </li></ul><ul><li>Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes </li></ul>
  31. 31. Live demo <ul><li>Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup </li></ul>
  32. 32. For more information <ul><li>MySQL documentation </li></ul><ul><li>5.0 documentation http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html </li></ul><ul><li>4.1 documentation http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html </li></ul>
  33. 33. Thank You! For more information: Issac Goldstand margol@mirimar.net http://www.beamartyr.net/ http://www.mirimar.net/

×