• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Percona Live 2012PPT: introduction-to-mysql-replication
 

Percona Live 2012PPT: introduction-to-mysql-replication

on

  • 7,858 views

Percona Live 2012PPT: introduction-to-mysql-replication

Percona Live 2012PPT: introduction-to-mysql-replication

Statistics

Views

Total Views
7,858
Views on SlideShare
4,361
Embed Views
3,497

Actions

Likes
30
Downloads
0
Comments
0

13 Embeds 3,497

http://www.mysqlops.com 3250
http://www.scoop.it 93
http://yananguo.gotoip4.com 71
http://www.mysqlpub.com 56
http://www.mysqlops.com. 11
http://www.mysqlops.net 5
http://localhost 3
http://web-01.mysqlops.com 2
http://mysqlops.net 2
http://tuicool.com 1
http://www.mysqlops.com&_=1330064854646 HTTP 1
http://translate.googleusercontent.com 1
http://www.tuicool.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Percona Live 2012PPT: introduction-to-mysql-replication Percona Live 2012PPT: introduction-to-mysql-replication Presentation Transcript

    • Introduction To MySQL Replication Kenny Gryp <kenny.gryp@percona.com> Percona Live Washington DC / 2012-01-11
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 2
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 3
    • What is Replication?Replication enables data from one MySQL database server (the master) to be replicated to one or more MySQL database servers (the slaves) (Source: http://dev.mysql.com/doc/refman/5.5/en/replication.html) www.percona.com 4
    • Replication Diagram APPLICATIONQueries MASTER SLAVE ... SQL Thread ... MyISAM MyISAM INNODB INNODB IO Thread BINLOGS RELAY LOGS www.percona.com 5
    • Replication Diagram APPLICATIONQueries Queries MASTER BINLOGS SLAVE ... SQL Thread ... MyISAM MyISAM INNODB INNODB IO Thread BINLOGS RELAY LOGS www.percona.com 6
    • ReplicationHappens at MySQL level, not Storage Engine Level (*NDB)Asynchronous! (Semi-sync available in 5.5)A server can have only 1 masterIO Thread: Fetches from masterSQL Thread: Executes on slaveSingle Threaded Execution (Expect enhancements in 5.6)Different schema’s are possible between master and slave(Watch out!!): different indexes storage engines data types columns www.percona.com 7
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 8
    • Binary Logs BINLOG.000001 BINLOG.index BINLOG.000002 BINLOG.000001 BINLOG.000003 BINLOG.000002BINLOG…. BINLOG.000003 www.percona.com 9
    • Binary LogsSet of filesContains all writes and schema changes!= REDO/Transaction logRotated when full (Set max_binlog_size)Incrementing numbers (000001,000002,000003,...)Relay Logs are also binary logs2 Formats: Statement Based (SBR) Row Based (RBR, since MySQL 5.1)Binary file www.percona.com 10
    • Binary Logs - Info Filesmaster.info Contains IO Thread information & connection informationrelay.info Contains SQL Thread information www.percona.com 11
    • Binary Log - Formats (binlog_format)Statement Based Replication (SBR): Writes statements to binary logs, slave executes the stmts (An update stmt that changes 1 row but reads many will also do the same on the slave) Some functions are non-deterministic and cause inconsistencies: UUID(), SYSDATE(), FOUND_ROWS(), UPDATE .. LIMIT without ORDER BY... Works for NOW(): timestamp included in binary log More complete list of issues: http://dev.mysql.com/doc/refman/5.5/en/replication-features.html www.percona.com 12
    • Binary Log - Formats (binlog_format)Row Based Replication (RBR, since 5.1): Write row changes (larger binlogs) Check Binlog_cache_disk_use, possibly increase binlog_cache_size Does not need to parse/execute queries, just make the changes necessary Much less/different issues compared to SBR: http://dev.mysql.com/doc/refman/5.5/en/replication-rbr- usage.htmlMixed: Combination of both: defaults to SBR, use RBRwhen necessary www.percona.com 13
    • Looking at Binary Log ContentsmysqlbinlogSHOW BINLOG EVENTS www.percona.com 14
    • Example SBR> SHOW GLOBAL VARIABLES LIKE binlog_format;+---------------+-----------+| Variable_name | Value |+---------------+-----------+| binlog_format | STATEMENT |+---------------+-----------+1 row in set (0.00 sec)> CREATE DATABASE replication;Query OK, 1 row affected (0.14 sec)> use replicationDatabase changed> CREATE TABLE repl (a int) ENGINE=innodb;Query OK, 0 rows affected (0.25 sec)> INSERT INTO repl VALUES (1);Query OK, 1 row affected (0.14 sec) www.percona.com 15
    • Example SBR - mysqlbinlog# mysqlbinlog mysql-bin.000193...# at 106#120106 15:19:13 server id 9999 end_log_pos 203 ! Query! thread_id=11CREATE DATABASE replication/*!*/;# at 203#120106 15:19:32 server id 9999 end_log_pos 312 ! Query! thread_id=11! exec_time=1! error_code=0use replication/*!*/;SET TIMESTAMP=1325859572/*!*/;CREATE TABLE repl (a INT) ENGINE=innodb/*!*/;# at 312#120106 15:19:55 server id 9999 end_log_pos 387 ! Query! thread_id=11! exec_time=0! error_code=0SET TIMESTAMP=1325859595/*!*/;BEGIN/*!*/;# at 387#120106 15:19:55 server id 9999 end_log_pos 484 ! Query! thread_id=11! exec_time=0! error_code=0SET TIMESTAMP=1325859595/*!*/;INSERT INTO repl VALUES (1)/*!*/;# at 484#120106 15:19:55 server id 9999 end_log_pos 511 ! Xid = 14 www.percona.com 16COMMIT/*!*/;
    • > SHOW BINLOG EVENTS FROM 106G*************************** 1. row *************************** Log_name: mysql-bin.000193 Example SBR - SHOW BINLOG EVENTS Pos: 106 Event_type: Query Server_id: 1End_log_pos: 203 Info: CREATE DATABASE replication*************************** 2. row *************************** Log_name: mysql-bin.000193 Pos: 203 Event_type: Query Server_id: 1End_log_pos: 312 Info: use `replication`; CREATE TABLE repl (a INT) ENGINE=innodb*************************** 3. row *************************** Log_name: mysql-bin.000193 Pos: 312 Event_type: Query Server_id: 1End_log_pos: 387 Info: BEGIN*************************** 4. row *************************** Log_name: mysql-bin.000193 Pos: 387 Event_type: Query Server_id: 1End_log_pos: 484 Info: use `replication`; INSERT INTO repl VALUES (1)*************************** 5. row *************************** Log_name: mysql-bin.000193 Pos: 484 Event_type: Xid Server_id: 1End_log_pos: 511 Info: COMMIT /* xid=14 */5 rows in set (0.00 sec) www.percona.com 17
    • Example RBR> SHOW GLOBAL VARIABLES LIKE binlog_format;+---------------+-------+| Variable_name | Value |+---------------+-------+| binlog_format | ROW |+---------------+-------+1 row in set (0.00 sec)> CREATE DATABASE replication;Query OK, 1 row affected (0.14 sec)> use replicationDatabase changed> CREATE TABLE repl (a int) ENGINE=innodb;Query OK, 0 rows affected (0.25 sec)> INSERT INTO repl VALUES (1);Query OK, 1 row affected (0.14 sec) www.percona.com 18
    • Example RBR - mysqlbinlog# mysqlbinlog mysql-bin.000193 --start-position=606...# at 606#120106 15:54:54 server id 1 end_log_pos 703 ! Query! thread_id=11! exec_time=0! error_code=0CREATE DATABASE replication/*!*/;# at 703#120106 15:55:02 server id 1 end_log_pos 812 ! Query! thread_id=11! exec_time=0! error_code=0use replication/*!*/;SET TIMESTAMP=1325861702/*!*/;CREATE TABLE repl (a int) ENGINE=innodb/*!*/;# at 812...# at 937#120106 15:55:06 server id 1 end_log_pos 937 ! Table_map: `replication`.`repl` mapped tonumber 17#120106 15:55:06 server id 1 end_log_pos 971 ! Write_rows: table id 17 flags: STMT_END_FBINLOG SgsHTxMBAAAAMgAAAKkDAAAAABEAAAAAAAEAC3JlcGxpY2F0aW9uAARyZXBsAAEDAAE=SgsHTxcBAAAAIgAAAMsDAAAAABEAAAAAAAEAAf/+AQAAAA==/*!*/;# at 971#120106 15:55:06 server id 1 end_log_pos 998 ! Xid = 34 www.percona.com 19COMMIT/*!*/;
    • Example RBR - mysqlbinlog --verbose# mysqlbinlog mysql-bin.000193 --verbose --verbose...# at 937#120106 15:55:06 server id 1 end_log_pos 937 ! Table_map: `replication`.`repl` mapped tonumber 17#120106 15:55:06 server id 1 end_log_pos 971 ! Write_rows: table id 17 flags: STMT_END_FBINLOG SgsHTxMBAAAAMgAAAKkDAAAAABEAAAAAAAEAC3JlcGxpY2F0aW9uAARyZXBsAAEDAAE=SgsHTxcBAAAAIgAAAMsDAAAAABEAAAAAAAEAAf/+AQAAAA==/*!*/;### INSERT INTO replication.repl### SET### @1=1 /* INT meta=0 nullable=1 is_null=0 */# at 971 www.percona.com 20
    • > SHOW BINLOG EVENTS FROM 606G*************************** 1. row *************************** Log_name: mysql-bin.000193 Example RBR - SHOW BINLOG EVENTS Pos: 606 Event_type: Query Server_id: 1End_log_pos: 703 Info: CREATE DATABASE replication*************************** 2. row *************************** Log_name: mysql-bin.000193 Pos: 703 Event_type: Query Server_id: 1End_log_pos: 812 Info: use `replication`; CREATE TABLE repl (a int) ENGINE=innodb...*************************** 4. row *************************** Log_name: mysql-bin.000193 Pos: 887 Event_type: Table_map Server_id: 1End_log_pos: 937 Info: table_id: 17 (replication.repl)*************************** 5. row *************************** Log_name: mysql-bin.000193 Pos: 937 Event_type: Write_rows Server_id: 1End_log_pos: 971 Info: table_id: 17 flags: STMT_END_F*************************** 6. row *************************** Log_name: mysql-bin.000193 Pos: 971 Event_type: Xid Server_id: 1End_log_pos: 998 Info: COMMIT /* xid=34 */ www.percona.com 216 rows in set (0.00 sec)
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 22
    • Setting up ReplicationPrerequisitesChange master/slave MySQL configurationConfigure ReplicationStart Replication/Check Status www.percona.com 23
    • PrerequisitesNo way to easily create slave with 1 commandIt’s required to Create/Restore consistent backup usingyour favorite backup tool with mysqldump, use --master-dataBinlog file and position from backup should be recorded File: mysql-bin.000008 Pos: 106 www.percona.com 24
    • Change master/slave ConfigurationOn the master: Enable Binary Logging: log-bin=log-bin Set A Server ID: server-id=1On the slave: Set A Server ID: server-id=2 www.percona.com 25
    • Why server-id ? APPLICATION Queries Queries MASTER BINLOGS SLAVE events get ... server-id=1 SQL Thread ... MyISAM MyISAM INNODB server-id != 2 INNODB IO Thread BINLOGS RELAY LOGSserver-id=1 server-id=2 www.percona.com 26
    • Why server-id ? APPLICATION Queries Queries MASTER server-id != 1 RELAY LOGS BINLOGS SLAVE events get server-id=2 SQL Thread IO Thread log_slave_updates log_slave_updates events get ... server-id=1 SQL Thread ... MyISAM MyISAM INNODB server-id != 2 INNODB IO Thread BINLOGS RELAY LOGSserver-id=1 server-id=2 www.percona.com 27
    • Why server-id ?Avoid events to be written more than oncereplicate_same_server_id does what it says www.percona.com 28
    • Configure ReplicationOn Master, add permissions:> GRANT REPLICATION SLAVE ON *.* TO‘repl’@‘slave’ IDENTIFIED BY ‘pass’;On Slave, configure replication:> CHANGE MASTER TO MASTER_HOST=‘master’,MASTER_USER=‘repl’, MASTER_PASSWORD=‘pass’,MASTER_LOG_FILE=‘mysql-bin.000008’,MASTER_LOG_POS=106; www.percona.com 29
    • slave> START SLAVE;slave> SHOW SLAVE STATUSG Slave_IO_State: Waiting for master to send event IO Thread State Start Replication Master_Host: master Master_User: repl Master_Port: 3306 Connect_Retry: Master_Log_File: 60 mysql-bin.000008 IO Thread Read Up To Read_Master_Log_Pos: 254 Relay_Log_File: Relay_Log_Pos: relay-bin.000002 399 SQL Thread Read Up To Relay_Master_Log_File: mysql-bin.000008 Slave_IO_Running: Slave_SQL_Running: Yes Yes Threads Running? Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 254 Relay_Log_Space: 566 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master:Master_SSL_Verify_Server_Cert: 0 No How Far Is Slave Behind? Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 www.percona.com 30 Last_SQL_Error:
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 31
    • CommandsSQL Commands Adminstrative Commands Diagnostics CommandsShell Commands mysqlbinlog www.percona.com 32
    • Administrative CommandsRotate binary log: FLUSH BINARY LOGSRotate relay log: FLUSH RELAY LOGSSTART/STOP SLAVE IO_THREAD/SQL_THREADRemove binary logs:PURGE MASTER LOGS TO ‘mysql-bin.000005’;PURGE MASTER LOGS BEFORE ‘2012-01-01 00:00:00’;Remove all binary logs: RESET MASTERRemove slave configuration and files: RESET SLAVE www.percona.com 33
    • Diagnostics CommandsOn Master SHOW MASTER STATUS SHOW PROCESSLIST SHOW SLAVE HOSTS SHOW BINLOG EVENTSOn Slave SHOW SLAVE STATUS SHOW PROCESSLIST www.percona.com 34
    • On Master:SHOW MASTER STATUSCurrent binary log file and position:master> SHOW MASTER STATUSG*************************** 1. row *************************** File: mysql-bin.000008 Position: 254 Binlog_Do_DB:Binlog_Ignore_DB: 1 row in set (0.00 sec) www.percona.com 35
    • On Master: SHOW PROCESSLISTFind Connected Slaves using SHOW PROCESSLIST:master> SHOW PROCESSLISTG...*************************** 2. row *************************** Id: 4 User: repl Host: localhost:43537 db: NULLCommand: Binlog Dump Time: 1264 State: Has sent all binlog to slave; waiting for binlog to beupdated Info: NULL... www.percona.com 36
    • On Master: SHOW SLAVE HOSTSShows connected slaves with their server-id:master> SHOW SLAVE HOSTS;+-----------+-----------+------+-----------+| Server_id | Host | Port | Master_id |+-----------+-----------+------+-----------+| 2 | slave | 3306 | 1 |+-----------+-----------+------+-----------+1 row in set (0.00 sec)Only works when slaves have configured:report-host, report-user, report-password,report-port www.percona.com 37
    • On Slave: SHOW PROCESSLISTSlave Thread Status:SLAVE> SHOW PROCESSLISTG*************************** 1. row *************************** Id: 5 User: system user Host: db: NULLCommand: Connect Time: 88611 State: Waiting for master to send event Info: NULL*************************** 2. row *************************** Id: 8 User: system user Host: db: NULLCommand: Connect Time: 83 State: Has read all relay log; waiting for the slave I/O thread to update it Info: NULL www.percona.com 38
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 39
    • Other Common Configuration OptionsFiltering: binlog-%, replicate-%Don’t start replication at start: skip_slave_startPut relay log events in it’s own binary log:log_slave_updatesDisallow writing on slaves: read_onlyAutomatically remove binlogs: expire_logs_daysChange binlog format: binlog_format www.percona.com 40
    • Filtering on Masterbinlog-do-db=..., binlog-ignore-db=...Warning! Different behavior between SBR and RBR: SBR: Log all statements executed if the ‘default database’ (use database) should be binlogged RBR: Log all row changes of the databases that should be binlogged www.percona.com 41
    • Filtering on Slavereplicate-do-db= & replicate-ignore-db=Warning! Different behavior between SBR and RBRSimilar to binlog-do/ignore-dbreplicate-do/ignore-table: Filter on table levelreplicate-wild-do/ignore-table: Use wildcards(%,_) to filter on table levelreplicate-rewrite-db=db1->db2 www.percona.com 42
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 43
    • Replication ArchitecturesMaster-SlaveMaster-MasterMulti Tiered ReplicationCircular Replication www.percona.com 44
    • Master-Slave Master Slave www.percona.com 45
    • Master-SlaveApplication Master Slave www.percona.com 45
    • Master-SlaveApplication Master Slave www.percona.com 45
    • Master-SlaveApplication Writes/Reads Master Slave www.percona.com 45
    • Master-SlaveApplication Writes/Reads Master Slave www.percona.com 45
    • Master-SlaveApplication Writes/Reads Master Reads Slave www.percona.com 45
    • Master-Slave MasterSlave Slave Slave www.percona.com 46
    • Master-SlaveApplication Master Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Master Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Writes/Reads Master Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Writes/Reads Master Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Writes/Reads MasterReads Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Writes/Reads MasterReads Slave Slave Slave www.percona.com 46
    • Master-SlaveApplication Writes/Reads MasterReads Slave Slave Slave www.percona.com 46
    • Master-SlaveScaling Reads: Slaves can take reads (remember asynchronous!) Reporting/Search queries can go to separate slave Does not take resources on master ‘live-site’-slave. (‘live site’: lower response time requirements than reports) Can have other indexes, even other tables (summary tables,...) Not much benefit when too much writesMaintenance (schema changes...) Upgrade slaves first: Take slave offline SET SQL_LOG_BIN=0; <-don’t put this session queries in ALTER TABLE ...; the binary log SET SQL_LOG_BIN=1; www.percona.com 47
    • Master-Slave: Reporting Queries Master Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Master Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Master Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Writes/Reads Master Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Writes/Reads Master Reads Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Writes/Reads Master Reads Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting QueriesApplication Writes/Reads Master Reads Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting Queries Application Writes/ReadsReporting Master Reads Slave Slave Slave www.percona.com 48
    • Master-Slave: Reporting Queries Application Writes/ReadsReporting Master Reads Slave Slave Slave www.percona.com 48
    • Master-Slave: Write IntensiveMaster Slaves Master Slaves Capacity left for READS Capacity left for READS Capacity taken by WRITES Capacity taken by WRITES www.percona.com 49
    • Master-MasterMaster Master www.percona.com 50
    • Master-MasterUsed for: High Availability Maintenance TasksWrite to 1 master, failover to the passive master whennecessaryPassive master is a real slave, can be used for readsScale Writes? NO! Every write still has to be performed on both mastersWriting to both masters at the same time? NO*! Can cause inconsistencies even with auto_increment_increment/ auto_increment_offset problems can happen www.percona.com 51
    • Master-Master: Reads & WritesApplication Writes/Reads Reads Master Master www.percona.com 52
    • Master-Master: MaintenanceApplicationWrites/Reads Master Master www.percona.com 53
    • Master-Master: MaintenanceApplicationWrites/Reads Master Master SET SQL_LOG_BIN=0; ALTER TABLE ...; ... SET SQL_LOG_BIN=1; www.percona.com 53
    • Master-Master: MaintenanceApplicationWrites/Reads Master Master www.percona.com 53
    • Master-Master: MaintenanceApplicationWrites/Reads Master Master SET SQL_LOG_BIN=0; ALTER TABLE ...; ... SET SQL_LOG_BIN=1; www.percona.com 53
    • Master-Master: Write to Both? Inconsistencies Application Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54
    • Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ UPDATE users SET where userid=1;email=‘gryp@percona.com’ where userid=1; Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54
    • Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ UPDATE users SET where userid=1;email=‘gryp@percona.com’ where userid=1; Master Master +--------+------------------+ +--------+-------------------+ | userid | email | | userid | email | +--------+------------------+ +--------+-------------------+ | 1 | gryp@percona.com | | 1 | kenny@percona.com | +--------+------------------+ +--------+-------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54
    • Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ where userid=1; Master Master UPDATE users SET email=‘gryp@percona.com’ where userid=1; +--------+------------------+ +--------+-------------------+ | userid | email | | userid | email | +--------+------------------+ +--------+-------------------+ | 1 | gryp@percona.com | | 1 | kenny@percona.com | +--------+------------------+ +--------+-------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54
    • Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ where userid=1; Master Master UPDATE users SET email=‘gryp@percona.com’ where userid=1; +--------+-------------------+ +--------+------------------+ | userid | email | | userid | email | +--------+-------------------+ +--------+------------------+ | 1 | kenny@percona.com | | 1 | gryp@percona.com | +--------+-------------------+ +--------+------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54
    • Master-Master: Write to Both? Replication Breaks Application Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 55
    • Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ INSERT INTO USERS WHERE userid=1; (email) VALUES (‘legryp@percona.com’) Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 55
    • Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ INSERT INTO USERS WHERE userid=1; (email) VALUES (‘legryp@percona.com’) Master Master +--------+--------------------+ +--------+--------------------+ | userid | email | | userid | email | +--------+--------------------+ +--------+--------------------+ | 1 | kg@percona.com | | 1 | legryp@percona.com | | 2 | legryp@percona.com | +--------+--------------------+ +--------+--------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 55
    • Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ WHERE userid=1; Master Master INSERT INTO USERS (email) VALUES (‘legryp@percona.com’) +--------+--------------------+ +--------+--------------------+ | userid | email | | userid | email | +--------+--------------------+ +--------+--------------------+ | 1 | kg@percona.com | | 1 | legryp@percona.com | | 2 | legryp@percona.com | +--------+--------------------+ +--------+--------------------+CREATE TABLE users (userid INT AUTO_INCREMENT, Error Duplicate entry legryp@percona.comemail VARCHAR(128) NOT NULL,PRIMARY KEY (userid), for key idx_email on query. DefaultUNIQUE KEY idx_email (email) database: test. Query: insert into); users(email) values (legryp@percona.com) www.percona.com 55
    • Multi Tiered Replication MasterSlave Slave Slave Slave Slave www.percona.com 56
    • Multi Tiered Replication2 Second query takes 6 seconds to reach the 3rd levelTop level slaves could be a master for some dbs/tables too Replication main data (users...) to top level slaves Top level slaves are sort of functionally partitioned www.percona.com 57
    • Multi Tiered Replication: Delay MasterSlave Slave Slave Slave Slave www.percona.com 58
    • Multi Tiered Replication: Delay Master 2 SecondsSlave Slave Slave Slave Slave www.percona.com 58
    • Multi Tiered Replication: Delay Master 2 Seconds 2 SecondsSlave Slave Slave Slave Slave www.percona.com 58
    • Multi Tiered Replication: Delay Master 2 Seconds 2 SecondsSlave Slave Slave 2 Seconds Slave Slave www.percona.com 58
    • Multi Tiered Replication: Top Level Slaves MASTER Master MASTER SEARCH LOGSlave Slave Slave SEARCH SEARCH Slave Slave www.percona.com 59
    • Multi Tiered Replication MasterSlave Slave Slave Slave Slave No Updates Anymore www.percona.com 60
    • Circular Replication MasterMaster Master Master www.percona.com 61
    • Circular Replication MasterMaster Master Master www.percona.com 62
    • Circular Replication MasterMaster Master Master www.percona.com 63
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 64
    • Common IssuesReplication LagReplication BreakingCrash Safe ReplicationConsistency www.percona.com 65
    • Replication Lag - CausesSingle threaded execution on slave!!!Long running queries with SBR: will take ~same time onslaveMany row changes with RBRMyISAM & table level locking: Long running SELECT onslave might block replicationSchema changes: 20min on master, 20min on slave <-- 20 min replication lag www.percona.com 66
    • Replication Lag - Tips 1Optimize DML StatementsAvoid DDL Statements in Replication (SQL_LOG_BIN=0)Hardware specifications usually need to be equal or betteron slaveMySQL 5.6:http://d2-systems.blogspot.com/2011/04/mysql-56x-feature-preview-multi.htmlEnsure BBU&Write Back Cache is used or lower durability:innodb_flush_log_at_trx_commit=2Change architecture: functional/horizontal partitioning www.percona.com 67
    • Replication Lag - Tips 2Try to avoid SQL Thread to need to do disk I/O: Replication Booster (Yoshinori Matsunobu) prefetches relay logs and converts queries to select and runs them, causing data to be cached before SQL thread reaches innodb_fake_changes http://www.percona.com/doc/percona-server/5.5/management/ innodb_fake_changes.html http://dom.as/2011/12/03/replication-prefetching/ www.percona.com 68
    • Replication Boosterhttp://yoshinorimatsunobu.blogspot.com/2011/10/making-slave-pre-fetching-work-better.html www.percona.com 69
    • Replication Breaking - ExamplesLast_Errno: 1146Last_Error: Error Table db.table doesnt exist on query.Default database: userdb. Query: ...Last_Errno: 1062Last_Error: Error Duplicate entry 1748099 for key 1 on query.Default database: db. Query: INSERT INTO table (url)VALUES(‘http://www.google.com)Last_Errno: 1053Last_Error: Query partially completed on the master (error onmaster: 1053) and was aborted. There is a chance that your masteris inconsistent at this point. If you are sure that your master isok, run this query manually on the slave and then restart theslave with SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE; .Query: ALTER TABLE ‘users’ ADD ‘name’ VARCHAR(128) NULL DEFAULT; www.percona.com 70
    • Replication Breaking - CausesWrites happening on SlavesSBR with bad statementsCrashes: Relay Log CorruptionDifferent schemasTemporary tablesMixing MyISAM with InnoDBDifferent behavior between MySQL versions (minor/major)Killing queries that change MyISAM tables will be partiallyexecuted, and break replication... www.percona.com 71
    • Replication Breaking - FixingQuestions to ask yourself: Why? Where? Impact?Quick ‘fix’:SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE; Causes 1 event to be skipped Causes inconsistencies!!!Look at: Check error & Investigate data on master/slave MySQL errorlogs SHOW SLAVE STATUSG mysqlbinlog on relaylogs/binarylogs and investigate pt-slave-restartMore On: http://www.percona.com/files/presentations/percona-live/london-2011/PLUK2011-diagnosing-and-fixing-mysql-replication.pdf www.percona.com 72
    • Crash Safe ReplicationOn master: sync_binlog=1 to fsync at every binlog write Performance impact, mainly on: Ext3 filesystem http://www.mysqlperformanceblog.com/2009/01/21/beware-ext3-and-sync-binlog-do-not-play-well-together/ High write-response time disksubsystem (no WriteBackCache& BBU)On slave: relay-log.info/master.info: not durable/consistent! When OS crash, old information might be in the file If InnoDB only: use innodb_overwrite_relay_log_info in Percona Server: http://www.percona.com/doc/percona-server/5.5/reliability/ innodb_recovery_update_relay_log.html www.percona.com 73
    • ConsistencyReplication can cause inconsistencies: Statement Based Replication Certain functions like UUID()... UPDATE/DELETE with LIMIT without ORDER BY unique key Writing to multiple masters Writing to slaves (by accident? use read_only) Failover done wrong MyISAM with killing queries and/ord MySQL Crashes Wrongly restored slaves Replication broke and used SQL_SLAVE_SKIP_COUNTER ...How to know if my data is consistent? www.percona.com 74
    • ConsistencyChecking Consistency: pt-table-checksum Checksums chunks of data on master and slaves Monitors replication lag Low checksum response time to lower impact on production Uses Replication to ensure consistent view of dataFixing Consistency: pt-table-sync Can checksum and use pt-table-checksum results Fixes inconsistencies www.percona.com 75
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 76
    • Replication Use CasesScale ReadsHigh AvailabilityBackups www.percona.com 77
    • High AvailabilityFailover: Virtual IPs Change IP in application serversWarning: Split-Brain possible!!! Ensure no writes happen on old master before moving Make sure new master is in syncScripts available to automate failover: MySQL-MHA http://code.google.com/p/mysql-master-ha/ MMM (not-recommended) http://mysql-mmm.org/ Percona-PRM http://www.mysqlperformanceblog.com/2011/11/29/percona-replication-manager-a- solution-for-mysql-high-availability-with-replication-using-pacemaker/ www.percona.com 78
    • High AvailabilityDue to asynchronous’ness, data might be lost:Semi-Synchronous Replication: ensure at least one slave’s IO thread has received the event before the transaction is committed. http://dev.mysql.com/doc/refman/5.5/en/replication-semisync.html www.percona.com 79
    • BackupsPoint In Time Recovery, using binary logsDelayed slaveBackup Slaves www.percona.com 80
    • Incremental / Point In Time RecoveryRestore Full Backup from your favorite backup toolReplay binary logs with mysqldump from the last positionof the backup# mysqlbinlog --start-position=15115 1sbinarylog | mysql# mysqlbinlog next-binary-logs | mysqlIf PITR, stop at the desired position/time:# mysqlbinlog --stop-position=102151 binarylog | mysql# mysqlbinlog --stop-datetime=“2012-01-01 23:00:00” lastbinarylog | mysql www.percona.com 81
    • Delayed SlaveDeliberately lag a slave for a predefined amount of timeDecrease recovery time in case of bad changes caused by: Humans (DROP DATABASE) Application (DELETE FROM table;)How to configure: pt-slave-delay --delay 60m (http://www.percona.com/doc/percona-toolkit/2.0/pt-slave-delay.html) 5.6 Built-in feature: > CHANGE MASTER TO MASTER_DELAY=3600;When a problem happens stop replication just before themalicious statement.START SLAVE UNTIL MASTER_LOG_FILE=log_name,MASTER_LOG_POS = pos; www.percona.com 82
    • Backup using SlavesOnline Backups might not be possible on active master:Too much impact to make online: InnoDB backup (using MySQL enterprise backup or XtraBackup) LVM Snapshots MyISAM tablesOffsite Backup www.percona.com 83
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 84
    • Other Replication ImplementationsTungsten Replicator Uses binary log Parallel replication Enhanced filtering Multi-master Replication between different databasesGalera Cluster Does not use binary log Synchronous replication Active-active master Parallel replication Percona XtraDB Cluster! http://www.mysqlperformanceblog.com/2012/01/09/ announcement-of-percona-xtradb-cluster-alpha-release/ www.percona.com 85
    • Tungsten Replicatorhttp://continuent.com/http://code.google.com/p/tungsten-replicator/ www.percona.com 86
    • Galera Clusterhttp://codership.com/ www.percona.com 87
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 88
    • Tools & More InfoPercona Toolkit http://www.percona.com/software/percona-toolkit/pt-slave-find, pt-table-checksum, pt-table-sync,pt-slave-delay, pt-slave-restart, pt-heartbeatPercona Server http://www.percona.com/software/percona-server/OpenArk kit http://code.openark.org/forge/openark-kitReplication Booster https://github.com/yoshinorim/replication-booster-for-mysqlHigh Availability MMM http://mysql-mmm.org/ Percona-PRM http://www.mysqlperformanceblog.com/2011/11/29/percona-replication-manager-a- solution-for-mysql-high-availability-with-replication-using-pacemaker/ MySQL-MHA http://code.google.com/p/mysql-master-ha/High Performance MySQL, 2nd Edition:http://shop.oreilly.com/product/9780596101718.do www.percona.com 89
    • MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 90
    • Kenny Gryp <kenny.gryp@percona.com> @gryp Were Hiring!www.percona.com/about-us/careers/