Percona Live 2012PPT： introduction-to-mysql-replication

Introduction To MySQL Replication Kenny Gryp <kenny.gryp@percona.com> Percona Live Washington DC / 2012-01-11

MySQL ReplicationReplication OverviewBinary LogsSetting Up ReplicationCommandsOther Common Configuration SettingsReplication ArchitecturesCommon IssuesReplication Use CasesOther Replication ImplementationsTools www.percona.com 2

What is Replication?Replication enables data from one MySQL database server (the master) to be replicated to one or more MySQL database servers (the slaves) (Source: http://dev.mysql.com/doc/refman/5.5/en/replication.html) www.percona.com 4

Replication Diagram APPLICATIONQueries MASTER SLAVE ... SQL Thread ... MyISAM MyISAM INNODB INNODB IO Thread BINLOGS RELAY LOGS www.percona.com 5

Replication Diagram APPLICATIONQueries Queries MASTER BINLOGS SLAVE ... SQL Thread ... MyISAM MyISAM INNODB INNODB IO Thread BINLOGS RELAY LOGS www.percona.com 6

ReplicationHappens at MySQL level, not Storage Engine Level (*NDB)Asynchronous! (Semi-sync available in 5.5)A server can have only 1 masterIO Thread: Fetches from masterSQL Thread: Executes on slaveSingle Threaded Execution (Expect enhancements in 5.6)Different schema’s are possible between master and slave(Watch out!!): different indexes storage engines data types columns www.percona.com 7

Binary Logs BINLOG.000001 BINLOG.index BINLOG.000002 BINLOG.000001 BINLOG.000003 BINLOG.000002BINLOG…. BINLOG.000003 www.percona.com 9

Binary LogsSet of filesContains all writes and schema changes!= REDO/Transaction logRotated when full (Set max_binlog_size)Incrementing numbers (000001,000002,000003,...)Relay Logs are also binary logs2 Formats: Statement Based (SBR) Row Based (RBR, since MySQL 5.1)Binary file www.percona.com 10

Binary Logs - Info Filesmaster.info Contains IO Thread information & connection informationrelay.info Contains SQL Thread information www.percona.com 11

Binary Log - Formats (binlog_format)Statement Based Replication (SBR): Writes statements to binary logs, slave executes the stmts (An update stmt that changes 1 row but reads many will also do the same on the slave) Some functions are non-deterministic and cause inconsistencies: UUID(), SYSDATE(), FOUND_ROWS(), UPDATE .. LIMIT without ORDER BY... Works for NOW(): timestamp included in binary log More complete list of issues: http://dev.mysql.com/doc/refman/5.5/en/replication-features.html www.percona.com 12

Binary Log - Formats (binlog_format)Row Based Replication (RBR, since 5.1): Write row changes (larger binlogs) Check Binlog_cache_disk_use, possibly increase binlog_cache_size Does not need to parse/execute queries, just make the changes necessary Much less/different issues compared to SBR: http://dev.mysql.com/doc/refman/5.5/en/replication-rbr- usage.htmlMixed: Combination of both: defaults to SBR, use RBRwhen necessary www.percona.com 13

Looking at Binary Log ContentsmysqlbinlogSHOW BINLOG EVENTS www.percona.com 14

Example SBR> SHOW GLOBAL VARIABLES LIKE binlog_format;+---------------+-----------+| Variable_name | Value |+---------------+-----------+| binlog_format | STATEMENT |+---------------+-----------+1 row in set (0.00 sec)> CREATE DATABASE replication;Query OK, 1 row affected (0.14 sec)> use replicationDatabase changed> CREATE TABLE repl (a int) ENGINE=innodb;Query OK, 0 rows affected (0.25 sec)> INSERT INTO repl VALUES (1);Query OK, 1 row affected (0.14 sec) www.percona.com 15

Example SBR - mysqlbinlog# mysqlbinlog mysql-bin.000193...# at 106#120106 15:19:13 server id 9999 end_log_pos 203 ! Query! thread_id=11CREATE DATABASE replication/*!*/;# at 203#120106 15:19:32 server id 9999 end_log_pos 312 ! Query! thread_id=11! exec_time=1! error_code=0use replication/*!*/;SET TIMESTAMP=1325859572/*!*/;CREATE TABLE repl (a INT) ENGINE=innodb/*!*/;# at 312#120106 15:19:55 server id 9999 end_log_pos 387 ! Query! thread_id=11! exec_time=0! error_code=0SET TIMESTAMP=1325859595/*!*/;BEGIN/*!*/;# at 387#120106 15:19:55 server id 9999 end_log_pos 484 ! Query! thread_id=11! exec_time=0! error_code=0SET TIMESTAMP=1325859595/*!*/;INSERT INTO repl VALUES (1)/*!*/;# at 484#120106 15:19:55 server id 9999 end_log_pos 511 ! Xid = 14 www.percona.com 16COMMIT/*!*/;

> SHOW BINLOG EVENTS FROM 106G*************************** 1. row *************************** Log_name: mysql-bin.000193 Example SBR - SHOW BINLOG EVENTS Pos: 106 Event_type: Query Server_id: 1End_log_pos: 203 Info: CREATE DATABASE replication*************************** 2. row *************************** Log_name: mysql-bin.000193 Pos: 203 Event_type: Query Server_id: 1End_log_pos: 312 Info: use `replication`; CREATE TABLE repl (a INT) ENGINE=innodb*************************** 3. row *************************** Log_name: mysql-bin.000193 Pos: 312 Event_type: Query Server_id: 1End_log_pos: 387 Info: BEGIN*************************** 4. row *************************** Log_name: mysql-bin.000193 Pos: 387 Event_type: Query Server_id: 1End_log_pos: 484 Info: use `replication`; INSERT INTO repl VALUES (1)*************************** 5. row *************************** Log_name: mysql-bin.000193 Pos: 484 Event_type: Xid Server_id: 1End_log_pos: 511 Info: COMMIT /* xid=14 */5 rows in set (0.00 sec) www.percona.com 17

Example RBR> SHOW GLOBAL VARIABLES LIKE binlog_format;+---------------+-------+| Variable_name | Value |+---------------+-------+| binlog_format | ROW |+---------------+-------+1 row in set (0.00 sec)> CREATE DATABASE replication;Query OK, 1 row affected (0.14 sec)> use replicationDatabase changed> CREATE TABLE repl (a int) ENGINE=innodb;Query OK, 0 rows affected (0.25 sec)> INSERT INTO repl VALUES (1);Query OK, 1 row affected (0.14 sec) www.percona.com 18

Example RBR - mysqlbinlog# mysqlbinlog mysql-bin.000193 --start-position=606...# at 606#120106 15:54:54 server id 1 end_log_pos 703 ! Query! thread_id=11! exec_time=0! error_code=0CREATE DATABASE replication/*!*/;# at 703#120106 15:55:02 server id 1 end_log_pos 812 ! Query! thread_id=11! exec_time=0! error_code=0use replication/*!*/;SET TIMESTAMP=1325861702/*!*/;CREATE TABLE repl (a int) ENGINE=innodb/*!*/;# at 812...# at 937#120106 15:55:06 server id 1 end_log_pos 937 ! Table_map: `replication`.`repl` mapped tonumber 17#120106 15:55:06 server id 1 end_log_pos 971 ! Write_rows: table id 17 flags: STMT_END_FBINLOG SgsHTxMBAAAAMgAAAKkDAAAAABEAAAAAAAEAC3JlcGxpY2F0aW9uAARyZXBsAAEDAAE=SgsHTxcBAAAAIgAAAMsDAAAAABEAAAAAAAEAAf/+AQAAAA==/*!*/;# at 971#120106 15:55:06 server id 1 end_log_pos 998 ! Xid = 34 www.percona.com 19COMMIT/*!*/;

Example RBR - mysqlbinlog --verbose# mysqlbinlog mysql-bin.000193 --verbose --verbose...# at 937#120106 15:55:06 server id 1 end_log_pos 937 ! Table_map: `replication`.`repl` mapped tonumber 17#120106 15:55:06 server id 1 end_log_pos 971 ! Write_rows: table id 17 flags: STMT_END_FBINLOG SgsHTxMBAAAAMgAAAKkDAAAAABEAAAAAAAEAC3JlcGxpY2F0aW9uAARyZXBsAAEDAAE=SgsHTxcBAAAAIgAAAMsDAAAAABEAAAAAAAEAAf/+AQAAAA==/*!*/;### INSERT INTO replication.repl### SET### @1=1 /* INT meta=0 nullable=1 is_null=0 */# at 971 www.percona.com 20

> SHOW BINLOG EVENTS FROM 606G*************************** 1. row *************************** Log_name: mysql-bin.000193 Example RBR - SHOW BINLOG EVENTS Pos: 606 Event_type: Query Server_id: 1End_log_pos: 703 Info: CREATE DATABASE replication*************************** 2. row *************************** Log_name: mysql-bin.000193 Pos: 703 Event_type: Query Server_id: 1End_log_pos: 812 Info: use `replication`; CREATE TABLE repl (a int) ENGINE=innodb...*************************** 4. row *************************** Log_name: mysql-bin.000193 Pos: 887 Event_type: Table_map Server_id: 1End_log_pos: 937 Info: table_id: 17 (replication.repl)*************************** 5. row *************************** Log_name: mysql-bin.000193 Pos: 937 Event_type: Write_rows Server_id: 1End_log_pos: 971 Info: table_id: 17 flags: STMT_END_F*************************** 6. row *************************** Log_name: mysql-bin.000193 Pos: 971 Event_type: Xid Server_id: 1End_log_pos: 998 Info: COMMIT /* xid=34 */ www.percona.com 216 rows in set (0.00 sec)

Setting up ReplicationPrerequisitesChange master/slave MySQL configurationConfigure ReplicationStart Replication/Check Status www.percona.com 23

PrerequisitesNo way to easily create slave with 1 commandIt’s required to Create/Restore consistent backup usingyour favorite backup tool with mysqldump, use --master-dataBinlog file and position from backup should be recorded File: mysql-bin.000008 Pos: 106 www.percona.com 24

Change master/slave ConfigurationOn the master: Enable Binary Logging: log-bin=log-bin Set A Server ID: server-id=1On the slave: Set A Server ID: server-id=2 www.percona.com 25

Why server-id ? APPLICATION Queries Queries MASTER BINLOGS SLAVE events get ... server-id=1 SQL Thread ... MyISAM MyISAM INNODB server-id != 2 INNODB IO Thread BINLOGS RELAY LOGSserver-id=1 server-id=2 www.percona.com 26

Why server-id ? APPLICATION Queries Queries MASTER server-id != 1 RELAY LOGS BINLOGS SLAVE events get server-id=2 SQL Thread IO Thread log_slave_updates log_slave_updates events get ... server-id=1 SQL Thread ... MyISAM MyISAM INNODB server-id != 2 INNODB IO Thread BINLOGS RELAY LOGSserver-id=1 server-id=2 www.percona.com 27

Why server-id ?Avoid events to be written more than oncereplicate_same_server_id does what it says www.percona.com 28

Configure ReplicationOn Master, add permissions:> GRANT REPLICATION SLAVE ON *.* TO‘repl’@‘slave’ IDENTIFIED BY ‘pass’;On Slave, configure replication:> CHANGE MASTER TO MASTER_HOST=‘master’,MASTER_USER=‘repl’, MASTER_PASSWORD=‘pass’,MASTER_LOG_FILE=‘mysql-bin.000008’,MASTER_LOG_POS=106; www.percona.com 29

slave> START SLAVE;slave> SHOW SLAVE STATUSG Slave_IO_State: Waiting for master to send event IO Thread State Start Replication Master_Host: master Master_User: repl Master_Port: 3306 Connect_Retry: Master_Log_File: 60 mysql-bin.000008 IO Thread Read Up To Read_Master_Log_Pos: 254 Relay_Log_File: Relay_Log_Pos: relay-bin.000002 399 SQL Thread Read Up To Relay_Master_Log_File: mysql-bin.000008 Slave_IO_Running: Slave_SQL_Running: Yes Yes Threads Running? Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 254 Relay_Log_Space: 566 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master:Master_SSL_Verify_Server_Cert: 0 No How Far Is Slave Behind? Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 www.percona.com 30 Last_SQL_Error:

CommandsSQL Commands Adminstrative Commands Diagnostics CommandsShell Commands mysqlbinlog www.percona.com 32

Administrative CommandsRotate binary log: FLUSH BINARY LOGSRotate relay log: FLUSH RELAY LOGSSTART/STOP SLAVE IO_THREAD/SQL_THREADRemove binary logs:PURGE MASTER LOGS TO ‘mysql-bin.000005’;PURGE MASTER LOGS BEFORE ‘2012-01-01 00:00:00’;Remove all binary logs: RESET MASTERRemove slave configuration and files: RESET SLAVE www.percona.com 33

Diagnostics CommandsOn Master SHOW MASTER STATUS SHOW PROCESSLIST SHOW SLAVE HOSTS SHOW BINLOG EVENTSOn Slave SHOW SLAVE STATUS SHOW PROCESSLIST www.percona.com 34

On Master:SHOW MASTER STATUSCurrent binary log file and position:master> SHOW MASTER STATUSG*************************** 1. row *************************** File: mysql-bin.000008 Position: 254 Binlog_Do_DB:Binlog_Ignore_DB: 1 row in set (0.00 sec) www.percona.com 35

On Master: SHOW PROCESSLISTFind Connected Slaves using SHOW PROCESSLIST:master> SHOW PROCESSLISTG...*************************** 2. row *************************** Id: 4 User: repl Host: localhost:43537 db: NULLCommand: Binlog Dump Time: 1264 State: Has sent all binlog to slave; waiting for binlog to beupdated Info: NULL... www.percona.com 36

On Slave: SHOW PROCESSLISTSlave Thread Status:SLAVE> SHOW PROCESSLISTG*************************** 1. row *************************** Id: 5 User: system user Host: db: NULLCommand: Connect Time: 88611 State: Waiting for master to send event Info: NULL*************************** 2. row *************************** Id: 8 User: system user Host: db: NULLCommand: Connect Time: 83 State: Has read all relay log; waiting for the slave I/O thread to update it Info: NULL www.percona.com 38

Other Common Configuration OptionsFiltering: binlog-%, replicate-%Don’t start replication at start: skip_slave_startPut relay log events in it’s own binary log:log_slave_updatesDisallow writing on slaves: read_onlyAutomatically remove binlogs: expire_logs_daysChange binlog format: binlog_format www.percona.com 40

Filtering on Masterbinlog-do-db=..., binlog-ignore-db=...Warning! Different behavior between SBR and RBR: SBR: Log all statements executed if the ‘default database’ (use database) should be binlogged RBR: Log all row changes of the databases that should be binlogged www.percona.com 41

Filtering on Slavereplicate-do-db= & replicate-ignore-db=Warning! Different behavior between SBR and RBRSimilar to binlog-do/ignore-dbreplicate-do/ignore-table: Filter on table levelreplicate-wild-do/ignore-table: Use wildcards(%,_) to filter on table levelreplicate-rewrite-db=db1->db2 www.percona.com 42

Replication ArchitecturesMaster-SlaveMaster-MasterMulti Tiered ReplicationCircular Replication www.percona.com 44

Master-Slave Master Slave www.percona.com 45

Master-SlaveApplication Master Slave www.percona.com 45

Master-SlaveApplication Writes/Reads Master Slave www.percona.com 45

Master-SlaveApplication Writes/Reads Master Reads Slave www.percona.com 45

Master-Slave MasterSlave Slave Slave www.percona.com 46

Master-SlaveApplication Master Slave Slave Slave www.percona.com 46

Master-SlaveApplication Writes/Reads Master Slave Slave Slave www.percona.com 46

Master-SlaveApplication Writes/Reads MasterReads Slave Slave Slave www.percona.com 46

Master-SlaveScaling Reads: Slaves can take reads (remember asynchronous!) Reporting/Search queries can go to separate slave Does not take resources on master ‘live-site’-slave. (‘live site’: lower response time requirements than reports) Can have other indexes, even other tables (summary tables,...) Not much benefit when too much writesMaintenance (schema changes...) Upgrade slaves first: Take slave offline SET SQL_LOG_BIN=0; <-don’t put this session queries in ALTER TABLE ...; the binary log SET SQL_LOG_BIN=1; www.percona.com 47

Master-Slave: Reporting Queries Master Slave Slave Slave www.percona.com 48

Master-Slave: Reporting QueriesApplication Master Slave Slave Slave www.percona.com 48

Master-Slave: Reporting QueriesApplication Writes/Reads Master Slave Slave Slave www.percona.com 48

Master-Slave: Reporting QueriesApplication Writes/Reads Master Reads Slave Slave Slave www.percona.com 48

Master-Slave: Reporting Queries Application Writes/ReadsReporting Master Reads Slave Slave Slave www.percona.com 48

Master-Slave: Write IntensiveMaster Slaves Master Slaves Capacity left for READS Capacity left for READS Capacity taken by WRITES Capacity taken by WRITES www.percona.com 49

Master-MasterMaster Master www.percona.com 50

Master-MasterUsed for: High Availability Maintenance TasksWrite to 1 master, failover to the passive master whennecessaryPassive master is a real slave, can be used for readsScale Writes? NO! Every write still has to be performed on both mastersWriting to both masters at the same time? NO*! Can cause inconsistencies even with auto_increment_increment/ auto_increment_offset problems can happen www.percona.com 51

Master-Master: Reads & WritesApplication Writes/Reads Reads Master Master www.percona.com 52

Master-Master: MaintenanceApplicationWrites/Reads Master Master www.percona.com 53

Master-Master: MaintenanceApplicationWrites/Reads Master Master SET SQL_LOG_BIN=0; ALTER TABLE ...; ... SET SQL_LOG_BIN=1; www.percona.com 53

Master-Master: MaintenanceApplicationWrites/Reads Master Master www.percona.com 53

Master-Master: MaintenanceApplicationWrites/Reads Master Master SET SQL_LOG_BIN=0; ALTER TABLE ...; ... SET SQL_LOG_BIN=1; www.percona.com 53

Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ UPDATE users SET where userid=1;email=‘gryp@percona.com’ where userid=1; Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54

Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ UPDATE users SET where userid=1;email=‘gryp@percona.com’ where userid=1; Master Master +--------+------------------+ +--------+-------------------+ | userid | email | | userid | email | +--------+------------------+ +--------+-------------------+ | 1 | gryp@percona.com | | 1 | kenny@percona.com | +--------+------------------+ +--------+-------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54

Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ where userid=1; Master Master UPDATE users SET email=‘gryp@percona.com’ where userid=1; +--------+------------------+ +--------+-------------------+ | userid | email | | userid | email | +--------+------------------+ +--------+-------------------+ | 1 | gryp@percona.com | | 1 | kenny@percona.com | +--------+------------------+ +--------+-------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54

Master-Master: Write to Both? Inconsistencies Application UPDATE users SET email=‘kenny@percona.com’ where userid=1; Master Master UPDATE users SET email=‘gryp@percona.com’ where userid=1; +--------+-------------------+ +--------+------------------+ | userid | email | | userid | email | +--------+-------------------+ +--------+------------------+ | 1 | kenny@percona.com | | 1 | gryp@percona.com | +--------+-------------------+ +--------+------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 54

Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ INSERT INTO USERS WHERE userid=1; (email) VALUES (‘legryp@percona.com’) Master Master +--------+----------------+ +--------+----------------+ | userid | email | | userid | email | +--------+----------------+ +--------+----------------+ | 1 | kg@percona.com | | 1 | kg@percona.com | +--------+----------------+ +--------+----------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 55

Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ INSERT INTO USERS WHERE userid=1; (email) VALUES (‘legryp@percona.com’) Master Master +--------+--------------------+ +--------+--------------------+ | userid | email | | userid | email | +--------+--------------------+ +--------+--------------------+ | 1 | kg@percona.com | | 1 | legryp@percona.com | | 2 | legryp@percona.com | +--------+--------------------+ +--------+--------------------+CREATE TABLE users (userid INT AUTO_INCREMENT,email VARCHAR(128) NOT NULL,PRIMARY KEY (userid),UNIQUE KEY idx_email (email)); www.percona.com 55

Master-Master: Write to Both? Replication Breaks Application UPDATE USERS SET email=‘legryp@percona.com’ WHERE userid=1; Master Master INSERT INTO USERS (email) VALUES (‘legryp@percona.com’) +--------+--------------------+ +--------+--------------------+ | userid | email | | userid | email | +--------+--------------------+ +--------+--------------------+ | 1 | kg@percona.com | | 1 | legryp@percona.com | | 2 | legryp@percona.com | +--------+--------------------+ +--------+--------------------+CREATE TABLE users (userid INT AUTO_INCREMENT, Error Duplicate entry legryp@percona.comemail VARCHAR(128) NOT NULL,PRIMARY KEY (userid), for key idx_email on query. DefaultUNIQUE KEY idx_email (email) database: test. Query: insert into); users(email) values (legryp@percona.com) www.percona.com 55

Multi Tiered Replication MasterSlave Slave Slave Slave Slave www.percona.com 56

Multi Tiered Replication2 Second query takes 6 seconds to reach the 3rd levelTop level slaves could be a master for some dbs/tables too Replication main data (users...) to top level slaves Top level slaves are sort of functionally partitioned www.percona.com 57

Multi Tiered Replication: Delay MasterSlave Slave Slave Slave Slave www.percona.com 58

Multi Tiered Replication: Delay Master 2 SecondsSlave Slave Slave Slave Slave www.percona.com 58

Multi Tiered Replication: Delay Master 2 Seconds 2 SecondsSlave Slave Slave Slave Slave www.percona.com 58

Multi Tiered Replication: Delay Master 2 Seconds 2 SecondsSlave Slave Slave 2 Seconds Slave Slave www.percona.com 58

Multi Tiered Replication: Top Level Slaves MASTER Master MASTER SEARCH LOGSlave Slave Slave SEARCH SEARCH Slave Slave www.percona.com 59

Multi Tiered Replication MasterSlave Slave Slave Slave Slave No Updates Anymore www.percona.com 60

Circular Replication MasterMaster Master Master www.percona.com 61

Common IssuesReplication LagReplication BreakingCrash Safe ReplicationConsistency www.percona.com 65

Replication Lag - CausesSingle threaded execution on slave!!!Long running queries with SBR: will take ~same time onslaveMany row changes with RBRMyISAM & table level locking: Long running SELECT onslave might block replicationSchema changes: 20min on master, 20min on slave <-- 20 min replication lag www.percona.com 66

Replication Lag - Tips 1Optimize DML StatementsAvoid DDL Statements in Replication (SQL_LOG_BIN=0)Hardware specifications usually need to be equal or betteron slaveMySQL 5.6:http://d2-systems.blogspot.com/2011/04/mysql-56x-feature-preview-multi.htmlEnsure BBU&Write Back Cache is used or lower durability:innodb_flush_log_at_trx_commit=2Change architecture: functional/horizontal partitioning www.percona.com 67

Replication Lag - Tips 2Try to avoid SQL Thread to need to do disk I/O: Replication Booster (Yoshinori Matsunobu) prefetches relay logs and converts queries to select and runs them, causing data to be cached before SQL thread reaches innodb_fake_changes http://www.percona.com/doc/percona-server/5.5/management/ innodb_fake_changes.html http://dom.as/2011/12/03/replication-prefetching/ www.percona.com 68

Replication Boosterhttp://yoshinorimatsunobu.blogspot.com/2011/10/making-slave-pre-fetching-work-better.html www.percona.com 69

Replication Breaking - ExamplesLast_Errno: 1146Last_Error: Error Table db.table doesnt exist on query.Default database: userdb. Query: ...Last_Errno: 1062Last_Error: Error Duplicate entry 1748099 for key 1 on query.Default database: db. Query: INSERT INTO table (url)VALUES(‘http://www.google.com)Last_Errno: 1053Last_Error: Query partially completed on the master (error onmaster: 1053) and was aborted. There is a chance that your masteris inconsistent at this point. If you are sure that your master isok, run this query manually on the slave and then restart theslave with SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE; .Query: ALTER TABLE ‘users’ ADD ‘name’ VARCHAR(128) NULL DEFAULT; www.percona.com 70

Replication Breaking - CausesWrites happening on SlavesSBR with bad statementsCrashes: Relay Log CorruptionDifferent schemasTemporary tablesMixing MyISAM with InnoDBDifferent behavior between MySQL versions (minor/major)Killing queries that change MyISAM tables will be partiallyexecuted, and break replication... www.percona.com 71

Replication Breaking - FixingQuestions to ask yourself: Why? Where? Impact?Quick ‘fix’:SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE; Causes 1 event to be skipped Causes inconsistencies!!!Look at: Check error & Investigate data on master/slave MySQL errorlogs SHOW SLAVE STATUSG mysqlbinlog on relaylogs/binarylogs and investigate pt-slave-restartMore On: http://www.percona.com/files/presentations/percona-live/london-2011/PLUK2011-diagnosing-and-fixing-mysql-replication.pdf www.percona.com 72

Crash Safe ReplicationOn master: sync_binlog=1 to fsync at every binlog write Performance impact, mainly on: Ext3 filesystem http://www.mysqlperformanceblog.com/2009/01/21/beware-ext3-and-sync-binlog-do-not-play-well-together/ High write-response time disksubsystem (no WriteBackCache& BBU)On slave: relay-log.info/master.info: not durable/consistent! When OS crash, old information might be in the file If InnoDB only: use innodb_overwrite_relay_log_info in Percona Server: http://www.percona.com/doc/percona-server/5.5/reliability/ innodb_recovery_update_relay_log.html www.percona.com 73

ConsistencyReplication can cause inconsistencies: Statement Based Replication Certain functions like UUID()... UPDATE/DELETE with LIMIT without ORDER BY unique key Writing to multiple masters Writing to slaves (by accident? use read_only) Failover done wrong MyISAM with killing queries and/ord MySQL Crashes Wrongly restored slaves Replication broke and used SQL_SLAVE_SKIP_COUNTER ...How to know if my data is consistent? www.percona.com 74

ConsistencyChecking Consistency: pt-table-checksum Checksums chunks of data on master and slaves Monitors replication lag Low checksum response time to lower impact on production Uses Replication to ensure consistent view of dataFixing Consistency: pt-table-sync Can checksum and use pt-table-checksum results Fixes inconsistencies www.percona.com 75

Replication Use CasesScale ReadsHigh AvailabilityBackups www.percona.com 77

High AvailabilityFailover: Virtual IPs Change IP in application serversWarning: Split-Brain possible!!! Ensure no writes happen on old master before moving Make sure new master is in syncScripts available to automate failover: MySQL-MHA http://code.google.com/p/mysql-master-ha/ MMM (not-recommended) http://mysql-mmm.org/ Percona-PRM http://www.mysqlperformanceblog.com/2011/11/29/percona-replication-manager-a- solution-for-mysql-high-availability-with-replication-using-pacemaker/ www.percona.com 78

High AvailabilityDue to asynchronous’ness, data might be lost:Semi-Synchronous Replication: ensure at least one slave’s IO thread has received the event before the transaction is committed. http://dev.mysql.com/doc/refman/5.5/en/replication-semisync.html www.percona.com 79

BackupsPoint In Time Recovery, using binary logsDelayed slaveBackup Slaves www.percona.com 80

Incremental / Point In Time RecoveryRestore Full Backup from your favorite backup toolReplay binary logs with mysqldump from the last positionof the backup# mysqlbinlog --start-position=15115 1sbinarylog | mysql# mysqlbinlog next-binary-logs | mysqlIf PITR, stop at the desired position/time:# mysqlbinlog --stop-position=102151 binarylog | mysql# mysqlbinlog --stop-datetime=“2012-01-01 23:00:00” lastbinarylog | mysql www.percona.com 81

Delayed SlaveDeliberately lag a slave for a predefined amount of timeDecrease recovery time in case of bad changes caused by: Humans (DROP DATABASE) Application (DELETE FROM table;)How to configure: pt-slave-delay --delay 60m (http://www.percona.com/doc/percona-toolkit/2.0/pt-slave-delay.html) 5.6 Built-in feature: > CHANGE MASTER TO MASTER_DELAY=3600;When a problem happens stop replication just before themalicious statement.START SLAVE UNTIL MASTER_LOG_FILE=log_name,MASTER_LOG_POS = pos; www.percona.com 82

Backup using SlavesOnline Backups might not be possible on active master:Too much impact to make online: InnoDB backup (using MySQL enterprise backup or XtraBackup) LVM Snapshots MyISAM tablesOffsite Backup www.percona.com 83

Other Replication ImplementationsTungsten Replicator Uses binary log Parallel replication Enhanced filtering Multi-master Replication between different databasesGalera Cluster Does not use binary log Synchronous replication Active-active master Parallel replication Percona XtraDB Cluster! http://www.mysqlperformanceblog.com/2012/01/09/ announcement-of-percona-xtradb-cluster-alpha-release/ www.percona.com 85

Tungsten Replicatorhttp://continuent.com/http://code.google.com/p/tungsten-replicator/ www.percona.com 86

Galera Clusterhttp://codership.com/ www.percona.com 87

Tools & More InfoPercona Toolkit http://www.percona.com/software/percona-toolkit/pt-slave-find, pt-table-checksum, pt-table-sync,pt-slave-delay, pt-slave-restart, pt-heartbeatPercona Server http://www.percona.com/software/percona-server/OpenArk kit http://code.openark.org/forge/openark-kitReplication Booster https://github.com/yoshinorim/replication-booster-for-mysqlHigh Availability MMM http://mysql-mmm.org/ Percona-PRM http://www.mysqlperformanceblog.com/2011/11/29/percona-replication-manager-a- solution-for-mysql-high-availability-with-replication-using-pacemaker/ MySQL-MHA http://code.google.com/p/mysql-master-ha/High Performance MySQL, 2nd Edition:http://shop.oreilly.com/product/9780596101718.do www.percona.com 89

Kenny Gryp <kenny.gryp@percona.com> @gryp Were Hiring!www.percona.com/about-us/careers/

http://www.mysqlops.com	3143
http://www.scoop.it	91
http://yananguo.gotoip4.com	71
http://www.mysqlpub.com	56
http://www.mysqlops.com.	11
http://www.mysqlops.net	5
http://localhost	3
http://web-01.mysqlops.com	2
http://mysqlops.net	2
http://tuicool.com	1
http://www.mysqlops.com&_=1330064854646 HTTP	1
http://translate.googleusercontent.com	1

Percona Live 2012PPT： introduction-to-mysql-replication

by mysqlops

Accessibility

Categories

Upload Details

Usage Rights

12 Embeds 3,387

Statistics

Percona Live 2012PPT： introduction-to-mysql-replication Presentation Transcript