Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
MySQL 5.6 Replication Webinar
1. MySQL 5.6 Replication:
Supporting High Scale Web & Cloud Services
Mat Keep
MySQL Product Management
mat.keep@oracle.com
Andrew Morgan
MySQL Product Management
andrew.morgan@oracle.com
2. <Insert Picture Here>
Agenda
• Replication Use Cases
• MySQL 5.6 Replication Features
• Resources to Get Started
Copyright 2013 Oracle Corporation | 27th February 2013 2
3. Industry Leaders Rely on MySQL
Cloud
Web & Enterprise OEM & ISVs
Copyright 2013 Oracle Corporation | 27th February 2013 3
4. ELASTIC SCALABILITY REAL TIME USER EXPERIENCE
ROCK SOLID RELIABILITY RAPID SERVICE INNOVATION
Driving new Database Demands
Copyright 2013 Oracle Corporation | 27th February 2013 4
5. • MySQL 5.6 builds on MySQL 5.5 by improving:
• Performance and Scalability
• Optimizer for better query execution times, diagnostics
• Performance Schema for better instrumentation
• InnoDB for better transactional throughput
• NoSQL API for better flexibility
• Replication largest set of enhancements ever released
MySQL Database 5.6: A Better MySQL.
Copyright 2013 Oracle Corporation | 27th February 2013 5
9. Multi-Threaded Slaves
• Increases slave throughput,
reducing lag
• All transactions received into
slave’s relay log
• Implements multiple SQL threads,
based on database
• Applies events to different
databases in parallel
• Great for systems which isolate
application data using databases –
e.g. multi-tenant
Copyright 2013 Oracle Corporation | 27th February 2013 9
SQL-A
I/O
Relay
Log
T1T2T3T4
SQL-B
SQL-C
DB-A DB-B DB-C
T1
T4
T2 T3
Coordinator
10. Multi-Threaded Slaves
• Throughput of slave increased by allowing multiple slave
threads:
• 0 – functionality disabled
• Up to 1024
• Exec_Master_Log_Posn in SHOW SLAVE STATUS now
shows the low-water-mark
• Configure using:
slave-parallel-workers=4
• Only ensures sequencing within a database
• Suitable if ordering of updates between databases is not required
Copyright 2013 Oracle Corporation | 27th February 2013 10
11. • Transactions applied in parallel on the slave
• Sequencing of transactions within a given
database will be the same
Multi-Threaded Slaves
Copyright 2013 Oracle Corporation | 27th February 2013 11
Master
INSERT INTO dba.tab1...
INSERT INTO dba.tab2...
INSERT INTO dbb.tab1...
INSERT INTO dbc.tab2...
INSERT INTO dba.tab1...
INSERT INTO dbb.tab3...
INSERT INTO dba.tab3...
INSERT INTO dbc.tab2...
INSERT INTO dba.tab4...
Slave thread1
dba.tab1
dba.tab2
dba.tab1
dba.tab3
dba.tab4
dbb.tab1
dbc.tab2
dbb.tab3
dbc.tab2
Slave thread2
12. Binlog Group Commit
• Increases replication throughput by
increasing performance of the
master
• Commits multiple transactions as a
group to Binlog on disk
• Finer grained locking; reducing lock
wait times
Copyright 2013 Oracle Corporation | 27th February 2013 12
Session
Binary
Log
Master
Database
T1 T2
T3 T4
Group
commit
13. Replication Binary Log Group Commit
• 4.8x faster than
MySQL 5.5
• 4.5x even when
sync_binlog=1
Copyright 2013 Oracle Corporation | 27th February 2013 13
https://blogs.oracle.com/MySQL/entry/mysql_5_6_replication_performance
0
2,000
4,000
6,000
8,000
10,000
12,000
32 64 128 256
TransactionsPerSecond
Connections
MySQL 5.5 vs 5.6: Master Performance
5.6 Sync=0
5.6 Sync=1
5.5 Sync=0
14. Replication Binary Log Group Commit
• Days of losing 50%+
master performance
are gone!
• 10% overhead when
configuring
sync_binlog=0
• Additional 5% when
configuring
sync_binlog=1
Copyright 2013 Oracle Corporation | 27th February 2013 14
0
2,000
4,000
6,000
8,000
10,000
12,000
32 64 128 256
TransactionsPerSecond
Connections
MySQL 5.6: Master Performance
No Binlog
Sync=0
Sync=1
https://blogs.oracle.com/MySQL/entry/mysql_5_6_replication_performance
15. Binary Log Group Commit
• Better handling of multiple writers by using finer grained locking
• Changes to InnoDB allow in-memory commits while binary log group
commit is running
• Flushing of binary log to disk controlled by these options:
• binlog-order-commits={ON,OFF}
Whether transaction commits must be recorded in the correct order in the
binary log. Performance may be better when turned off as transactions can
be committed to the binary log in parallel.
• binlog-max-flush-queue-time
Use to reduce latency of commits in highly loaded systems; this is the target
max time (in microseconds) and is not guaranteed.
Copyright 2013 Oracle Corporation | 27th February 2013 15
16. Optimized Row Based Replication
• Increases replication throughput for master and slave
• Reduces Binlog size, memory requirements & network bandwidth
• Only replicates those elements of the Row image that
have changed
Primary Key Changed Columns
Copyright 2013 Oracle Corporation | 27th February 2013 16
17. Optimized Row Based Replication
• Default is to include full before & after image for
every changed row
• New option: binlog-row-image=minimal
• Reduces space:
• Inserts: only after image
• Deletes: only before image
• Updates: after image (modified columns) + primary key or
indexed columns of before image
Copyright 2013 Oracle Corporation | 27th February 2013 17
19. Global Transaction Identifiers
• Simple to track & compare replication across the cluster
– Unique identifier for each transaction written to the Binlog
• Automatically identify the most up-to-date slave for failover
• Deploy n-tier replication hierarchies
Master
GTID=123456
GTID=123456
GTID=123456 GTID=123456
Copyright 2013 Oracle Corporation | 27th February 2013 19
20. Before Global Transaction IDs
Copyright 2013 Oracle Corporation | 27th February 2013 20
Master
{my-bin.000132,
782}
{my-bin.000101,
873}
{my-bin.000099,
123}
{my-bin.00088,
1027}
21. Before Global Transaction IDs
Failover & Recovery
Copyright 2013 Oracle Corporation | 27th February 2013 21
Master
{my-bin.000132,
782}
{my-bin.000101,
873}
{my-bin.000099,
123}
{my-bin.00088,
1027}
CHANGE MASTER TO??
Must convert
{file,pos} from
old master to one
from the new master
22. Crash-Safe Slaves & Binlog
Before:
– Transaction Data: in tables
– Replication Info: in files
MySQL 5.6
– Transaction Data: in tables
– Replication Info: in tables
Data
Position Info
CRASH!
Time
Data
Position Info
Time
• Automatic recovery of a slave and Binlog
after a failure
• Binlog and table data are transactionally
consistent
• Resumes replication without Dev/Op
intervention
• Automatically rolling back replication to last
committed event
• Eliminates risk of data loss or corruption
Atomic
Atomic
Copyright 2013 Oracle Corporation | 27th February 2013 22
23. Crash-Safe Slaves
• Writes and reads back only complete events or transactions from
the binary log
• master.info & relay-log.info files can be replaced with
tables
• ACID transaction prevent them getting out of sync with InnoDB tables
• --master-info-repository=TABLE
• --relay-log-info-repository=TABLE
mysql.slave_master_info
mysql.slave_relay_log_info
Copyright 2013 Oracle Corporation | 27th February 2013 23
25. Replication Event Checksums
• Ensures replicated data is correct,
consistent and accessible
• Detects corrupt replication events
before they’re applied
• Returns an error
• Protects entire replication path
• Memory
• Disk
• Network
• Bugs
Copyright 2013 Oracle Corporation | 27th February 2013 25
Master
#
Slave
#
26. Replication Event Checksums
• Implemented in the binary and relay logs
• New mysqld options:
• binlog-checksum=CRC32
Checksums written to the binary log
• master-verify-checksum=1
Master validates checksum read from the binary log
• slave-sql-verify-checksum=1
Slave validates checksum when reading from the relay log
Copyright 2013 Oracle Corporation | 27th February 2013 26
28. MySQL Utilities
Automate common Dev/Ops tasks
– Replication: provisioning, testing, monitoring and failover
– Database comparisons: consistency checking
– Database administration: users, connections, tables, etc
– New utilities in development, ie log analysis
Implemented as Python scripts, plug-in for MySQL Workbench
– Also available from LaunchPad
– Extensible to include custom scripting
Resources: Documentation & Community Forum
– http://dev.mysql.com/doc/workbench/en/mysql-utils-man.html
– http://forums.mysql.com/list.php?155
Copyright 2013 Oracle Corporation | 27th February 2013 28
29. Utility Workflow for Replication
• Replicate: Initiates Replication to the new slave
• Check: Validates the replication configuration and operation
• Show: Display Replication topology
• Fail-Over & Admin: Detects and failovers (or switches) from
master to slave. Status monitoring
Check Show
Fail-Over
& AdminReplicate
Copyright 2013 Oracle Corporation | 27th February 2013 29
30. mysqlreplicate Utility
• Starts replication process
• User can provide login and
connection parameters for the
master
• Configurable replication start points:
current position / GTID, specific
event, entire binlog
• Checks storage engine compatibility
• Enables fast, simple scaling
New Slave
Login,
Connection &
Start-Point
Master
Replication configured,
binlog position
Repl
Copyright 2013 Oracle Corporation | 27th February 2013 30
31. mysqlreplicate example
$ mysqlreplicate --master=root:billy@localhost:3306
--slave=root:billy@localhost:3307
# master on localhost: ... connected.
# slave on localhost: ... connected.
Repl
Copyright 2013 Oracle Corporation | 27th February 2013 31
32. mysqlrplcheck example
$ mysqlrplcheck --master=root@host1:3310
--slave=root@host2:3311
# master on host1: ... connected.
# slave on host2: ... connected. Test Description Status
-------------------------------------------------------------
Checking for binary logging on master [pass]
Are there binlog exceptions? [pass]
Replication user exists? [pass]
Checking server_id values [pass]
Is slave connected to master? [pass]
Check master information file [pass]
Checking InnoDB compatibility [pass]
Checking storage engines compatibility [pass]
Checking lower_case_table_names settings [pass]
Checking slave delay (seconds behind master) [pass]
# ...done.
Check
Copyright 2013 Oracle Corporation | 27th February 2013 32
33. mysqlrplshow Utility
• Discovers & displays replication
topology
• Shows slaves attached to each
master
• Labels each slave with hostname and
port number
• Displays multiple topologies, including
hierarchical and circular
• Displays results in a graph or list
• Enables user to monitor topology
on-demand
$ mysqlrplshow --master=root@localhost:3311
# master on localhost: ... connected.
# Finding slaves for master: localhost:3311
# Replication Topology Graph::
localhost:3306 (MASTER)
|
+--- localhost:3307 - (SLAVE)
|
+--- localhost:3308 - (SLAVE + MASTER)
|
+--- localhost:3313 - (SLAVE)
Show
Copyright 2013 Oracle Corporation | 27th February 2013 33
34. Use of mysqlfailover
mysqlfailover --master=root@black:3306 --discover-slaves-login=root --rediscover
MySQL Replication Failover Utility
Failover Mode = auto Next Interval = Wed Aug 15 13:19:30 2012
Master Information
------------------
Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB
black-bin.000001 2586
GTID Executed Set
A0F7E82D-3554-11E2-9949-080027685B56:1-5
Replication Health Status
+---------+-------+---------+--------+------------+---------+
| host | port | role | state | gtid_mode | health |
+---------+-------+---------+--------+------------+---------+
| black | 3306 | MASTER | UP | ON | OK |
| blue | 3306 | SLAVE | UP | ON | OK |
| green | 3306 | SLAVE | UP | ON | OK |
| brown | 3306 | SLAVE | UP | ON | OK |
| red | 3306 | SLAVE | UP | ON | OK |
+---------+-------+---------+--------+------------+---------+
Q-quit R-refresh H-health G-GTID Lists U-UUIDs
Fail-Over
Copyright 2013 Oracle Corporation | 27th February 2013 34
35. mysqlfailover – master failed
Failover starting...
# Candidate slave blue:3306 will become the new master.
# Preparing candidate for failover.
# Creating replication user if it does not exist.
# Stopping slaves.
# Performing STOP on all slaves.
# Switching slaves to new master.
# Starting slaves.
# Performing START on all slaves.
# Checking slaves for errors.
# Failover complete.
# Discovering slaves for master at jane:3306
Failover console will restart in 5 seconds.
Fail-Over
Copyright 2013 Oracle Corporation | 27th February 2013 35
36. Use of mysqlfailover
MySQL Replication Failover Utility
Failover Mode = auto Next Interval = Wed Aug 15 13:19:30 2012
Master Information
------------------
Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB
black-bin.000001 2586
GTID Executed Set
A0F7E82D-3554-11E2-9949-080027685B56:1-5
Replication Health Status
+---------+-------+---------+--------+------------+---------+
| host | port | role | state | gtid_mode | health |
+---------+-------+---------+--------+------------+---------+
| blue | 3306 | MASTER | UP | ON | OK |
| green | 3306 | SLAVE | UP | ON | OK |
| brown | 3306 | SLAVE | UP | ON | OK |
| red | 3306 | SLAVE | UP | ON | OK |
+---------+-------+---------+--------+------------+---------+
Q-quit R-refresh H-health G-GTID Lists U-UUIDs
Fail-Over
Copyright 2013 Oracle Corporation | 27th February 2013 36
37. Replication Administration Utility
• Perform switchover to eliminate downtime
during planned maintenance
• Start and stop slaves
• Slave discovery & monitoring
• Slave status, thread status
• Replication processing, including any lag
• Configure slave promotion policies
• Non GTID-functionality can be used with pre-
MySQL 5.6 releases
Master
Slaves
Administration
Utility
Status &
Switchover
Admin
Copyright 2013 Oracle Corporation | 27th February 2013 37
38. Planned switchover to new master with mysqlrpladmin
mysqlrpladmin --master=root@blue:3306 --new-master=root@black:3306 --
demote-master --discover-slaves-login=root switchover
# Discovering slaves for master at blue:3306
# Checking privileges.
# …
#
# Replication Topology Health:
+---------+-------+---------+--------+------------+---------+
| host | port | role | state | gtid_mode | health |
+---------+-------+---------+--------+------------+---------+
| black | 3306 | MASTER | UP | ON | OK |
| blue | 3306 | SLAVE | UP | ON | OK |
| green | 3306 | SLAVE | UP | ON | OK |
| brown | 3306 | SLAVE | UP | ON | OK |
| red | 3306 | SLAVE | UP | ON | OK |
+---------+-------+---------+--------+------------+---------+
# ...done.
Switch-
Over
Copyright 2013 Oracle Corporation | 27th February 2013 38
39. Time Delayed Replication
• Configure time period before replication
events applied to slave
• Per-slave, via execution of SQL Thread
• Second-level granularity, up to 68 years!
• Protects against operational error
• Dropping a table, etc.
• Allows database to be inspected
without loading a back-up
Copyright 2013 Oracle Corporation | 27th February 2013 39
Master
:3306
Binary
Logs
Relay
Log
Relay
Log
Slave
1
:3307
Slave2
:3308
10 Minute Delay
41. Time Delayed Replication – Doh!
slave2> STOP SLAVE;
Breath slowly!
master> SHOW BINLOG EVENTSG
*************************** 12. row ***************************
Log_name: ws2-bin.000001 Pos: 984 Event_type: Query Server_id: 1
End_log_pos: 1096 Info: use `clusterdb`; DROP TABLE `towns` /* generated by
server */
slave2> START SLAVE UNTIL
-> MASTER_LOG_FILE='ws2-bin.000001',
-> MASTER_LOG_POS=984;
Copyright 2013 Oracle Corporation | 27th February 2013 41
42. Time Delayed Replication – Doh! (with GTIDs)
slave2> STOP SLAVE;
Breath slowly!
master> SHOW BINLOG EVENTSG
*************************** 12. row ***************************
Log_name: ws2-bin.000001 Pos: 940 Event_type: Gtid Server_id: 1 End_log_pos:
984 Info: SET @@SESSION.GTID_NEXT= '4B2CBA63-8082-11E1-BE2D-F0DEF11A08B7:666'
*************************** 13. row ***************************
Log_name: ws2-bin.000001 Pos: 984 Event_type: Query Server_id: 1 End_log_pos:
1096 Info: use `clusterdb`; DROP TABLE `towns` /* generated by server */
slave2> SET GTID_NEXT=“SET @@SESSION.GTID_NEXT=
“4B2CBA63-8082-11E1-BE2D-F0DEF11A08B7:666”;
slave2> COMMIT;
slave2> START SLAVE;
Copyright 2013 Oracle Corporation | 27th February 2013 42
43. Remote Binary Log Backup
• Creates real-time backup of binary log on remote server
• Adds a “raw” flag to the mysqlbinlog command
• Sent via the replication channel
$> mysqlbinlog --read-from-remote-
server --raw –h secret_server -P 3306
-u root mysql-bin.000001
Copyright 2013 Oracle Corporation | 27th February 2013 43
44. Informational Log Events
• Simplifies debugging and auditing when using Row
Based Replication
• Original query written to the Binlog
• Replicated with row event to the slave
• Activate with
binlog-rows-query-log-events=TRUE
master> INSERT INTO simples VALUES (20),
(21),(22);
Copyright 2013 Oracle Corporation | 27th February 2013 44
45. Informational Log Events
BEFORE: mysqlbinlog (-v):
### INSERT INTO clusterdb.simples
### SET
### @1=20 /* INT meta=0 nullable=0 is_null=0 */
### INSERT INTO clusterdb.simples
### SET
### @1=21 /* INT meta=0 nullable=0 is_null=0 */
### INSERT INTO clusterdb.simples
### SET
### @1=22 /* INT meta=0 nullable=0 is_null=0 */
Copyright 2013 Oracle Corporation | 27th February 2013 45
46. Informational Log Events
AFTER: mysqlbinlog (-vv):
# at 443
#111128 16:04:24 server id 1 end_log_pos 504 Rows_query
# INSERT INTO simples VALUES (20),(21),(22)
# at 504
Copyright 2013 Oracle Corporation | 27th February 2013 46
47. Next Steps
Copyright 2013 Oracle Corporation | 27th February 2013 47
www.mysql.com/why-mysql/white-papers/mysql-replication-introduction/ www.mysql.com/why-mysql/white-papers/mysql-replication-tutorial/
• Evaluate the new features
• Questions & Feedback: forums.mysql.com/list.php?26
• Bugs: bugs.mysql.com/
• Replication documentation
• dev.mysql.com/doc/refman/5.6/en/replication.html
• Learn more about HA Solutions for MySQL
• mysql.com/why-mysql/white-papers/mysql_wp_ha_strategy_guide.php