MySQL replication has evolved a lot in 5.6 ,5.7 and 8.0. This presentation focus on the changes made in parallel replication. It covers MySQL 8.0. It was presented at Mydbops database meetup on 04-08-2016 in Bangalore.

1. Evolution of MySQL Parallel
replication
Mydbops Database Meetup (Bangalore)
Presenter
Karthik P R
Founder Mydbops
www.mydbops.com info@mydbops.com

2. About Mydbops
● Founded in 2015, HQ in Bangalore India with 150+ customer base across the globe.
● Mydbops is on Database Consulting with core specialization on MySQL and MongoDB Administration and
Support.
● We have expert team with 20+ certified DBA’s providing full time support and currently managing 300+
servers.
● Mydbops was created with a motto of developing a DevOPS model for Database administration offering
24*7 expert remote DBA support.
● We help organisations to architect and scale systems in MySQL/MongoDB by implementing the advanced
technologies in industry which are completely open source.

3. Mydbops is into MySQL/MongoDB Support and Consulting. It is founded by experts
who have scaled database at Yahoo! ,Percona and Datavail. We are providing an
expert level support and 24*7 monitoring for MySQL databases and its related
technologies like MariaDB , Percona ( also clustering ) . We support modern
database technologies in MySQL which includes Galera ( Clustering ), Group
Replication , SQL aware Load balancers like Maxscale / ProxySQL.
About Mydbops

4. `
mysqlsupport@mydbops.com
Our Clients
www.mydbops.com

5. CEO / DB Architect
About Me

6. ● Introduction : MySQL Replication
● Basics of Binary log .
● Parallel Replication
● Summary
Table of Contents

7. ● One of the most popular feature
● Native and inbuilt
● Logical Replication
● Asynchronous in state.
● Semi Sync and Flow control ( Group Replication )
● Read Scalability
● Native high availability feature.
Introduction : MySQL Replication

8. Introduction : MySQL Replication

9. ● Binary log is the key for replication
● Filtered Replication
○ Binlog filter
○ Relay log filter
● Possible Architecture
○ Master - Slave
○ Master - Master
○ Multi master ( Multi Source )
○ Intermediate Slave
○ Group Replication
● No parallelism by default ( Single threaded )
Introduction : MySQL Replication

10. ● Journal of the master writes.
● Contains the DDL and other writes.
● Records the statements in the order of transaction commit.
● Works independent of the engine.
● Formats : ROW and Statement
● Reproduced on the slaves as relay log
Basics of Binary log

11. Basics of Binary log
Execute Binlog Commit
Two Phase Commit (2PC)
1. Changes are collected in transaction cache ( per connection )
2. Transaction is committed to storage engine
3. Transaction cache is written to binary log (commit).

12. Sample binlog content ( MySQL 8.0 )
BEGIN
/*!*/;
# at 825
#180803 23:57:16 server id 101 end_log_pos 893 CRC32 0xbc8bc6b1 Table_map: `sbtest1`.`sbtest6` mapped to
number 82
# at 893
#180803 23:57:16 server id 101 end_log_pos 1118 CRC32 0xdfabee92 Write_rows: table id 82 flags: STMT_END_F
### INSERT INTO `sbtest1`.`sbtest6`
### SET
### @1=1000001 /* INT meta=0 nullable=0 is_null=0 */
### @2=501910 /* INT meta=0 nullable=0 is_null=0 */
###
@3='93213299669-86838228032-85626473166-33684960785-51919150459-30647403687-71837739046-18605717775-51951633177-01
652473758' /* STRING(480) meta=61152 nullable=0 is_null=0 */
### @4='10076769068-66969333322-72197014465-67237974816-92730387311' /* STRING(240) meta=65264 nullable=0
is_null=0 */
# at 1118
#180803 23:57:16 server id 101 end_log_pos 1149 CRC32 0x8a6136cc Xid = 7532
COMMIT/*!*/;
# at 1149
Basics of Binary log

13. Parallel Replication

14. Need for parallel replication.
● Effective usage of multi core machines ( avoid single core writes in slaves )
● Use the modern disk and storage efficiently
○ RAID 1 ( 2 disks 2 IOs parallel)
○ SSD ( write IOs in parallel )
● Faster replication ( minimal lag )
Slowness happens at SQL_thread ( SQL applier )
Parallel Replication

15. MySQL 5.6 MySQL 5.7 MySQL 8.0
Parallel Replication
WriteSet
Transaction writing different
tuples
Schema Based
Schema local Transactions
Logical Clock
Binary log group commits

16. Schema based

17. MySQL 5.6 ( Schema based )
● Two transactions of different schema can be parallelised
● Transaction are distributed based on per database basis.
● The transaction ordering can be different master and slave.
○ Master A1, B1, A2, B2, A3,B3
○ Slave A1, B1, A2, A3, B2, B3
Parallel Replication

18. Parallel Replication
Parallel Threads = N
Coordinator Thread = 1
Worker Threads = (N-1)
Schema based parallel replication

19. Schema based ( MySQL 5.6)
● How to Facilitate ?
○ slave_parallel_workers=N ( N > 1)
○ slave_parallel_type=”DATABASE” ( default )
● Impact
○ Gaps in transaction order
○ Replication Crash Recovery ( GTID can be a saviour )
○ Beware of backup used.
Parallel Replication

20. Logical Clock

21. Logical clock ( MySQL 5.7)
● Based on binlog group commit
● Binlog_group_commit_sync_delay and binlog_group_commit_sync_no_delay_count.
● Parallelism interval is based on last_committed and sequence number in binary logs
● Tuning need binlog analysis ( no counters inbuilt )
● Slave_preserve_commit_order avoids gaps.
https://jfg-mysql.blogspot.com/2017/02/metric-for-tuning-parallel-replication-mysql-5-7.html
Parallel Replication

22. Logical clock ( MySQL 5.7)
Parallel Replication
Last_committed : Start with 0 and reset at end of binlog
Sequence_number : Start with 1 and reset at end of binlog

23. Schema based ( MySQL 5.7)
● How to Facilitate ?
○ slave_parallel_workers=N ( N > 1)
○ slave_parallel_type=”LOGICAL_CLOCK” ( default )
○ Tune the binlog group commit.
● Impact
○ Can slow down the master writes.
○ Gaps in transaction order ( Slave_preserve_commit_order can avoid it ).
○ slave_preserve_commit order needs log_slave_updates.
○ longer transactions can block or slow down
Parallel Replication

24. Writeset

25. Write set ( MySQL 8.0 and 5.7.22)
● Transaction that affect different rows ( tuples ) can be parallelized.
● The dependency of each transaction is tracked ( history ).
● The sequence number of the last transaction which updates current row is tracked.
● Faster and better mode of parallel replication.
write set : Any transaction with different tuples can be parallelized.
write set session : Updates from same session can’t be reordered
Parallel Replication

26. Parallel Replication
Master - Writes

27. Parallel Replication
Slave ( Writeset )

28. Parallel Replication
Slave ( Writeset Session)

29. Write set ( MySQL 8.0)
● How to Facilitate ?
○ slave_parallel_workers=N ( N > 1)
○ slave_parallel_type=”LOGICAL_CLOCK” ( default )
○ binlog_transaction_dependency_tracking= writeset/writeset_session
○ transaction_writeset_extraction= XXHASH64/MURMUR32
To Disable write set :
● binlog_transaction_dependency_tracking= COMMIT_ORDER
Parallel Replication

30. Limitations :
● Primary key is needed on all table.
● Do not work well with tables with foreign keys.
● Works only with the ROW based replication ( Default in 8.0).
Additional Info :
● Used in group replication
● 5.7.22 has writeset ( backported from MySQL 8.0)
Parallel Replication

31. Summary

32. Summary :
● Parallel replication minimize the lag.
● It’s not easy to setup ( tune ) but worth the efforts.
● Schema based can be used in multi tenant environments.
● Logical Clock (5.7) needs more tuning.
● Writeset (8.0) gives better performance.
Summary

33. General tips over parallel replication:
● Use only InnoDB tables.
● Enable GTID’s for crash safe replication.
● relay_log_recovery should be enabled.
● master_info and slave_info_repositorty should be in table.
● ROW based replication is faster and better for most cases.
● Ensure your tables have a primary key.
General Tips

34. ● https://jfg-mysql.blogspot.com Jean-Francois Slides and blogs on parallel
replication.
● https://mysqlhighavailability.com/improving-the-parallel-applier-with-writeset-ba
sed-dependency-tracking/ Writeset replication.
● http://mysqlmusings.blogspot.com/2012/06/binary-log-group-commit-in-mysql-5
6.html Binlog Group commit.
Resources

35. info@mydbops.com
www.mydbops.com
080-48505683
Contact us

36. Thank You !!!