Open source India - MySQL Labs: Multi-Source Replication

1. MySQL Labs : Multi Source Replication Shivji Kumar Jha, Software Developer, MySQL Replication Team 1 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

2. Safe Harbour Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

3. MySQL Labs features are not fit for production. They are provided solely for testing purposes, to try the latest bug fixes and generally to keep up with the development. Please, do not use these binaries in production. Instead, install them on a spare server. If you are looking for production ready binaries, please visit MySQL Downloads. 3 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

4.  Agenda Background: Why Use Replication?  Multi-source Replication – Introduction – Use cases – Internals – Monitoring  4 Reading More about Multi-source Replication | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

5. Background: What is Replication Used For?  Read scale-out S M S More reads? More slaves! M S S write clients write clients 5 read clients | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | read clients

6. What about Write scale-out? M? M More writes? More Masters? M? M? write clients Enter MySQL Fabric ... 6 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | write clients

7. Background: What is Replication Used For?  Redundancy: If master crashes, promote slave to master B A B Uh Oh! C B Whew! Crash A A C C B is the new master 7 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

8. Background: What is Replication Used For?  On-line Backup and Reporting S M   write clients  8 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |  business intelligent client apps reporting client apps big queries client apps

9. Background: What is Replication Used For? A C A B B C Image from www.ginkgomaps.com 9 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

11. Multi-Source Replication: Introduction  A Slave can have more than one master. M1 M2 M3 M4 11 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S

12. Multi-Source Replication: Introduction  A Slave can have more than one master. db1 12 M1 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

13. Multi-Source Replication: Introduction  A Slave can have more than one master. db1 db2 13 M1 M2 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

14. Multi-Source Replication: Introduction  A Slave can have more than one master. db1 db2 M2 db3 M3 db4 14 M1 M4 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

15. Multi-Source Replication: Introduction  A Slave can have more than one master. db1 M1 db2 M2 S db3 db4 15 M3 M4 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | db1 db2 db3 db4

17. Multi-Source Replication: Use Cases  The main use cases of Multi-source replication are related to data aggregation. 17 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

18. Multi-Source Replication: Use Cases Database 1 M1 Database 2 Database 1, 2, 3 M2 Database 3 M3 18 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S Business Intelligence Data analytics Backup etc

19. Multi-Source Replication: Use Cases Shard 1 M Shard 2 Full table M Shard 3 M 19 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S

21. Multi-Source Replication: More details..  Slave can have more than one master. M1 – Receive transactions from several MySQL servers simultaneously. M2 S – Apply transactions from different masters simultaneously. M3 – No conflict detection or resolution. M4 21 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

22. Multi-Source Replication: What is a Channel?  A channel is an an abstraction for a sender-receiver-applier path. Client Insert... Receiver thread Sender thread Applier thread binary log binary log Insert... A relay log Insert... Insert... Network Channel 22 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | B

23. Multi-Source Replication: How Many Channels?  Number of channels on slave = Number of sources. M1 Network relay log1 M2 binary log Network relay log2 M3 23 binary log binary log Network relay log3 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S binary log

24. Multi-Source Replication: Slave Appliers  Each channel has its own single-threaded slave applier. M1 Network relay log1 M2 binary log Network relay log2 M3 24 binary log binary log Network relay log3 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S binary log

25. Multi-Source Replication: Slave Appliers  Each channel has its own multi-threaded slave applier. M1 Network relay log1 M2 binary log Network relay log2 M3 25 binary log binary log Network relay log3 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | S binary log

26. Adding / Configuring A Channel  Each channel can be configured individually.  To add or alter a channel configuration use: CHANGE MASTER TO master_def … FOR CHANNEL=”<channel_name>” 26 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

27. Replication Commands on a Channel  The FOR CHANNEL=”<channel_name>” clause is consistent across different replication commands. – start slave [...] FOR CHANNEL=”<channel_name>" – stop slave [thread_types] FOR CHANNEL=”<channel_name>” – reset slave [all] FOR CHANNEL=”<channel_name>” – flush relay logs FOR CHANNEL=”<channel_name>” – show relay log events FOR CHANNEL=”<channel_name>” ... 27 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

28. Working With All Channels  The FOR ALL CHANNELS clause. – START SLAVE [thread_type] FOR ALL CHANNELS – STOP SLAVE [thread_type] 28 FOR ALL CHANNELS | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

29. Compatibility With Other Replication Features Multi-Threaded Slaves Global Transaction Identifiers Yes Semi-Synchronous Replication Yes Crash Safe Slaves Yes Filters 29 Yes Yes | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

31. Replication Monitoring: P_S Replication Tables Slave Status Connection Configuration Connection Status Execution Configuration Execution Status Applier / Coordinator Status 31 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. | Workers Status

32. Replication Monitoring: P_S Replication Tables  We have six performance schema tables for replication (MySQL-5.7.2): – replication_connection_configuration – replication_connection_status One row per CHANNEL – replication_execute_configuration – replication_execute_status – replication_execute_status_by_coordinator – replication_execute_status_by_worker One row per CHANNEL per WORKER  Consistent semantics across tables. Lets explore one of them. 32 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

33. Replication Monitoring: Connection Status mysql> select * from performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME : CHANNEL1 SOURCE_UUID : 7cff7406-23ca-11e3-ac3e-5c260a83b12b THREAD_ID : 13 SERVICE_STATE : ON RECEIVED_TRANSACTION_SET : 7cff7406-23ca-11e3-ac3e-5c260a83b12b:1-4 LAST_ERROR_NUMBER :0 LAST_ERROR_MESSAGE : LAST_ERROR_TIMESTAMP : 0000-00-00 00:00:00 *************************** 2. row *************************** CHANNEL_NAME : CHANNEL2 ... One row per CHANNEL 33 33 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

34. Replication Monitoring: Connection Status mysql> select * from performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME : CHANNEL1 SOURCE_UUID : 7cff7406-23ca-11e3-ac3e-5c260a83b12b THREAD_ID : 13 SERVICE_STATE : ON RECEIVED_TRANSACTION_SET : 7cff7406-23ca-11e3-ac3e-5c260a83b12b:1-4 LAST_ERROR_NUMBER :0 LAST_ERROR_MESSAGE : Receiver thread & LAST_ERROR_TIMESTAMP : 0000-00-00 00:00:00 *************************** 2. row *************************** its service state CHANNEL_NAME : CHANNEL2 ... 34 34 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

35. Replication Monitoring: Connection Status mysql> select * from performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME : CHANNEL1 SOURCE_UUID : 7cff7406-23ca-11e3-ac3e-5c260a83b12b THREAD_ID : 13 SERVICE_STATE : ON RECEIVED_TRANSACTION_SET : 7cff7406-23ca-11e3-ac3e-5c260a83b12b:1-4 LAST_ERROR_NUMBER :0 LAST_ERROR_MESSAGE : LAST_ERROR_TIMESTAMP : 0000-00-00 00:00:00 *************************** 2. row *************************** CHANNEL_NAME : CHANNEL2 Set of transactions received ... through this channel 35 35 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

36. Replication Monitoring: Connection Status mysql> select * from performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME : CHANNEL1 SOURCE_UUID : 7cff7406-23ca-11e3-ac3e-5c260a83b12b THREAD_ID : 13 SERVICE_STATE : ON RECEIVED_TRANSACTION_SET : 7cff7406-23ca-11e3-ac3e-5c260a83b12b:1-4 LAST_ERROR_NUMBER : 1045 LAST_ERROR_MESSAGE : error connecting to master 'replssl@127.0.0.1:13000' ... LAST_ERROR_TIMESTAMP : 2013-11-04 13:37:23 *************************** 2. row *************************** CHANNEL_NAME : CHANNEL2 ... Oops! There was an error in the receiver thread on this channel 36 36 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

37. Replication Monitoring: Connection Status mysql> select * from performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME : CHANNEL1 Receiver SOURCE_UUID : 7cff7406-23ca-11e3-ac3e-5c260a83b12b thread on this channel stopped serving... THREAD_ID : NULL SERVICE_STATE : OFF RECEIVED_TRANSACTION_SET : 7cff7406-23ca-11e3-ac3e-5c260a83b12b:1-4 LAST_ERROR_NUMBER : 1045 LAST_ERROR_MESSAGE : error connecting to master 'replssl@127.0.0.1:13000' ... LAST_ERROR_TIMESTAMP : 2013-11-04 13:37:23 *************************** 2. row *************************** CHANNEL_NAME : CHANNEL2 ... 37 37 | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

39. Read More About Multi Source Replication    39 Read more about Multi-source replication from the design notes: http://dev.mysql.com/worklog/task/?id=1697 Read a feature preview of Multi-source replication on Rith's blog: http://on-mysql-replication.blogspot.in/2013/09/feature-preview-mysqlmulti-source-replication.html Read more about Multi-source replication on Rith's blog: http://on-mysql-replication.blogspot.in/2013/09/mysql-labs-multi-sour ce-replication.html | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

40. Read More About Multi Source Replication   40 Official MySQL documentation for replication P_S tables: http://dev.mysql.com/doc/refman/5.7/en/performance-schema-replicati on-tables.html Read more about replication P_S tables on Shiv's blog: http://shivjijha.blogspot.com/2013/09/Monitoring-Replication-with-t he-NEW-performance-schema-tables.html | Copyright © 2013, Oracle and/or its affiliates. All rights reserved. | Open Source India | Bangalore, India, November 13 th, 2013. |

Open source India - MySQL Labs: Multi-Source Replication

Shivji kumar Jha

Open source India - MySQL Labs: Multi-Source Replication