Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mix ‘n’ Match Async and Group Replication for Advanced Replication Setups

750 views

Published on

A look into how you can migrate to group replication and combine it with other replication features

Published in: Software
  • Be the first to comment

Mix ‘n’ Match Async and Group Replication for Advanced Replication Setups

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. Mix ‘n’ Match Async and Group Replication for Advanced Replication Setups Pedro Gomes (pedro.gomes@oracle.com) Software Engineer 4th of February Oracle / Fosdem 2017 1
  2. 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 24th of February Oracle / Fosdem 2017
  3. 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda 4th of February Oracle / Fosdem 2017
  4. 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Background Mix it! Migrations Conclusion Program Agenda 4th of February Oracle / Fosdem 2017 4 1 2 3 4
  5. 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Background 4th of February Oracle / Fosdem 2017 1
  6. 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | In the beggining there was Async Replication 64th of February Oracle / Fosdem 2017 S1 S2 S3 Master MySQL Connector Application MySQL Connector Application Writes Reads Async replication connection In the beginning there was asynchronous replication. You write in the master where events are logged and sent to the slaves where they are queued and applied. This can be used for data safety on master crashes or read scalability
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Semi sync Replication 74th of February Oracle / Fosdem 2017 S1 S2 S3 Master MySQL Connector Application MySQL Connector Application ACK ACK On 5.5 we introduced semi sync. The idea is that when the master commits it waits for the slaves to queue the transaction or to prepare the commit. You can wait on all or on a set number of slaves.
  8. 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Multi Source Replication 84th of February Oracle / Fosdem 2017 S1 S2 S3 Master MySQL Connector Application MySQL Connector Application Master MySQL Connector Application On early 5.7 we introduced Multi source. It introduced the notion of channel, i.e., a replication sources. Slaves can now have different channels that will receive data from different masters. Conflicts are not detected. This can be used for data aggregation or full backups of sharded data
  9. 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Group Replication 94th of February Oracle / Fosdem 2017 Master MySQL Connector Application MySQL Connector Application Master MySQL Connector Application Master Group We now present Group replication. When you write in a member, the information flows to the other members in a ordered way, meaning all members get the same transactions in the same order. More on the next slide
  10. 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication • What is MySQL Group Replication? “Multi-master update everywhere replication plugin for MySQL with built-in automatic distributed recovery, conflict detection and group membership.” • What does the MySQL Group Replication plugin do for the user? – Removes the need for handling server fail-over. – Provides fault tolerance. – Enables update everywhere setups. – Automates group reconfiguration (handling of crashes, failures, re-connects). – Provides a highly available replicated database. 104th of February Oracle / Fosdem 2017
  11. 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Mix it! 4th of February Oracle / Fosdem 2017 2 2.1 Basic scenarios
  12. 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Read out Scalability 124th of February Oracle / Fosdem 2017 Master MySQL Connector Application MySQL Connector Application Master MySQL Connector Application Master S1 S2 S3 If you are afraid of the 9 members limitation and you want to scale your reads, you can add more slaves that replicate from the group. Remember that the group has automatic failover, but not your slaves.
  13. 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Aggregate some data 134th of February Oracle / Fosdem 2017 Master MySQL Connector Applicaton Master Master S Master Master Master Multi source can also be used to aggregate data from 2 groups. Again you can do some data analyses or have a full backup of your data.
  14. 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Mix it! 4th of February Oracle / Fosdem 2017 2 2.2 Advanced scenarios
  15. 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Inter data center replication 154th of February Oracle / Fosdem 2017 Master Master Master Master Master Master If you have different groups on different databases you can still replication from one to another using asynchronous replication. Here we use 2 pairs of servers to share the load of this task among the members, but you can use only one circular pair or have even more connections if desired.
  16. 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrations 4th of February Oracle / Fosdem 2017 3 How to migrate from asynchronous replication3.1
  17. 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 174th of February Oracle / Fosdem 2017 Master Master Master Master S1 S2 ?
  18. 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 184th of February Oracle / Fosdem 2017 M1 M2 S1 S2 MySQL Connector Application Here I chose to start the group in a new provisioned member but you can also start by doing this in on of the existing slaves . M2 will replicate from M1 using async replication.
  19. 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 194th of February Oracle / Fosdem 2017 M1 M2 S1 S2 MySQL Connector Application On by one you can start moving the slaves into the group. As all info applied into M1 is disseminated to the group, you can starting migrating reads to it and the migrated slaves.
  20. 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 204th of February Oracle / Fosdem 2017 M1 M2 S2 MySQL Connector Application S1 Eventually all slaves will be migrated to the group. Now comes the trick part.
  21. 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 214th of February Oracle / Fosdem 2017 M1 M2 S2 MySQL Connector Application S1 If your workload allows it, start migrating some writes to the group gradually. If not, you need to wait for the delay from M1 to M2 to be nearly 0, block writes on M1 and then switch to M2 when that last N transactions from M1 are replicated.
  22. 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to migrate from asynchronous scenarios 224th of February Oracle / Fosdem 2017 M1 M2 S2 MySQL Connector Application S1 Now you just need to add M1 to the group. Distributed recovery will make the member catch up to the writes it missed in the transition to the group.
  23. 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrations 4th of February Oracle / Fosdem 2017 3 Different servers3.2
  24. 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 244th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application This one is a less common case, but still interesting as a migration example. What if you want to merge shards or two applications into a group to make it HA?
  25. 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 254th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application Start replicating from one of the servers into the other. One thing to consider here: the member that becomes a slave has to deal with this increased load, but on the other hand, we must eventually block writes to the M1 for a brief moment.
  26. 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 264th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application Start a group with M2
  27. 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 274th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application M3 We add a new member here for 2 reasons: 1. The resulting group with only two members, M1 and M2, could suffer from split brain problems if one of them died. 2. There is a moment where M1 will stop serving requests and at that moment M2 would take all the load alone.
  28. 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 284th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application M3 If your workload allows it, start migrating some M1 writes to the group gradually. If not, you need to wait for the delay from M1 to M2 to be nearly 0, block writes on M1 and then switch to M2 when that last N transactions are replicated.
  29. 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication 294th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application M3 You can now join M1 to the group, being probably best to do some provisioning here
  30. 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication (alternative) 304th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application S Another alternative is to add a multi source slave that will replicate from both members.
  31. 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication (alternative) 314th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application S Start a new group on this slave.
  32. 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication (alternative) 324th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application S If your workload allows it, start migrating some writes to S gradually. If not, you need to wait for the delay from M2 to S to be nearly 0, block writes on M2 and then switch to S. Add M2 to the group.
  33. 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrate disjoint servers into Group replication (alternative) 334th of February Oracle / Fosdem 2017 M1 MySQL Connector Application M2 MySQL Connector Application S When M2 can take writes from its original application repeat the previous step with M1.
  34. 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Migrations 4th of February Oracle / Fosdem 2017 3 Going Back3.3
  35. 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | From Group Replication to Asynchronous topologies 354th of February Oracle / Fosdem 2017 M2 M1 MySQL Connector Application M3
  36. 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | From Group Replication to Asynchronous topologies 364th of February Oracle / Fosdem 2017 M2 M1 MySQL Connector Application S3 Just stop group replication in one member and configure it as a slave of the one the original members. If using multi master direct all writes to this member that is now a master.
  37. 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | From Group Replication to Asynchronous topologies 374th of February Oracle / Fosdem 2017 S2 M1 MySQL Connector Application S3 Repeat the same process for the other members until only the master remains in the group. At that point stop group replication on the master as well.
  38. 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Conclusion 4th of February Oracle / Fosdem 2017 4 This presentation only contains examples of possible setups. Other possibilities may exist, just try it
  39. 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Where to go from here? • GA Packages – http://www.mysql.com/downloads/ • Documentation – http://dev.mysql.com/doc/refman/5.7/en/group-replication.html • Blogs from the Engineers (news, technical information, and much more) – http://mysqlhighavailability.com 394th of February Oracle / Fosdem 2017
  40. 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.4th of February Oracle / Fosdem 2017 41 @distributedpete Twitter: Thanks!

×