Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MySQL Group Replication

492 views

Published on

Talk on MySQL Group Replication in FossAsia 2016 in Singapore

Published in: Software
  • Be the first to comment

MySQL Group Replication

  1. 1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |Sunday,March 20, 2016 FOSSASIA'2016 -Singapore Enhanced High Availability using MySQL Group Replication Manish Kumar (manish.4.kumar@oracle.com) Senior Member Technical Staf 1
  2. 2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  3. 3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Program Agenda Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  4. 4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Program Agenda The Theory How things work! How to Use Conclusion 1 2 3 4 4Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  5. 5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | The theory1 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  6. 6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | M S S S S M write clients read clients read clients write clients More reads? More slaves! Read scale-out Background: What is Replication Used For? 6Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  7. 7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | C B A C B ACrashCrash C B A B is the new master Uh Oh! Whew! Redundancy: If master crashes, promote slave to master Background: What is Replication Used For? 7Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  8. 8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication 8 M M M M M Replication Group Clients Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  9. 9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication • What is MySQL Group Replication? “Multi-master update everywhere replication plugin for MySQL with built-in automatic distributed recovery, conflict detection and group membership.” • What does the MySQL Group Replication plugin do for the user? – Removes the need for handling server fail-over. – Provides fault tolerance. – Enables update everywhere setups. – Automates group reconfiguration (handling of crashes, failures, re-connects) Provides a highly available replicated database. 9Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  10. 10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Use Cases • Elastic Replication – Environments that require a very fluid replication infrastructure, where the number of servers has to grow or shrink dynamically and with as little pain as possible. • Highly Available Shards – Sharding is a popular approach to achieve write scale-out. Users can use MySQL Group Replication to implement highly available shards. Each shard can map into a Replication Group. • Alternative to Master-Slave replication – It may be that a single master server makes it a single point of contention. Writing to an entire group may prove more scalable under certain circumstances. 10Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  11. 11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | The theory behind it... • Implementation based in Replicated Database State Machines – Group Communication Primitives resenble properties of Databases – Distributed systems meet Databases: Pedone, Guerraoui and Schiper paper • Deferred update replication: before committing locally we certify in all nodes – In order to implement it one needs Atomic Broadcast – Necessary to avoid inconsistent Certification outcome. Endured by Atomic Broadcast:minimum requirement for DBSM. • Membership Service – Group Members: It allows one to know in a moment in time all the members that are participating in the protocol, associated with a logical identifier (view id) – View Synchrony: Ensure that messages from past views are all delivered before a new view is installed. 11Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  12. 12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | How things work!2 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  13. 13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Lets add a node to an existing group: Setup • Create a user for distributed recovery • Server needs to be started with the correct configuration: 13 ./bin/mysqld --no-defaults --basedir=. --datadir=<DATADIR_LOCATION> –P <PORT> --socket=mysqld<ID>.sock --log-bin=master-bin --server-id=<ID>         --gtid-mode=on --enforce-gtid-consistency --log-slave-updates    --binlog-checksum=NONE --binlog-format=row                       --master-info-repository=TABLE --relay-log-info-repository=TABLE --transaction-write-set-extraction=MURMUR32                      --plugin-dir=lib/plugin --plugin-load=group_replication.so ./bin/mysqld --no-defaults --basedir=. --datadir=<DATADIR_LOCATION> –P <PORT> --socket=mysqld<ID>.sock --log-bin=master-bin --server-id=<ID>         --gtid-mode=on --enforce-gtid-consistency --log-slave-updates    --binlog-checksum=NONE --binlog-format=row                       --master-info-repository=TABLE --relay-log-info-repository=TABLE --transaction-write-set-extraction=MURMUR32                      --plugin-dir=lib/plugin --plugin-load=group_replication.so ./bin/mysql -uroot -h 127.0.0.1 -P 13001 -p --prompt='server1>'   server1> CREATE USER 'rpl_user'@'%' IDENTIFIED BY 'rpl_pass'; GRANT REPLICATION SLAVE ON *.* TO rpl_user@'%'; ./bin/mysql -uroot -h 127.0.0.1 -P 13001 -p --prompt='server1>'   server1> CREATE USER 'rpl_user'@'%' IDENTIFIED BY 'rpl_pass'; GRANT REPLICATION SLAVE ON *.* TO rpl_user@'%'; Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  14. 14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Lets add a node to a group : Setup (2) • Now lets configure the replication user for recovery... • ...and the communication backbone 14 SET GLOBAL group_replication_recovery_user='rpl_user'; SET GLOBAL group_replication_recovery_password='rpl_pass'; SET GLOBAL group_replication_recovery_user='rpl_user'; SET GLOBAL group_replication_recovery_password='rpl_pass'; SET GLOBAL group_replication_group_name= <valid UUID>; SET GLOBAL group_replication_local_address=<this node address:port for the communication backbone>; SET GLOBAL group_replication_peer_addresses= <comma-separated list of all other nodes in the group>; SET GLOBAL group_replication_bootstrap_group= 0; SET GLOBAL group_replication_group_name= <valid UUID>; SET GLOBAL group_replication_local_address=<this node address:port for the communication backbone>; SET GLOBAL group_replication_peer_addresses= <comma-separated list of all other nodes in the group>; SET GLOBAL group_replication_bootstrap_group= 0; Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  15. 15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Start me up! • Server that joins the group will automatically synchronize with the others. • It will retrieve the difference from its data to the data of the group members • Hint: provision the new node with data before joining an existing group 15 M M M M M N I want to play with you START GROUP REPLICATION; ONLINE Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  16. 16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Start me up! (2) 16 M M M M M N ONLINE RECOVERING SELECT * FROM performance_schema.replication_group_membersG M M M M M N ONLINE Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  17. 17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Well, I need to go now... • If a server leaves the group, the others will automatically be informed. 17 M M M M M M My machine needs maintenance or a system crash happens Each membership configuration is identified by a view_id view_id: 4 STOP GROUP REPLICATION; Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  18. 18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | The world keeps spinning until... • If a server leaves the group, the others will automatically be informed. • Server that (re)joins the group will automatically synchronize with the others. 18 M M M M M view_id: 5 M M M M M M RECOVERING -> ONLINE view_id: 6 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  19. 19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | How to use3 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  20. 20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Multi-Master update everywhere! • Any two transactions on different servers can write to the same tuple. • Conflicts will be detected and dealt with. – First committer wins rule. 20 M M M M M UPDATE t1 SET a=4 WHERE a=2UPDATE t1 SET a=3 WHERE a=1 OKOK M M M M M UPDATE t1 SET a=2 WHERE a=1UPDATE t1 SET a=3 WHERE a=1 OK Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  21. 21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Full GTID support! • All group members share the same UUID, the group name. 21 M M M M M INSERT y; Will have GTID: group_name:2 INSERT x; Will have GTID: group_name:1 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  22. 22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Management • Monitor group replication stats though Performance Schema tables. 22 mysql> SELECT * FROM performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier GROUP_NAME: 8a94f357-aab4-11df-86ab-c80aa9429563 SOURCE_UUID: 8a94f357-aab4-11df-86ab-c80aa9429563 THREAD_ID: NULL SERVICE_STATE: ON ... mysql> SELECT * FROM performance_schema.replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier GROUP_NAME: 8a94f357-aab4-11df-86ab-c80aa9429563 SOURCE_UUID: 8a94f357-aab4-11df-86ab-c80aa9429563 THREAD_ID: NULL SERVICE_STATE: ON ... mysql> SELECT * FROM performance_schema.replication_group_member_statsG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier VIEW_ID: 1428497631:3 MEMBER_ID: e38fdea8-dded-11e4-b211-e8b1fc3848de COUNT_TRANSACTIONS_IN_QUEUE: 0 COUNT_TRANSACTIONS_CHECKED: 12 COUNT_CONFLICTS_DETECTED: 5 COUNT_TRANSACTIONS_VALIDATING: 6 TRANSACTIONS_COMMITTED_ALL_MEMBERS: 8a84f397-aaa4-18df-89ab- c70aa9823561:1-7 LAST_CONFLICT_FREE_TRANSACTION: 8a84f397-aaa4-18df-89ab-c70aa9823561:7 mysql> SELECT * FROM performance_schema.replication_group_member_statsG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier VIEW_ID: 1428497631:3 MEMBER_ID: e38fdea8-dded-11e4-b211-e8b1fc3848de COUNT_TRANSACTIONS_IN_QUEUE: 0 COUNT_TRANSACTIONS_CHECKED: 12 COUNT_CONFLICTS_DETECTED: 5 COUNT_TRANSACTIONS_VALIDATING: 6 TRANSACTIONS_COMMITTED_ALL_MEMBERS: 8a84f397-aaa4-18df-89ab- c70aa9823561:1-7 LAST_CONFLICT_FREE_TRANSACTION: 8a84f397-aaa4-18df-89ab-c70aa9823561:7 mysql> SELECT * FROM performance_schema.replication_group_membersG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier MEMBER_ID: 597dbb72-3e2c-11e4-9d9d-ecf4bb227f3b MEMBER_HOST: nightfury MEMBER_PORT: 13000 MEMBER_STATE: ONLINE *************************** 2. row *************************** ... mysql> SELECT * FROM performance_schema.replication_group_membersG *************************** 1. row *************************** CHANNEL_NAME: group_replication_applier MEMBER_ID: 597dbb72-3e2c-11e4-9d9d-ecf4bb227f3b MEMBER_HOST: nightfury MEMBER_PORT: 13000 MEMBER_STATE: ONLINE *************************** 2. row *************************** ... Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  23. 23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Conclusion4 Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  24. 24. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Summary • Cloud Friendly – Great technology for deployments where elasticity is a requirement, such as cloud based infrastructures. • Integrated – With server core through a well defined API. – With GTIDs, row based replication, performance schema tables. • Autonomic and Operations Friendly – It is self-healing: no administrative overhead for handling server fail-overs. – Provides fault-tolerance, enables multi-master update everywhere and a dependable MySQL service. • Lab releases provide a sneak peek at what is coming - a new replication plugin and exciting new infrastructure: MySQL Group Replication and MySQL Router. 24Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  25. 25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | • Strong development cycles and continuous community engagement through regular lab releases. Releases 25 2014-Apr-06 Labs release: 0.3.0 2014-Aug-06 Labs release: 0.4.0 2015-Sep-14 Labs release: 0.5.0 Introduces new communication engine! 2015-Oct-22 Labs release: 0.6.0 2016-Jan-13 Labs release: 0.7.0 WINDOWS SUPPORT Sunday,March 20, 2016 FOSSASIA'2016 -Singapore
  26. 26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Where to go from here? • Packages – http://labs.mysql.com • Blogs from the Engineers (news, technical information, and much more) – http://mysqlhighavailability.com 26Sunday,March 20, 2016 FOSSASIA'2016 -Singapore

×