1 / 148
2 / 148
ROOUG, December 2017
Introduction to MySQL InnoDB Cluster
MySQL High Availability made easy
 
 
 
Frédéric Descamps - MySQL Community Manager - Oracle
3 / 148
 
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purpose only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied up in
making purchasing decisions. The development, release and timing of any features or
functionality described for Oracle´s product remains at the sole discretion of Oracle.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
4 / 148
http://about.me/lefred
Who am I ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
5 / 148
Frédéric Descamps
@lefred
MySQL Evangelist
Hacking MySQL since 3.23
devops believer
MySQL Community Manager since May
2016
living in Belgium 🇧🇪
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
6 / 148
7 / 148
100%
 
virtually all organizations require their
most critical systems to be highly
available!
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
8 / 148
 
  "High Availability becomes a core first
class feature of MySQL!"
MySQL InnoDB Cluster
 
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
9 / 148
MySQL InnoDB Cluster: vision
  "A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
10 / 148
MySQL InnoDB Cluster: vision
  "A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
InnoDB
cluster
Ease-of-Use
Extreme Scale-Out
Out-of-Box Solution
Built-in HA
High Performance
Everything Integrated
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
11 / 148
One Product: MySQL
All components created together
Tested together
Packaged together
Easy to Use
One client: MySQL Shell
Easy packaging
Integrated orchestration
Homogenous servers
Flexible and Modern
SQL and NoSQL together
Protocol Buffers
Asynchronous API
Developer friendly
Support Read/Write Scale Out
Shared clusters
Federated system of N replica sets
supporting cross shard
operations
Each replica set manages a shard
MySQL InnoDB Cluster: goals
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
12 / 148
Our vision in 4 steps
MySQL Document Store
Relational & Document Models
MySQL HA
Out-Of-Box HA
Read Scale-Out
Async Replication + Auto Failover
Write Scale-Out
Sharding
S1 S3
S2 S4
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
13 / 148
Step 2´s Architecture
Application
MySQL Connector
MySQL Router
MySQL Shell
InnoDB
cluster
Application
MySQL Connector
MySQL Router
Mp
M
M
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
14 / 148
Step 3´s Architecture
Application
MySQL Connector
MySQL Router
MySQL Shell
InnoDB
cluster
Application
MySQL Connector
MySQL Router
Mp
M
M
S1 S2 S3 S4 S...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
15 / 148
Step 4´s Architecture
Application
MySQL Connector
MySQL Router
MySQL Shell
Application
MySQL Connector
MySQL Router
InnoDB
cluster
Mp
M
M
S2 S3 S4 S...S1
InnoDB
cluster
Mp
M
M
S1 S2 S3 S4 S...
InnoDB
cluster
Mp
M
M
S1 S2 S3 S4 S...
Application
MySQL Connector
MySQL Router
Application
MySQL Connector
MySQL Router
Application
MySQL Connector
MySQL Router
replicaset1
replicaset2
replicaset3
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
16 / 148
Group Replication: heart of MySQL InnoDB
Cluster
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
17 / 148
Group Replication: heart of MySQL InnoDB
Cluster
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
18 / 148
MySQL Group Replication
but what is it ?!?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
19 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
20 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
GR is an implementation of Replicated Database State Machine theory
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
21 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
GR is an implementation of Replicated Database State Machine theory
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
22 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
GR is an implementation of Replicated Database State Machine theory
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
GR implements conflict detection and resolution
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
23 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
GR is an implementation of Replicated Database State Machine theory
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
GR implements conflict detection and resolution
GR allows automatic distributed recovery
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
24 / 148
MySQL Group Replication
but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
GR is an implementation of Replicated Database State Machine theory
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
GR implements conflict detection and resolution
GR allows automatic distributed recovery
Supported on all MySQL platforms !!
Linux, Windows, Solaris, OSX, FreeBSD
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
25 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
26 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
27 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Paxos based protocol
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
28 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Paxos based protocol
its task: deliver messages across the distributed system:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
29 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Paxos based protocol
its task: deliver messages across the distributed system:
atomically
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
30 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Paxos based protocol
its task: deliver messages across the distributed system:
atomically
in Total Order
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
31 / 148
MySQL Group Communication System (GCS)
MySQL Xcom protocol
Replicated Database State Machine
Paxos based protocol
its task: deliver messages across the distributed system:
atomically
in Total Order
MySQL Group Replication receives the Ordered 'tickets' from this GCS subsystem.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
32 / 148
And for users ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
33 / 148
And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
34 / 148
And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
GR provides fault tolerance
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
35 / 148
And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
GR provides fault tolerance
GR enables update-everywhere setups
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
36 / 148
And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
GR provides fault tolerance
GR enables update-everywhere setups
GR handles crashes, failures, re-connetcs automatically
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
37 / 148
And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
GR provides fault tolerance
GR enables update-everywhere setups
GR handles crashes, failures, re-connetcs automatically
Allows an easy setup of a MySQL service high available !
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
38 / 148
OK, but how does it work ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
39 / 148
OK, but how does it work ?
it´s just ...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
40 / 148
OK, but how does it work ?
it´s just ...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
41 / 148
OK, but how does it work ?
it´s just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
42 / 148
OK, but how does it work ?
it´s just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
not that easy to understand... right ? As a picture is worth a 1000 words, let´s illustrate
this...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
43 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
44 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
45 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
46 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
47 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
48 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
49 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
50 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
51 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
52 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
53 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
54 / 148
MySQL Group Replication
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
55 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
56 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
57 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
58 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
59 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
60 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
61 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
62 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
63 / 148
MySQL Group Replication (full transaction)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
64 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
65 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
66 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
67 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
68 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
69 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
70 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
71 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
72 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
73 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
74 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
75 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
76 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
77 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
78 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
79 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
80 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
81 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
82 / 148
Group Replication : Total Order Delivery - GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
83 / 148
Group Replication: return commit
Asynchronous Replication:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
84 / 148
Group Replication: return from commit (2)
Semi-Sync Replication:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
85 / 148
Group Replication: return from commit (3)
Group Replication:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
86 / 148
Group Replication : Optimistic Locking
Group Replication uses optimistic locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
87 / 148
Group Replication : Optimistic Locking
Group Replication uses optimistic locking
during a transaction, local (InnoDB) locking happens
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
88 / 148
Group Replication : Optimistic Locking
Group Replication uses optimistic locking
during a transaction, local (InnoDB) locking happens
optimistically assumes there will be no conflicts across nodes
(no communication between nodes necessary)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
89 / 148
Group Replication : Optimistic Locking
Group Replication uses optimistic locking
during a transaction, local (InnoDB) locking happens
optimistically assumes there will be no conflicts across nodes
(no communication between nodes necessary)
cluster-wide conflict resolution happens only at COMMIT, during certification
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
90 / 148
Group Replication : Optimistic Locking
Group Replication uses optimistic locking
during a transaction, local (InnoDB) locking happens
optimistically assumes there will be no conflicts across nodes
(no communication between nodes necessary)
cluster-wide conflict resolution happens only at COMMIT, during certification
Let´s first have a look at the traditional locking to compare.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
91 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
92 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
93 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
94 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
95 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
96 / 148
Traditional locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
97 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
98 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
99 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
100 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
101 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
102 / 148
Optimistic Locking
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
103 / 148
Optimistic Locking
The system returns error 149 as certification failed:
ERROR 1180 (HY000): Got error 149 during COMMIT
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
104 / 148
Such conflicts happen only when using multi-
primary group !
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
105 / 148
Certification
Certification is the process that only needs to answer the following unique question:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
106 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
107 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
108 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
109 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
110 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
111 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
112 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
pass: enter in the apply queue
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
113 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
pass: enter in the apply queue
fail: drop the transaction
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
114 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
pass: enter in the apply queue
fail: drop the transaction
serialized by the total order in GCS/XCOM + GTID
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
115 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
pass: enter in the apply queue
fail: drop the transaction
serialized by the total order in GCS/XCOM + GTID
cost is based on trx size (# rows & # keys)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
116 / 148
Certification
Certification is the process that only needs to answer the following unique question:
can the write (transaction) be applied ?
based on unapplied earlier transactions
such conflicts must come for other members/nodes
happens on every member/node
should be deterministic on every node
results are not reported to the group
pass: enter in the apply queue
fail: drop the transaction
serialized by the total order in GCS/XCOM + GTID
cost is based on trx size (# rows & # keys)
first committer (and ack from majority) wins rule
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
117 / 148
Drawbacks of optimistic locking
having a first-committer-wins system means conflicts will more likely happen when
writing on multiple members with:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
118 / 148
Drawbacks of optimistic locking
having a first-committer-wins system means conflicts will more likely happen when
writing on multiple members with:
large transactions
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
119 / 148
Drawbacks of optimistic locking
having a first-committer-wins system means conflicts will more likely happen when
writing on multiple members with:
large transactions
long running transactions
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
120 / 148
Drawbacks of optimistic locking
having a first-committer-wins system means conflicts will more likely happen when
writing on multiple members with:
large transactions
long running transactions
hotspot records
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
121 / 148
Default = Single Primary Mode
By default, MySQL InnoDB Cluster runs in Single Primary Mode.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
122 / 148
Default = Single Primary Mode
By default, MySQL InnoDB Cluster runs in Single Primary Mode.
mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
123 / 148
Default = Single Primary Mode
By default, MySQL InnoDB Cluster runs in Single Primary Mode.
mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
In Single Primary Mode, a single member acts as the writable master (PRIMARY) and
the rest of the members act as hot-standbys (SECONDARY).
The group itself coordinates and configures itself automatically to determine which
member will act as the PRIMARY, through a leader election mechanism.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
124 / 148
Single Primary Mode - new in 8.0.2
group_replication_member_weight: influence the primary member election in Single-
Primary mode (from 0 to 100).
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
125 / 148
Single Primary Mode - new in 8.0.2
group_replication_member_weight: influence the primary member election in Single-
Primary mode (from 0 to 100).
more visibility
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
126 / 148
Single Primary Mode - new in 8.0.2
group_replication_member_weight: influence the primary member election in Single-
Primary mode (from 0 to 100).
more visibility
mysql-sql> select * from performance_schema.replication_group_members;
+--------------+--------------+--------+------+---------+-------------+-------+
| CHANNEL_NAME | MEMBER_ID | M_HOST | PORT | M_STATE | MEMBER_ROLE | M_VER |
+--------------+--------------+--------+------+---------+-------------+-------+
| gr_applier | ...-8d90-... | mysql1 | 3306 | ONLINE | SECONDARY | 8.0.2 |
| gr_applier | ...-be23-... | mysql2 | 3306 | ONLINE | SECONDARY | 8.0.2 |
| gr_applier | ...-a747-... | mysql3 | 3306 | ONLINE | PRIMARY | 8.0.2 |
+--------------+--------------+--------+------+---------+-------------+-------+
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
127 / 148
New in MySQL 8.0.3
Group Replication flow control variables now correctly permit you to have some
members in a group that do not affect the minimum throughput of the flow-control
mechanism
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
128 / 148
New in MySQL 8.0.3
Group Replication flow control variables now correctly permit you to have some
members in a group that do not affect the minimum throughput of the flow-control
mechanism
GCS instrumentation in Performance_Schema
thread/group_rpl/THD_Gcs_xcom_control
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
129 / 148
New in MySQL 8.0.3
Group Replication flow control variables now correctly permit you to have some
members in a group that do not affect the minimum throughput of the flow-control
mechanism
GCS instrumentation in Performance_Schema
thread/group_rpl/THD_Gcs_xcom_control
Auto-setting Read Only when leaving the Group
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
130 / 148
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
131 / 148
MySQL Router (GA!)
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
132 / 148
MySQL Router (GA!)
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
MySQL Router doesn´t require any specific configuration. It configures itself
automatically (bootstrap) using MySQL InnoDB Cluster´s metadata.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
133 / 148
MySQL Router (GA!)
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
MySQL Router doesn´t require any specific configuration. It configures itself
automatically (bootstrap) using MySQL InnoDB Cluster´s metadata.
The MySQL Router development will be focusing on sharding.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
134 / 148
ProxySQL has native support for Group
Replication which makes it a good choice
for advanced users.
ProxySQL
If you need some specific features that are not yet available in MySQL Router, like
transparent R/W splitting, then you can use your software of choice.
We are also collaborating with ProxySQL.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
135 / 148
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
136 / 148
MySQL Shell (GA!)
The MySQL Shell is an interactive Javascript, Python, or SQL interface supporting
development and administration for the MySQL Server and is a component of the MySQL
Server. You can use the MySQL Shell to perform data queries and updates as well as
various administration operations.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
137 / 148
MySQL Shell (2)
The MySQL Shell provides:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
138 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
139 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Document and Relational Models
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
140 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Document and Relational Models
CRUD Document and Relational APIs via scripting
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
141 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Document and Relational Models
CRUD Document and Relational APIs via scripting
Traditional Table, JSON, Tab Separated output results formats
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
142 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Document and Relational Models
CRUD Document and Relational APIs via scripting
Traditional Table, JSON, Tab Separated output results formats
MySQL Standard and X Protocols
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
143 / 148
MySQL Shell (2)
The MySQL Shell provides:
Both Interactive and Batch operations
Document and Relational Models
CRUD Document and Relational APIs via scripting
Traditional Table, JSON, Tab Separated output results formats
MySQL Standard and X Protocols
and more...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
144 / 148
don´t forget, this is a HA solution !
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
145 / 148
Demo ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
146 / 148
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
147 / 148
Thank you !
Any Questions ?
more at http://lefred.be/content/category/mysql/group-replication/
http://mysqlhighavailability.com/
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
148 / 148

Introduction to MySQL InnoDB Cluster

  • 1.
  • 2.
  • 3.
    ROOUG, December 2017 Introductionto MySQL InnoDB Cluster MySQL High Availability made easy       Frédéric Descamps - MySQL Community Manager - Oracle 3 / 148
  • 4.
      Safe Harbor Statement Thefollowing is intended to outline our general product direction. It is intended for information purpose only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied up in making purchasing decisions. The development, release and timing of any features or functionality described for Oracle´s product remains at the sole discretion of Oracle. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 4 / 148
  • 5.
    http://about.me/lefred Who am I? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 5 / 148
  • 6.
    Frédéric Descamps @lefred MySQL Evangelist HackingMySQL since 3.23 devops believer MySQL Community Manager since May 2016 living in Belgium 🇧🇪 Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 6 / 148
  • 7.
  • 8.
    100%   virtually all organizationsrequire their most critical systems to be highly available! Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 8 / 148
  • 9.
        "High Availabilitybecomes a core first class feature of MySQL!" MySQL InnoDB Cluster   Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 9 / 148
  • 10.
    MySQL InnoDB Cluster:vision   "A single product - MySQL - with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use." Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 10 / 148
  • 11.
    MySQL InnoDB Cluster:vision   "A single product - MySQL - with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use." InnoDB cluster Ease-of-Use Extreme Scale-Out Out-of-Box Solution Built-in HA High Performance Everything Integrated Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 11 / 148
  • 12.
    One Product: MySQL Allcomponents created together Tested together Packaged together Easy to Use One client: MySQL Shell Easy packaging Integrated orchestration Homogenous servers Flexible and Modern SQL and NoSQL together Protocol Buffers Asynchronous API Developer friendly Support Read/Write Scale Out Shared clusters Federated system of N replica sets supporting cross shard operations Each replica set manages a shard MySQL InnoDB Cluster: goals Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 12 / 148
  • 13.
    Our vision in4 steps MySQL Document Store Relational & Document Models MySQL HA Out-Of-Box HA Read Scale-Out Async Replication + Auto Failover Write Scale-Out Sharding S1 S3 S2 S4 Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 13 / 148
  • 14.
    Step 2´s Architecture Application MySQLConnector MySQL Router MySQL Shell InnoDB cluster Application MySQL Connector MySQL Router Mp M M Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 14 / 148
  • 15.
    Step 3´s Architecture Application MySQLConnector MySQL Router MySQL Shell InnoDB cluster Application MySQL Connector MySQL Router Mp M M S1 S2 S3 S4 S... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 15 / 148
  • 16.
    Step 4´s Architecture Application MySQLConnector MySQL Router MySQL Shell Application MySQL Connector MySQL Router InnoDB cluster Mp M M S2 S3 S4 S...S1 InnoDB cluster Mp M M S1 S2 S3 S4 S... InnoDB cluster Mp M M S1 S2 S3 S4 S... Application MySQL Connector MySQL Router Application MySQL Connector MySQL Router Application MySQL Connector MySQL Router replicaset1 replicaset2 replicaset3 Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 16 / 148
  • 17.
    Group Replication: heartof MySQL InnoDB Cluster Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 17 / 148
  • 18.
    Group Replication: heartof MySQL InnoDB Cluster Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 18 / 148
  • 19.
    MySQL Group Replication butwhat is it ?!? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 19 / 148
  • 20.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 20 / 148
  • 21.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL GR is an implementation of Replicated Database State Machine theory Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 21 / 148
  • 22.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL GR is an implementation of Replicated Database State Machine theory GR allows to write on all Group Members (cluster nodes) simultaneously while retaining consistency Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 22 / 148
  • 23.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL GR is an implementation of Replicated Database State Machine theory GR allows to write on all Group Members (cluster nodes) simultaneously while retaining consistency GR implements conflict detection and resolution Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 23 / 148
  • 24.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL GR is an implementation of Replicated Database State Machine theory GR allows to write on all Group Members (cluster nodes) simultaneously while retaining consistency GR implements conflict detection and resolution GR allows automatic distributed recovery Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 24 / 148
  • 25.
    MySQL Group Replication butwhat is it ?!? GR is a plugin for MySQL, made by MySQL and packaged with MySQL GR is an implementation of Replicated Database State Machine theory GR allows to write on all Group Members (cluster nodes) simultaneously while retaining consistency GR implements conflict detection and resolution GR allows automatic distributed recovery Supported on all MySQL platforms !! Linux, Windows, Solaris, OSX, FreeBSD Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 25 / 148
  • 26.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 26 / 148
  • 27.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 27 / 148
  • 28.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Paxos based protocol Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 28 / 148
  • 29.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Paxos based protocol its task: deliver messages across the distributed system: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 29 / 148
  • 30.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Paxos based protocol its task: deliver messages across the distributed system: atomically Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 30 / 148
  • 31.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Paxos based protocol its task: deliver messages across the distributed system: atomically in Total Order Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 31 / 148
  • 32.
    MySQL Group CommunicationSystem (GCS) MySQL Xcom protocol Replicated Database State Machine Paxos based protocol its task: deliver messages across the distributed system: atomically in Total Order MySQL Group Replication receives the Ordered 'tickets' from this GCS subsystem. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 32 / 148
  • 33.
    And for users? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 33 / 148
  • 34.
    And for users? no longer necessary to handle server fail-over manually or with a complicated script Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 34 / 148
  • 35.
    And for users? no longer necessary to handle server fail-over manually or with a complicated script GR provides fault tolerance Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 35 / 148
  • 36.
    And for users? no longer necessary to handle server fail-over manually or with a complicated script GR provides fault tolerance GR enables update-everywhere setups Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 36 / 148
  • 37.
    And for users? no longer necessary to handle server fail-over manually or with a complicated script GR provides fault tolerance GR enables update-everywhere setups GR handles crashes, failures, re-connetcs automatically Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 37 / 148
  • 38.
    And for users? no longer necessary to handle server fail-over manually or with a complicated script GR provides fault tolerance GR enables update-everywhere setups GR handles crashes, failures, re-connetcs automatically Allows an easy setup of a MySQL service high available ! Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 38 / 148
  • 39.
    OK, but howdoes it work ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 39 / 148
  • 40.
    OK, but howdoes it work ? it´s just ... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 40 / 148
  • 41.
    OK, but howdoes it work ? it´s just ... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 41 / 148
  • 42.
    OK, but howdoes it work ? it´s just ... ... no, in fact the writesets replication is synchronous and then certification and apply of the changes are local to each nodes and asynchronous. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 42 / 148
  • 43.
    OK, but howdoes it work ? it´s just ... ... no, in fact the writesets replication is synchronous and then certification and apply of the changes are local to each nodes and asynchronous. not that easy to understand... right ? As a picture is worth a 1000 words, let´s illustrate this... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 43 / 148
  • 44.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 44 / 148
  • 45.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 45 / 148
  • 46.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 46 / 148
  • 47.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 47 / 148
  • 48.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 48 / 148
  • 49.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 49 / 148
  • 50.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 50 / 148
  • 51.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 51 / 148
  • 52.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 52 / 148
  • 53.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 53 / 148
  • 54.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 54 / 148
  • 55.
    MySQL Group Replication Copyright@ 2017 Oracle and/or its affiliates. All rights reserved. 55 / 148
  • 56.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 56 / 148
  • 57.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 57 / 148
  • 58.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 58 / 148
  • 59.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 59 / 148
  • 60.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 60 / 148
  • 61.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 61 / 148
  • 62.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 62 / 148
  • 63.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 63 / 148
  • 64.
    MySQL Group Replication(full transaction) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 64 / 148
  • 65.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 65 / 148
  • 66.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 66 / 148
  • 67.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 67 / 148
  • 68.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 68 / 148
  • 69.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 69 / 148
  • 70.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 70 / 148
  • 71.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 71 / 148
  • 72.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 72 / 148
  • 73.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 73 / 148
  • 74.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 74 / 148
  • 75.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 75 / 148
  • 76.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 76 / 148
  • 77.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 77 / 148
  • 78.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 78 / 148
  • 79.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 79 / 148
  • 80.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 80 / 148
  • 81.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 81 / 148
  • 82.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 82 / 148
  • 83.
    Group Replication :Total Order Delivery - GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 83 / 148
  • 84.
    Group Replication: returncommit Asynchronous Replication: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 84 / 148
  • 85.
    Group Replication: returnfrom commit (2) Semi-Sync Replication: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 85 / 148
  • 86.
    Group Replication: returnfrom commit (3) Group Replication: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 86 / 148
  • 87.
    Group Replication :Optimistic Locking Group Replication uses optimistic locking Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 87 / 148
  • 88.
    Group Replication :Optimistic Locking Group Replication uses optimistic locking during a transaction, local (InnoDB) locking happens Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 88 / 148
  • 89.
    Group Replication :Optimistic Locking Group Replication uses optimistic locking during a transaction, local (InnoDB) locking happens optimistically assumes there will be no conflicts across nodes (no communication between nodes necessary) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 89 / 148
  • 90.
    Group Replication :Optimistic Locking Group Replication uses optimistic locking during a transaction, local (InnoDB) locking happens optimistically assumes there will be no conflicts across nodes (no communication between nodes necessary) cluster-wide conflict resolution happens only at COMMIT, during certification Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 90 / 148
  • 91.
    Group Replication :Optimistic Locking Group Replication uses optimistic locking during a transaction, local (InnoDB) locking happens optimistically assumes there will be no conflicts across nodes (no communication between nodes necessary) cluster-wide conflict resolution happens only at COMMIT, during certification Let´s first have a look at the traditional locking to compare. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 91 / 148
  • 92.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 92 / 148
  • 93.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 93 / 148
  • 94.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 94 / 148
  • 95.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 95 / 148
  • 96.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 96 / 148
  • 97.
    Traditional locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 97 / 148
  • 98.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 98 / 148
  • 99.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 99 / 148
  • 100.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 100 / 148
  • 101.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 101 / 148
  • 102.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 102 / 148
  • 103.
    Optimistic Locking Copyright @2017 Oracle and/or its affiliates. All rights reserved. 103 / 148
  • 104.
    Optimistic Locking The systemreturns error 149 as certification failed: ERROR 1180 (HY000): Got error 149 during COMMIT Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 104 / 148
  • 105.
    Such conflicts happenonly when using multi- primary group ! Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 105 / 148
  • 106.
    Certification Certification is theprocess that only needs to answer the following unique question: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 106 / 148
  • 107.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 107 / 148
  • 108.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 108 / 148
  • 109.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 109 / 148
  • 110.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 110 / 148
  • 111.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 111 / 148
  • 112.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 112 / 148
  • 113.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group pass: enter in the apply queue Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 113 / 148
  • 114.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group pass: enter in the apply queue fail: drop the transaction Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 114 / 148
  • 115.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group pass: enter in the apply queue fail: drop the transaction serialized by the total order in GCS/XCOM + GTID Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 115 / 148
  • 116.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group pass: enter in the apply queue fail: drop the transaction serialized by the total order in GCS/XCOM + GTID cost is based on trx size (# rows & # keys) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 116 / 148
  • 117.
    Certification Certification is theprocess that only needs to answer the following unique question: can the write (transaction) be applied ? based on unapplied earlier transactions such conflicts must come for other members/nodes happens on every member/node should be deterministic on every node results are not reported to the group pass: enter in the apply queue fail: drop the transaction serialized by the total order in GCS/XCOM + GTID cost is based on trx size (# rows & # keys) first committer (and ack from majority) wins rule Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 117 / 148
  • 118.
    Drawbacks of optimisticlocking having a first-committer-wins system means conflicts will more likely happen when writing on multiple members with: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 118 / 148
  • 119.
    Drawbacks of optimisticlocking having a first-committer-wins system means conflicts will more likely happen when writing on multiple members with: large transactions Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 119 / 148
  • 120.
    Drawbacks of optimisticlocking having a first-committer-wins system means conflicts will more likely happen when writing on multiple members with: large transactions long running transactions Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 120 / 148
  • 121.
    Drawbacks of optimisticlocking having a first-committer-wins system means conflicts will more likely happen when writing on multiple members with: large transactions long running transactions hotspot records Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 121 / 148
  • 122.
    Default = SinglePrimary Mode By default, MySQL InnoDB Cluster runs in Single Primary Mode. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 122 / 148
  • 123.
    Default = SinglePrimary Mode By default, MySQL InnoDB Cluster runs in Single Primary Mode. mysql> show global variables like 'group_replication_single_primary_mode'; +---------------------------------------+-------+ | Variable_name | Value | +---------------------------------------+-------+ | group_replication_single_primary_mode | ON | +---------------------------------------+-------+ Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 123 / 148
  • 124.
    Default = SinglePrimary Mode By default, MySQL InnoDB Cluster runs in Single Primary Mode. mysql> show global variables like 'group_replication_single_primary_mode'; +---------------------------------------+-------+ | Variable_name | Value | +---------------------------------------+-------+ | group_replication_single_primary_mode | ON | +---------------------------------------+-------+ In Single Primary Mode, a single member acts as the writable master (PRIMARY) and the rest of the members act as hot-standbys (SECONDARY). The group itself coordinates and configures itself automatically to determine which member will act as the PRIMARY, through a leader election mechanism. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 124 / 148
  • 125.
    Single Primary Mode- new in 8.0.2 group_replication_member_weight: influence the primary member election in Single- Primary mode (from 0 to 100). Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 125 / 148
  • 126.
    Single Primary Mode- new in 8.0.2 group_replication_member_weight: influence the primary member election in Single- Primary mode (from 0 to 100). more visibility Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 126 / 148
  • 127.
    Single Primary Mode- new in 8.0.2 group_replication_member_weight: influence the primary member election in Single- Primary mode (from 0 to 100). more visibility mysql-sql> select * from performance_schema.replication_group_members; +--------------+--------------+--------+------+---------+-------------+-------+ | CHANNEL_NAME | MEMBER_ID | M_HOST | PORT | M_STATE | MEMBER_ROLE | M_VER | +--------------+--------------+--------+------+---------+-------------+-------+ | gr_applier | ...-8d90-... | mysql1 | 3306 | ONLINE | SECONDARY | 8.0.2 | | gr_applier | ...-be23-... | mysql2 | 3306 | ONLINE | SECONDARY | 8.0.2 | | gr_applier | ...-a747-... | mysql3 | 3306 | ONLINE | PRIMARY | 8.0.2 | +--------------+--------------+--------+------+---------+-------------+-------+ Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 127 / 148
  • 128.
    New in MySQL8.0.3 Group Replication flow control variables now correctly permit you to have some members in a group that do not affect the minimum throughput of the flow-control mechanism Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 128 / 148
  • 129.
    New in MySQL8.0.3 Group Replication flow control variables now correctly permit you to have some members in a group that do not affect the minimum throughput of the flow-control mechanism GCS instrumentation in Performance_Schema thread/group_rpl/THD_Gcs_xcom_control Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 129 / 148
  • 130.
    New in MySQL8.0.3 Group Replication flow control variables now correctly permit you to have some members in a group that do not affect the minimum throughput of the flow-control mechanism GCS instrumentation in Performance_Schema thread/group_rpl/THD_Gcs_xcom_control Auto-setting Read Only when leaving the Group Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 130 / 148
  • 131.
    Copyright @ 2017Oracle and/or its affiliates. All rights reserved. 131 / 148
  • 132.
    MySQL Router (GA!) MySQLRouter is lightweight middleware that provides transparent routing between your application and backend MySQL Servers. It can be used for a wide variety of use cases, such as providing high availability and scalability by effectively routing database traffic to appropriate backend MySQL Servers. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 132 / 148
  • 133.
    MySQL Router (GA!) MySQLRouter is lightweight middleware that provides transparent routing between your application and backend MySQL Servers. It can be used for a wide variety of use cases, such as providing high availability and scalability by effectively routing database traffic to appropriate backend MySQL Servers. MySQL Router doesn´t require any specific configuration. It configures itself automatically (bootstrap) using MySQL InnoDB Cluster´s metadata. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 133 / 148
  • 134.
    MySQL Router (GA!) MySQLRouter is lightweight middleware that provides transparent routing between your application and backend MySQL Servers. It can be used for a wide variety of use cases, such as providing high availability and scalability by effectively routing database traffic to appropriate backend MySQL Servers. MySQL Router doesn´t require any specific configuration. It configures itself automatically (bootstrap) using MySQL InnoDB Cluster´s metadata. The MySQL Router development will be focusing on sharding. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 134 / 148
  • 135.
    ProxySQL has nativesupport for Group Replication which makes it a good choice for advanced users. ProxySQL If you need some specific features that are not yet available in MySQL Router, like transparent R/W splitting, then you can use your software of choice. We are also collaborating with ProxySQL. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 135 / 148
  • 136.
    Copyright @ 2017Oracle and/or its affiliates. All rights reserved. 136 / 148
  • 137.
    MySQL Shell (GA!) TheMySQL Shell is an interactive Javascript, Python, or SQL interface supporting development and administration for the MySQL Server and is a component of the MySQL Server. You can use the MySQL Shell to perform data queries and updates as well as various administration operations. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 137 / 148
  • 138.
    MySQL Shell (2) TheMySQL Shell provides: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 138 / 148
  • 139.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 139 / 148
  • 140.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Document and Relational Models Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 140 / 148
  • 141.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Document and Relational Models CRUD Document and Relational APIs via scripting Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 141 / 148
  • 142.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Document and Relational Models CRUD Document and Relational APIs via scripting Traditional Table, JSON, Tab Separated output results formats Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 142 / 148
  • 143.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Document and Relational Models CRUD Document and Relational APIs via scripting Traditional Table, JSON, Tab Separated output results formats MySQL Standard and X Protocols Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 143 / 148
  • 144.
    MySQL Shell (2) TheMySQL Shell provides: Both Interactive and Batch operations Document and Relational Models CRUD Document and Relational APIs via scripting Traditional Table, JSON, Tab Separated output results formats MySQL Standard and X Protocols and more... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 144 / 148
  • 145.
    don´t forget, thisis a HA solution ! Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 145 / 148
  • 146.
    Demo ? Copyright @2017 Oracle and/or its affiliates. All rights reserved. 146 / 148
  • 147.
    Copyright @ 2017Oracle and/or its affiliates. All rights reserved. 147 / 148
  • 148.
    Thank you ! AnyQuestions ? more at http://lefred.be/content/category/mysql/group-replication/ http://mysqlhighavailability.com/ Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 148 / 148