MySQL High Availability
Deployment Practice
马楚成 (Ivan Ma)
Copyright © 2020 Oracle and/or its affiliates.
MySQL Principal Solution Engineer
Oracle MySQL GBU, Oracle LLC
2020-06-13
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 3
MySQL Sessions – Day 2
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Agenda
4
MySQL Latest Update
MySQL High Availability(Past, Present and Future )
Deployment Practice
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
https://www.facebook.com/groups/hkmysqlusergroup/
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Version and LifeCycle
• MySQL Enterprise Edition
– MySQL 5.6
• Feb-2021 (Extended Support End)
– MySQL 5.7
• Oct-2020 (Premier Support End)
– Using MySQL 8.0
• NDB Cluster (Carrier Grade Edition)
– NDB Cluster 8.0 (New!)
• Jan-2020 Released
7
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Updates
8
GIS / Spaital
JSON / Document Store
Reliability (MySQL Replication, MySQL InnoDB Cluster)
MySQL
Shell
MySQL
Router
MySQL
Enterprise
Backup
MySQL
Enterprise
Monitor
MySQL
Enterprise
Workbench
MySQL Security – Role, Dual Password, Encryption TDE, Firewall, Fine grained Audit …
Recursive CTE
Windows Functions
Tables….
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Document Store: Architecture
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
X DevAPI Implemented in connectors for
C++, Java, .Net, Node.js, Python, PHP
working with Communities
New MySQL Shell
Python, JavaScript & SQL modes
Admin functions
Supports CRUD operations
Operation Document Relational
Create Collection.add() Table.insert()
Read Collection.find() Table.select()
Update Collection.modify() Table.update()
Delete Collection.remove() Table.delete()
Use SQL, CRUD APIs – Document (NoSQL) and
Relational (SQL), or “All of the Above”
All of this is in addition to the Classic APIs
https://dev.mysql.com/doc/refman/8.0/en/mysql-shell-
tutorial-javascript.html
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
js> session.createSchema('name')
js> use name
js> db.getCollections()
js> db.createCollection(‘myCollection')
js> db.getCollections()
js> db.myCollection.add({“param1":“value1", “param2":“value2”})
js> db.myCollection.find()
js> db.myCollection.find().limit(1)
js> db.myCollection.find("_id = '00005af018430000000000000002'")
js> db.myCollection.modify("_id = '1234'").set(“param",“value")
js> db.myCollection.remove("_id = '1234'")
js> session.startTransaction()
js> …
js> session.rollback()
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
MySQL Document Store cheat sheet
Create
Read
Update
Delete
Transactions
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Past, Present and Future (MySQL HA)
• In the Past,
– Setting up Replication topology was done manually, taking many steps
• including user management, restoring backups, configuring replication...
– MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture
– This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time
automating
12
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Past, Present and Future (MySQL HA)
• In the Past,
– Setting up Replication topology was done manually, taking many steps
• including user management, restoring backups, configuring replication...
– MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture
– This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time
automating
• 2016 - MySQL InnoDB Cluster
– Group Replication: Automatic membership changes, network partition handling, consistency...
– Shell to provide a powerful interface that helps in automating an integrating all components
– InnoDB CLONE to automatically provision members, fully integrated in InnoDB
13
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Past, Present and Future (MySQL HA)
• In the Past,
– Setting up Replication topology was done manually, taking many steps
• including user management, restoring backups, configuring replication...
– MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture
– This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time
automating
• 2016 - MySQL InnoDB Cluster
– Group Replication: Automatic membership changes, network partition handling, consistency...
– Shell to provide a powerful interface that helps in automating an integrating all components
– InnoDB CLONE to automatically provision members, fully integrated in InnoDB
• 2020 - MySQL InnoDB Replicaset
– ‘classic', 'asynchronous' Replication based Solution, fully integrated
14
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL InnoDB Cluster
• "A single product — MySQL — with high availability and scaling features
baked in; providing an integrated end-to-end solution that is easy to use."
15
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Group Replication – The core component
16
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL InnoDB Cluster – Use Cases
17
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Router
Transparent Access to Database Arch.
– "provide transparent routing between your application and back-end
MySQL Servers"
• Transparent client connection routing
– Load balancing
– Application connection failover
– Little to no configuration needed
• Stateless design offers easy HA client routing
– Router as part of the application stack
• Integration into InnoDB Cluster & InnoDB ReplicaSet
– Understands Group Replication & Replication topology
• Currently TCP Port each for PRIMARY and NON-PRIMARY traffic
18
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL InnoDB Cluster & ReplicaSet
19
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL ReplicaSet
• Fully integrated MySQL Router
– Automatic Routing
• Ease of use with MySQL Shell
– Configuring, Adding, Removing members
– Automatic Member Provisioning (CLONE)
• Replication Architecture:
– (manual) Switchover & Failover
– (asynchronous) Read Scaleout
– 'Simple' Replication architecture:
• no network/hardware requirements
• Providing Availability on PRIMARY when issues with secondaries or network
20
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Ease Of Use
21
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL
InnoDB
cluster
MySQL Enterprise Monitor
…
mysql-js> cluster.status()
{
"clusterName": “NewAppCluster",
"defaultReplicaSet": {
"status": "Cluster tolerant to up to ONE failure.",
"topology": {
“hanode1:3306": {
"address": “hanode1:3306",
"status": "ONLINE",
"role": "HA",
"mode": "R/W",
MySQL InnoDB Cluster: Architecture
22
MySQL Router
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 23
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Deployment Examples
MySQL InnoDB Cluster
24
https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-production-deployment.html
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Single Data Center
25
Active Data Center
ClusterDC1
MySQL
Router
MySQL
Router
MySQL
Router
MySQL InnoDB Cluster
MySQL
Shell
MySQL
Enterprise
Monitor
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
DC1 ➔ DC2
MySQL InnoDB Cluster Deployment
26
Async Replication
Active Data Center Backup Data Center
ClientsClusterDC1 ClusterDC2
MySQL
Router
MySQL
Router
MySQL
Router
MySQL
Enterprise
Monitor
MySQL
Shell
MySQL
RouterMySQL
Router
MySQL
Router
MySQL
Router
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
DC1 ➔ DC2
MySQL InnoDB Cluster Deployment
27
Async Replication
Active Data Center Backup Data Center
ClientsClusterDC1 ClusterDC2
MySQL
Router
MySQL
Router
MySQL
Router
MySQL
Router
MySQL
Router
MySQL
Enterprise
Monitor
MySQL
Shell
Load Balancer Load Balancer
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Machine 1
Machine 2
Machine 3
Machine 4
Deployment Sample Illustration
28
Active Data Center
ClusterDC1
MySQL
Router
MySQL
Router
MySQL
Router
MySQLInnoDBCluster
MySQL
Shell
MySQL
Enterprise
Monitor
MySQLInnoDBCluster
MySQLInnoDBCluster
DataCenter2
Replication
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Summary
• MySQL High Availability
– MySQL Replication
• Asynchronous Replication
• Semi-Sync Replication
– MySQL InnoDB Cluster
• Group Replication + MySQL Router + MySQL Shell
• Deployment Practices
29
20200613 my sql-ha-deployment

20200613 my sql-ha-deployment

  • 1.
    MySQL High Availability DeploymentPractice 马楚成 (Ivan Ma) Copyright © 2020 Oracle and/or its affiliates. MySQL Principal Solution Engineer Oracle MySQL GBU, Oracle LLC 2020-06-13
  • 2.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  • 3.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | 3 MySQL Sessions – Day 2
  • 4.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Agenda 4 MySQL Latest Update MySQL High Availability(Past, Present and Future ) Deployment Practice
  • 5.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. |
  • 6.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | https://www.facebook.com/groups/hkmysqlusergroup/
  • 7.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Version and LifeCycle • MySQL Enterprise Edition – MySQL 5.6 • Feb-2021 (Extended Support End) – MySQL 5.7 • Oct-2020 (Premier Support End) – Using MySQL 8.0 • NDB Cluster (Carrier Grade Edition) – NDB Cluster 8.0 (New!) • Jan-2020 Released 7
  • 8.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Updates 8 GIS / Spaital JSON / Document Store Reliability (MySQL Replication, MySQL InnoDB Cluster) MySQL Shell MySQL Router MySQL Enterprise Backup MySQL Enterprise Monitor MySQL Enterprise Workbench MySQL Security – Role, Dual Password, Encryption TDE, Firewall, Fine grained Audit … Recursive CTE Windows Functions Tables….
  • 9.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Document Store: Architecture Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
  • 10.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | X DevAPI Implemented in connectors for C++, Java, .Net, Node.js, Python, PHP working with Communities New MySQL Shell Python, JavaScript & SQL modes Admin functions Supports CRUD operations Operation Document Relational Create Collection.add() Table.insert() Read Collection.find() Table.select() Update Collection.modify() Table.update() Delete Collection.remove() Table.delete() Use SQL, CRUD APIs – Document (NoSQL) and Relational (SQL), or “All of the Above” All of this is in addition to the Classic APIs https://dev.mysql.com/doc/refman/8.0/en/mysql-shell- tutorial-javascript.html
  • 11.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | js> session.createSchema('name') js> use name js> db.getCollections() js> db.createCollection(‘myCollection') js> db.getCollections() js> db.myCollection.add({“param1":“value1", “param2":“value2”}) js> db.myCollection.find() js> db.myCollection.find().limit(1) js> db.myCollection.find("_id = '00005af018430000000000000002'") js> db.myCollection.modify("_id = '1234'").set(“param",“value") js> db.myCollection.remove("_id = '1234'") js> session.startTransaction() js> … js> session.rollback() Copyright @ 2018 Oracle and/or its affiliates. All rights reserved. MySQL Document Store cheat sheet Create Read Update Delete Transactions
  • 12.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Past, Present and Future (MySQL HA) • In the Past, – Setting up Replication topology was done manually, taking many steps • including user management, restoring backups, configuring replication... – MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture – This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time automating 12
  • 13.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Past, Present and Future (MySQL HA) • In the Past, – Setting up Replication topology was done manually, taking many steps • including user management, restoring backups, configuring replication... – MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture – This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time automating • 2016 - MySQL InnoDB Cluster – Group Replication: Automatic membership changes, network partition handling, consistency... – Shell to provide a powerful interface that helps in automating an integrating all components – InnoDB CLONE to automatically provision members, fully integrated in InnoDB 13
  • 14.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Past, Present and Future (MySQL HA) • In the Past, – Setting up Replication topology was done manually, taking many steps • including user management, restoring backups, configuring replication... – MySQL offered the technical pieces, leaving it up to the user to setup an (always customized) architecture – This requires technical components ... bringing lot's of work for DBA's and experts, who spent their time automating • 2016 - MySQL InnoDB Cluster – Group Replication: Automatic membership changes, network partition handling, consistency... – Shell to provide a powerful interface that helps in automating an integrating all components – InnoDB CLONE to automatically provision members, fully integrated in InnoDB • 2020 - MySQL InnoDB Replicaset – ‘classic', 'asynchronous' Replication based Solution, fully integrated 14
  • 15.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL InnoDB Cluster • "A single product — MySQL — with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use." 15
  • 16.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Group Replication – The core component 16
  • 17.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL InnoDB Cluster – Use Cases 17
  • 18.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Router Transparent Access to Database Arch. – "provide transparent routing between your application and back-end MySQL Servers" • Transparent client connection routing – Load balancing – Application connection failover – Little to no configuration needed • Stateless design offers easy HA client routing – Router as part of the application stack • Integration into InnoDB Cluster & InnoDB ReplicaSet – Understands Group Replication & Replication topology • Currently TCP Port each for PRIMARY and NON-PRIMARY traffic 18
  • 19.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL InnoDB Cluster & ReplicaSet 19
  • 20.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL ReplicaSet • Fully integrated MySQL Router – Automatic Routing • Ease of use with MySQL Shell – Configuring, Adding, Removing members – Automatic Member Provisioning (CLONE) • Replication Architecture: – (manual) Switchover & Failover – (asynchronous) Read Scaleout – 'Simple' Replication architecture: • no network/hardware requirements • Providing Availability on PRIMARY when issues with secondaries or network 20
  • 21.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Ease Of Use 21
  • 22.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | MySQL Connector Application MySQL Router MySQL Connector Application MySQL Router MySQL Shell MySQL Connector Application MySQL Router MySQL Connector Application MySQL InnoDB cluster MySQL Enterprise Monitor … mysql-js> cluster.status() { "clusterName": “NewAppCluster", "defaultReplicaSet": { "status": "Cluster tolerant to up to ONE failure.", "topology": { “hanode1:3306": { "address": “hanode1:3306", "status": "ONLINE", "role": "HA", "mode": "R/W", MySQL InnoDB Cluster: Architecture 22 MySQL Router
  • 23.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | 23
  • 24.
    Copyright © 2017,Oracle and/or its affiliates. All rights reserved. | Deployment Examples MySQL InnoDB Cluster 24 https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-production-deployment.html
  • 25.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Single Data Center 25 Active Data Center ClusterDC1 MySQL Router MySQL Router MySQL Router MySQL InnoDB Cluster MySQL Shell MySQL Enterprise Monitor
  • 26.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | DC1 ➔ DC2 MySQL InnoDB Cluster Deployment 26 Async Replication Active Data Center Backup Data Center ClientsClusterDC1 ClusterDC2 MySQL Router MySQL Router MySQL Router MySQL Enterprise Monitor MySQL Shell MySQL RouterMySQL Router MySQL Router MySQL Router
  • 27.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | DC1 ➔ DC2 MySQL InnoDB Cluster Deployment 27 Async Replication Active Data Center Backup Data Center ClientsClusterDC1 ClusterDC2 MySQL Router MySQL Router MySQL Router MySQL Router MySQL Router MySQL Enterprise Monitor MySQL Shell Load Balancer Load Balancer
  • 28.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Machine 1 Machine 2 Machine 3 Machine 4 Deployment Sample Illustration 28 Active Data Center ClusterDC1 MySQL Router MySQL Router MySQL Router MySQLInnoDBCluster MySQL Shell MySQL Enterprise Monitor MySQLInnoDBCluster MySQLInnoDBCluster DataCenter2 Replication
  • 29.
    Copyright © 2019,Oracle and/or its affiliates. All rights reserved. | Summary • MySQL High Availability – MySQL Replication • Asynchronous Replication • Semi-Sync Replication – MySQL InnoDB Cluster • Group Replication + MySQL Router + MySQL Shell • Deployment Practices 29