MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool

Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Miguel Araújo
Senior Software Developer
MySQL Middleware and Clients
pre-FOSDEM’19 MySQL Day - February 01, 2019
MySQL Shell
A DevOps-engineer day with MySQL’s
development and administration tool

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
2

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. 3
Who am I?

whoami
Miguel Araújo
• MySQL Senior Software Developer
• Lead developer for the AdminAPI at the
MySQL Shell team
• MySQL InnoDB Cluster evangelist
• Living in Portugal
• http://about.me/miguel-araujo
4

DevOps: What is it?
As simple as:
 Engineers (developers) and operations working together to:
Automate and monitor processes
 Share knowledge and collaborate
Get things done faster
Deliver faster!

DevOps-Engineer: …and that is?
6

¯_(ツ)_/¯
7

8
OK, actually it’s someone that can:
 Do system administration (sysadmin)
 Code
 Understand testing
 Automate processes
 Talk! (bridge between devs and ops)

MySQL Shell: Introduction
• Interactive multi-language console interface that supports development and
administration for the MySQL Server
• Can be used to perform data queries or updates, and administration operations:
 Interactively or through batch operations
 With scriptable “DevOps” APIs
 Unified Interface for MySQL Developers and DBAs: one tool!
• Intuitive, flexible, powerful and easy to use!
9

MySQL Shell: Features (Dev)
• Multi-language support
– JavaScript, Python and SQL
– Both interactive and batch operations (scripting)
10
MySQL
Development

• Multi-language support
– JavaScript, Python and SQL
– Both interactive and batch operations (scripting)
• Document and relational model
– Supports the classic relational model
– Modern fluent API for the MySQL Document Store
• CRUD and Relational
11
MySQL
Development

• Scripting
– Execute scripts from:
• Interactive REPL interface
• Directly from the cmdline
• From Python, JS, or SQL script files
– Import any installed module (in Python)
MySQL
Development

MySQL Shell: Features (Dev/Ops)
• APIs
 MySQL DBs
– ‘mysql’ object
– Interaction with MySQL Server using the relational model
 Document Store
– ‘mysqlx’ object
– Interact with MySQL Server using the X DevAPI
MySQL
Development

MySQL Shell: Features (Dev/Ops)
• APIs
 ShellAPI
– ‘shell’ object
– General purpose functions and properties
 AdminAPI
– ‘dba’ object
– Set up and management of InnoDB clusters
 Shell utilities
– ‘util’ object
– Upgrade checker
– Import JSON documents
MySQL
Administration

A common use-case

A common use-case
Company X starts a new business unit
16
• A DBMS is needed

A common use-case
17
• The infraestructure must be prepared

A common use-case
18
• The DBMS must be deployed and managed

A common use-case
19
• The world’s most popular open-source database is suggested

A common use-case
20
 MySQL is approved!

A common use-case
21
 MySQL is approved!
Call the devOps-engineer!

Meet Bill
The devOps-engineer
22
This is Bill.
Bill is a MySQL user.
Bill knows the Shell is the devOps tool for MySQL.
Bill uses the Shell to make his life easier.
Bill is smart.
Be like Bill.

A common use-case
Prepare the DBMS
23
1. Package installation
2. Directories management
3. Set up security
4. Configure server settings
5. Create accounts with the right permissions
6. ...

A common use-case
Prepare the DBMS
24
• Bill writes a JS script that can be used by others or even automated

A common use-case
Prepare the DBMS
25
• Interacts with the database through the session object

A common use-case
Prepare the DBMS
26
• Connect to localhost:3310
• Execute create_account.js with or without extra arguments forwarded to
the script (sys.argv[])
• Rely on the Shell prompt utility

A common use-case
Upgrade the DBMS
27
• A major new MySQL version (8.0) is released with many improvements and
features
• Bill needs to upgrade the whole setup but is afraid it gets broken...

A common use-case
Upgrade the DBMS
28
• A major new MySQL version (8.0) is released with many improvements and
features
• Bill needs to upgrade the whole setup but is afraid it gets broken...
Shell to the rescue!

MySQL Shell: Features (Ops)
• Upgrade checker
– Verify whether instances are ready or not
for upgrading to MySQL 8
– Available through the util object
29
mysql-js> util.checkForServerUpgrade(“root@localhost:3306”);
MySQL
Administration

A common use-case
Upgrade the DBMS
30

• API Command line integration
– cmdline syntax to invoke built-in Shell commands
– Requires minimal extra typing, quoting and escaping
– Supported objects: dba, cluster, shell, shell.options
and util
Syntax: mysqlsh [OPTIONS] [URI] -- <object_name> <method_name> [argument_list]
31
$ mysqlsh root@myserver:33060 –- util check-for-server-upgrade
MySQL
Administration

A common use-case
Upgrade the DBMS
32
• Bill is smart
• Bill is a devOps-engineer
• Bill automates processes, by using continuous automation frameworks
• How could Bill ensure that the MySQL Servers deployments are always
using the latest available version?

Shell meets continuous automation
33
• Let’s use as an example:
service { ‘mysql’:
ensure => latest,
enable => true,
require => Package[‘mysql-server-community’],
}

34
What if current is 5.7 and latest is 8.0? Will it break everything?
ensure => latest,
enable => true,
}
!
• Let’s use as an example:

35
ensure => latest,
enable => true,
before => Exec [‘upgrade checker’]
}
exec { “upgrade checker”:
command => “mysqlsh -- util check-for-server-upgrade { --user=root --
host=localhost --port=3306} --password=‘myPwd’”
require => Service[“mysqlsh”]
}
• Use MySQL Shell upgrade checker

• Customizable prompts
– Display context information
– Custom/dynamic tags that react to context
• Eg. export PRODUCTION_SERVERS=192.168.42.16
MySQL Shell: The DBA toolbox
Other features and tweaks
36

• Secure credential manager
• Passwords queried from a secure password store (uses OS native APIs)
• Optional auto-save for typed passwords (default will prompt)
• History
• Typed command history
• Pager
• Pass SQL results through an external screen paging program
37

• Built-in help
– Documentation for included APIs, shell commands, SQL, etc.
38

A common use-case: part 2
The business unit is growing
39
• The systems are vulnerable to downtime
• Downtime causes high losses...
• Critical data can’t be lost and must ”always” be available!
• High Availability becomes mandatory

40
• The systems are vulnerable to downtime
• Downtime causes high losses...
• Critical data can’t be lost and must ”always” be available!
• High Availability becomes mandatory
Bill wonders what kind of HA support MySQL has...
?

InnoDB cluster

InnoDB cluster
 MySQL Servers running Group
Replication

InnoDB cluster
Replication
 App Servers with MySQL Router

InnoDB cluster
Replication
 MySQL Shell
 Set up, Manage, Orchestrate IDC

InnoDB cluster
Replication
 MySQL Shell
 Set up, Manage, Orchestrate IDC
 Clients / Applications

• Administration API: AdminAPI
– Creation and Management of InnoDB Clusters
– Hides the complexity of:
• Configuration
• Provisioning
• Orchestration
– Simple and straight-forward
– Doesn’t require MySQL expertise
– Flexible, powerful and secure
– Available in both JavaScript and Python
47
MySQL
Administration

48
• Bill needs to set up an HA deployment for the company needs
• Using the Shell and the AdminAPI he can achieve that very easily!

49
$ mysqlsh
mysql-js> dba.configureInstance(“root@host1:3306”, {clusterAdmin: “bill”})

50
$ mysqlsh
mysql-js> var cluster = dba.createCluster(“Cluster1”)

51
$ mysqlsh
mysql-js> var cluster = dba.createCluster(“Cluster1”)
mysql-js> cluster.addInstance(“bill@host2:3306”)

HA in 5 minutes!
52
 JSON object describing the structure of
the Cluster
 Includes general cluster information
 Topology:
 ReplicaSets
 Instance members

HA in 5 minutes!
53
 JSON object describing the status of the
Cluster
 Includes general cluster status
 Status information about each ReplicaSet
member

The requirements change
54
• A strange behaviour was seen in the application leading to user complaints

55
• Ops traced down the issue to the database... And found possible
inconsistent reads

56
inconsistent reads
• Finally someone noticed that whenever a primary failover happens, stale
data was being read from the newly elected primary causing the strange
“inconsistencies”

57
inconsistent reads
• Finally someone noticed that whenever a primary failover happens, stale
data was being read from the newly elected primary causing the strange
“inconsistencies”
• Direction asks for a solution even if it compromises performance!

Checking the cluster options
58
 Verify the current cluster settings

Checking the cluster options
59
 Verify the current cluster settings
 JSON object listing configuration options
of the Cluster
 Includes global cluster options
 Includes configuration options of each cluster
member

Failover consistency
60
The failover consistency option can have 2 values:
• EVENTUAL (default)
 Read queries allowed in the new primary (even if backlog is being applied)
 Writes are blocked
 Possible stale reads
• BEFORE_ON_PRIMARY_FAILOVER
 Read and write queries blocked in the new primary (meanwhile backlog is being applied)
 “read your writes”

Change the failover consistency
61
The AdminAPI allows changing a configuration option:
1) For the whole cluster
 <Cluster.>setOption(option, value)
2) In a cluster member
 <Cluster.>setInstanceOption(instance, option, value)

Change the failover consistency
62
The AdminAPI allows changing a configuration option:
1) For the whole cluster
 <Cluster.>setOption(option, value)
2) In a cluster member
 <Cluster.>setInstanceOption(instance, option, value)

Maintenance
63
• Maintenance tasks on servers are common...
• Bill needs to perform a maintenance task on the current primary instance
• Which implies some sort of disruption...
• Idealy, Bill would rather manually elect a new primary to avoid any
disruption

Maintenance
64
• Maintenance tasks on servers are common...
• Bill needs to perform a maintenance task on the current primary instance
• Which implies some sort of disruption...
• Idealy, Bill would rather manually elect a new primary to avoid any
disruption
But... How?
?

Elect a new primary
65
 <Cluster.>setPrimaryInstance(instance)
By using the Shell and the AdminAPI!

The requirements change... again
66
• Analysis of the dataset and its usage, concluded that the workload could be
spread accross the cluster members:

67
– No concurrent DDL ops in the same object

68
– No foreign key constraints

69
– No large transactions, etc.

70
• Decision to use Multi-Primary mode, even if not recommended, is taken.

71
• Decision to use Multi-Primary mode, even if not recommended, is taken.
But that implies the downtime of re-creating the whole cluster!
!

 <Cluster.>switchToMultiPrimaryMode()
Switch to Multi-Primary mode
72
No restarts nor downtime!

Switch to Multi-Primary mode
73
• All cluster members become
writable (R/W)
 Note:
Switch to single-primary mode available with:
 <Cluster.>switchToSinglePrimaryMode()

• Suddently somes instances are expelled from the cluster
• ... even though ops report that the network is OK!
• But further investigation concludes the network is actually flaky
• Some members are being expelled due to very small network cuts of 2
minutes
Troubleshooting
74

• Suddently somes instances are expelled from the cluster
• ... even though ops report that the network is OK!
• But further investigation concludes the network is actually flaky
• Some members are being expelled due to very small network cuts of 2
minutes
What can be done?
Troubleshooting
75
?

• Group Replication’s failure detector window is configurable
• Allows delays of suspensions of previously active cluster members
• Configurable using the AdminAPI!
Defining the expel timeout
76

A common use-case
Other useful features
77
• Define the next primary instance “in line”
• memberWeight = x
– Integer: [0, 100]
• Define the behaviour of an instance whenever it drops out of the cluster
• exitStateAction = y
– ABORT_SERVER
– READ_ONLY
• Extended Cluster.status()
• extended: true
• queryMembers: true

Summary
• MySQL Shell brings together developers and DBAs
• Unified interface: one tool!
• Development and Administration APIs and features
• MySQL InnoDB Cluster is THE built-in HA solution for MySQL:
• Full-stack: High Availability out-of-the-box
• Easy to use: usability as a top concern
• Full control of the Cluster using the AdminAPI
78

Resources
• MySQL Shell Userguide:
– https://dev.mysql.com/doc/refman/en/mysql-shell.html
• MySQL InnoDB Cluster Userguide:
– http://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html
• APIs Reference manuals
– JavaScript: https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/
– Python: https://dev.mysql.com/doc/dev/mysqlsh-api-python/
• Blogging
– http://mysqlserverteam.com/category/high-availability
79

Thank you!
Any questions?
80

MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool

MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool

Similar to MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool (20)

More from Miguel Araújo

More from Miguel Araújo (10)

Recently uploaded

Recently uploaded (20)

MySQL Shell - A DevOps-engineer day with MySQL’s development and administration tool