MySQL Community Meetup in China : Innovation driven by the Community

August 2017
MySQL Innovation
Evolution is inevitable & why the MySQL Users Base´s feedback is
important
from 5.7 to 8.0, InnoDB Cluster, Document Store

Frédéric Descamps - MySQL Evangeslist - Oracle
4 / 217

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purpose only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied up in
making purchasing decisions. The development, release and timing of any features or
functionality described for Oracle´s product remains at the sole discretion of Oracle.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
5 / 217

about.me/lefred
Who am I ?
6 / 217

Frédéric Descamps -
弗雷德
@lefred
MySQL Evangelist
Hacking MySQL since 3.23
devops believer
part of MySQL Team since May 2016
living in Belgium 🇧🇪
7 / 217

MySQL 5.7 -> 8.0
Innovation
8 / 217

MySQL 5.7 (GA)
3x Better Performance
Replication Enhancements
Optimizer Cost Model
JSON Support
Improved Security
SYS & Performance Schema
GIS

MySQL InnoDB Cluster (GA)
MySQL Group Replication
MySQL Router
MySQL Shell
MySQL 8.0 (DMR)
Data Dictionary
Roles
Unicode
CTEs
Window Functions
Security
Replication
SysSchema
GIS
MySQL Innovation: 5.7 -> 8.0
9 / 217

100%

virtually all organizations require their
most critical systems to be highly available
!
10 / 217

100%

virtually all organizations require their
most critical systems to be highly available
!
11 / 217

"High Availability becomes a core first
class feature of MySQL!"
MySQL InnoDB Cluster

12 / 217

MySQL InnoDB Cluster: vision
"A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
13 / 217

MySQL InnoDB Cluster: vision
"A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
InnoDB
cluster
Ease-of-Use
Extreme Scale-Out
Out-of-Box Solution
Built-in HA
High Performance
Everything Integrated
14 / 217

One Product: MySQL
All components created together
Tested together
Packaged together
Easy to Use
One client: MySQL Shell
Easy packaging
Integrated orchestration
Homogenous servers
Flexible and Modern
SQL and NoSQL together
Protocol Buffers
Asynchronous API
Developer friendly
Support Read/Write Scale Out
Shared clusters
Federated system of N replica sets
supporting cross shard
operations
Each replica set manages a shard
MySQL InnoDB Cluster: goals
15 / 217

MySQL InnoDB Cluster: Users Base Feedback
Examples of improvements suggested by our users:
Disallow changes to members that have left the group (8.0.2) - BUG#84728,
BUG#84795, BUG#84733
Influence primary election by assigning promotion weights (8.0.2).
More information on Performance Schema tables (8.0.2).
Fine tuning options for the flow control mechanism (8.0.2).
16 / 217

MySQL powers the Web !
17 / 217

MySQL powers the Web !
and many more...
18 / 217

Web applications evolution
19 / 217

Everything is already Mobile first or moving to mobile very soon
20 / 217

Everything is already Mobile first or moving to mobile very soon

...and it impacts our roadmap and new features priority
21 / 217

Evolution Driving Roadmap: 1995 vs 2017

There was no local.
Now everything is
contextual to me as
a user.
The Internet was
global, but full of
silos. Now we are
more connected than
ever.
In the past APIs were
cumbersome,
powered by SOAP
and XML-RPC.
There is now an
increased focus on
privacy and
protecting
information.
22 / 217

Evolution Driving Roadmap: 1995 vs 2017

There was no local.
Now everything is
contextual to me as
a user.
The Internet was
global, but full of
silos. Now we are
more connected than
ever.
In the past APIs were
cumbersome,
powered by SOAP
and XML-RPC.
There is now an
increased focus on
privacy and
protecting
information.
23 / 217

Evolution Driving Roadmap: MySQL 5.7 recap
Switched to
Boost.Geometry
library
R+Tree indexes for
InnoDB
Spatial Typed for
InnoDB
Optimized InnoDB
Temporary Tables
Optimized Packed
Sort Buffer
Switched InnoDB to
DYNAMIC row format
by default
Native JSON Data
Type
20+ JSON Functions
Generated Columns
for Functional Index
JSON Comparator
Short-hand JSON
operators
Secure by Default
Password Expiration
Password Policy
Remove old
password hash
Encrypted
Tablespaces, TDE
(Enterprise)
Firewall (Enterprise)
Audit (Enterprise)
24 / 217

Evolution Driving Roadmap: MySQL 8.0
Geography support
Spatial Reference
System (SRS)
Support
SQL/MM Information
Schema views
Standard compliant
axis ordering in
import/export
functions
Utf8mb4 as default
Up to 16x Faster
Performance
Based on Unicode 9.0
New collations with
Accents/Case sensitivity
Japanese Collation
Aggregate, Table
and Pretty Print
Functions
Sort Performance
Improvements
Optimizer support
for in-place update
JSON Shell
Roles
Encrypted REDO Log
and UNDO Space
Finer grained
system privileges
Atomicity in
Privilege
statements
25 / 217

Overview of improvements & new features
What's new in MySQL 8.0 ?
26 / 217

Fully Function, Properly Architected
Roles
Create and Drop Roles, Grant to Roles
Grant Roles to Roles. Grant Roles to
Users
Limit Hosts that can use roles, Define
Default Roles
And even visualize Roles with SQL
function ROLES_GRAHPHML()
MySQL 8.0: Security Roles
27 / 217

MySQL 8.0 vs MySQL 5.7 utf8mb4
UTF8MB4 is now the default charset set !
28 / 217

MySQL 8.0 vs MySQL 5.7 utf8mb3
No need to use the short-cut as in 5.7
29 / 217

MySQL 8.0: Boosts Developer Productivity

Improve support for the SQL standard
30 / 217

MySQL 8.0: Boosts Developer Productivity
New! CTEs and Recursive CTEs
New! Window Functions
New! SKIP LOCKED and NOWAIT
UUIDs and Bitwise Functions
31 / 217

WITH RECURSIVE qn AS
( SELECT 1 AS a
UNION ALL
SELECT 1+a FROM qn WHERE a<10
)
SELECT * FROM qn;
MySQL 8.0: recursive CTE (example)

32 / 217

WITH RECURSIVE qn AS
( SELECT 1 AS a
UNION ALL
SELECT 1+a FROM qn WHERE a<10
)
SELECT * FROM qn;
+------+
| a |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+------+
10 rows in set (0.01 sec)
MySQL 8.0: recursive CTE (example)

33 / 217

SELECT row_number() over ()
as `row_num`,
`name`, `country`
FROM friends
ORDER BY country;
MySQL 8.0: Window Functions (example)

34 / 217

SELECT row_number() over ()
as `row_num`,
`name`, `country`
FROM friends
ORDER BY country;
+---------+--------+----------+
| row_num | name | country |
+---------+--------+----------+
| 1 | dim0 | Belgium |
| 2 | Kenny | Belgium |
| 3 | Luis | Portugal |
| 4 | Emily | USA |
| 5 | Stoker | USA |
+---------+--------+----------+
5 rows in set (0.00 sec)
MySQL 8.0: Window Functions (example)

35 / 217

SELECT seat_no
FROM seats
JOIN seat_rows USING (row_no)
WHERE seat_no IN (3,4)
AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seats_rows NOWAIT;
MySQL 8.0: better handing of hot rows

36 / 217

SELECT seat_no
FROM seats
JOIN seat_rows USING (row_no)
WHERE seat_no IN (3,4)
AND seat_rows.row_no IN (12)
AND booked = 'NO'
FOR UPDATE OF seats SKIP LOCKED
FOR SHARE OF seats_rows NOWAIT;
SKIP LOCKED = non deterministically
skip over locked rows
NOWAIT = error immediately if a row is
already locked
MySQL 8.0: better handing of hot rows

37 / 217

Do more with a fewer number of servers
Take more advantages of new hardware:
SSD
multi-core processors
Greater density per server
Operator Friendliness
Increase Reliability
Improved Replication Throughput
38 / 217

MySQL 8.0: Makes Scale Easier
Better Observability: improving PFS (more instruments, better performance)
New operations
Higher Reliability: system tables are now in InnoDB and is used for Data Dictionary
Parallel Replication
39 / 217

MySQL 8.0: Performance Schema
SQL error instrumentation
Indexes
Configuration Variables
New! Data Locks
New! Histograms
40 / 217

PFS : Performance Comparison
41 / 217

PFS : Performance Comparison
Over 30x faster!
42 / 217

MySQL 8.0: Invisible Indexes
Indexes are "hidden" to the MySQL Optimizer
not the same as "disabled indexes"
contents are fully up to date and maintained by DML
Two use cases:
Soft Delete (Recycle Bin)
Staged Rollout
43 / 217

MySQL 8.0: Persist Configuration
Persist GLOBAL Server Variables
SET PERSIST max_connections = 500;
Examples Include:
Of ine_mode
Read_Only
Requires no filesystem access
New! Captures timestamp and change user
44 / 217

MySQL 8.0: Parallel Replication
MySQL 5.7 added support for logical clock parallel replication
in early version (<5.7.3), the logical clock is group commit based
in next versions, the logical clock is interval based
MySQL 8.0 adds support for Write Set parallelism identification
45 / 217

MySQL 8.0: Replication Improvements
46 / 217

MySQL 8.0: Replication Improvements
47 / 217

Source code now documented with
Doxygen
Plugin Infrastructure!
Expanded Query Hints Support
Improved Scan Query Performance
Improved BLOB Storage
Improved Memcached Interface
Cost Model Improvements
Scalabibilty Improvements
Atomicity in Privileges
Parser Refactoring
Improvements to Temporary Tables
C++11 and Toolchain
Improvements
GTID_PURGED always settable
Persistent Auto Increment
Native InnoDB Data Dictionary
Improved Information Schema
performance
New! SQL Grouping Function
New! Optimizer Trace detailed sort
statistics
New! Descending Indexes
New! Smaller Package Downloads
New! JSON Aggregate, Pretty print
functions
New! JSON performance
improvements
New! Expended Query Hints
New! Improved usability of cost
constant configuration
Group Replication
New! Transaction Save Point
support in Group Replication
New! Improved Replication Applier
Lag Statistics in Performance
Schema
New! Per Multi-source Channel
Replication Filters
New! Atomic DDL extended to the
Binary Log
New! Performance Improvements
on the Replication Applier
New! Parallel Replication Applier
Policy
New! Binary Log Management
Enhancements
New! Additional Metadata into the
Binary Log
All these features plus...
48 / 217

Our vision in 4 steps
MySQL Document Store
Relational & Document Models
MySQL HA
Out-Of-Box HA
Read Scale-Out
Async Replication + Auto Failover
Write Scale-Out
Sharding
E1 E3
E2 E4
50 / 217

Step 2´s Architecture
Application
MySQL Connector
MySQL Router
MySQL Shell
InnoDB
cluster
Application
MySQL Connector
MySQL Router
Mp
M
M
51 / 217

Application
MySQL Connector
MySQL Router
MySQL Shell
InnoDB
cluster
Application
MySQL Connector
MySQL Router
Mp
M
M
S1 S2 S3 S4 S...
52 / 217

Application
MySQL Connector
MySQL Router
MySQL Shell
Application
MySQL Connector
MySQL Router
InnoDB
cluster
Mp
M
M
S2 S3 S4 S...S1
InnoDB
cluster
Mp
M
M
S1 S2 S3 S4 S...
InnoDB
cluster
Mp
M
M
S1 S2 S3 S4 S...
Application
MySQL Connector
MySQL Router
Application
MySQL Connector
MySQL Router
Application
MySQL Connector
MySQL Router
replicaset1
replicaset2
replicaset3
53 / 217

Group Replication: heart of MySQL InnoDB
Cluster
54 / 217

Group Replication: heart of MySQL InnoDB
Cluster
55 / 217

But before going further...
56 / 217

Database High Availability == data redundancy
57 / 217

Database High Availability == data redundancy
The best way to achieve data redundancy with MySQL is to use replication !
58 / 217

MySQL HA Architectures with asynchronous
replication
It exists a lot of possible architectures and as much tools to enforce them.
59 / 217

replication
They are all based on the same principle:
60 / 217

replication
1 master
61 / 217

replication
1 master
1 or many replicas (slaves)
62 / 217

replication
1 master
1 or many replicas (slaves)
If the master as an issue, the most accurate slave needs to takeover the role and be
promoted as master for all the remaining slaves still online.
63 / 217

Automation
64 / 217

Automation
And this is the most complicated part...
65 / 217

Automation
It exists a large amount of external tools that can be used to achieve this task. Most of
them are only compatible with GNU/Linux.
66 / 217

Automation
Their problem is that they create complexity in the architecture's design. You need to be
DBA & sysadmin to manage those solutions:
67 / 217

Automation
MySQL-Utilities (mysqlrplcheck, mysqlrpladmin)
68 / 217

Automation
Pacemaker with MySQL dedicated OCF
69 / 217

Automation
Pacemaker with MySQL dedicated OCF
custom solutions...
70 / 217

but what is it ?!?
71 / 217

but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
72 / 217

but what is it ?!?
GR is an implementation of Replicated Database State Machine theory
73 / 217

but what is it ?!?
GR uses a Paxos based protocol
74 / 217

but what is it ?!?
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
75 / 217

but what is it ?!?
GR implements conflict detection and resolution
76 / 217

but what is it ?!?
GR allows automatic distributed recovery
77 / 217

but what is it ?!?
GR allows automatic distributed recovery
Supported on all MySQL platforms !!
Linux, Windows, Solaris, OSX, FreeBSD
78 / 217

And for users ?
79 / 217

And for users ?
no longer necessary to handle server fail-over manually or with a complicated script
80 / 217

And for users ?
GR provides fault tolerance
81 / 217

And for users ?
GR enables update-everywhere setups
82 / 217

And for users ?
GR handles crashes, failures, re-connetcs automatically
83 / 217

And for users ?
GR handles crashes, failures, re-connetcs automatically
Allows an easy setup of a MySQL service high available !
84 / 217

OK, but how does it work ?
85 / 217

it's just ...
86 / 217

it's just ...
87 / 217

it's just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
88 / 217

it's just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
not that easy to understand... right ? As a picture is worth a 1000 words, let's illustrate
this...
89 / 217

90 / 217

91 / 217

92 / 217

93 / 217

94 / 217

95 / 217

96 / 217

97 / 217

98 / 217

99 / 217

100 / 217

101 / 217

Certification
Certification is the process that only needs to answer the following unique question:
102 / 217

Certification
can the write (transaction) be applied ?
103 / 217

Certification
based on unapplied earlier transactions
104 / 217

Certification
such conflicts must come for other members/nodes
105 / 217

Certification
happens on every member/node
106 / 217

Certification
should be deterministic on every node
107 / 217

Certification
results are not reported to the group
108 / 217

Certification
pass: enter in the apply queue
109 / 217

Certification
fail: drop the transaction
110 / 217

Certification
serialized by the total order in GCS/XCOM + GTID
111 / 217

Certification
serialized by the total order in GCS/XCOM + GTID
cost is based on trx size (# rows & # keys)
112 / 217

GTID
GTIDs are the same as those used by asynchronous replication.
mysql> SELECT * FROM performance_schema.replication_connection_statusG
************************** 1. row ***************************
CHANNEL_NAME: group_replication_applier
GROUP_NAME: afb80f36-2bff-11e6-84e0-0800277dd3bf
SOURCE_UUID: afb80f36-2bff-11e6-84e0-0800277dd3bf
THREAD_ID: NULL
SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
RECEIVED_TRANSACTION_SET: afb80f36-2bff-11e6-84e0-0800277dd3bf:1-57,
f037578b-46b1-11e6-8005-08002774c31b:1-48937
LAST_ERROR_NUMBER: 0
LAST_ERROR_MESSAGE:
LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
113 / 217

but transactions use the Group´s GTID
mysql> show master statusG
************************** 1. row ***************************
File: mysql4-bin.000001
Position: 1501
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set: afb80f36-2bff-11e6-84e0-0800277dd3bf:1-57,
f037578b-46b1-11e6-8005-08002774c31b:1-48937
114 / 217

Requirements
exclusively works with InnoDB tables only
115 / 217

Requirements
every tables must have a PK defined
116 / 217

Requirements
only IPV4 is supported
117 / 217

Requirements
a good network with low latency is important
118 / 217

Requirements
maximum of 9 members per group
119 / 217

Requirements
maximum of 9 members per group
log-bin must be enabled and only ROW format is supported
120 / 217

Requirements (2)
enable GTIDs
121 / 217

Requirements (2)
enable GTIDs
replication meta-data must be stored in system tables
--master-info-repository=TABLE
--relay-log-info-repository=TABLE
122 / 217

Requirements (2)
enable GTIDs
writesets extraction must be enabled
--transaction-write-set-extraction=XXHASH64
123 / 217

Requirements (2)
enable GTIDs
writesets extraction must be enabled
--transaction-write-set-extraction=XXHASH64
log-slave-updates must also be enabled
124 / 217

Limitations
binlog checksum is not supported (but we have checksum in the replication channel)
--binlog-checksum=NONE
125 / 217

Limitations
savepoints were not supported before 5.7.19 & 8.0.1
126 / 217

Limitations
SERIALIZABLE is not supported as transaction isolation level
127 / 217

Limitations
SERIALIZABLE is not supported as transaction isolation level
http://lefred.be/content/mysql-group-replication-limitations-savepoints/
http://lefred.be/content/mysql-group-replication-and-table-design/
128 / 217

Default = Single Primary Mode
By default, MySQL InnoDB Cluster runs in Single Primary Mode.
129 / 217

mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
130 / 217

mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
In Single Primary Mode, a single member acts as the writable master (PRIMARY) and
the rest of the members act as hot-standbys (SECONDARY).
The group itself coordinates and configures itself automatically to determine which
member will act as the PRIMARY, through a leader election mechanism.
131 / 217

Single Primary Mode - new in 8.0.2
group_replication_member_weight: influence the primary member election in Single-
Primary mode (from 0 to 100).
132 / 217

more visibility
133 / 217

135 / 217

MySQL Router (GA!)
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
136 / 217

MySQL Router (GA!)
MySQL Router doesn´t require any specific configuration. It configures itself
automatically (bootstrap) using MySQL InnoDB Cluster´s metadata.
137 / 217

MySQL Router (GA!)
MySQL Router doesn´t require any specific configuration. It configures itself
automatically (bootstrap) using MySQL InnoDB Cluster´s metadata.
The MySQL Router development will be focusing on sharding.
138 / 217

ProxySQL has native support for Group
Replication which makes it a good choice
for advanced users.
ProxySQL
If you need some specific features that are not yet available in MySQL Router, like
transparent R/W splitting, then you can use your software of choice.
We are also collaborating with ProxySQL.
139 / 217

140 / 217

MySQL Shell (GA!)
The MySQL Shell is an interactive Javascript, Python, or SQL interface supporting
development and administration for the MySQL Server and is a component of the MySQL
Server. You can use the MySQL Shell to perform data queries and updates as well as
various administration operations.
141 / 217

MySQL Shell (2)
The MySQL Shell provides:
142 / 217

MySQL Shell (2)
Both Interactive and Batch operations
143 / 217

MySQL Shell (2)
Document and Relational Models
144 / 217

MySQL Shell (2)
CRUD Document and Relational APIs via scripting
145 / 217

MySQL Shell (2)
Traditional Table, JSON, Tab Separated output results formats
146 / 217

MySQL Shell (2)
MySQL Standard and X Protocols
147 / 217

MySQL Shell (2)
MySQL Standard and X Protocols
and more...
148 / 217

MySQL Shell (3)
PREVIEW: the new Shell´s prompt is nice and clear !
149 / 217

don´t forget, this is a HA solution !
150 / 217

Deploying a MySQL InnoDB Cluster using MySQL
Shell´s adminAPI:
mysql-js> var i1 = 'root@instance01:3306';
mysql-js> dba.checkInstanceCon guration(i1);
mysql-js> shell.connect(i1);
mysql-js> var cluster = dba.createCluster('ChinaCluster');
mysql-js> cluster.checkInstanceState(i2);
mysql-js> cluster.addInstance(i2);
mysql-js> cluster.checkInstanceState(i3);
mysql-js> cluster.addInstance(i3);
151 / 217

DEMO !
152 / 217

RDBMS & MySQL
Relational Databases
154 / 217

Data Integrity
155 / 217

Data Integrity
normalization
156 / 217

Data Integrity
normalization
constraints (foreign keys, ...)
157 / 217

Data Integrity
normalization
Atomicity, Consistency, Isolation, Durability
158 / 217

Data Integrity
normalization
ACID compliant
159 / 217

Data Integrity
normalization
ACID compliant
transactions
160 / 217

Data Integrity
normalization
ACID compliant
transactions
SQL
161 / 217

Data Integrity
normalization
ACID compliant
transactions
SQL
powerfull query language
162 / 217

NoSQL & MySQL
NoSQL Databases
163 / 217

NoSQL or Document Store
Schemaless
164 / 217

Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
165 / 217

Schemaless
very quick initial development
166 / 217

Schemaless
Flexible data structure
167 / 217

Schemaless
embedded arrays or objects
168 / 217

Schemaless
valid solution when natural data can´t be optimally modeled into a relational
model
169 / 217

Schemaless
model
objects persistence without the use of any ORM - mapping object-oriented
170 / 217

Schemaless
model
JSON
171 / 217

Schemaless
model
JSON
close to frontend
172 / 217

Schemaless
model
JSON
close to frontend
native in JS
173 / 217

Schemaless
model
JSON
close to frontend
native in JS
easy to learn
174 / 217

How DBAs see data How Developers see
data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Fede
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.80000305175
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
175 / 217

DMBS or NoSQL ?
176 / 217

DMBS or NoSQL ?
Why not both ?
177 / 217

MySQL as Document Store
Requirements
178 / 217

MySQL as Document Store: requirements
179 / 217

1. support of JSON datatype
180 / 217

2. CRUD Operations
181 / 217

2. CRUD Operations
3. X-Plugin
182 / 217

2. CRUD Operations
3. X-Plugin
4. X-Protocol
183 / 217

2. CRUD Operations
3. X-Plugin
4. X-Protocol
extends and replaces the traditional MySQL protocol
184 / 217

2. CRUD Operations
3. X-Plugin
4. X-Protocol
supports asynchronous communication to the server
185 / 217

2. CRUD Operations
3. X-Plugin
4. X-Protocol
supports multiple API calls like Javascript, Python, SQL...
186 / 217

2. CRUD Operations
3. X-Plugin
4. X-Protocol
supports multiple API calls like Javascript, Python, SQL...
5. MySQL Shell
187 / 217

Starting using MySQL DS in few minutes
188 / 217

Starting using MySQL DS in few minutes
This is the new MySQL shell as it will be in 8 ;-)
189 / 217

Migration from MongoDB to MySQL DS
For this example, I will use the well known restaurants collection:
We have two options:
1. We need to dump the data to a file and we will use MySQL Shell with the Python
interpreter to load the data.
1. We retrieve the data from MongoDB and load it to MySQL via the Shell in javascript
190 / 217

Dump and load using MySQL Shell & Python
this example is the courtesy of @datacharmer:
https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo --quiet --eval 'DBQuery.shellBatchSize=300;
var all=db.restaurants. nd() ; all' | perl -pe
's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
191 / 217

Dump and load using bash and MySQL Shell

$ mongo --quiet --eval 'DBQuery.shellBatchSize=30000;
var all=db.restaurants. nd() ; all'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g'
| head -n -1 | while read line
do
mysqlsh root@127.0.0.1 --password='' -e "rec=$line;
session.getSchema('docstore').getCollection('restaurants').add(rec).execute()"
done
192 / 217

Let´s query
193 / 217

Let´s query
That´s too much records to show it here... let´s limit it
194 / 217

195 / 217

Some more examples
196 / 217

Some more examples
Let´s add a selection criteria:
197 / 217

Syntax slightly different than MongoDB
> db.restaurants. nd({"cuisine": "French",
"borough": { $not: /^Manhattan/} },
{"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2)
{ "borough" : "Queens", "cuisine" : "French",
"name" : "La Baraka Restaurant" }
{ "borough" : "Queens", "cuisine" : "French",
"name" : "Air France Lounge" }
198 / 217

CRUD operations
199 / 217

what about my old SQL ?
The hidden part of the iceberg
200 / 217

JSON datatype is behind the scene
native datatype (since 5.7.8)
201 / 217

JSON values are stored in MySQL tables using UTF8MB4
202 / 217

Conversion from "native" SQL types to JSON values
203 / 217

JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
204 / 217

JSON_SEARCH,...)
Generated/virtual columns
205 / 217

JSON_SEARCH,...)
Indexing JSON data
206 / 217

JSON_SEARCH,...)
Indexing JSON data
Foreign Keys to JSON data
207 / 217

JSON_SEARCH,...)
Indexing JSON data
Foreign Keys to JSON data
SQL Views to JSON data
208 / 217

Example
209 / 217

Example
210 / 217

Example (2)
211 / 217

Example (2)

212 / 217

Example (3): explain
213 / 217

Example (4): add index
214 / 217

what do I gain ?
Conclusion
215 / 217

schemaless
flexible data structure
easy to start (CRUD)
Conclusion
This is the best of the two worlds in one product !
Data integrity
ACID Compliant
SQL
216 / 217

Thank you - 谢谢 !
Any Questions ?
217 / 217

MySQL Community Meetup in China : Innovation driven by the Community

More Related Content

What's hot

Similar to MySQL Community Meetup in China : Innovation driven by the Community

More from Frederic Descamps

Recently uploaded

MySQL Community Meetup in China : Innovation driven by the Community