30 mins to discover MySQL Document Store a Percona University in Ghent, Belgium

1. 1 / 71

2. 2 / 71

3. Percona University, Ghent, BelgiumJune
2017
MySQL Document Store
How to replace a NoSQL database by MySQL without effort but with a lot of
gains ?
Frédéric Descamps - MySQL Community Manager - Oracle
3 / 71

4.
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purpose only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied up in
making purchasing decisions. The development, release and timing of any features or
functionality described for Oracle´s product remains at the sole discretion of Oracle.
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
4 / 71

5. about.me/lefred
Who am I ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
5 / 71

6. Frédéric Descamps
@lefred
MySQL Evangelist
Hacking MySQL since 3.23
devops believer
MySQL Community Manager since May
2016
living in Belgium 🇧🇪
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
6 / 71

7. 7 / 71

8. RDBMS & MySQL
Relational Databases
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
8 / 71

9. Relational Databases
Data Integrity
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
9 / 71

10. Relational Databases
Data Integrity
normalization
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
10 / 71

11. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
11 / 71

12. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
12 / 71

13. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
ACID compliant
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
13 / 71

14. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
ACID compliant
transactions
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
14 / 71

15. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
ACID compliant
transactions
SQL
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
15 / 71

16. Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
ACID compliant
transactions
SQL
powerfull query language
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
16 / 71

17. NoSQL & MySQL
NoSQL Databases
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
17 / 71

18. NoSQL or Document Store
Schemaless
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
18 / 71

19. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
19 / 71

20. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
20 / 71

21. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
21 / 71

22. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
22 / 71

23. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
23 / 71

24. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
objects persistance without the use of any ORM- mappingoobject-oriented
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
24 / 71

25. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
objects persistance without the use of any ORM- mappingoobject-oriented
JSON
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
25 / 71

26. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
objects persistance without the use of any ORM- mappingoobject-oriented
JSON
close to frontend
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
26 / 71

27. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
objects persistance without the use of any ORM- mappingoobject-oriented
JSON
close to frontend
native in JS
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
27 / 71

28. NoSQL or Document Store
Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can't be modelized optimaly into a relational
model
objects persistance without the use of any ORM- mappingoobject-oriented
JSON
close to frontend
native in JS
easy to learn
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
28 / 71

29. How DBAs see data How Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
29 / 71

30. DMBS or NoSQL ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
30 / 71

31. DMBS or NoSQL ?
Why not both ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
31 / 71

32. MySQL as Document Store
Requirements
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
32 / 71

33. MySQL as Document Store: requirements
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
33 / 71

34. MySQL as Document Store: requirements
1.support of JSON datatype
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
34 / 71

35. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
35 / 71

36. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
36 / 71

37. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
4.X-Protocol
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
37 / 71

38. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
4.X-Protocol
extends and replaces the traditional MySQL protocol
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
38 / 71

39. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
4.X-Protocol
extends and replaces the traditional MySQL protocol
supports asynchronous communication to the server
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
39 / 71

40. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
4.X-Protocol
extends and replaces the traditional MySQL protocol
supports asynchronous communication to the server
supports multiple API calls like Javascript, Python, SQL...
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
40 / 71

41. MySQL as Document Store: requirements
1.support of JSON datatype
2.CRUD Operations
3.X-Plugin
4.X-Protocol
extends and replaces the traditional MySQL protocol
supports asynchronous communication to the server
supports multiple API calls like Javascript, Python, SQL...
5.MySQL Shell
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
41 / 71

42. Starting using MySQL DS in few minutes
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
42 / 71

43. Starting using MySQL DS in few minutes
This is the newMySQL shell as it will be in 8 ;-)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
43 / 71

44. Migration from MongoDB to MySQL DS
For this example, I will use the well known restaurants collection:
We have two options:
1.We need to dump the data to a file and we will use MySQL Shell with the Python
interpreter to load the data.
1.We retrieve the data fromMongoDB and load it to MySQL via the Shell in javascript
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
44 / 71

45. Dump and load using MySQL Shell & Python
thisexampleisthecourtesyof@datacharmer:
https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo --quiet --eval 'DBQuery.shellBatchSize=300;
var all=db.restaurants. nd() ; all' | perl -pe
's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
45 / 71

46. Dump and load using bash and MySQL Shell
$ mongo --quiet --eval 'DBQuery.shellBatchSize=30000;
var all=db.restaurants. nd() ; all'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g'
| head -n -1 | while read line
do
mysqlsh root@127.0.0.1 --password='' -e "rec=$line;
session.getSchema('docstore').getCollection('restaurants').add(rec).execute()"
done
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
46 / 71

47. Let's query
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
47 / 71

48. Let's query
That's too much records to showit here... let's limit it
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
48 / 71

49. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
49 / 71

50. Some more examples
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
50 / 71

51. Some more examples
Let's add a selection criteria:
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
51 / 71

52. Syntax slightly different than MongoDB
> db.restaurants. nd({"cuisine": "French",
"borough": { $not: /^Manhattan/} },
{"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2)
{ "borough" : "Queens", "cuisine" : "French",
"name" : "La Baraka Restaurant" }
{ "borough" : "Queens", "cuisine" : "French",
"name" : "Air France Lounge" }
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
52 / 71

53. CRUD operations
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
53 / 71

54. what about my old SQL ?
The hidden part of the iceberg
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
54 / 71

55. JSON datatype is behind the scene
native datatype (since 5.7.8)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
55 / 71

56. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
56 / 71

57. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
57 / 71

58. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
58 / 71

59. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
Generated/virtual columns
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
59 / 71

60. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
Generated/virtual columns
Indexing JSON data
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
60 / 71

61. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
Generated/virtual columns
Indexing JSON data
Foreign Keys to JSON data
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
61 / 71

62. JSON datatype is behind the scene
native datatype (since 5.7.8)
JSON values are stored in MySQL tables using UTF8MB4
Conversion from"native" SQL types to JSON values
JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH,...)
Generated/virtual columns
Indexing JSON data
Foreign Keys to JSON data
SQL Views to JSON data
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
62 / 71

63. Example
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
63 / 71

64. Example
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
64 / 71

65. Example (2)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
65 / 71

66. Example (2)
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
66 / 71

67. Example (3): explain
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
67 / 71

68. Example (4): add index
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
68 / 71

69. what's do I gain ?
Conclusion
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
69 / 71

70. schemaless
flexible data structure
easy to start (CRUD)
Conclusion
This is the best of the two worlds in one product !
Data integrity
ACID Compliant
SQL
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
70 / 71

71. Thank you !
Any Questions ?
Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.
71 / 71