Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to MySQL Document Store

2,508 views

Published on

30 mins to discover MySQL Document Store a Percona University in Ghent, Belgium

Published in: Technology
  • Be the first to comment

Introduction to MySQL Document Store

  1. 1. 1 / 71
  2. 2. 2 / 71
  3. 3. Percona University, Ghent, BelgiumJune 2017 MySQL Document Store How to replace a NoSQL database by MySQL without effort but with a lot of gains ?       Frédéric Descamps - MySQL Community Manager - Oracle 3 / 71
  4. 4.   Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purpose only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied up in making purchasing decisions. The development, release and timing of any features or functionality described for Oracle´s product remains at the sole discretion of Oracle. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 4 / 71
  5. 5. about.me/lefred Who am I ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 5 / 71
  6. 6. Frédéric Descamps @lefred MySQL Evangelist Hacking MySQL since 3.23 devops believer MySQL Community Manager since May 2016 living in Belgium 🇧🇪 Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 6 / 71
  7. 7. 7 / 71
  8. 8. RDBMS & MySQL Relational Databases Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 8 / 71
  9. 9. Relational Databases Data Integrity Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 9 / 71
  10. 10. Relational Databases Data Integrity normalization Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 10 / 71
  11. 11. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 11 / 71
  12. 12. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Atomicity, Consistency, Isolation, Durability Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 12 / 71
  13. 13. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Atomicity, Consistency, Isolation, Durability ACID compliant Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 13 / 71
  14. 14. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 14 / 71
  15. 15. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions SQL Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 15 / 71
  16. 16. Relational Databases Data Integrity normalization constraints (foreign keys, ...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions SQL powerfull query language Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 16 / 71
  17. 17. NoSQL & MySQL NoSQL Databases Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 17 / 71
  18. 18. NoSQL or Document Store Schemaless Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 18 / 71
  19. 19. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 19 / 71
  20. 20. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 20 / 71
  21. 21. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 21 / 71
  22. 22. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 22 / 71
  23. 23. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 23 / 71
  24. 24. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model objects persistance without the use of any ORM- mappingoobject-oriented Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 24 / 71
  25. 25. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model objects persistance without the use of any ORM- mappingoobject-oriented JSON Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 25 / 71
  26. 26. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model objects persistance without the use of any ORM- mappingoobject-oriented JSON close to frontend Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 26 / 71
  27. 27. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model objects persistance without the use of any ORM- mappingoobject-oriented JSON close to frontend native in JS Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 27 / 71
  28. 28. NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types, ... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can't be modelized optimaly into a relational model objects persistance without the use of any ORM- mappingoobject-oriented JSON close to frontend native in JS easy to learn Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 28 / 71
  29. 29. How DBAs see data How Developers see data { "GNP" : 249704, "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : 10239000, "LifeExpectancy" : 77.8000030517578 }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" } } Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 29 / 71
  30. 30. DMBS or NoSQL ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 30 / 71
  31. 31. DMBS or NoSQL ? Why not both ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 31 / 71
  32. 32. MySQL as Document Store Requirements Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 32 / 71
  33. 33. MySQL as Document Store: requirements Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 33 / 71
  34. 34. MySQL as Document Store: requirements 1.support of JSON datatype Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 34 / 71
  35. 35. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 35 / 71
  36. 36. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 36 / 71
  37. 37. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin 4.X-Protocol Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 37 / 71
  38. 38. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin 4.X-Protocol extends and replaces the traditional MySQL protocol Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 38 / 71
  39. 39. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin 4.X-Protocol extends and replaces the traditional MySQL protocol supports asynchronous communication to the server Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 39 / 71
  40. 40. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin 4.X-Protocol extends and replaces the traditional MySQL protocol supports asynchronous communication to the server supports multiple API calls like Javascript, Python, SQL... Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 40 / 71
  41. 41. MySQL as Document Store: requirements 1.support of JSON datatype 2.CRUD Operations 3.X-Plugin 4.X-Protocol extends and replaces the traditional MySQL protocol supports asynchronous communication to the server supports multiple API calls like Javascript, Python, SQL... 5.MySQL Shell Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 41 / 71
  42. 42. Starting using MySQL DS in few minutes Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 42 / 71
  43. 43. Starting using MySQL DS in few minutes This is the newMySQL shell as it will be in 8 ;-) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 43 / 71
  44. 44. Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection: We have two options: 1.We need to dump the data to a file and we will use MySQL Shell with the Python interpreter to load the data. 1.We retrieve the data fromMongoDB and load it to MySQL via the Shell in javascript Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 44 / 71
  45. 45. Dump and load using MySQL Shell & Python thisexampleisthecourtesyof@datacharmer: https://www.slideshare.net/datacharmer/mysql-documentstore $ mongo --quiet --eval 'DBQuery.shellBatchSize=300; var all=db.restaurants. nd() ; all' | perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 45 / 71
  46. 46. Dump and load using bash and MySQL Shell   $ mongo --quiet --eval 'DBQuery.shellBatchSize=30000; var all=db.restaurants. nd() ; all' | perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' | head -n -1 | while read line do mysqlsh root@127.0.0.1 --password='' -e "rec=$line; session.getSchema('docstore').getCollection('restaurants').add(rec).execute()" done Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 46 / 71
  47. 47. Let's query Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 47 / 71
  48. 48. Let's query That's too much records to showit here... let's limit it Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 48 / 71
  49. 49. Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 49 / 71
  50. 50. Some more examples Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 50 / 71
  51. 51. Some more examples Let's add a selection criteria: Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 51 / 71
  52. 52. Syntax slightly different than MongoDB > db.restaurants. nd({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) { "borough" : "Queens", "cuisine" : "French", "name" : "La Baraka Restaurant" } { "borough" : "Queens", "cuisine" : "French", "name" : "Air France Lounge" } Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 52 / 71
  53. 53. CRUD operations Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 53 / 71
  54. 54. what about my old SQL ? The hidden part of the iceberg Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 54 / 71
  55. 55. JSON datatype is behind the scene native datatype (since 5.7.8) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 55 / 71
  56. 56. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 56 / 71
  57. 57. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 57 / 71
  58. 58. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,...) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 58 / 71
  59. 59. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,...) Generated/virtual columns Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 59 / 71
  60. 60. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,...) Generated/virtual columns Indexing JSON data Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 60 / 71
  61. 61. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,...) Generated/virtual columns Indexing JSON data Foreign Keys to JSON data Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 61 / 71
  62. 62. JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from"native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH,...) Generated/virtual columns Indexing JSON data Foreign Keys to JSON data SQL Views to JSON data Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 62 / 71
  63. 63. Example Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 63 / 71
  64. 64. Example Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 64 / 71
  65. 65. Example (2) Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 65 / 71
  66. 66. Example (2)   Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 66 / 71
  67. 67. Example (3): explain Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 67 / 71
  68. 68. Example (4): add index Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 68 / 71
  69. 69. what's do I gain ? Conclusion Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 69 / 71
  70. 70. schemaless flexible data structure easy to start (CRUD) Conclusion This is the best of the two worlds in one product ! Data integrity ACID Compliant SQL Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 70 / 71
  71. 71. Thank you ! Any Questions ? Copyright @ 2017 Oracle and/or its affiliates. All rights reserved. 71 / 71

×