Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PyConX - Python & MySQL 8.0 Document Store

185 views

Published on

How to use the new MySQL 8.0 Document Store with Python.

Easy CRUD operations, JSON validations, analytics with SQL only when needed, full ACID and more !

Published in: Technology
  • Be the first to comment

PyConX - Python & MySQL 8.0 Document Store

  1. 1. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 1/104 1 / 104
  2. 2. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 2/104 2 / 1042 / 104
  3. 3. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 3/104   Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purpose only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied up in making purchasing decisions. The development, release and timing of any features or functionality described for Oracle´s product remains at the sole discretion of Oracle. Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 3 / 104
  4. 4. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 4/104 about me - http://about.me/lefred Who am I ? Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 4 / 104
  5. 5. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 5/104 Frédéric Descamps @lefred MySQL Evangelist Hacking MySQL since 3.23 devops believer living in Belgium 🇧🇪 http://lefred.be   Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 5 / 104
  6. 6. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 6/104 MySQL and JSON How MySQL works with JSON Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 6 / 104
  7. 7. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 7/104 JavaScript Object Notation (JSON) { "_id": "5ad5b645f88c5bb8fe3fd337", "name": "Morris Park Bake Shop", "grades": [ { "date": "2014-03-03T00:00:00Z", "grade": "A", "score": 2 }, ], "address": { "coord": [ -73.856077, 40.848447 ], "street": "Morris Park Ave", "zipcode": "10462", "building": "1007" }, "borough": "Bronx", "cuisine": "Bakery", "restaurant_id": "30075445" } Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 7 / 104
  8. 8. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 8/104 8.0 MySQL has Native JSON Support JSON data type since MySQL 5.7 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 8 / 104
  9. 9. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 9/104 8.0 MySQL has Native JSON Support JSON data type since MySQL 5.7 Stored as binary object Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 9 / 104
  10. 10. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 10/104 8.0 MySQL has Native JSON Support JSON data type since MySQL 5.7 Stored as binary object Support for partial updates in MySQL 8.0 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 10 / 104
  11. 11. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 11/104 8.0 MySQL has Native JSON Support JSON data type since MySQL 5.7 Stored as binary object Support for partial updates in MySQL 8.0 28 JSON functions Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 11 / 104
  12. 12. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 12/104 8.0 ... and support JSON Paths $.address.coord[*] $ the root of the document Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 12 / 104
  13. 13. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 13/104 8.0 ... and support JSON Paths $.address.coord[*] $ the root of the document . path leg separator Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 13 / 104
  14. 14. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 14/104 8.0 ... and support JSON Paths $.address.coord[*] $ the root of the document . path leg separator * wildcard: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 14 / 104
  15. 15. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 15/104 8.0 ... and support JSON Paths $.address.coord[*] $ the root of the document . path leg separator * wildcard: .* all members in the object [*] all values in the array [prefix]**suffix path beginning with prefix and ending with suffix Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 15 / 104
  16. 16. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 16/104 MySQL Document Store the Solution Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 16 / 104
  17. 17. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 17/104 Built on the MySQL JSON Data type and Proven MySQL Server Technology Provides a schema flexible JSON Document Store No SQL required No need to define all possible attributes, tables, etc. Uses new X DevAPI Can leverage generated column to extract JSON values into materialized columns that can be indexed for fast SQL searches. Document can be ~1GB It's a column in a row of a table It cannot exceed max_allowed_packet Allows use of modern programming styles No more embedded strings of SQL in your code Easy to read Also works with relational Tables Proven MySQL Technology Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 17 / 104
  18. 18. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 18/104 X Protocol Connectors Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 18 / 104
  19. 19. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 19/104 X DevAPI Developed from scratch for modern development Supports: SQL NoSQL - JSON documents NoSQL - SQL tables Uniform API across programming languages We provide connectors for C++, Java, .Net, Node.js, Python, PHP working with Communities to help them supporting it too Supported in the MySQL Shell Full ACID support Savepoints Connection pools (as of 8.0.13) Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 19 / 104
  20. 20. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 20/104 PY JS SQL > MySQL 8.0 Upgrade Checker Auto Completion & Command History Output Formats (table, json, tabs) Prompt Themes Batch Execution 5.7 8.0 JS Document Store X dev API SQL CLI InnoDB Cluster importJSON JSJS Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 20 / 104
  21. 21. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 21/104 Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 21 / 104
  22. 22. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 22/104 Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 22 / 104
  23. 23. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 23/104 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 23 / 104
  24. 24. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 24/104 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 24 / 104
  25. 25. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 25/104 Let´s query Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 25 / 104
  26. 26. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 26/104 Let´s query That´s too much records to show here... Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 26 / 104
  27. 27. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 27/104 Let´s query That´s too much records to show here... Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 27 / 104
  28. 28. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 28/104 Let´s query That´s too much records to show here... Let´s add a limit to it Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 28 / 104
  29. 29. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 29/104 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 29 / 104
  30. 30. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 30/104 Some more examples Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 30 / 104
  31. 31. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 31/104 Some more examples Let´s add a selection criteria: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 31 / 104
  32. 32. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 32/104 Using IN... Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 32 / 104
  33. 33. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 33/104 Syntax slightly different than MongoDB Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 33 / 104
  34. 34. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 34/104 Syntax slightly different than MongoDB > db.restaurants. nd({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) { "borough" : "Queens", "cuisine" : "French", "name" : "La Baraka Restaurant" } { "borough" : "Queens", "cuisine" : "French", "name" : "Air France Lounge" } Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 34 / 104
  35. 35. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 35/104 CRUD operations Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 35 / 104
  36. 36. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 36/104 CRUD operations for collections Add a document Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 36 / 104
  37. 37. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 37/104 CRUD operations for collections Add a document collection.add({ name: 'fred', age: 42 }) .add({ name: 'dave', age: 23 }) .execute() collection.add([ { name: 'dimo', age: 50 }, { name: 'kenny', age: 25 } ]).execute() Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 37 / 104
  38. 38. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 38/104 CRUD operations for collections Modify a document Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 38 / 104
  39. 39. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 39/104 CRUD operations for collections Modify a document collection.modify('name = :name') .bind('name', 'fred') .set('age', 43) .sort('name ASC') .limit(1) .execute() collection.modify('name = :name') .bind('name', 'fred') .patch({ age: 43, active: false }) .sort('name DESC') .limit(1) .execute() Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 39 / 104
  40. 40. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 40/104 CRUD operations for collections Remove a document Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 40 / 104
  41. 41. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 41/104 CRUD operations for collections Remove a document collection.remove('name = :name') .bind('name', 'fred') .sort('age ASC') .limit(1) .execute() Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 41 / 104
  42. 42. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 42/104 MySQL Document Store Objects Summary Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 42 / 104
  43. 43. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 43/104 All you need to know is here: https://dev.mysql.com/doc/x-devapi-userguide/en/crud-operations-overview.html Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 43 / 104
  44. 44. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 44/104 MySQL Connector Python The Python connector for MySQL Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 44 / 104
  45. 45. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 45/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 45 / 104
  46. 46. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 46/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 46 / 104
  47. 47. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 47/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Dual license Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 47 / 104
  48. 48. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 48/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Dual license Support MySQL Server 5.5, 5.6, 5.7 and 8.0 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 48 / 104
  49. 49. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 49/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Dual license Support MySQL Server 5.5, 5.6, 5.7 and 8.0 SQL and NoSQL support Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 49 / 104
  50. 50. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 50/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Dual license Support MySQL Server 5.5, 5.6, 5.7 and 8.0 SQL and NoSQL support Table and JSON Document support Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 50 / 104
  51. 51. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 51/104 NoSQL SQL   MySQL Connector/Python 8 GA since April 2018 Maintained by Oracle Dual license Support MySQL Server 5.5, 5.6, 5.7 and 8.0 SQL and NoSQL support Table and JSON Document support All you need to know is here: https://dev.mysql.com/doc/connector-python/en/ Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 51 / 104
  52. 52. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 52/104 MySQL Connector/Python 8 - 3 APIs Choice of Three APIs API Python Module Comment PEP249 Python Database API mysql.connector The traditional API C Extension API _mysql_connector Similar to the MySQL C API MySQL X DevAPI mysqlx New in 8.0, both SQL and NoSQL Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 52 / 104
  53. 53. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 53/104   MySQL Connect/Python and Django MySQL Connector/Python includes also a mysql.connector.django module that provides a Django back end for MySQL DATABASES = { 'default': { 'NAME': 'user_data', 'ENGINE': 'mysql.connector.django', 'USER': 'mysql_user', 'PASSWORD': 'password', 'OPTIONS': { 'autocommit': True, }, } } Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 53 / 104
  54. 54. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 54/104 MySQL Connector/Python 8 - installation easiest, use pip: use a MySQL repository for your distribution (fedora, ubuntu, debian, ...) Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 54 / 104
  55. 55. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 55/104 MySQL Connector/Python 8 - installation (2) or go to https://dev.mysql.com/downloads/connector/python/ Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 55 / 104
  56. 56. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 56/104 What time is it ? Time to code ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 56 / 104
  57. 57. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 57/104 Some coding... import mysqlx session = mysqlx.get_session( { 'host': 'localhost', 'port': 33060, 'user': 'fred', 'password': 'fred' }) db = session.get_schema('docstore') col = db.get_collection('all_recs') restaurants = col. nd("cuisine='Italian'"). elds("name, cuisine, borough") .limit(3).execute() for restaurant in restaurants.fetch_all(): print restaurant session.close() Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 57 / 104
  58. 58. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 58/104 If it was not yet obvious, you can now use MySQL without SQL ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 58 / 104
  59. 59. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 59/104 OK we have Document Store, CRUD and ACID but what makes MySQL Document Store unique ? Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 59 / 104
  60. 60. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 60/104 Challenge: list the best restaurant of each type of food and show the top 10, with the best one first !   don't forget that all these restaurants are just JSON documents Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 60 / 104
  61. 61. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 61/104 NoSQL as SQL - aggregation Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 61 / 104
  62. 62. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 62/104 NoSQL as SQL - aggregation Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 62 / 104
  63. 63. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 63/104 NoSQL as SQL - aggregation Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 63 / 104
  64. 64. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 64/104 import mysqlx session = mysqlx.get_session( { 'host': 'localhost', 'port': 33060, 'user': 'fred', 'password': 'fred' }) db = session.get_schema('docstore') col = db.get_collection('all_recs') restaurants = col. nd("cuisine='Italian'"). elds("name, cuisine, borough") .limit(3).execute() for restaurant in restaurants.fetch_all(): print restaurant result = session.sql('''WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM docstore.all_recs) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10''').execute() for restaurant in result.fetch_all(): print "%s - %s - %d " % (restaurant[0], restaurant[1], restaurant[2]) session.close() Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 64 / 104
  65. 65. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 65/104 Output Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 65 / 104
  66. 66. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 66/104 And back to JSON And of course it's possible to return the result as a JSON document: result = session.sql('''select JSON_PRETTY(JSON_ARRAYAGG(JSON_OBJECT( "name", name, "cuisine", cuisine, "score", avg_score))) from (WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM docstore.all_recs) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10) as t''').execute() row = result.fetch_one() print row[0] Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 66 / 104
  67. 67. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 67/104 And back to JSON - output Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 67 / 104
  68. 68. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 68/104 some extras... The hidden part of the iceberg Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 68 / 104
  69. 69. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 69/104 _id Every document has a unique identifier called the document ID. It can be manually assigned when adding a document or generated and assigned to the document automatically ! Since MySQL 8.0.11, the _id is generated by the server. To get the list of automatically generated IDs use the result.getGeneratedIDs() method. The _id is made of 3 parts, all hex encoded: prefix timestamp when the MySQL Server instance was started auto-increment counter Globally unique IDs Optimized for InnoDB storage Examples 00005cc17b700000000000000003 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 69 / 104
  70. 70. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 70/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 70 / 104
  71. 71. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 71/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: innodb_ ush_log_at_trx_commit = 1 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 71 / 104
  72. 72. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 72/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 72 / 104
  73. 73. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 73/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 73 / 104
  74. 74. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 74/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1 transaction_isolation = REPEATABLE-READ|READ- COMMITTED|... Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 74 / 104
  75. 75. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 75/104 Document Store Full ACID ! It relies on the proven MySQL InnoDB´s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1 transaction_isolation = REPEATABLE-READ|READ- COMMITTED|... We do care about your data ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 75 / 104
  76. 76. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 76/104 What time is it now ? Time for some more examples ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 76 / 104
  77. 77. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 77/104 More fun with The Walking Dead Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 77 / 104
  78. 78. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 78/104 Example: All Episodes of Season 1 Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 78 / 104
  79. 79. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 79/104 Example: All First Episodes of Each Season Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 79 / 104
  80. 80. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 80/104 Example: Speed Up Episode's Name Lookup Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 80 / 104
  81. 81. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 81/104 Example: Speed Up Episode's Name Lookup Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 81 / 104
  82. 82. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 82/104 Example: Speed Up Episode's Name Lookup Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 82 / 104
  83. 83. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 83/104 Example: Transaction Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 83 / 104
  84. 84. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 84/104 Example: Do more with SQL List the amount of episodes by season: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 84 / 104
  85. 85. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 85/104 Example: Do more with SQL (2) Episode statistics for each season: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 85 / 104
  86. 86. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 86/104 Example: Do more with SQL (3) Number of days between episodes: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 86 / 104
  87. 87. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 87/104 JSON Data validation It's possible to have constraints between collections (or tables): Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 87 / 104
  88. 88. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 88/104 JSON Data validation (2) Let's see in SQL how the 2 collections look like: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 88 / 104
  89. 89. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 89/104 JSON Data validation (3) Let's add one virtual column in each collection: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 89 / 104
  90. 90. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 90/104 JSON Data validation (4) Now we can index the reference and create the constraint: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 90 / 104
  91. 91. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 91/104 JSON Data validation (4) Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 91 / 104
  92. 92. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 92/104 JSON Data validation (5) And now even in pure NoSQL CRUD, we can see the data validation in action: Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 92 / 104
  93. 93. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 93/104 JSON Data validation (6) - NEW 8.0.16 Since MySQL 8.0.16, CHECK constraints are now supported ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 93 / 104
  94. 94. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 94/104 JSON Data validation (6) - NEW 8.0.16 Since MySQL 8.0.16, CHECK constraints are now supported ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 94 / 104
  95. 95. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 95/104 JSON Data validation (6) - NEW 8.0.16 Since MySQL 8.0.16, CHECK constraints are now supported ! Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 95 / 104
  96. 96. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 96/104 what do I gain ? Conclusion Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 96 / 104
  97. 97. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 97/104 schemaless flexible data structure easy to start (CRUD) Conclusion This is the best of the two worlds in one product ! Data integrity ACID Compliant Transactions SQL Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 97 / 104
  98. 98. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 98/104 98 / 104
  99. 99. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 99/104 8.0 http://lefred.be/content/top-10-reasons-for- nosql-with-mysql/ Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 99 / 104
  100. 100. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 100/104 8.0 https://www.slideshare.net/lefred.descamps/mysql- shell-the-best-dba-tool https://github.com/lefred/mysql- shell-mydba https://github.com/lefred/mysql- shell-innotop More with MySQL Shell & Python Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 100 / 104
  101. 101. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 101/104 MySQL Shell Reporting Framework - 8.0.16 Now it's also possible to create user-defined reports directly in the Shell. They can be written in Python and called the same way on every operating systems. No need for extra libraries ! More Info: https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-reporting.html https://mysql.wisborg.dk/2019/04/26/mysql-shell-8-0-16-built-in-reports/ https://mysql.wisborg.dk/2019/04/27/mysql-shell-8-0-16-user-defined-reports/ Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 101 / 104
  102. 102. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 102/104 Credits Thank you Olivier Dasini for some TV Show examples: http://dasini.net/blog/2019/04/02/mysql-json-document-store/ Thank you Jesper Wisborg Krogh for inspiration extending the MySQL Shell Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 102 / 104
  103. 103. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 103/104 103 / 104
  104. 104. 5/2/2019 Python & MySQL 8.0 Document Store file:///home/fred/ownCloud/Presentations/ORACLE/PyconX/Python e MySQL 8.0 Document Store/Python e MySQL 8.0 Document Store.html#49 104/104 Thank you ! Any Questions ? share your 💕 for MySQL on social media using @mysql #MySQL8isGreat #MySQL Copyright @ 2019 Oracle and/or its affiliates. All rights reserved. 104 / 104

×