Successfully reported this slideshow.
Your SlideShare is downloading. ×

MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 82 Ad

MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019

Download to read offline

MySQL an be used as a NoSQL JSON Document Store as well as its well known ability as a SQL Relational Data Base. This presentation covers why you would want to use NoSQL and JSON and how to combine it what the relational data you already have

MySQL an be used as a NoSQL JSON Document Store as well as its well known ability as a SQL Relational Data Base. This presentation covers why you would want to use NoSQL and JSON and how to combine it what the relational data you already have

Advertisement
Advertisement

More Related Content

Similar to MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019 (20)

Advertisement

More from Dave Stokes (20)

Advertisement

MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019

  1. 1. MySQL Document Store - A NoSQL JSON Document Database Dave Stokes @stoker david.stokes@oracle.com Elephantdolphin.blogger.com OpensourceDBA.wordpress.com
  2. 2. MySQL Without the SQL - Oh My! Dave Stokes @stoker david.stokes@oracle.com Elephantdolphin.blogger.com OpensourceDBA.wordpress.com
  3. 3. 3
  4. 4. Safe Harbor Agreement THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE. 4
  5. 5. MySQL Community Edition 5
  6. 6. NoSQL & SQL 6 Together! MySQL Document Store
  7. 7. Relational Databases 7
  8. 8. Relational Databases ● Data Integrity ○ Normalization ○ constraints (foreign keys, ...) ● Atomicity, Consistency, Isolation, Durability ○ ACID compliant ○ transactions ● SQL ○ powerful query language 8
  9. 9. Relational Databases ● Need to set up tables BEFORE use ● Relations, indexes, data normalization, query optimizations ● Hard to change on the fly ● Need a DBA or someone who has DBA skills 9
  10. 10. NoSQL or Document Store 10
  11. 11. NoSQL or Document Store ● Schemaless ○ No schema design, no normalization, no foreign keys, no data types, … ○ Very quick initial development ● Flexible data structure ○ Embedded arrays or objects ○ Valid solution when natural data can not be modelized optimally into a relational model ○ Objects persistence without the use of any ORM - *mapping object- oriented* 11
  12. 12. NoSQL or Document Store ● JSON ● close to frontend ● native in JS ● easy to learn 12
  13. 13. How DBAs see data as opposed to how Developers see data { "GNP" : 249704, "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : 10239000, "LifeExpectancy" : 77.8000030517578 }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" } } 13
  14. 14. What if there was a way to provide both SQL and NoSQL on one stable platform that has proven stability on well know technology with a large Community and a diverse ecosystem ? With the MySQL Document Store it is now an option! 14
  15. 15. A Solution for all Developers: schemaless ★ rapid prototyping & simpler APIs ★ document model ★ transactions Operations: ★ performance management/visibility ★ robust replication, backup, restore ★ comprehensive tooling ecosystem ★ simpler application schema upgrades 15 Business Owner: ★ don't lose my data == ACID trx ★ capture all my data = extensible/schemaless ★ product on schedule/time to market = rapid development
  16. 16. Built on the MySQL JSON Data type and Proven MySQL Server Technology 16 ★ Provides a schema flexible JSON Document Store ★ No SQL required ★ No need to define all possible attributes, tables, etc. ★ Uses new X DevAPI ★ Can leverage generated column to extract JSON values into materialized columns that can be indexed for fast SQL searches.
  17. 17. Built on the MySQL JSON Data type and Proven MySQL Server Technology 17 ★ Document can be ~1GB ○ It's a column in a row of a table ★ Allows use of modern programming styles ○ No more embedded strings of SQL in your code ○ Easy to read ★ Also works with relational Tables ★ Proven MySQL Technology
  18. 18. ★ C++ ★ Java ★ .Net ★ Node.js ★ JavaScript ★ Python ★ PHP ○ Working with other Communities to help them supporting it too 18 Connectors for
  19. 19. ★ Command Completion ★ Python, JavaScripts & SQL modes ★ Admin functions ★ New Util object ★ A new high-level session concept that can scale from single MySQL Server to a multiple server environment 19 New MySQL Shell
  20. 20. ★ Non-blocking, asynchronous calls follow common language patterns ★ Supports CRUD operations 20 New Model
  21. 21. 21 X Protocol built on Google Protobufs
  22. 22. 22 Architecture of both Old and New Protocols
  23. 23. 23 How Your Application will work with InnoDB Cluster
  24. 24. But what does this look like in PHP?? 24
  25. 25. JavaScript 25 // Connecting to MySQL Server and working with a Collection var mysqlx = require('mysqlx'); // Connect to server var mySession = mysqlx.getSession( { host: 'localhost', port: 33060, user: 'user', password: 'password'} ); var myDb = mySession.getSchema('test'); // Create a new collection 'my_collection' var myColl = myDb.createCollection('my_collection'); // Insert documents myColl.add({_id: '1', name: 'Sakila', age: 15}).execute(); myColl.add({_id: '2', name: 'Susanne', age: 24}).execute(); myColl.add({_id: '3', name: 'User', age: 39}).execute(); // Find a document var docs = myColl.find('name like :param1 AND age < :param2').limit(1). bind('param1','S%').bind('param2',20).execute(); // Print document print(docs.fetchOne()); // Drop the collection myDb.dropCollection('my_collection');
  26. 26. Python 26 # Connecting to MySQL Server and working with a Collection from mysqlsh import mysqlx # Connect to server mySession = mysqlx.get_session( { 'host': 'localhost', 'port': 33060, 'user': 'user', 'password': 'password'} ) myDb = mySession.get_schema('test') # Create a new collection 'my_collection' myColl = myDb.create_collection('my_collection') # Insert documents myColl.add({'_id': '1', 'name': 'Sakila', 'age': 15}).execute() myColl.add({'_id': '2', 'name': 'Susanne', 'age': 24}).execute() myColl.add({'_id': '3', 'name': 'User', 'age': 39}).execute() # Find a document docs = myColl.find('name like :param1 AND age < :param2') .limit(1) .bind('param1','S%') .bind('param2',20) .execute() # Print document doc = docs.fetch_one() print doc
  27. 27. Node.JS 27 // Connecting to MySQL Server and working with a Collection var mysqlx = require('@mysql/xdevapi'); var db; // Connect to server mysqlx .getSession({ user: 'user', password: 'password', host: 'localhost', port: '33060', }) .then(function (session) { db = session.getSchema('test'); // Create a new collection 'my_collection' return db.createCollection('my_collection'); }) .then(function (myColl) { // Insert documents return Promise .all([ myColl.add({ name: 'Sakila', age: 15 }).execute(), myColl.add({ name: 'Susanne', age: 24 }).execute(), myColl.add({ name: 'User', age: 39 }).execute() ]) .then(function () { // Find a document return myColl .find('name like :name && age < :age') .bind({ name: 'S%', age: 20 }) .limit(1) .execute(function (doc) { // Print document console.log(doc); }); }); }) .then(function(docs) { // Drop the collection return db.dropCollection('my_collection'); }) .catch(function(err) { // Handle error });
  28. 28. C++ 28 // Connect to server var mySession = MySQLX.GetSession("server=localhost;port=33060;user=user;password=password;"); var myDb = mySession.GetSchema("test"); // Create a new collection "my_collection" var myColl = myDb.CreateCollection("my_collection"); // Insert documents myColl.Add(new { name = "Sakila", age = 15}).Execute(); myColl.Add(new { name = "Susanne", age = 24}).Execute(); myColl.Add(new { name = "User", age = 39}).Execute(); // Find a document var docs = myColl.Find("name like :param1 AND age < :param2").Limit(1) .Bind("param1", "S%").Bind("param2", 20).Execute(); // Print document Console.WriteLine(docs.FetchOne()); // Drop the collection myDb.DropCollection("my_collection");
  29. 29. Java 29 // Connect to server Session mySession = new SessionFactory().getSession("mysqlx://localhost:33060/test?user=user&password=password"); Schema myDb = mySession.getSchema("test"); // Create a new collection 'my_collection' Collection myColl = myDb.createCollection("my_collection"); // Insert documents myColl.add("{"name":"Sakila", "age":15}").execute(); myColl.add("{"name":"Susanne", "age":24}").execute(); myColl.add("{"name":"User", "age":39}").execute(); // Find a document DocResult docs = myColl.find("name like :name AND age < :age") .bind("name", "S%").bind("age", 20).execute(); // Print document DbDoc doc = docs.fetchOne(); System.out.println(doc); // Drop the collection myDB.dropCollection("test", "my_collection");
  30. 30. 30 New Shell
  31. 31. Starting using MySQL in few minutes 31
  32. 32. Quickly add a document 32
  33. 33. Find that document 33
  34. 34. Fast modifications 34
  35. 35. Shell info 35
  36. 36. For this example, I will use the well known restaurants collection: We need to dump the data to a file and we will use the MySQL Shell with the Python interpreter to load the data. Migration from MongoDB to MySQL Document Store 36
  37. 37. Dump and load using MySQL Shell & Python This example is inspired by @datacharmer's work: https://www.slideshare.net/datacharmer/mysql-documentstore $ mongo quiet eval 'DBQuery.shellBatchSize=30000; db.restaurants.find().shellPrint()' | perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json 37
  38. 38. Or use new bulk loader in 8.0.13 38
  39. 39. BSON Support Now, it supports the conversion of the following additional BSON types: ■ Date ■ Timestamp ■ NumberDecimal ■ NumberLong ■ NumberInt ■ Regular Expression ■ Binary 39 > util.importJson("/path_to_file/neighborhoods_mongo.json", {schema: "test", collection: "neighborhoods", convertBsonTypes: true});
  40. 40. 40
  41. 41. 41 Let’s query Too many records to show here … let’s limit it! restaurants.find().limit(1)
  42. 42. 42 More Examples! restaurants.find().fields([“name”,”cuisine”]).limit(2 )
  43. 43. 43 Comparing Syntax: MongoDB vs MYSQL MongoDB: > db.restaurants.find({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) MySQL: >restaurants.find(“cuisine=’French’ AND borough!=’Manhattan’”).fields([“name”,”cuisine”,”borough” ]).limit(2)
  44. 44. 44 CRUD Operations
  45. 45. 45 Add a Document
  46. 46. 46 Modify a Document
  47. 47. 47 Remove a Document
  48. 48. 48 Find a Document
  49. 49. 49 MySQL Document Store Objects Summary
  50. 50. MySQL Document Store is Fully ACID Compliant 50
  51. 51. MySQL Document Store is Fully ACID Compliant 51
  52. 52. What about old SQL? The Hidden Part of the Iceberg 52
  53. 53. ★ Native data type (since 5.7.8) ★ JSON values are stored in MySQL tables using UTF8MB4 ★ Conversion from "native" SQL types to JSON values ★ JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES, ...) ★ Generated/virtual columns ○ Indexing JSON data ○ Foreign Keys to JSON data ○ SQL Views to JSON data JSON datatype is behind the scene 53
  54. 54. How Does It Work?? 54
  55. 55. What does a collection look like on the server ? 55
  56. 56. Every document has a unique identifier called the document ID, which can be thought of as the equivalent of a table's primary key. The document ID value can be manually assigned when adding a document. If no value is assigned, a document ID is generated and assigned to the document automatically ! Use getDocumentId() or getDocumentIds() to get _ids(s) _id 56
  57. 57. Mapping to SQL Examples createCollection('mycollection') versus CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4; 57
  58. 58. Mapping to SQL Examples mycollection.add({‘test’: 1234}) versus INSERT INTO `test`.`mycoll` (doc) VALUES ( JSON_OBJECT( 'test',1234)); 58
  59. 59. More Mapping to SQL Examples mycollection.find("test > 100") Versus SELECT doc FROM `test`.`mycoll` WHERE (JSON_EXTRACT(doc,'$.test') >100); 59
  60. 60. 60 SQL and JSON Example
  61. 61. It's also possible to create indexes without using SQL syntax 61
  62. 62. SQL and JSON Example (2): validation 62
  63. 63. SQL and JSON Example (3): explain 63
  64. 64. SQL and JSON Example (3): explain 64
  65. 65. SQL and JSON Example (4): add index 65
  66. 66. SQL and JSON Example (4): add index 66
  67. 67. [ { "date": { "$date": 1416009600000 }, "grade": "Z", "score": 38 }, { "date": { "$date": 1398988800000 }, "grade": "A", "score": 10 }, { "date": { "$date": 1362182400000 }, "grade": "A", "score": 7 }, { "date": { "$date": 1328832000000 }, "grade": "A", "score": 13 } ] 67 Embedded Arrays of values can be messy to traverse.
  68. 68. SQL and JSON Example (5): arrays 68
  69. 69. $.grades[0] $.greades[1 to 2] $.grades[first] $.grades[last] $.grades[first to last - 1] 69 Arrays are now simple
  70. 70. NoSQL as SQL 70 JSON_TABLE turns your un- structured JSON data into a temporary structured table!
  71. 71. NoSQL as SQL 71 This temporary structured table can be treated like any other table -- LIMIT, WHERE, GROUP BY ...
  72. 72. 72 More Sophisticated Analysis
  73. 73. Find the top 10 restaurants by grade for each cuisine 73 WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM restaurants) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10; This query uses a Common Table Expression (CTE) and a Windowing Function to rank the average scores of each restaurant, by each cuisine assembled in a JSON_TABLE
  74. 74. Conclusion: What Do I Gain? 74
  75. 75. This is the best of the two worlds in one product ! ● Data integrity ● ACID Compliant ● Transactions ● SQL ● Schemaless ● flexible data structure ● easy to start (CRUD) 75
  76. 76. Mutable Data!! Reduce Many to many joins Replace ‘stub’ tables Change on the fly, aggregate new data 76
  77. 77. Non JSON Data Transforms to JSON 77
  78. 78. And JSON to Relational SELECT country_name, IndyYear FROM countryinfo, JSON_TABLE(doc,"$" COLUMNS (country_name CHAR(60) PATH "$.Name",IndyYear INT PATH "$.IndepYear")) AS stuff WHERE IndyYear > 1992; +----------------+----------+ | country_name | IndyYear | +----------------+----------+ | Czech Republic | 1993 | | Eritrea | 1993 | | Palau | 1994 | | Slovakia | 1993 | 78
  79. 79. GeoJSON support too! mysql> SELECT ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2); +-------------------------------------------------------------+ | ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2) | +-------------------------------------------------------------+ | {"type": "Point", "coordinates": [11.11, 12.22]} | +-------------------------------------------------------------+ 79
  80. 80. New in MySQL 8.0 1. True Data Dictionary 2. Default UTF8MB4 3. Windowing Functions, CTEs, Lateral Derived Joins 4. InnoDB SKIPPED LOCK and NOWAIT 5. Instant Add Column 6. Histograms 7. Resource Groups 8. Better optimizer with new temporary table engine 9. True Descending Indexes 10.3D GIS 11.JSON Enhancements 80
  81. 81. Please buy my book! If you deal with the JSON Data Type or have an interest in the MySQL Document Store, this text is a great guide with many examples to help you understand the complexities and opportunities with a native JSON Data Type 81
  82. 82. Thanks! Contact info: Dave Stokes David.Stokes@Oracle.com @Stoker slideshare.net/davidmstokes Elepantdolphin.blogger.com opensourcedba.Wordpress.com 82

Editor's Notes

  • What is new in MySQL 8.0

×