DS
Uploaded byDave Stokes
PDF, PPTX234 views

Datacon LA - MySQL without the SQL - Oh my!

The document discusses MySQL's transition to a document store functionality that integrates NoSQL capabilities with traditional SQL operations, providing a flexible and schemaless data structure using JSON. It highlights benefits for developers, including rapid prototyping and simplified APIs, while ensuring ACID compliance for business users. It also explains the new functionalities and coding examples across various programming languages for working with the MySQL document store.

Embed presentation

Download as PDF, PPTX
MySQL Without the SQL Oh My! Dave Stokes @stoker david.stokes@oracle.com Elephantdolphin.blogger.com Slides -> slideshare.net/DaveStokes
2
Safe Harbor Agreement THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE. 3
MySQL Community Edition 4
NoSQL & SQL 5 Together! MySQL Document Store
Relational Databases 6
Relational Databases ● Original Goal was to save data with minimal duplication ● Disks were expensive ● … and slow ● 45 years old ● Introduced the concept of accessing many records with a single command 7
Relational Databases ● Data Integrity ○ Normalization ○ constraints (foreign keys, ...) ● Atomicity, Consistency, Isolation, Durability ○ ACID compliant ○ transactions ● SQL ○ powerful query language 8
Relational Databases ● Need to set up tables BEFORE use ● Relations, indexes, data normalization, query optimizations ● Hard to change on the fly ● Need a DBA or someone who has DBA skills ● This can be a chokepoint 9
NoSQL or Document Store 10
NoSQL or Document Store ● Schemaless ○ No schema design, no normalization, no foreign keys, no data types, … ○ Very quick initial development ● Flexible data structure ○ Embedded arrays or objects ○ Valid solution when natural data can not be modeled optimally into a relational model ○ Objects persistence without the use of any ORM - *mapping object-oriented* 11
NoSQL or JSON Document Store ● JSON ● close to frontend ● native in JS ● easy to learn 12
How DBAs see data as opposed to how Developers see data { "GNP" : 249704, "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : 10239000, "LifeExpectancy" : 77.8000030517578 }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" } } 13
What if there was a way to provide both SQL and NoSQL on one stable platform that has proven stability on well know technology with a large Community and a diverse ecosystem ? With the MySQL Document Store it is now an option! 14
A Solution for all Developers: schemaless ★ rapid prototyping & simpler APIs ★ document model ★ transactions Operations: ★ performance management/visibility ★ robust replication, backup, restore ★ comprehensive tooling ecosystem ★ simpler application schema upgrades 15 Business Owner: ★ don't lose my data == ACID trx ★ capture all my data = extensible/schemaless ★ product on schedule/time to market = rapid development
Built on the MySQL JSON Data type and Proven MySQL Server Technology 16 ★ Provides a schema flexible JSON Document Store ★ No SQL required ★ No need to define all possible attributes, tables, etc. ★ Uses new MySQL X DevAPI ★ Can leverage generated column to extract JSON values into materialized columns that can be indexed for fast SQL searches.
Built on the MySQL JSON Data type and Proven MySQL Server Technology 17 ★ Document can be ~1GB ○ It's a column in a row of a table ★ Allows use of modern programming styles ○ No more embedded strings of SQL in your code ○ Easy to read ★ Also works with relational Tables ★ Proven MySQL Technology
★ C++ ★ Java ★ .Net ★ Node.js ★ JavaScript ★ Python ★ PHP ○ Working with other Communities to help them support it too 18 Connectors for
★ Command Completion ★ Python, JavaScripts & SQL modes ★ Admin functions ★ New Util object ★ A new high-level session concept that can scale from single MySQL Server to a multiple server environment 19 New MySQL Shell
★ Non-blocking, asynchronous calls follow common language patterns ★ Send out many queries and process other things until they return ★ Supports CRUD operations ★ Concentrate on basic functions ★ Easily scale from one server to InnoDB cluster w/o changing application! 20 New Model
21 X Protocol built on Google Protobufs Not an ORM!!
22 Architecture of both Old and New Protocols
23 How Your Application will work with InnoDB Cluster
But what does this look like in PHP?? 24
JavaScript 25 // Connecting to MySQL Server and working with a Collection var mysqlx = require('mysqlx'); // Connect to server var mySession = mysqlx.getSession( { host: 'localhost', port: 33060, user: 'user', password: 'password'} ); var myDb = mySession.getSchema('test'); // Create a new collection 'my_collection' var myColl = myDb.createCollection('my_collection'); // Insert documents myColl.add({_id: '1', name: 'Sakila', age: 15}).execute(); myColl.add({_id: '2', name: 'Susanne', age: 24}).execute(); myColl.add({_id: '3', name: 'User', age: 39}).execute(); // Find a document var docs = myColl.find('name like :param1 AND age < :param2').limit(1). bind('param1','S%').bind('param2',20).execute(); // Print document print(docs.fetchOne()); // Drop the collection myDb.dropCollection('my_collection'); No SQL!!
Python 26 # Connecting to MySQL Server and working with a Collection from mysqlsh import mysqlx # Connect to server mySession = mysqlx.get_session( { 'host': 'localhost', 'port': 33060, 'user': 'user', 'password': 'password'} ) myDb = mySession.get_schema('test') # Create a new collection 'my_collection' myColl = myDb.create_collection('my_collection') # Insert documents myColl.add({'_id': '1', 'name': 'Sakila', 'age': 15}).execute() myColl.add({'_id': '2', 'name': 'Susanne', 'age': 24}).execute() myColl.add({'_id': '3', 'name': 'User', 'age': 39}).execute() # Find a document docs = myColl.find('name like :param1 AND age < :param2') .limit(1) .bind('param1','S%') .bind('param2',20) .execute() # Print document doc = docs.fetch_one() print doc
Node.JS 27 // Connecting to MySQL Server and working with a Collection var mysqlx = require('@mysql/xdevapi'); var db; // Connect to server mysqlx .getSession({ user: 'user', password: 'password', host: 'localhost', port: '33060', }) .then(function (session) { db = session.getSchema('test'); // Create a new collection 'my_collection' return db.createCollection('my_collection'); }) .then(function (myColl) { // Insert documents return Promise .all([ myColl.add({ name: 'Sakila', age: 15 }).execute(), myColl.add({ name: 'Susanne', age: 24 }).execute(), myColl.add({ name: 'User', age: 39 }).execute() ]) .then(function () { // Find a document return myColl .find('name like :name && age < :age') .bind({ name: 'S%', age: 20 }) .limit(1) .execute(function (doc) { // Print document console.log(doc); }); }); }) .then(function(docs) { // Drop the collection return db.dropCollection('my_collection'); }) .catch(function(err) { // Handle error });
C++ 28 // Connect to server var mySession = MySQLX.GetSession("server=localhost;port=33060;user=user;password=password;"); var myDb = mySession.GetSchema("test"); // Create a new collection "my_collection" var myColl = myDb.CreateCollection("my_collection"); // Insert documents myColl.Add(new { name = "Sakila", age = 15}).Execute(); myColl.Add(new { name = "Susanne", age = 24}).Execute(); myColl.Add(new { name = "User", age = 39}).Execute(); // Find a document var docs = myColl.Find("name like :param1 AND age < :param2").Limit(1) .Bind("param1", "S%").Bind("param2", 20).Execute(); // Print document Console.WriteLine(docs.FetchOne()); // Drop the collection myDb.DropCollection("my_collection");
Java 29 // Connect to server Session mySession = new SessionFactory().getSession("mysqlx://localhost:33060/test?user=user&password=password"); Schema myDb = mySession.getSchema("test"); // Create a new collection 'my_collection' Collection myColl = myDb.createCollection("my_collection"); // Insert documents myColl.add("{"name":"Sakila", "age":15}").execute(); myColl.add("{"name":"Susanne", "age":24}").execute(); myColl.add("{"name":"User", "age":39}").execute(); // Find a document DocResult docs = myColl.find("name like :name AND age < :age") .bind("name", "S%").bind("age", 20).execute(); // Print document DbDoc doc = docs.fetchOne(); System.out.println(doc); // Drop the collection myDB.dropCollection("test", "my_collection");
30 New Shell
Starting using MySQL in few minutes 31
Quickly add a document 32
Find that document 33
Fast modifications 34
Shell info 35
For this example, I will use the well known restaurants collection: We need to dump the data to a file and we will use the MySQL Shell with the Python interpreter to load the data. Migration from MongoDB to MySQL Document Store 36
Dump and load using MySQL Shell & Python This example is inspired by @datacharmer's work: https://www.slideshare.net/datacharmer/mysql-documentstore $ mongo quiet eval 'DBQuery.shellBatchSize=30000; db.restaurants.find().shellPrint()' | perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json 37
Or use new bulk loader in 8.0.13 38Parallel import introduced in 8.0.17
BSON Support Now, it supports the conversion of the following additional BSON types: ■ Date ■ Timestamp ■ NumberDecimal ■ NumberLong ■ NumberInt ■ Regular Expression ■ Binary 39 > util.importJson("/path_to_file/neighborhoods_mongo.json", {schema: "test", collection: "neighborhoods", convertBsonTypes: true});
40
41 Let’s query Too many records to show here … let’s limit it! restaurants.find().limit(1)
42 More Examples! restaurants.find().fields([“name”,”cuisine”]).limit(2)
43 Comparing Syntax: MongoDB vs MYSQL MongoDB: > db.restaurants.find({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) MySQL: >restaurants.find(“cuisine=’French’ AND borough!=’Manhattan’”).fields([“name”,”cuisine”,”borough” ]).limit(2)
44 CRUD Operations
45 Add a Document
46 Modify a Document
47 Remove a Document
48 Find a Document
49 MySQL Document Store Objects Summary
MySQL Document Store is Fully ACID Compliant 50
MySQL Document Store is Fully ACID Compliant 51
How Does It Work?? 52
What does a collection look like on the server ? 53
Every document has a unique identifier called the document ID, which can be thought of as the equivalent of a table's primary key. The document ID value can be manually assigned when adding a document. If no value is assigned, a document ID is generated and assigned to the document automatically ! Use getDocumentId() or getDocumentIds() to get _ids(s) _id 54
Mapping to SQL Examples createCollection('mycollection') versus CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4; 55
Mapping to SQL Examples mycollection.add({‘test’: 1234}) versus INSERT INTO `test`.`mycoll` (doc) VALUES ( JSON_OBJECT( 'test',1234)); 56
More Mapping to SQL Examples mycollection.find("test > 100") Versus SELECT doc FROM `test`.`mycoll` WHERE (JSON_EXTRACT(doc,'$.test') >100); 57
58 SQL and JSON Example
It's also possible to create indexes without using SQL syntax 59
SQL and JSON Example (3): explain 60
SQL and JSON Example (3): explain 61
SQL and JSON Example (4): add index 62
SQL and JSON Example (4): add index 63
[ { "date": { "$date": 1416009600000 }, "grade": "Z", "score": 38 }, { "date": { "$date": 1398988800000 }, "grade": "A", "score": 10 }, { "date": { "$date": 1362182400000 }, "grade": "A", "score": 7 }, { "date": { "$date": 1328832000000 }, "grade": "A", "score": 13 } ] 64 Embedded Arrays of values can be messy to traverse.
SQL and JSON Example (5): arrays 65
$.grades[0] $.grades[1 to 2] $.grades[first] $.grades[last] $.grades[first to last - 1] 66 Arrays are now simple
NoSQL as SQL 67 JSON_TABLE turns your un-structured JSON data into a temporary structured table!
NoSQL as SQL 68 This temporary structured table can be treated like any other table -- LIMIT, WHERE, GROUP BY ...
69 More Sophisticated Analysis
Find the top 10 restaurants by grade for each cuisine 70 WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM restaurants) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10; This query uses a Common Table Expression (CTE) and a Windowing Function to rank the average scores of each restaurant, by each cuisine assembled in a JSON_TABLE
No SQL Consumed In This Query!! 71 $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll();
JSON Validation 72
JSON Validation The Problem Unlike strictly types relational databases there is no data normalization or ‘rigor’ applied to that data. There is also no native way to do range checks And there is no way have required fields 73
JSON-Schema.org The Problem Unlike strictly types relational databases there is no data normalization or ‘rigor’ applied to that data. There is also no native way to do range checks And there is no way have required fields JSON-Shema.org is work to fix that 74
JSON Validation set @s='{"type": "object", "properties": { "myage": { "type" : "number", "minimum": 28, "maximum": 99 } } }'; set @d='{ "myage": 33}'; select JSON_SCHEMA_VALID(@s,@d); +--------------------------+ | JSON_SCHEMA_VALID(@s,@d) | +--------------------------+ | 1 | +--------------------------+ 1 row in set (0.00 sec) 75
JSON Validation Report select JSON_PRETTY(JSON_SCHEMA_VALIDATION_REPORT(@s,@d))G *************************** 1. row *************************** JSON_PRETTY(JSON_SCHEMA_VALIDATION_REPORT(@s,@d)): { "valid": false, "reason": "The JSON document location '#/myage' failed requirement 'minimum' at JSON Schema location '#/properties/myage'", "schema-location": "#/properties/myage", "document-location": "#/myage", "schema-failed-keyword": "minimum" } 76
JSON Check Constraint CREATE TABLE `testx` ( `col` JSON, CONSTRAINT `myage_inRange` CHECK (JSON_SCHEMA_VALID('{"type": "object", "properties": { "myage": { "type" : "number", "minimum": 28, "maximum": 99 } },"required": ["myage"] }', `col`) = 1) ); 77
JSON Check Constraint mysql> insert into testx values('{"myage":27}'); ERROR 3819 (HY000): Check constraint 'myage_inRange' is violated. mysql> insert into testx values('{"myage":97}'); Query OK, 1 row affected (0.02 sec) 78
Multi Value Indexes 79
Index JSON Arrays { "user":"Bob", "user_id":31, "zipcode":[94477,94536] } CREATE TABLE customers ( id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, modified DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, custinfo JSON, INDEX zips( (CAST(custinfo->'$.zip' AS UNSIGNED ARRAY)) ) ); Mutli Value Indexes allow you to go past the 1:1 relation to index the data in JSON arrays. And there are three special functions what can make use of MVIs when used on the right side of a WHERE clause -- MEMBER OF(), JSON_CONTAINS(), and JSON_OVERLAPS() 80
Conclusion: What Do I Gain? 81
This is the best of the two worlds in one product ! ● Data integrity ● ACID Compliant ● Transactions ● SQL ● Schemaless ● flexible data structure ● easy to start (CRUD) 82
Mutable Data!! Reduce Many to many joins Replace ‘stub’ tables Change on the fly, aggregate new data 83
Non JSON Data Transforms to JSON 84
GeoJSON support too! mysql> SELECT ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2); +-------------------------------------------------------------+ | ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2) | +-------------------------------------------------------------+ | {"type": "Point", "coordinates": [11.11, 12.22]} | +-------------------------------------------------------------+ 85
New in MySQL 8.0 1. True Data Dictionary 2. Default UTF8MB4 3. Windowing Functions, CTEs, Lateral Derived Joins 4. InnoDB SKIPPED LOCK and NOWAIT 5. Instant Add Column 6. Histograms 7. Resource Groups 8. Better optimizer with new temporary table engine 9. True Descending Indexes 10. 3D GIS 11. JSON Enhancements 86
Please buy my book! If you deal with the JSON Data Type or have an interest in the MySQL Document Store, this text is a great guide with many examples to help you understand the complexities and opportunities with a native JSON Data Type – SECOND EDITION now Available on 87
Test Drive MySQL Database Service For Free Today Get $300 in credits and try MySQL Database Service free for 30 days. https://www.oracle.com/cloud/free/ Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted88
“We have saved around 40% of our costs and are able to reinvest that back into the business. And we are scaling across EMEA, and that’s basically all because of Oracle.” —Asser Smidt CEO and Cofounder, BotSupply Startups get cloud credits and a 70% discount for 2 years, global exposure via marketing, events, digital promotion, and media, plus access to mentorship, capital and Oracle’s 430,000+ customers Customers meet vetted startups in transformative spaces that help them stay ahead of their competition Oracle stays at the competitive edge of innovation with solutions that complement its technology stack Oracle for Startups - enroll at oracle.com/startup A Virtuous Cycle of Innovation, Everybody Wins.
Thanks! Contact info: Dave Stokes David.Stokes@Oracle.com @Stoker slideshare.net/davidmstokes Elepantdolphin.blogger.com 90

More Related Content

PDF
Json within a relational database
byDave Stokes
 
PDF
JavaScript and Friends August 20th, 20201 -- MySQL Shell and JavaScript
byDave Stokes
 
PDF
MySQL 8.0 New Features -- September 27th presentation for Open Source Summit
byDave Stokes
 
PDF
Longhorn PHP - MySQL Indexes, Histograms, Locking Options, and Other Ways to ...
byDave Stokes
 
PDF
Open Source World June '21 -- JSON Within a Relational Database
byDave Stokes
 
PPTX
Discover the Power of the NoSQL + SQL with MySQL
byDave Stokes
 
PPTX
cPanel now supports MySQL 8.0 - My Top Seven Features
byDave Stokes
 
PDF
MySQL Replication Update - DEbconf 2020 presentation
byDave Stokes
 
Json within a relational database
byDave Stokes
 
JavaScript and Friends August 20th, 20201 -- MySQL Shell and JavaScript
byDave Stokes
 
MySQL 8.0 New Features -- September 27th presentation for Open Source Summit
byDave Stokes
 
Longhorn PHP - MySQL Indexes, Histograms, Locking Options, and Other Ways to ...
byDave Stokes
 
Open Source World June '21 -- JSON Within a Relational Database
byDave Stokes
 
Discover the Power of the NoSQL + SQL with MySQL
byDave Stokes
 
cPanel now supports MySQL 8.0 - My Top Seven Features
byDave Stokes
 
MySQL Replication Update - DEbconf 2020 presentation
byDave Stokes
 

What's hot

PDF
Php &amp; my sql - how do pdo, mysq-li, and x devapi do what they do
byDave Stokes
 
PDF
Midwest PHP Presentation - New MSQL Features
byDave Stokes
 
PDF
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
byDave Stokes
 
PPTX
Confoo 2021 - MySQL Indexes & Histograms
byDave Stokes
 
PPTX
MySQL Utilities -- Cool Tools For You: PHP World Nov 16 2016
byDave Stokes
 
PDF
MySQL without the SQL -- Cascadia PHP
byDave Stokes
 
PDF
Ohio Linux Fest -- MySQL's NoSQL
byDave Stokes
 
PPTX
Develop PHP Applications with MySQL X DevAPI
byDave Stokes
 
PPTX
Confoo 2021 -- MySQL New Features
byDave Stokes
 
PDF
MySQL Utilities -- PyTexas 2015
byDave Stokes
 
PPTX
A Step by Step Introduction to the MySQL Document Store
byDave Stokes
 
PDF
MySQL 8.0 Operational Changes
byDave Stokes
 
PPTX
MySQL 8 -- A new beginning : Sunshine PHP/PHP UK (updated)
byDave Stokes
 
PDF
PNWPHP -- What are Databases so &#%-ing Difficult
byDave Stokes
 
PDF
MySQL 8 Tips and Tricks from Symfony USA 2018, San Francisco
byDave Stokes
 
PPTX
Open Source 1010 and Quest InSync presentations March 30th, 2021 on MySQL Ind...
byDave Stokes
 
PDF
Dutch PHP Conference 2021 - MySQL Indexes and Histograms
byDave Stokes
 
PPTX
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
byDave Stokes
 
PPTX
Validating JSON -- Percona Live 2021 presentation
byDave Stokes
 
PDF
Data Love Conference - Window Functions for Database Analytics
byDave Stokes
 
Php &amp; my sql - how do pdo, mysq-li, and x devapi do what they do
byDave Stokes
 
Midwest PHP Presentation - New MSQL Features
byDave Stokes
 
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
byDave Stokes
 
Confoo 2021 - MySQL Indexes & Histograms
byDave Stokes
 
MySQL Utilities -- Cool Tools For You: PHP World Nov 16 2016
byDave Stokes
 
MySQL without the SQL -- Cascadia PHP
byDave Stokes
 
Ohio Linux Fest -- MySQL's NoSQL
byDave Stokes
 
Develop PHP Applications with MySQL X DevAPI
byDave Stokes
 
Confoo 2021 -- MySQL New Features
byDave Stokes
 
MySQL Utilities -- PyTexas 2015
byDave Stokes
 
A Step by Step Introduction to the MySQL Document Store
byDave Stokes
 
MySQL 8.0 Operational Changes
byDave Stokes
 
MySQL 8 -- A new beginning : Sunshine PHP/PHP UK (updated)
byDave Stokes
 
PNWPHP -- What are Databases so &#%-ing Difficult
byDave Stokes
 
MySQL 8 Tips and Tricks from Symfony USA 2018, San Francisco
byDave Stokes
 
Open Source 1010 and Quest InSync presentations March 30th, 2021 on MySQL Ind...
byDave Stokes
 
Dutch PHP Conference 2021 - MySQL Indexes and Histograms
byDave Stokes
 
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
byDave Stokes
 
Validating JSON -- Percona Live 2021 presentation
byDave Stokes
 
Data Love Conference - Window Functions for Database Analytics
byDave Stokes
 

Similar to Datacon LA - MySQL without the SQL - Oh my!

PDF
Node.js and the MySQL Document Store
byRui Quelhas
 
PDF
MySQL Document Store (Oracle Code Warsaw 2018)
byVittorio Cioe
 
PDF
MySQL Without the SQL - Oh My! August 2nd presentation at Mid Atlantic Develo...
byDave Stokes
 
PPTX
Discover The Power of NoSQL + MySQL with MySQL
byDave Stokes
 
PDF
MySQL Document Store -- SCaLE 17x Presentation
byDave Stokes
 
PPTX
MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019
byDave Stokes
 
PDF
MySQL as a Document Store
byDave Stokes
 
PDF
MySQL Without The SQL -- Oh My! PHP Detroit July 2018
byDave Stokes
 
PPTX
MySQL Without the SQL -- Oh My! Longhorn PHP Conference
byDave Stokes
 
ODP
MySQL Without the MySQL -- Oh My!
byDave Stokes
 
PPTX
Making MySQL Agile-ish
byDave Stokes
 
PDF
MySQL Document Store
byMario Beck
 
PDF
20171104 hk-py con-mysql-documentstore_v1
byIvan Ma
 
PDF
MySQL Document Store - A Document Store with all the benefts of a Transactona...
byOlivier DASINI
 
PDF
Oracle Code Event - MySQL JSON Document Store
byMark Swarbrick
 
PDF
MySQL Document Store and Node.JS
byReggie Burnett
 
PDF
MySQL document_store
byGiuseppe Maxia
 
PDF
MySQL Day Paris 2016 - MySQL as a Document Store
byOlivier DASINI
 
PDF
Introduction to MySQL Document Store
byFrederic Descamps
 
ODP
MongoDB & PHP
bySanjeev Shrestha
 
Node.js and the MySQL Document Store
byRui Quelhas
 
MySQL Document Store (Oracle Code Warsaw 2018)
byVittorio Cioe
 
MySQL Without the SQL - Oh My! August 2nd presentation at Mid Atlantic Develo...
byDave Stokes
 
Discover The Power of NoSQL + MySQL with MySQL
byDave Stokes
 
MySQL Document Store -- SCaLE 17x Presentation
byDave Stokes
 
MySQL Without the SQL - Oh My! -> MySQL Document Store -- Confoo.CA 2019
byDave Stokes
 
MySQL as a Document Store
byDave Stokes
 
MySQL Without The SQL -- Oh My! PHP Detroit July 2018
byDave Stokes
 
MySQL Without the SQL -- Oh My! Longhorn PHP Conference
byDave Stokes
 
MySQL Without the MySQL -- Oh My!
byDave Stokes
 
Making MySQL Agile-ish
byDave Stokes
 
MySQL Document Store
byMario Beck
 
20171104 hk-py con-mysql-documentstore_v1
byIvan Ma
 
MySQL Document Store - A Document Store with all the benefts of a Transactona...
byOlivier DASINI
 
Oracle Code Event - MySQL JSON Document Store
byMark Swarbrick
 
MySQL Document Store and Node.JS
byReggie Burnett
 
MySQL document_store
byGiuseppe Maxia
 
MySQL Day Paris 2016 - MySQL as a Document Store
byOlivier DASINI
 
Introduction to MySQL Document Store
byFrederic Descamps
 
MongoDB & PHP
bySanjeev Shrestha
 

More from Dave Stokes

PPTX
Upgrading to MySQL 8.0 webinar slides November 27th, 2019
byDave Stokes
 
PDF
MySQL 8.0 Graphical Information System - Mid Atlantic Developers Conference
byDave Stokes
 
PPTX
MySQL 8.0 from December London Open Source Database Meetup
byDave Stokes
 
PDF
MySQL New Features -- Sunshine PHP 2020 Presentation
byDave Stokes
 
PPTX
MySQL 8 - UKOUG Techfest Brighton December 2nd, 2019
byDave Stokes
 
PDF
Confoo 202 - MySQL Group Replication and ReplicaSet
byDave Stokes
 
PDF
Oracle CodeOne Foreign Keys Support in MySQL 8.0
byDave Stokes
 
PDF
Database basics for new-ish developers -- All Things Open October 18th 2021
byDave Stokes
 
PDF
Windowing Functions - Little Rock Tech Fest 2019
byDave Stokes
 
Upgrading to MySQL 8.0 webinar slides November 27th, 2019
byDave Stokes
 
MySQL 8.0 Graphical Information System - Mid Atlantic Developers Conference
byDave Stokes
 
MySQL 8.0 from December London Open Source Database Meetup
byDave Stokes
 
MySQL New Features -- Sunshine PHP 2020 Presentation
byDave Stokes
 
MySQL 8 - UKOUG Techfest Brighton December 2nd, 2019
byDave Stokes
 
Confoo 202 - MySQL Group Replication and ReplicaSet
byDave Stokes
 
Oracle CodeOne Foreign Keys Support in MySQL 8.0
byDave Stokes
 
Database basics for new-ish developers -- All Things Open October 18th 2021
byDave Stokes
 
Windowing Functions - Little Rock Tech Fest 2019
byDave Stokes
 

Recently uploaded

PDF
GTI Solution Overview.pdf useful for security solution
by33fastfast
 
PDF
Batman Forever album sličice.pdf panini album
byStefanFilipovic9
 
PDF
automobili.pdf panini album slicice ......
byStefanFilipovic9
 
PDF
investor pitch - iapploud | independent music platform
byVenkat Subramanian
 
PPTX
Social_Media_Good_or_Bad_B2Plus_Lesson.pptx
byJuliaRutkowska4
 
PPTX
Creating content that converts into leads.
bySEONutri
 
GTI Solution Overview.pdf useful for security solution
by33fastfast
 
Batman Forever album sličice.pdf panini album
byStefanFilipovic9
 
automobili.pdf panini album slicice ......
byStefanFilipovic9
 
investor pitch - iapploud | independent music platform
byVenkat Subramanian
 
Social_Media_Good_or_Bad_B2Plus_Lesson.pptx
byJuliaRutkowska4
 
Creating content that converts into leads.
bySEONutri
 

Datacon LA - MySQL without the SQL - Oh my!

  • 1.
    MySQL Without the SQL OhMy! Dave Stokes @stoker david.stokes@oracle.com Elephantdolphin.blogger.com Slides -> slideshare.net/DaveStokes
  • 2.
  • 3.
    Safe Harbor Agreement THEFOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE. 3
  • 4.
  • 5.
  • 6.
  • 7.
    Relational Databases ● OriginalGoal was to save data with minimal duplication ● Disks were expensive ● … and slow ● 45 years old ● Introduced the concept of accessing many records with a single command 7
  • 8.
    Relational Databases ● DataIntegrity ○ Normalization ○ constraints (foreign keys, ...) ● Atomicity, Consistency, Isolation, Durability ○ ACID compliant ○ transactions ● SQL ○ powerful query language 8
  • 9.
    Relational Databases ● Needto set up tables BEFORE use ● Relations, indexes, data normalization, query optimizations ● Hard to change on the fly ● Need a DBA or someone who has DBA skills ● This can be a chokepoint 9
  • 10.
  • 11.
    NoSQL or DocumentStore ● Schemaless ○ No schema design, no normalization, no foreign keys, no data types, … ○ Very quick initial development ● Flexible data structure ○ Embedded arrays or objects ○ Valid solution when natural data can not be modeled optimally into a relational model ○ Objects persistence without the use of any ORM - *mapping object-oriented* 11
  • 12.
    NoSQL or JSONDocument Store ● JSON ● close to frontend ● native in JS ● easy to learn 12
  • 13.
    How DBAs seedata as opposed to how Developers see data { "GNP" : 249704, "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : 10239000, "LifeExpectancy" : 77.8000030517578 }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" } } 13
  • 14.
    What if therewas a way to provide both SQL and NoSQL on one stable platform that has proven stability on well know technology with a large Community and a diverse ecosystem ? With the MySQL Document Store it is now an option! 14
  • 15.
    A Solution forall Developers: schemaless ★ rapid prototyping & simpler APIs ★ document model ★ transactions Operations: ★ performance management/visibility ★ robust replication, backup, restore ★ comprehensive tooling ecosystem ★ simpler application schema upgrades 15 Business Owner: ★ don't lose my data == ACID trx ★ capture all my data = extensible/schemaless ★ product on schedule/time to market = rapid development
  • 16.
    Built on theMySQL JSON Data type and Proven MySQL Server Technology 16 ★ Provides a schema flexible JSON Document Store ★ No SQL required ★ No need to define all possible attributes, tables, etc. ★ Uses new MySQL X DevAPI ★ Can leverage generated column to extract JSON values into materialized columns that can be indexed for fast SQL searches.
  • 17.
    Built on theMySQL JSON Data type and Proven MySQL Server Technology 17 ★ Document can be ~1GB ○ It's a column in a row of a table ★ Allows use of modern programming styles ○ No more embedded strings of SQL in your code ○ Easy to read ★ Also works with relational Tables ★ Proven MySQL Technology
  • 18.
    ★ C++ ★ Java ★.Net ★ Node.js ★ JavaScript ★ Python ★ PHP ○ Working with other Communities to help them support it too 18 Connectors for
  • 19.
    ★ Command Completion ★Python, JavaScripts & SQL modes ★ Admin functions ★ New Util object ★ A new high-level session concept that can scale from single MySQL Server to a multiple server environment 19 New MySQL Shell
  • 20.
    ★ Non-blocking, asynchronouscalls follow common language patterns ★ Send out many queries and process other things until they return ★ Supports CRUD operations ★ Concentrate on basic functions ★ Easily scale from one server to InnoDB cluster w/o changing application! 20 New Model
  • 21.
    21 X Protocol builton Google Protobufs Not an ORM!!
  • 22.
    22 Architecture of bothOld and New Protocols
  • 23.
    23 How Your Applicationwill work with InnoDB Cluster
  • 24.
    But what doesthis look like in PHP?? 24
  • 25.
    JavaScript 25 // Connectingto MySQL Server and working with a Collection var mysqlx = require('mysqlx'); // Connect to server var mySession = mysqlx.getSession( { host: 'localhost', port: 33060, user: 'user', password: 'password'} ); var myDb = mySession.getSchema('test'); // Create a new collection 'my_collection' var myColl = myDb.createCollection('my_collection'); // Insert documents myColl.add({_id: '1', name: 'Sakila', age: 15}).execute(); myColl.add({_id: '2', name: 'Susanne', age: 24}).execute(); myColl.add({_id: '3', name: 'User', age: 39}).execute(); // Find a document var docs = myColl.find('name like :param1 AND age < :param2').limit(1). bind('param1','S%').bind('param2',20).execute(); // Print document print(docs.fetchOne()); // Drop the collection myDb.dropCollection('my_collection'); No SQL!!
  • 26.
    Python 26 # Connectingto MySQL Server and working with a Collection from mysqlsh import mysqlx # Connect to server mySession = mysqlx.get_session( { 'host': 'localhost', 'port': 33060, 'user': 'user', 'password': 'password'} ) myDb = mySession.get_schema('test') # Create a new collection 'my_collection' myColl = myDb.create_collection('my_collection') # Insert documents myColl.add({'_id': '1', 'name': 'Sakila', 'age': 15}).execute() myColl.add({'_id': '2', 'name': 'Susanne', 'age': 24}).execute() myColl.add({'_id': '3', 'name': 'User', 'age': 39}).execute() # Find a document docs = myColl.find('name like :param1 AND age < :param2') .limit(1) .bind('param1','S%') .bind('param2',20) .execute() # Print document doc = docs.fetch_one() print doc
  • 27.
    Node.JS 27 // Connectingto MySQL Server and working with a Collection var mysqlx = require('@mysql/xdevapi'); var db; // Connect to server mysqlx .getSession({ user: 'user', password: 'password', host: 'localhost', port: '33060', }) .then(function (session) { db = session.getSchema('test'); // Create a new collection 'my_collection' return db.createCollection('my_collection'); }) .then(function (myColl) { // Insert documents return Promise .all([ myColl.add({ name: 'Sakila', age: 15 }).execute(), myColl.add({ name: 'Susanne', age: 24 }).execute(), myColl.add({ name: 'User', age: 39 }).execute() ]) .then(function () { // Find a document return myColl .find('name like :name && age < :age') .bind({ name: 'S%', age: 20 }) .limit(1) .execute(function (doc) { // Print document console.log(doc); }); }); }) .then(function(docs) { // Drop the collection return db.dropCollection('my_collection'); }) .catch(function(err) { // Handle error });
  • 28.
    C++ 28 // Connectto server var mySession = MySQLX.GetSession("server=localhost;port=33060;user=user;password=password;"); var myDb = mySession.GetSchema("test"); // Create a new collection "my_collection" var myColl = myDb.CreateCollection("my_collection"); // Insert documents myColl.Add(new { name = "Sakila", age = 15}).Execute(); myColl.Add(new { name = "Susanne", age = 24}).Execute(); myColl.Add(new { name = "User", age = 39}).Execute(); // Find a document var docs = myColl.Find("name like :param1 AND age < :param2").Limit(1) .Bind("param1", "S%").Bind("param2", 20).Execute(); // Print document Console.WriteLine(docs.FetchOne()); // Drop the collection myDb.DropCollection("my_collection");
  • 29.
    Java 29 // Connectto server Session mySession = new SessionFactory().getSession("mysqlx://localhost:33060/test?user=user&password=password"); Schema myDb = mySession.getSchema("test"); // Create a new collection 'my_collection' Collection myColl = myDb.createCollection("my_collection"); // Insert documents myColl.add("{"name":"Sakila", "age":15}").execute(); myColl.add("{"name":"Susanne", "age":24}").execute(); myColl.add("{"name":"User", "age":39}").execute(); // Find a document DocResult docs = myColl.find("name like :name AND age < :age") .bind("name", "S%").bind("age", 20).execute(); // Print document DbDoc doc = docs.fetchOne(); System.out.println(doc); // Drop the collection myDB.dropCollection("test", "my_collection");
  • 30.
  • 31.
    Starting using MySQLin few minutes 31
  • 32.
    Quickly add adocument 32
  • 33.
  • 34.
  • 35.
  • 36.
    For this example,I will use the well known restaurants collection: We need to dump the data to a file and we will use the MySQL Shell with the Python interpreter to load the data. Migration from MongoDB to MySQL Document Store 36
  • 37.
    Dump and loadusing MySQL Shell & Python This example is inspired by @datacharmer's work: https://www.slideshare.net/datacharmer/mysql-documentstore $ mongo quiet eval 'DBQuery.shellBatchSize=30000; db.restaurants.find().shellPrint()' | perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json 37
  • 38.
    Or use newbulk loader in 8.0.13 38Parallel import introduced in 8.0.17
  • 39.
    BSON Support Now, itsupports the conversion of the following additional BSON types: ■ Date ■ Timestamp ■ NumberDecimal ■ NumberLong ■ NumberInt ■ Regular Expression ■ Binary 39 > util.importJson("/path_to_file/neighborhoods_mongo.json", {schema: "test", collection: "neighborhoods", convertBsonTypes: true});
  • 40.
  • 41.
    41 Let’s query Too manyrecords to show here … let’s limit it! restaurants.find().limit(1)
  • 42.
  • 43.
    43 Comparing Syntax: MongoDBvs MYSQL MongoDB: > db.restaurants.find({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) MySQL: >restaurants.find(“cuisine=’French’ AND borough!=’Manhattan’”).fields([“name”,”cuisine”,”borough” ]).limit(2)
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
    49 MySQL Document StoreObjects Summary
  • 50.
    MySQL Document Storeis Fully ACID Compliant 50
  • 51.
    MySQL Document Storeis Fully ACID Compliant 51
  • 52.
    How Does ItWork?? 52
  • 53.
    What does acollection look like on the server ? 53
  • 54.
    Every document hasa unique identifier called the document ID, which can be thought of as the equivalent of a table's primary key. The document ID value can be manually assigned when adding a document. If no value is assigned, a document ID is generated and assigned to the document automatically ! Use getDocumentId() or getDocumentIds() to get _ids(s) _id 54
  • 55.
    Mapping to SQLExamples createCollection('mycollection') versus CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4; 55
  • 56.
    Mapping to SQLExamples mycollection.add({‘test’: 1234}) versus INSERT INTO `test`.`mycoll` (doc) VALUES ( JSON_OBJECT( 'test',1234)); 56
  • 57.
    More Mapping toSQL Examples mycollection.find("test > 100") Versus SELECT doc FROM `test`.`mycoll` WHERE (JSON_EXTRACT(doc,'$.test') >100); 57
  • 58.
  • 59.
    It's also possibleto create indexes without using SQL syntax 59
  • 60.
    SQL and JSONExample (3): explain 60
  • 61.
    SQL and JSONExample (3): explain 61
  • 62.
    SQL and JSONExample (4): add index 62
  • 63.
    SQL and JSONExample (4): add index 63
  • 64.
    [ { "date": { "$date": 1416009600000 }, "grade":"Z", "score": 38 }, { "date": { "$date": 1398988800000 }, "grade": "A", "score": 10 }, { "date": { "$date": 1362182400000 }, "grade": "A", "score": 7 }, { "date": { "$date": 1328832000000 }, "grade": "A", "score": 13 } ] 64 Embedded Arrays of values can be messy to traverse.
  • 65.
    SQL and JSONExample (5): arrays 65
  • 66.
  • 67.
    NoSQL as SQL67 JSON_TABLE turns your un-structured JSON data into a temporary structured table!
  • 68.
    NoSQL as SQL68 This temporary structured table can be treated like any other table -- LIMIT, WHERE, GROUP BY ...
  • 69.
  • 70.
    Find the top10 restaurants by grade for each cuisine 70 WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM restaurants) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10; This query uses a Common Table Expression (CTE) and a Windowing Function to rank the average scores of each restaurant, by each cuisine assembled in a JSON_TABLE
  • 71.
    No SQL ConsumedIn This Query!! 71 $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll();
  • 72.
  • 73.
    JSON Validation The Problem Unlikestrictly types relational databases there is no data normalization or ‘rigor’ applied to that data. There is also no native way to do range checks And there is no way have required fields 73
  • 74.
    JSON-Schema.org The Problem Unlike strictlytypes relational databases there is no data normalization or ‘rigor’ applied to that data. There is also no native way to do range checks And there is no way have required fields JSON-Shema.org is work to fix that 74
  • 75.
    JSON Validation set @s='{"type":"object", "properties": { "myage": { "type" : "number", "minimum": 28, "maximum": 99 } } }'; set @d='{ "myage": 33}'; select JSON_SCHEMA_VALID(@s,@d); +--------------------------+ | JSON_SCHEMA_VALID(@s,@d) | +--------------------------+ | 1 | +--------------------------+ 1 row in set (0.00 sec) 75
  • 76.
    JSON Validation Report selectJSON_PRETTY(JSON_SCHEMA_VALIDATION_REPORT(@s,@d))G *************************** 1. row *************************** JSON_PRETTY(JSON_SCHEMA_VALIDATION_REPORT(@s,@d)): { "valid": false, "reason": "The JSON document location '#/myage' failed requirement 'minimum' at JSON Schema location '#/properties/myage'", "schema-location": "#/properties/myage", "document-location": "#/myage", "schema-failed-keyword": "minimum" } 76
  • 77.
    JSON Check Constraint CREATETABLE `testx` ( `col` JSON, CONSTRAINT `myage_inRange` CHECK (JSON_SCHEMA_VALID('{"type": "object", "properties": { "myage": { "type" : "number", "minimum": 28, "maximum": 99 } },"required": ["myage"] }', `col`) = 1) ); 77
  • 78.
    JSON Check Constraint mysql>insert into testx values('{"myage":27}'); ERROR 3819 (HY000): Check constraint 'myage_inRange' is violated. mysql> insert into testx values('{"myage":97}'); Query OK, 1 row affected (0.02 sec) 78
  • 79.
  • 80.
    Index JSON Arrays {"user":"Bob", "user_id":31, "zipcode":[94477,94536] } CREATE TABLE customers ( id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, modified DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, custinfo JSON, INDEX zips( (CAST(custinfo->'$.zip' AS UNSIGNED ARRAY)) ) ); Mutli Value Indexes allow you to go past the 1:1 relation to index the data in JSON arrays. And there are three special functions what can make use of MVIs when used on the right side of a WHERE clause -- MEMBER OF(), JSON_CONTAINS(), and JSON_OVERLAPS() 80
  • 81.
  • 82.
    This is thebest of the two worlds in one product ! ● Data integrity ● ACID Compliant ● Transactions ● SQL ● Schemaless ● flexible data structure ● easy to start (CRUD) 82
  • 83.
    Mutable Data!! Reduce Manyto many joins Replace ‘stub’ tables Change on the fly, aggregate new data 83
  • 84.
    Non JSON DataTransforms to JSON 84
  • 85.
    GeoJSON support too! mysql>SELECT ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2); +-------------------------------------------------------------+ | ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2) | +-------------------------------------------------------------+ | {"type": "Point", "coordinates": [11.11, 12.22]} | +-------------------------------------------------------------+ 85
  • 86.
    New in MySQL8.0 1. True Data Dictionary 2. Default UTF8MB4 3. Windowing Functions, CTEs, Lateral Derived Joins 4. InnoDB SKIPPED LOCK and NOWAIT 5. Instant Add Column 6. Histograms 7. Resource Groups 8. Better optimizer with new temporary table engine 9. True Descending Indexes 10. 3D GIS 11. JSON Enhancements 86
  • 87.
    Please buy mybook! If you deal with the JSON Data Type or have an interest in the MySQL Document Store, this text is a great guide with many examples to help you understand the complexities and opportunities with a native JSON Data Type – SECOND EDITION now Available on 87
  • 88.
    Test Drive MySQL DatabaseService For Free Today Get $300 in credits and try MySQL Database Service free for 30 days. https://www.oracle.com/cloud/free/ Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted88
  • 89.
    “We have savedaround 40% of our costs and are able to reinvest that back into the business. And we are scaling across EMEA, and that’s basically all because of Oracle.” —Asser Smidt CEO and Cofounder, BotSupply Startups get cloud credits and a 70% discount for 2 years, global exposure via marketing, events, digital promotion, and media, plus access to mentorship, capital and Oracle’s 430,000+ customers Customers meet vetted startups in transformative spaces that help them stay ahead of their competition Oracle stays at the competitive edge of innovation with solutions that complement its technology stack Oracle for Startups - enroll at oracle.com/startup A Virtuous Cycle of Innovation, Everybody Wins.
  • 90.