Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PHP, The X DevAPI, and the MySQL Document Store -- Benelux PHP Confernece 2019

63 views

Published on

How to use the PHP PECL MySQL X DevAPI to access the MySQL Document Store with the PHP Programming Language

Published in: Internet
  • Be the first to comment

PHP, The X DevAPI, and the MySQL Document Store -- Benelux PHP Confernece 2019

  1. 1. PHP, The X DevAPI, and the MySQL Document Store Dave Stokes @Stoker David.Stokes@Oracle.com https://elephantdolphin.blogspot.com/ https://slideshare.net/davidmstokes
  2. 2. Safe Harbor Agreement THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE. 2
  3. 3. Small History Lesson Here 3
  4. 4. Programmers Tasks ● 20 years ago ○ Editor - vi, vim, emacs ○ Compiler - 1 or two languages ○ Debugger ○ Browser (maybe) ○ XML (maybe) ○ HTML (growing need) ○ Source Control System ○ IDE (you wish!) ○ Documentation (low priority) 4
  5. 5. Programmers Tasks ● 20 years ago ○ Editor - vi, vim, emacs ○ Compiler - 1 or two languages ○ Debugger ○ Browser (maybe) ○ XML (maybe) ○ HTML (growing need) ○ Source Control System ○ IDE (you wish!) ○ Documentation (low priority) ● Today ○ GIT ○ Tests ○ CI ○ Small(er) stuff ■ Containers, VMs, micro services, etc. ○ Main language & Framework ○ 7 or 8 JavaScript Frameworks ○ Data Store (SQL & NoSQL) 5
  6. 6. Programmers Tasks ● 20 years ago ○ Editor - vi, vim, emacs ○ Compiler - 1 or two languages ○ Debugger ○ Browser (maybe) ○ XML (maybe) ○ HTML (growing need) ○ Source Control System ○ IDE (you wish!) ○ Documentation (low priority) ● Today ○ GIT ○ Tests ○ CI ○ Small(er) stuff ■ Containers, VMs, micro services, etc. ○ Main language & Framework ○ 7 or 8 JavaScript Frameworks ○ Data Store (SQL & NoSQL) ○ JSON ○ SSH/TLS ○ Encryption ○ Bash & Powershell ○ Markdown ○ Cloud 6
  7. 7. Programmers Tasks ● 20 years ago ○ Editor - vi, vim, emacs ○ Compiler - 1 or two languages ○ Debugger ○ Browser (maybe) ○ XML (maybe) ○ HTML (growing need) ○ Source Control System ○ IDE (you wish!) ○ Documentation (low priority) ● Today ○ GIT ○ Tests ○ CI ○ Small(er) stuff ■ Containers, VMs, micro services, etc. ○ Main language & Framework ○ 7 or 8 JavaScript Frameworks ○ Data Store (SQL & NoSQL) ○ JSON ○ SSH/TLS ○ Encryption ○ Bash & Powershell ○ Markdown ○ Cloud ○ Debugger ○ Multiple Browsers ○ Multiple IDEs ○ Documentation (still low priority) ○ Mentoring (giving or receiving) ○ More frameworks ○ More third party libraries ○ Embedded, Android, IOS, etc. ○ Key management ○ Slack ○ Repository tools ○ * as a Service (*aaS) ○ Whatever the latest craze is from The Register, Inforworld, Slashdot 7
  8. 8. Programmers Tasks ● 20 years ago ○ Editor - vi, vim, emacs ○ Compiler - 1 or two languages ○ Debugger ○ Browser (maybe) ○ XML (maybe) ○ HTML (growing need) ○ Source Control System ○ IDE (you wish!) ○ Documentation (low priority) ● Today ○ GIT ○ Tests ○ CI ○ Small(er) stuff ■ Containers, VMs, micro services, etc. ○ Main language & Framework ○ 7 or 8 JavaScript Frameworks ○ Data Store (SQL & NoSQL) ○ JSON ○ SSH/TLS ○ Encryption ○ Bash & Powershell ○ Markdown ○ Cloud ○ Debugger ○ Multiple Browsers ○ Multiple IDEs ○ Documentation (still low priority) ○ Mentoring (giving or receiving) ○ More frameworks ○ More third party libraries ○ Embedded, Android, IOS, etc. ○ Key management ○ Slack ○ Repository tools ○ * as a Service (*aaS) ○ Whatever the latest craze is from The Register, Inforworld, Slashdot ○ YAML and other markup file syntax ○ The newest Google tool ○ etc 8
  9. 9. 9 BTW This was the big concern of 1999!
  10. 10. Constantly Increasing Learning Curve 10
  11. 11. 2% In my not so scientific study over the past eight years using informal informational gathering techniques, roughly TWO PERCENT of developers receive any formal training in Structured Query Language (SQL), Relational Calculus, Data Normalization, or other database technologies. And 100% of that 2% wonder why their queries stink! 11
  12. 12. Impedance Mismatch 12
  13. 13. Declarative Language Buried in a OO/Procedural A big problem for many developers is that they are used to Object Oriented and/or Procedural programming languages. SQL is declarative programming language and embedding SQL in PHP is an object- relational impedance mismatch. // Formulate Query // This is the best way to perform an SQL query // For more examples, see mysql_real_escape_string() $query = sprintf("SELECT firstname, lastname, address, age FROM friends WHERE firstname='%s' AND lastname='%s'", mysql_real_escape_string($firstname), mysql_real_escape_string($lastname)); // Perform Query $result = mysql_query($query); http://us1.php.net/manual/en/function.mysql-query.php 13 Note: the ‘mysql’ connector has been deprecated and is not supported in PHP 7 and was replaced by ‘mysqli’ but there is still a lot of code with ‘mysql’ around
  14. 14. Other Issues 1. UGLY 2. Hard to have help from your IDE 3. Extra level of complexity / opportunity to fail 4. Badly organized data & queries
  15. 15. And Yet More Issues ● Relational tables need to be set up ● Indexes ● Data mutability ● Need to rely on a DBA (or someone who has that role) ● Can’t start coding much of project before data format is know ● ORMs -- more complexity and another layer to maintain 15
  16. 16. X DevAPI & The MySQL Document Store 16
  17. 17. So what if there was a way to ... ● Use schemaless JSON documents so you do not have to normalize data and code before you know the complete schema ● Not have to embed SQL strings in your code ● Use a modern programming style API ● Be able to use the JSON data from SQL or NoSQL - ○ Best of both worlds 17
  18. 18. 18
  19. 19. The X DevAPI wraps powerful concepts in a simple API. A new high-level session concept enables you to write code that can transparently scale from single MySQL Server to a multiple server environment. →Read operations are simple and easy to understand. →Non-blocking, asynchronous calls follow common host language patterns. →The X DevAPI introduces a new, modern and easy-to-learn way to work with your data. Documents are stored in Collections and have their dedicated CRUD operation set. →Work with your existing domain objects or generate code based on structure definitions for strictly typed languages. →Focus is put on working with data via CRUD operations. →Modern practices and syntax styles are used to get away from traditional SQL-String-Building. 19
  20. 20. Scale from Single Server to Cluster w/o Code Change The code that is needed to connect to a MySQL document store looks a lot like the traditional MySQL connection code, but now applications can establish logical sessions to MySQL server instances running the X Plugin. Sessions are produced by the mysqlx factory, and the returned Sessions can encapsulate access to one or more MySQL server instances running X Plugin. Applications that use Session objects by default can be deployed on both single server setups and database clusters with no code changes. 20
  21. 21. var mysqlx = require('mysqlx'); // Connect to server on localhost var mySession = mysqlx.getSession( { host: 'localhost', port: 33060, user: 'user', password: 'password' } ); var myDb = mySession.getSchema('test'); // Use the collection 'my_collection' var myColl = myDb.getCollection('my_collection'); // Specify which document to find with Collection.find() and // fetch it from the database with .execute() var myDocs = myColl.find('name like :param').limit(1). bind('param', 'S%').execute(); // Print document print(myDocs.fetchOne()); mySession.close(); 21
  22. 22. PECL MySQl X DevAPI http://php.net/manual/en/book.mysql-xdevapi.php 22
  23. 23. Installation (Ubuntu 18.04 & PHP 7.2) Note: The MySQL X DevAPI PECL extension is not bundled with PHP. // Dependencies $ apt install build-essential libprotobuf-dev libboost-dev openssl protobuf- compiler // PHP with the desired extensions; php7.2-dev is required to compile $ apt install php7.2-cli php7.2-dev php7.2-mysql php7.2-pdo php7.2-xml // Compile the extension $ pecl install mysql_xdevapi // Use the 'phpenmod' command $ phpenmod -v 7.2 -s ALL mysql_xdevapi 23
  24. 24. A JAON document is a data structure composed of key-value pairs and is the fundamental structure for using MySQL as document store. This document shows that the values of keys can be simple data types, such as integers or strings, but can also contain other documents, arrays, and lists of documents. For example, the geography key's value consists of multiple key-value pairs. A JSON document is represented internally using the MySQL binary JSON object, through the JSON MySQL datatype. The most important differences between a document and the tables known from traditional relational databases are that the structure of a document does not have to be defined in advance, and a collection can contain multiple documents with different structures. Relational tables on the other hand require that their structure be defined, and all rows in the table must contain the same columns. { "GNP": .6, "IndepYear": 1967, "Name": "Sealand", "_id": "SEA", "demographics": { "LifeExpectancy": 79, "Population": 27 }, "geography": { "Continent": "Europe", "Region": "British Islands", "SurfaceArea": 193 }, "government": { "GovernmentForm": "Monarchy", "HeadOfState": "Michael Bates" } } 24
  25. 25. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $marco = [ "name" => "Marco", "age" => 19, "job" => "Programmer" ]; $mike = [ "name" => "Mike", "age" => 39, "job" => "Manager" ]; $schema = $session->getSchema("testxx"); $collection = $schema->createCollection("example1"); $collection = $schema->getCollection("example1"); $collection->add($marco, $mike)->execute(); var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 25 An Example
  26. 26. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $marco = [ "name" => "Marco", "age" => 19, "job" => "Programmer" ]; $mike = [ "name" => "Mike", "age" => 39, "job" => "Manager" ]; $schema = $session->getSchema("testxx"); $collection = $schema->createCollection("example1"); $collection = $schema->getCollection("example1"); $collection->add($marco, $mike)->execute(); var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 26 The URI specifies the details of the connection - Protocol - Username - Authentication String - Lost - Port The X Plugin listens at port 33060
  27. 27. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $marco = [ "name" => "Marco", "age" => 19, "job" => "Programmer" ]; $mike = [ "name" => "Mike", "age" => 39, "job" => "Manager" ]; $schema = $session->getSchema("testxx"); $collection = $schema->createCollection("example1"); $collection = $schema->getCollection("example1"); $collection->add($marco, $mike)->execute(); var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 27 1. Connect to the schema ‘testxx’ 2. Create a collection for JSON documents named ‘example1’ 3. Use the collection ‘example1’
  28. 28. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $marco = [ "name" => "Marco", "age" => 19, "job" => "Programmer" ]; $mike = [ "name" => "Mike", "age" => 39, "job" => "Manager" ]; $schema = $session->getSchema("testxx"); $collection = $schema->createCollection("example1"); $collection = $schema->getCollection("example1"); $collection->add($marco, $mike)->execute(); var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 28 Add records to the collection
  29. 29. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $marco = [ "name" => "Marco", "age" => 19, "job" => "Programmer" ]; $mike = [ "name" => "Mike", "age" => 39, "job" => "Manager" ]; $schema = $session->getSchema("testxx"); $collection = $schema->createCollection("example1"); $collection = $schema->getCollection("example1"); $collection->add($marco, $mike)->execute(); var_dump($collection->find("name = 'Mike'")->execute()->fetchOne()); 29 Find the record where the name = ‘Mike’
  30. 30. The Emphasis is on CRUD 30 Operation form Description db.name.add() The add() method inserts one document or more documents into the named collection. db.name.find() The find() method returns some or all documents in the named collection. db.name.modify() The modify() method updates documents in the named collection. db.name.remove() The remove() method deletes one document or a list of documents from the named collection. CRUD EBNF Definitions - https://dev.mysql.com/doc/x-devapi-userguide/en/mysql-x- crud-ebnf-definitions.html
  31. 31. find() 31
  32. 32. No more messy strings $SQLQuery = “SELECT * FROM people WHERE job LIKE “ . $job . “AND age > $age”; Versus $collection = $schema- >getCollection("people"); $result = $collection ->find('job like :job and age > :age') ->bind(['job' => 'Butler', 'age' => 16]) ->execute(); 32
  33. 33. Easier to read/comprehend than SQL $result = $collection ->remove('age > :age_from and age < :age_to') ->bind(['age_from' => 20, 'age_to' => 50]) ->limit(2) ->execute(); 33 Easy to add filters like SORT, LIMIT, HAVING GROUP BY
  34. 34. Indexes on collections $collection->createIndex( 'myindex1', '{"fields": [{ "field": "$.name", "type": "TEXT(25)", "required": true}], "unique": false}' ); 34
  35. 35. Got Tables? You can also use the MySQL Document Store with Relational Tables 35
  36. 36. Quick Example using a table #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://root:hidave@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll(); print_r($row); 36
  37. 37. The URI Connection for a Session #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://root:hidave@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll(); $row = $result->fetchAll(); print_r($row); 37
  38. 38. Get schema and table #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://root:hidave@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll(); print_r($row); 38
  39. 39. The Query #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://root:hidave@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $schema = $session->getSchema("world"); $table = $schema->getTable("city"); $row = $table->select('Name','District') ->where('District like :district') ->bind(['district' => 'Texas']) ->limit(25) ->execute()->fetchAll(); ; print_r($row); 39
  40. 40. What if I Don’t Want to Rewrite old queries? $session->sql("CREATE DATABASE addressbook")->execute(); 40
  41. 41. Okay I can use Collections and Tables BUT WHAT ABOUT BOTH!?!?!?! 41
  42. 42. You can also use a Collection as a table! $collection = $schema->getCollection("people"); $table = $schema->getCollectionAsTable("people"); 42
  43. 43. #!/bin/php <?php $session = mysql_xdevapigetSession("mysqlx://myuser:mypass@localhost:33060"); if ($session === NULL) { die("Connection could not be established"); } $schema = $session->getSchema("nyeats"); $table = $schema->getTable("restaurants"); $sqlx = 'WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM restaurants) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10;’; $row->sql($sqlx)->execute()-fetchAll();’ print_r($row); 43 You can also use Windowing Functions for Advanced Analytics
  44. 44. One of the advantages of using MySQL as a NoSQL Document store is that you can use SQL analytics on your data! 44
  45. 45. Combine CTEs, Windowing Functions, & JSON_TABLE WITH cte1 AS (SELECT doc->>"$.name" AS name, doc->>"$.cuisine" AS cuisine, (SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT PATH "$.score")) AS r) AS avg_score FROM restaurants) SELECT *, RANK() OVER (PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank` FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10; JSON_TABLE turns unstructured JSON documents in to temporary relational tables that can be processed with SQL Windowing Function for analytics Common Table Expression make it easy to write sub-queries 45
  46. 46. 46
  47. 47. The X Plugin … is a shared object that is installed by default in MySQL 8.0 and must be loaded in 5.7 mysqlsh -u user -h localhost --classic --dba enableXProtocol Or mysql> INSTALL PLUGIN mysqlx SONAME 'mysqlx.so'; It listens on port 33060 so make sure you open firewall for 3306 (old MySQL) and 33060 (X Plugin). And it supports SSL/TLS!!! 47
  48. 48. The New MySQL Shell 48 Built In JavaScript and Python interpreters let you work with you data in the MySQL Shell. Plus you get command completion, great help facilities, the ability to check for server upgrades, and the ability to administrate a InnoDB Clusters. And you can also use SQL
  49. 49. Built in JSON Bulk Loader 49
  50. 50. 50 InnoDB Cluster MySQL InnoDB cluster provides a complete high availability solution for MySQL. Each MySQL server instance runs MySQL Group Replication, which provides the mechanism to replicate data within InnoDB clusters, with built-in failover. AdminAPI removes the need to work directly with Group Replication in InnoDB clusters MySQL Router can automatically configure itself based on the cluster you deploy, connecting client applications transparently to the server instances. Multiple secondary server instances are replicas of the primary. If the primary fails, a secondary is automatically promoted to the role of primary. MySQL Router detects this and forwards client applications to the new primary. Advanced users can also configure a cluster to have multiple-primaries.
  51. 51. Questions and Answers plus Additional Resources ● More Info on MySQL Document Store ○ PHP PECL Extension for X DevAPI ■ http://php.net/manual/en/book.mysql-xdevapi.php ○ MySQL Document Store ■ https://dev.mysql.com/doc/refman/8.0/en/document-store.html ○ X DevAPI User Guide ■ https://dev.mysql.com/doc/x-devapi-userguide/en/ ○ Dev.MySQL.com for Downloads and Other Doces ○ X DevAPI Tutorial for Sunshine PHP ■ https://github.com/davidmstokes/PHP-X-DevAPI ● David.Stokes@Oracle.com ○ https://elephantdolphin.blogspot.com/ ○ Slides at https://slideshare.net/davidmstokes ○ @Stoker ○ MySQL & JSON - A Practical Programming Guide 51

×