Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mql to-sql - a json-based rdbms query language

5,281 views

Published on

  • Be the first to comment

Mql to-sql - a json-based rdbms query language

  1. 1. MQL-to-SQL a JSON-based query language for RDBMS data access from AJAX applications
  2. 2. Welcome! <ul><li>Roland Bouman, Leiden, the Netherlands
  3. 3. Ex-MySQL, ex-Sun Microsystems
  4. 4. Web & BI developer at Strukton Rail
  5. 5. Author </li><ul><li>“Pentaho Solutions”
  6. 6. “Pentaho Kettle Solutions” </li></ul><li>http://rpbouman.blogspot.com , @rolandbouman </li></ul>
  7. 7. Agenda <ul><li>The data access problem
  8. 8. What's wrong with SQL?
  9. 9. The Metaweb Query Language (MQL)
  10. 10. MQL-to-SQL
  11. 11. Demo
  12. 12. Questions </li></ul>
  13. 13. Agenda <ul><li>The data access problem
  14. 14. What's wrong with SQL?
  15. 15. The Metaweb Query Language (MQL)
  16. 16. MQL-to-SQL
  17. 17. Demo
  18. 18. Questions </li></ul>
  19. 19. The Data Access Problem <ul><li>Web application </li></ul><ul><li>Classical Client/Server </li></ul>SQL Resultset SQL Resultset RDBMS HTTP Browser Desktop Application ?
  20. 20. The Data Access Problem <ul><li>Build a service </li><ul><li>Protocol on top of HTTP: REST, RPC
  21. 21. Data format: JSON, XML </li></ul><li>Advantage: controlled data access
  22. 22. Disadvantage: controlled=limited </li></ul>
  23. 23. The Data Access Problem <ul><li>Advantage: unlimited data access
  24. 24. Disadvantage: unlimited=uncontrolled </li></ul>SQL SQL Resultset JSON <ul><li>Build a query service </li><ul><li>Example: dbslayer </li></ul></ul>
  25. 25. Agenda <ul><li>The data access problem
  26. 26. What's wrong with SQL?
  27. 27. The Metaweb Query Language (MQL)
  28. 28. MQL-to-SQL
  29. 29. Demo
  30. 30. Questions </li></ul>
  31. 31. What's wrong with SQL? <ul><li>Nothing of course, it's great!!! </li><ul><li>very powerful
  32. 32. expressive syntax
  33. 33. declarative
  34. 34. designed for the RDBMS
  35. 35. portable
  36. 36. well supported </li></ul></ul>
  37. 37. What's wrong with SQL? <ul><li>SQL is very powerful </li><ul><li>...way too powerful </li></ul><li>SQL has expressive syntax </li><ul><li>...and too hard to parse and generate </li></ul><li>SQL is declarative </li><ul><li>...but not enough </li></ul><li>SQL is designed for the RDBMS </li><ul><li>...can't escape relational model </li></ul></ul>
  38. 38. SELECT f.title , f.description , f.release_year , f.length , f.rating , a.first_name , a.last_name , l.name FROM film f INNER JOIN language l ON f.language_id = l.id LEFT JOIN film_actor fa ON f.id = fa.film_id LEFT JOIN actor a ON fa.actor_id = a.id SQL not declarative enough Spoken in Has cast Starring in
  39. 39. SELECT f.title , f.description , f.release_year , f.length , f.rating , a.first_name , a.last_name , spoken_in.name FROM film f RELATE spoken_in language l , has_cast film_actor fa RELATE starring_in actor a SQL not declarative enough Spoken in Has cast Starring in
  40. 40. SQL can't escape the RM Unnormalized data Split off – multivalued attributes – repeating groups First normal form (1NF) Higher normal forms (>= 3NF) Remove redundancy Denormalized data (1NF) SQL queries: – JOIN operation ??? Header – line 1a – line 2b header line 1a line 2b header 1 2 line 1a line 2b header line 1a header line 2b
  41. 41. SQL can't escape the RM
  42. 42. SELECT f.* , l.*, ol.* , (SELECT * FROM film_category fc INNER JOIN category c ON fc.category_id = c.id WHERE fc.film_id = f.film_id) , (SELECT * FROM film_actor fa INNER JOIN actor a ON fa.actor_id = a.id WHERE fa.film_id = f.film_id) FROM film f INNER JOIN language l ON f.language_id = l.id INNER JOIN language ol ON f.original_language_id = ol.id SQL can't escape the RM
  43. 43. SELECT f.* , l.*, ol.* , c.*, a.* FROM film f INNER JOIN language l ON f.language_id = l.id INNER JOIN language ol ON f.original_language_id = ol.id LEFT JOIN film_category fc ON f.id = fc.film_id LEFT JOIN category c ON fc.category_id = c.id LEFT JOIN film_actor fa ON f.id = fa.film_id LEFT JOIN actor a ON fa.actor_id = a.id SQL can't escape the RM
  44. 44. How do we get rid of SQL? <ul><li>Object relational mapper?
  45. 45. NoSQL? </li><ul><li>Document databases
  46. 46. ACID
  47. 47. Schema flexibility </li></ul><li>Can we preserve the RM? </li></ul>
  48. 48. Agenda <ul><li>The data access problem
  49. 49. What's wrong with SQL?
  50. 50. The Metaweb Query Language (MQL)
  51. 51. MQL-to-SQL
  52. 52. Demo
  53. 53. Questions </li></ul>
  54. 54. The Metaweb Query Language <ul><li>MQL – rhymes with “pickle”
  55. 55. Native query language of Freebase </li><ul><li>http://www.freebase.com/
  56. 56. http://www.freebase.com/docs </li></ul><li>JSON over HTTP </li><ul><li>Natural fit for AJAX applications </li></ul></ul>
  57. 57. Freebase MQL Services <ul><li>http://api.freebase.com/api/service/mqlread? query ={&quot;query&quot;:[{ Your MQL here }]}
  58. 58. http://api.freebase.com/api/service/mqlread? queries ={&quot;q1&quot;: {&quot;query&quot;:[{ Your MQL here }]}, &quot;q2&quot;: {&quot;query&quot;:[{ Your MQL here }]}}
  59. 59. http://api.freebase.com/api/service/mqlwrite </li></ul>
  60. 60. JSON <ul><li>Subset of JavaScript </li><ul><li>Object : set of property/value pairs
  61. 61. Scalar: boolean, null, number, string
  62. 62. Array: list of values </li></ul></ul>{ &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : &quot;ACADAMY DINOSAUR&quot; , &quot;language&quot; : { &quot;language_id&quot; : 1 , &quot;language_name&quot; : &quot;English&quot; } , &quot;category&quot; :[ &quot;Action&quot; , &quot;Adventure&quot; ] }
  63. 63. JSON is great for AJAX <ul><li>Modern Web applications </li><ul><li>Asynchronous JavaScript and XML
  64. 64. ...except it's JSON rather than XML </li></ul><li>1 to 1 runtime representation </li><ul><li>XML requires mapping, DOM traversal
  65. 65. JSON support in all modern browsers </li></ul><li>Cross-domain requests (JSONP) </li><ul><li><script> injection </li></ul></ul>
  66. 66. MQL Queries <ul><li>JSON-based </li><ul><li>Both Query and Result </li></ul><li>MQL is “query by example” </li><ul><li>MQL Query is an object template
  67. 67. Finds objects matching the template
  68. 68. Fills in value placeholders in template </li></ul><li>Query & Result symmetry </li><ul><li>Easy to parse, easy to generate </li></ul></ul>
  69. 69. Basic MQL query { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : &quot;ACADAMY DINOSAUR&quot; , &quot;language&quot; : { &quot;language_id&quot; : 1 , &quot;language_name&quot; : &quot;English&quot; }, &quot;category&quot;: [ &quot;Action&quot; , &quot;Adventure&quot; ] } { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : null , &quot;language&quot; : { &quot;language_id&quot; : null , &quot;name&quot; : null }, &quot;category&quot; : [] } Query: Result: <ul><li>Property/value pairs act as filter
  70. 70. Value placeholders return data </li></ul>
  71. 71. Meta: Objects & Properties <ul><li>Objects are bags of properties
  72. 72. Properties </li><ul><li>name/value pairs
  73. 73. Single- or multi-valued (array)
  74. 74. Value can be scalar or object </li></ul><li>No explicit relationships </li><ul><li>Implicit: properties with object values </li></ul></ul>
  75. 75. Meta: Types <ul><li>Types define a bag of properties </li><ul><li>Objects may have multiple types
  76. 76. Types are grouped in domains </li></ul><li>In MQL, use the type property </li><ul><li>type : &quot;/domainname/typename&quot; </li></ul><li>Property has exactly 1 expected type </li></ul>
  77. 77. MQL Joins/subqueries <ul><li>Implicit via object-type properties
  78. 78. Can't specify join conditions </li></ul>
  79. 79. MQL operators <ul><li>Property/value act as equals operator
  80. 80. Explicit operators for advanced filters
  81. 81. != , < , <= , >= , > </li><ul><li>Have their usual meaning </li></ul><li>|= and !|= test for membership </li><ul><li>&quot;one of&quot; and &quot;not one of&quot; </li></ul><li>~= for word pattern matching </li></ul>
  82. 82. MQL operators { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot;: 1 } { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id < &quot; : 2 , &quot;title ~= &quot; : &quot;ACADEMY&quot; , &quot;language != &quot; : &quot;Danish&quot; , &quot;category |= &quot; : [ &quot;Action&quot;, &quot;Adventure&quot; ], &quot;film_id&quot; : null } Query: Result: <ul><li>Operators follow the property name
  83. 83. Operations not returned in the result </li></ul>
  84. 84. MQL property aliases { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : null , &quot;title&quot; : null , &quot;1:category&quot; : &quot;Action&quot;, &quot;2:category&quot; : &quot;Adventure&quot; } Query: <ul><li>Implement an AND operation </li></ul>
  85. 85. MQL directives <ul><li>Reserved property names </li><ul><li>limit
  86. 86. return: count
  87. 87. cursor
  88. 88. * wildcard </li></ul></ul>
  89. 89. Agenda <ul><li>The data access problem
  90. 90. What's wrong with SQL?
  91. 91. The Metaweb Query Language (MQL)
  92. 92. MQL-to-SQ L
  93. 93. Demo
  94. 94. Questions </li></ul>
  95. 95. MQL-to-SQL <ul><li>Implementation of mqlread service </li><ul><li>PHP using PDO database access
  96. 96. MySQL, PostgreSQL, SQLite, Oracle
  97. 97. http://code.google.com/p/mql-to-sql
  98. 98. GPL
  99. 99. Lacks many features, has many bugs
  100. 100. don't use in hospitals and airplanes </li></ul></ul>
  101. 101. MQL-to-SQL <ul><li>MQL domain maps to a SQL schema
  102. 102. MQL type maps to a SQL table
  103. 103. MQL property maps to </li><ul><li>SQL column
  104. 104. Relationship (foreign key) </li></ul><li>Mappings defined in JSON file </li></ul>
  105. 105. MQL-to-SQL index.php config.php <?php include 'config.php' ; ...code... ?> <?php $connection_file_name = ''; $metadata_file_name = ''; ?>
  106. 106. MQL-to-SQL metadata file { &quot;domains&quot; : { &quot;sakila&quot; : { &quot;schema_name&quot; : &quot;&quot; , &quot;types&quot; : { &quot;actor&quot; : { &quot;properties&quot; : { &quot;actor_id&quot; : { &quot;column_name&quot; : &quot;actor_id&quot; , &quot;nullable&quot; : false , &quot;type&quot; : &quot;/type/int&quot; }, &quot;starring_in&quot; : { &quot;type&quot; : &quot;/sakila/film_actor&quot; , &quot;direction&quot; : &quot;referenced<-referencing&quot; , &quot;join_condition&quot; : [ { &quot;referencing_column&quot; : &quot;actor_id&quot; , &quot;referenced_column&quot; : &quot;actor_id&quot; },
  107. 107. Agenda <ul><li>The data access problem
  108. 108. What's wrong with SQL?
  109. 109. The Metaweb Query Language (MQL)
  110. 110. MQL-to-SQL
  111. 111. Demo
  112. 112. Questions </li></ul>
  113. 113. Agenda <ul><li>The data access problem
  114. 114. What's wrong with SQL?
  115. 115. The Metaweb Query Language (MQL)
  116. 116. MQL-to-SQL
  117. 117. Demo
  118. 118. Questions </li></ul>
  119. 119. Finally <ul><li>I need your feedback
  120. 120. Love to see this inside a RDBMS </li><ul><li>drizzle plugin?
  121. 121. MySQL plugin?
  122. 122. PostgreSQL SP language? </li></ul><li>http://code.google.com/p/mql-to-sql/
  123. 123. Online query editor </li></ul>

×