MQL-to-SQL  a JSON-based query language  for RDBMS data access from AJAX  applications
Welcome! <ul><li>Roland Bouman, Leiden, the Netherlands
Ex-MySQL, ex-Sun Microsystems
Web & BI developer at Strukton Rail
Author </li><ul><li>“Pentaho Solutions”
“Pentaho Kettle Solutions” </li></ul><li>http://rpbouman.blogspot.com , @rolandbouman </li></ul>
Agenda <ul><li>The data access problem
What's wrong with SQL?
The Metaweb Query Language (MQL)
MQL-to-SQL
Demo
Questions </li></ul>
Agenda <ul><li>The data access problem
What's wrong with SQL?
The Metaweb Query Language (MQL)
MQL-to-SQL
Demo
Questions </li></ul>
The Data Access Problem <ul><li>Web application </li></ul><ul><li>Classical Client/Server </li></ul>SQL Resultset SQL Resu...
The Data Access Problem <ul><li>Build a service </li><ul><li>Protocol on top of HTTP: REST, RPC
Data format: JSON, XML </li></ul><li>Advantage: controlled data access
Disadvantage: controlled=limited </li></ul>
The Data Access Problem <ul><li>Advantage: unlimited data access
Disadvantage: unlimited=uncontrolled </li></ul>SQL SQL Resultset JSON <ul><li>Build a query service </li><ul><li>Example: ...
Agenda <ul><li>The data access problem
What's wrong with SQL?
The Metaweb Query Language (MQL)
MQL-to-SQL
Demo
Questions </li></ul>
What's wrong with SQL? <ul><li>Nothing of course, it's great!!! </li><ul><li>very powerful
expressive syntax
declarative
designed for the RDBMS
portable
well supported </li></ul></ul>
What's wrong with SQL? <ul><li>SQL is very powerful </li><ul><li>...way too powerful </li></ul><li>SQL has expressive synt...
SELECT  f.title ,  f.description ,  f.release_year ,  f.length ,  f.rating ,  a.first_name ,  a.last_name ,  l.name FROM  ...
SELECT f.title ,  f.description ,  f.release_year ,  f.length ,  f.rating ,  a.first_name ,  a.last_name ,  spoken_in.name...
Upcoming SlideShare
Loading in...5
×

Mql to-sql - a json-based rdbms query language

4,161

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,161
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
26
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Mql to-sql - a json-based rdbms query language

  1. 1. MQL-to-SQL a JSON-based query language for RDBMS data access from AJAX applications
  2. 2. Welcome! <ul><li>Roland Bouman, Leiden, the Netherlands
  3. 3. Ex-MySQL, ex-Sun Microsystems
  4. 4. Web & BI developer at Strukton Rail
  5. 5. Author </li><ul><li>“Pentaho Solutions”
  6. 6. “Pentaho Kettle Solutions” </li></ul><li>http://rpbouman.blogspot.com , @rolandbouman </li></ul>
  7. 7. Agenda <ul><li>The data access problem
  8. 8. What's wrong with SQL?
  9. 9. The Metaweb Query Language (MQL)
  10. 10. MQL-to-SQL
  11. 11. Demo
  12. 12. Questions </li></ul>
  13. 13. Agenda <ul><li>The data access problem
  14. 14. What's wrong with SQL?
  15. 15. The Metaweb Query Language (MQL)
  16. 16. MQL-to-SQL
  17. 17. Demo
  18. 18. Questions </li></ul>
  19. 19. The Data Access Problem <ul><li>Web application </li></ul><ul><li>Classical Client/Server </li></ul>SQL Resultset SQL Resultset RDBMS HTTP Browser Desktop Application ?
  20. 20. The Data Access Problem <ul><li>Build a service </li><ul><li>Protocol on top of HTTP: REST, RPC
  21. 21. Data format: JSON, XML </li></ul><li>Advantage: controlled data access
  22. 22. Disadvantage: controlled=limited </li></ul>
  23. 23. The Data Access Problem <ul><li>Advantage: unlimited data access
  24. 24. Disadvantage: unlimited=uncontrolled </li></ul>SQL SQL Resultset JSON <ul><li>Build a query service </li><ul><li>Example: dbslayer </li></ul></ul>
  25. 25. Agenda <ul><li>The data access problem
  26. 26. What's wrong with SQL?
  27. 27. The Metaweb Query Language (MQL)
  28. 28. MQL-to-SQL
  29. 29. Demo
  30. 30. Questions </li></ul>
  31. 31. What's wrong with SQL? <ul><li>Nothing of course, it's great!!! </li><ul><li>very powerful
  32. 32. expressive syntax
  33. 33. declarative
  34. 34. designed for the RDBMS
  35. 35. portable
  36. 36. well supported </li></ul></ul>
  37. 37. What's wrong with SQL? <ul><li>SQL is very powerful </li><ul><li>...way too powerful </li></ul><li>SQL has expressive syntax </li><ul><li>...and too hard to parse and generate </li></ul><li>SQL is declarative </li><ul><li>...but not enough </li></ul><li>SQL is designed for the RDBMS </li><ul><li>...can't escape relational model </li></ul></ul>
  38. 38. SELECT f.title , f.description , f.release_year , f.length , f.rating , a.first_name , a.last_name , l.name FROM film f INNER JOIN language l ON f.language_id = l.id LEFT JOIN film_actor fa ON f.id = fa.film_id LEFT JOIN actor a ON fa.actor_id = a.id SQL not declarative enough Spoken in Has cast Starring in
  39. 39. SELECT f.title , f.description , f.release_year , f.length , f.rating , a.first_name , a.last_name , spoken_in.name FROM film f RELATE spoken_in language l , has_cast film_actor fa RELATE starring_in actor a SQL not declarative enough Spoken in Has cast Starring in
  40. 40. SQL can't escape the RM Unnormalized data Split off – multivalued attributes – repeating groups First normal form (1NF) Higher normal forms (>= 3NF) Remove redundancy Denormalized data (1NF) SQL queries: – JOIN operation ??? Header – line 1a – line 2b header line 1a line 2b header 1 2 line 1a line 2b header line 1a header line 2b
  41. 41. SQL can't escape the RM
  42. 42. SELECT f.* , l.*, ol.* , (SELECT * FROM film_category fc INNER JOIN category c ON fc.category_id = c.id WHERE fc.film_id = f.film_id) , (SELECT * FROM film_actor fa INNER JOIN actor a ON fa.actor_id = a.id WHERE fa.film_id = f.film_id) FROM film f INNER JOIN language l ON f.language_id = l.id INNER JOIN language ol ON f.original_language_id = ol.id SQL can't escape the RM
  43. 43. SELECT f.* , l.*, ol.* , c.*, a.* FROM film f INNER JOIN language l ON f.language_id = l.id INNER JOIN language ol ON f.original_language_id = ol.id LEFT JOIN film_category fc ON f.id = fc.film_id LEFT JOIN category c ON fc.category_id = c.id LEFT JOIN film_actor fa ON f.id = fa.film_id LEFT JOIN actor a ON fa.actor_id = a.id SQL can't escape the RM
  44. 44. How do we get rid of SQL? <ul><li>Object relational mapper?
  45. 45. NoSQL? </li><ul><li>Document databases
  46. 46. ACID
  47. 47. Schema flexibility </li></ul><li>Can we preserve the RM? </li></ul>
  48. 48. Agenda <ul><li>The data access problem
  49. 49. What's wrong with SQL?
  50. 50. The Metaweb Query Language (MQL)
  51. 51. MQL-to-SQL
  52. 52. Demo
  53. 53. Questions </li></ul>
  54. 54. The Metaweb Query Language <ul><li>MQL – rhymes with “pickle”
  55. 55. Native query language of Freebase </li><ul><li>http://www.freebase.com/
  56. 56. http://www.freebase.com/docs </li></ul><li>JSON over HTTP </li><ul><li>Natural fit for AJAX applications </li></ul></ul>
  57. 57. Freebase MQL Services <ul><li>http://api.freebase.com/api/service/mqlread? query ={&quot;query&quot;:[{ Your MQL here }]}
  58. 58. http://api.freebase.com/api/service/mqlread? queries ={&quot;q1&quot;: {&quot;query&quot;:[{ Your MQL here }]}, &quot;q2&quot;: {&quot;query&quot;:[{ Your MQL here }]}}
  59. 59. http://api.freebase.com/api/service/mqlwrite </li></ul>
  60. 60. JSON <ul><li>Subset of JavaScript </li><ul><li>Object : set of property/value pairs
  61. 61. Scalar: boolean, null, number, string
  62. 62. Array: list of values </li></ul></ul>{ &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : &quot;ACADAMY DINOSAUR&quot; , &quot;language&quot; : { &quot;language_id&quot; : 1 , &quot;language_name&quot; : &quot;English&quot; } , &quot;category&quot; :[ &quot;Action&quot; , &quot;Adventure&quot; ] }
  63. 63. JSON is great for AJAX <ul><li>Modern Web applications </li><ul><li>Asynchronous JavaScript and XML
  64. 64. ...except it's JSON rather than XML </li></ul><li>1 to 1 runtime representation </li><ul><li>XML requires mapping, DOM traversal
  65. 65. JSON support in all modern browsers </li></ul><li>Cross-domain requests (JSONP) </li><ul><li><script> injection </li></ul></ul>
  66. 66. MQL Queries <ul><li>JSON-based </li><ul><li>Both Query and Result </li></ul><li>MQL is “query by example” </li><ul><li>MQL Query is an object template
  67. 67. Finds objects matching the template
  68. 68. Fills in value placeholders in template </li></ul><li>Query & Result symmetry </li><ul><li>Easy to parse, easy to generate </li></ul></ul>
  69. 69. Basic MQL query { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : &quot;ACADAMY DINOSAUR&quot; , &quot;language&quot; : { &quot;language_id&quot; : 1 , &quot;language_name&quot; : &quot;English&quot; }, &quot;category&quot;: [ &quot;Action&quot; , &quot;Adventure&quot; ] } { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : 1 , &quot;title&quot; : null , &quot;language&quot; : { &quot;language_id&quot; : null , &quot;name&quot; : null }, &quot;category&quot; : [] } Query: Result: <ul><li>Property/value pairs act as filter
  70. 70. Value placeholders return data </li></ul>
  71. 71. Meta: Objects & Properties <ul><li>Objects are bags of properties
  72. 72. Properties </li><ul><li>name/value pairs
  73. 73. Single- or multi-valued (array)
  74. 74. Value can be scalar or object </li></ul><li>No explicit relationships </li><ul><li>Implicit: properties with object values </li></ul></ul>
  75. 75. Meta: Types <ul><li>Types define a bag of properties </li><ul><li>Objects may have multiple types
  76. 76. Types are grouped in domains </li></ul><li>In MQL, use the type property </li><ul><li>type : &quot;/domainname/typename&quot; </li></ul><li>Property has exactly 1 expected type </li></ul>
  77. 77. MQL Joins/subqueries <ul><li>Implicit via object-type properties
  78. 78. Can't specify join conditions </li></ul>
  79. 79. MQL operators <ul><li>Property/value act as equals operator
  80. 80. Explicit operators for advanced filters
  81. 81. != , < , <= , >= , > </li><ul><li>Have their usual meaning </li></ul><li>|= and !|= test for membership </li><ul><li>&quot;one of&quot; and &quot;not one of&quot; </li></ul><li>~= for word pattern matching </li></ul>
  82. 82. MQL operators { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot;: 1 } { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id < &quot; : 2 , &quot;title ~= &quot; : &quot;ACADEMY&quot; , &quot;language != &quot; : &quot;Danish&quot; , &quot;category |= &quot; : [ &quot;Action&quot;, &quot;Adventure&quot; ], &quot;film_id&quot; : null } Query: Result: <ul><li>Operators follow the property name
  83. 83. Operations not returned in the result </li></ul>
  84. 84. MQL property aliases { &quot;type&quot; : &quot;/sakila/film&quot; , &quot;film_id&quot; : null , &quot;title&quot; : null , &quot;1:category&quot; : &quot;Action&quot;, &quot;2:category&quot; : &quot;Adventure&quot; } Query: <ul><li>Implement an AND operation </li></ul>
  85. 85. MQL directives <ul><li>Reserved property names </li><ul><li>limit
  86. 86. return: count
  87. 87. cursor
  88. 88. * wildcard </li></ul></ul>
  89. 89. Agenda <ul><li>The data access problem
  90. 90. What's wrong with SQL?
  91. 91. The Metaweb Query Language (MQL)
  92. 92. MQL-to-SQ L
  93. 93. Demo
  94. 94. Questions </li></ul>
  95. 95. MQL-to-SQL <ul><li>Implementation of mqlread service </li><ul><li>PHP using PDO database access
  96. 96. MySQL, PostgreSQL, SQLite, Oracle
  97. 97. http://code.google.com/p/mql-to-sql
  98. 98. GPL
  99. 99. Lacks many features, has many bugs
  100. 100. don't use in hospitals and airplanes </li></ul></ul>
  101. 101. MQL-to-SQL <ul><li>MQL domain maps to a SQL schema
  102. 102. MQL type maps to a SQL table
  103. 103. MQL property maps to </li><ul><li>SQL column
  104. 104. Relationship (foreign key) </li></ul><li>Mappings defined in JSON file </li></ul>
  105. 105. MQL-to-SQL index.php config.php <?php include 'config.php' ; ...code... ?> <?php $connection_file_name = ''; $metadata_file_name = ''; ?>
  106. 106. MQL-to-SQL metadata file { &quot;domains&quot; : { &quot;sakila&quot; : { &quot;schema_name&quot; : &quot;&quot; , &quot;types&quot; : { &quot;actor&quot; : { &quot;properties&quot; : { &quot;actor_id&quot; : { &quot;column_name&quot; : &quot;actor_id&quot; , &quot;nullable&quot; : false , &quot;type&quot; : &quot;/type/int&quot; }, &quot;starring_in&quot; : { &quot;type&quot; : &quot;/sakila/film_actor&quot; , &quot;direction&quot; : &quot;referenced<-referencing&quot; , &quot;join_condition&quot; : [ { &quot;referencing_column&quot; : &quot;actor_id&quot; , &quot;referenced_column&quot; : &quot;actor_id&quot; },
  107. 107. Agenda <ul><li>The data access problem
  108. 108. What's wrong with SQL?
  109. 109. The Metaweb Query Language (MQL)
  110. 110. MQL-to-SQL
  111. 111. Demo
  112. 112. Questions </li></ul>
  113. 113. Agenda <ul><li>The data access problem
  114. 114. What's wrong with SQL?
  115. 115. The Metaweb Query Language (MQL)
  116. 116. MQL-to-SQL
  117. 117. Demo
  118. 118. Questions </li></ul>
  119. 119. Finally <ul><li>I need your feedback
  120. 120. Love to see this inside a RDBMS </li><ul><li>drizzle plugin?
  121. 121. MySQL plugin?
  122. 122. PostgreSQL SP language? </li></ul><li>http://code.google.com/p/mql-to-sql/
  123. 123. Online query editor </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×