Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too

175 views

Published on

Presented at All Things Open 2018
Presented by Matthew Groves with Couchbase
10/23/18 - 3:00 PM - Databases track

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too

  1. 1. Querying NoSQL with SQL HAVING Your JSON Cake and SELECTing it too
  2. 2. Querying JSON with JavaScript/JSON 2
  3. 3. Querying JSON with SQL 3
  4. 4. 4 AGENDA 01/ Why NoSQL? 02/ JSON Data Modeling 03/ SQL for JSON 04/ Ecosystem and convergence 05/ Q&A
  5. 5. Where am I? 5 • All Things Open • https://allthingsopen.org/ • https://twitter.com/AllThingsOpen
  6. 6. Who am I? 6 • Matthew D. Groves • Developer Advocate for Couchbase • @mgroves on Twitter • Podcast and blog: https://crosscuttingconcerns.com • "I am not an expert, but I am an enthusiast." – Alan Stevens by @natelovett
  7. 7. Querying NoSQL with SQL HAVING Your JSON Cake and SELECTing it too
  8. 8. Major Enterprises Across Industries are Adopting NoSQL CommunicationsTechnology Travel & Hospitality Media & Entertainment E-Commerce & Digital Advertising Retail & Apparel Games & GamingFinance & Business Services
  9. 9. Why NoSQL? 9 1
  10. 10. Why NoSQL? Scalability 12
  11. 11. Why NoSQL? Scalability 13
  12. 12. Why NoSQL? Flexibility 14
  13. 13. Why NoSQL? Performance 15
  14. 14. Why NoSQL? Availability 16
  15. 15. JSON Data Modeling 17 2
  16. 16. Models for Representing Data 18 Data Concern Relational Model JSON Document Model Rich Structure Relationships Value Evolution Structure Evolution
  17. 17. Properties of Real-World Data 19
  18. 18. Modeling Data in a Relational World 20 Billing ConnectionsPurchases Contacts Customer
  19. 19. CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Table: Customer { "Name" : "Jane Smith", "DOB" : "1990-01-30” } Customer DocumentKey: CBL2015
  20. 20. ©2017 Couchbase Inc. 25 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "cardnum" : "5827-2842…", "expiry" : "2019-03", "cardType" : "visa", "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } DocumentKey: CBL2015 Custom erID Name DOB Cardnum Expiry CardType CBL201 5 Jane Smith 1990-01- 30 5827- 2842… 2019-03 visa CustomerI D ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith CustomerI D item amt CBL2015 mac 2823.5 2 CBL2015 ipad2 623.52 CustomerI D ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith Contacts Customer ConnectionsPurchases
  21. 21. { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } DocumentKey: CBL2016 CustomerID Name DOB CBL2016 Bob Jones 1980-01-29 Custome rID Type Cardnu m Expiry CBL201 6 visa 5927… 2020-03 CBL201 6 maste r 6273… 2019-11 CustomerI D ConnId Name CBL2016 XYZ987 Joe Smith CBL2016 SKR007 Sam Smith CustomerI D item amt CBL2016 mac 2823.5 2 CBL2016 ipad2 623.52 CustomerI D ConnId Name CBL2016 XYZ98 7 Joe Smith CBL2016 SKR00 7 Sam Smith Contacts Customer Billing ConnectionsPurchases
  22. 22. Models for Representing Data 27 Data Concern Relational Model JSON Document Model Rich Structure • Multiple flat tables • Assembly / disassembly  Documents  No (or less) assembly required Relationships  Represented  Queries with SQL  Represented  Queried…with? Value Evolution  Data can be updated  Data can be updated Structure Evolution  Uniform, rigid, enforced  Manual disruptive change  Flexible  Dynamic change  Increased app responsibility
  23. 23. SQL for JSON 28 3
  24. 24. Why SQL for JSON? 29
  25. 25. Key/Value 30
  26. 26. MapReduce 31
  27. 27. Some other declarative query 32
  28. 28. Result: Querying is Inadequate 33 Query 'customer' objects from database For each 'customer' object Find all the 'order' objects for the 'customer' Calculator the total amount for each 'order' Sum up the grand total amount for all 'orders' If grand total amount > $10000, extract 'customer' data Add 'customer' to the high-value customer list Sort the high-value customer list Example: • Customers have orders. Orders have items. Items have prices. • Find High-Value Customers with Orders > $10000
  29. 29. What would SQL for JSON look like? 35
  30. 30. Route document example 36 route_5966 ← key { "type": "route", ← document type identifier "airlineid": "airline_24", ← foreign key "sourceairport": "MCO", "destinationairport": "SEA", "equipment": "737", "schedule": [ {"day": 1, "utc": "13:25:00", "flight": "AA788"}, {"day": 4, "utc": "13:25:00", "flight": "AA419"}, {"day": 5, "utc": "13:25:00", "flight": "AA519"} ] }
  31. 31. Airline document example 37 airline_24 ← key { "active": "Y", "callsign": "AMERICAN", "country": "United States", "iata": "AA", "icao": "AAL", "name": "American Airlines", "type": "airline" ← document type identifier }
  32. 32. SELECT 38
  33. 33. SELECT: JOIN 39
  34. 34. SELECT: Aggregation 40
  35. 35. UNNEST (intra-document JOIN) 42
  36. 36. UNNEST (intra-document JOIN) 43
  37. 37. SQL: Tables in, tables out 44 ResultSe t
  38. 38. SQL: JSON in, JSON out 45 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo Result Documents Orders Customer CosmosD b SQL
  39. 39. N1QL (Couchbase) 46
  40. 40. SQL (DocumentDb CosmosDb) 47
  41. 41. SQL and N1QL and CosmosDb 48 SQL CosmosDb SQL ANSI SQL standard JSON stuff
  42. 42. Other JSON "Stuff" 49 Ranging over collections • WHERE ANY c IN children SATISFIES c.age > 10 END • WHERE EVERY r IN ratings SATISFIES r > 3 END Mapping with filtering • ARRAY c.name FOR c IN children WHEN c.age > 10 END Deep traversal, SET, and UNSET • WHERE ANY node WITHIN request SATISFIES node.type = “xyz” END • UPDATE doc UNSET c.field1 FOR c WITHIN doc END Dynamic Construction • SELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object • SELECT [ a, b ] FROM … // Dynamic array Nested traversal • SELECT x.y.z, a[0] FROM a.b.c …
  43. 43. DEMO 50
  44. 44. Data Modification 51
  45. 45. Indexing: CREATE INDEX and DROP INDEX 52
  46. 46. Indexing 53
  47. 47. Indexing: Plan Visualization 54
  48. 48. Find High-Value Customers with Orders > $10000 55 Query 'customer' objects from database For each 'customer' object Find all the 'order' objects for the 'customer' Calculator the total amount for each 'order' Sum up the grand total amount for all 'orders' If grand total amount > $10000, extract 'customer' data Add 'customer' to the high-value customer list Sort the high- value customer list SELECT Customers.ID, Customers.Name, SUM(OrderLine.Amount) FROM `Orders` UNNEST Orders.LineItems AS OrderLine JOIN `Customers` ON KEYS Orders.CustID GROUP BY Customers.ID, Customers.Name HAVING SUM(OrderLine.Amount) > 10000 ORDER BY SUM(OrderLine.Amount) DESC
  49. 49. Summary: SQL and SQL for JSON 56 Query Features SQL SQL for JSON Statements  SELECT, INSERT, UPDATE, DELETE, MERGE  SELECT, INSERT, UPDATE, DELETE, MERGE Query Operations  Select, Join, Project, Subqueries  Strict Schema  Strict Type checking  Select, Join, Project, Subqueries  Nest & Unnest  No Type Mismatch Errors!  JSON keys act as columns Schema  Predetermined Columns  Fully addressable JSON  Flexible document structure Data Types  SQL Data types  Conversion Functions  JSON Data types  Conversion Functions Query Processing  INPUT: Sets of Tuples  OUPUT: Set of Tuples  INPUT: Sets of JSON  OUTPUT: Set of JSON
  50. 50. Summary: N1QL vs CosmosDb 57 Query Features N1QL CosmosDB SQL SELECT Yes Yes INSERT, UPDATE, DELETE, MERGE Yes No Intra-document join Yes: UNNEST Yes: JOIN Inter-document join Yes: JOIN, NEST No Aggregation (GROUP BY, SUM,MAX,MIN) Yes No GROUP BY Stored Procedures, Triggers, UDF "Events" (JavaScript with embedded N1QL) Kinda (JavaScript)
  51. 51. SQL for JSON 58 SQL++ N1QL http://tinyurl.com/UCSDsql
  52. 52. Try SQL for JSON 66 http://tiny.cc/n1ql http://tiny.cc/cosmosdb
  53. 53. Couchbase Plug 67 • Go to Couchbase.com to download Couchbase • Enter to win a $100 gift card here: [redacted]
  54. 54. Where do you find us? 68 •blog.couchbase.com •@mgroves •@couchbasedev
  55. 55. Frequently Asked Questions 69 1. How is Couchbase different than Mongo? 2. Is Couchbase the same thing as CouchDb? 3. How tall are you? Do you play basketball? 4. What is the Couchbase licensing situation? 5. Is Couchbase a Managed Cloud Service (DBaaS)?
  56. 56. Managed Cloud Server (DBaaS) 70 < Back Short answer: No. Longer answer: Kinda. https://zdatainc.com/couchbase-managed-services/ Longest answer: See me after class.
  57. 57. MongoDB vs Couchbase 71 • Architecture • Memory first architecture • Master-master architecture • Auto-sharding • Features • SQL (N1QL) • Full Text Search • Mobile & Sync < Back
  58. 58. Licensing 72 < Back Couchbase Server Community • Open source (Apache 2) • Binary release is one release behind Enterprise (except major versions) • Free to use in dev/test/qa/prod • Forum support only Couchbase Server Enterprise • Mostly open source (Apache 2) • Some features not available on Community (XDCR TLS, MDS, Rack Zone, etc) • Free to use in dev/test/qa • Need commercial license for prod • Paid support provided
  59. 59. CouchDB and Couchbase 73 < Back memcached

×