Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL

812 views

Published on


Abstract

NoSQL databases bring the benefits of schema flexibility and
elastic scaling to the enterprise. Until recently, these benefits have
come at the expense of giving up rich declarative querying as
represented by SQL.

In today’s world of agile business, developers and organizations need
the benefits of both NoSQL and SQL in a single platform. NoSQL
(document) databases provide schema flexibility; fast lookup; and
elastic scaling. SQL-based querying provides expressive data access
and transformation; separation of querying from modeling and storage;
and a unified interface for applications, tools, and users.

Developers need to deliver applications that can easily evolve,
perform, and scale. Otherwise, the cost, effort, and delay in keeping
up with changing business needs will become significant disadvantages.

Organizations need sophisticated and rapid access to their operational data, in
order to maintain insight into their business. This access should
support both pre-defined and ad-hoc querying, and should integrate
with standard analytical tools.

This talk will cover how to build applications that combine the
benefits of NoSQL and SQL to deliver agility, performance, and
scalability. It includes:

-  N1QL, which extends SQL to JSON
-  JSON data modeling
-  Indexing and performance
-  Transparent scaling
-  Integration and ecosystem

You will walk away with an understanding of the design patterns and
best practices for effective utilization of NoSQL document
databases - all using open-source technologies.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL

  1. 1. Bringing SQL to NoSQL: Rich, Declarative Querying for NoSQL Gerald Sangudi | @sangudi | Chief Architect | Couchbase Keshav Murthy | @rkeshavmurthy | Director | Couchbase Team @N1QL
  2. 2. ©2016 Couchbase Inc. 2 Agenda  SQL  NoSQL – Motivation – Rich Data and JSON – Landscape  SQL for NoSQL – Motivation – Considerations  N1QL: Bringing SQL to NoSQL – Enterprises using N1QL – Features – Indexing – Architecture – Ecosystem  Q & A
  3. 3. ©2015 Couchbase Inc. 3
  4. 4. SQL
  5. 5. ©2016 Couchbase Inc. 5 SQL Structured Query Language is a special-purpose programming language designed for managing data held in a relational database management system (RDBMS). Originally based upon relational algebra and tuple relational calculus, SQL consists of a data definition language, data manipulation language, and a data control language. Source: https://en.wikipedia.org/wiki/SQL
  6. 6. ©2016 Couchbase Inc. 6 SQL: Core Features  Based on Relational Data Model  Declarative Query Language  SELECT, UPDATE, DELETE, INSERT, MERGE  Arithmetic, Logical, Set operators  Data types: Numerical, Character, Date,Time, etc.  Schema design & management Huge Impact of SQL on Business Applications
  7. 7. ©2016 Couchbase Inc. 7 SQL: Database Features for Application Development  Transactions  ACID  Features that enrich SQL  Stored procedures, triggers, views, indexes, catalogs  Optimizers  4GL  Extensions, integrations  Spatial,Text, Queues
  8. 8. NoSQL
  9. 9. NoSQL: Motivation
  10. 10. ©2016 Couchbase Inc. 10 Why NoSQL? Agility Scalability Performance Availability
  11. 11. ©2016 Couchbase Inc. 11 Why NoSQL? Agility  Schema flexibility  Easier management of change  In the business requirements  In the structure of the data  Change is inevitable
  12. 12. ©2016 Couchbase Inc. 12 Why NoSQL? Scalability  Elastic scaling  Size your cluster for today  Scale out on demand  Cost effective scaling  Commodity hardware  On premise or on cloud  Scale OUT instead of Scale UP
  13. 13. ©2016 Couchbase Inc. 13 Why NoSQL? Scalability: Elastic Scaling STORAGE Managed Cache Cluster Manager NoSQL Node STORAGE Managed Cache Cluster Manager NoSQL Node STORAGE Managed Cache Cluster Manager NoSQL Node STORAGE Managed Cache Cluster Manager NoSQL Node STORAGE Managed Cache Cluster Manager NoSQL Node STORAGE Managed Cache Cluster Manager NoSQL Node
  14. 14. ©2016 Couchbase Inc. 14 Why NoSQL? Performance  NoSQL systems are optimized for specific access patterns  Low response time for web & mobile user experience  Millisecond latency  Consistently high throughput to handle growth
  15. 15. ©2016 Couchbase Inc. 15 Why NoSQL? Availability  Built-in replication and fail-over  No application downtime when hardware fails  Online maintenance & upgrade  No application downtime
  16. 16. ©2016 Couchbase Inc. 16 CAPTheorem - 101  Each distributed system can only provide two properties simultaneously.  PartitionTolerance is required in our systems  Choice is between Availability and Consistency Availability Partition Tolerance Consistency
  17. 17. NoSQL: Rich Data and JSON
  18. 18. ©2016 Couchbase Inc. 18 Properties of Real-World Data  Rich structure – Attributes, Sub-structure  Relationships – To other data  Value evolution – Data is updated  Structure evolution – Data is reshaped Customer Name DOB Billing Connections Purchases
  19. 19. ©2016 Couchbase Inc. 19 Modeling Data in RelationalWorld Billing ConnectionsPurchases Contacts Customer  Rich structure  Normalize & JOIN Queries  Relationships  JOINS and Constraints  Value evolution  INSERT, UPDATE, DELETE  Structure evolution  ALTERTABLE  Application Downtime  Application Migration  ApplicationVersioning
  20. 20. ©2016 Couchbase Inc. 20 JSON 101 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "address" : { "Street" : "10, Downing Street", "City" : "San Francico”, “State” : “California”, “zip” :94401 } } • Used to represent object data in text • Representation • "Key":"Value" • DataTypes: • Number, Strings, Boolean, objects, Arrays, NULL • Hierarchical
  21. 21. ©2016 Couchbase Inc. 21 Using JSON For RealWorld Data CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Table: Customer { "Name" : "Jane Smith", "DOB" : "1990-01-30” } Customer DocumentKey: CBL2015  The primary (CustomerID) becomes the DocumentKey  Column name-Column value become KEY-VALUE pair. { "Name" : { "fname": "Jane ", "lname": "Smith” } "DOB" : "1990-01-30” } OR
  22. 22. ©2016 Couchbase Inc. 22 Using JSON to Store Data CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Table: Customer { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" } ] } Customer DocumentKey: CBL2015 CustomerID Type Cardnum Expiry CBL2015 visa 5827… 2019-03 Table: Billing  Rich Structure & Relationships – Billing information is stored as a sub-document – There could be more than a single credit card. So, use an array.
  23. 23. ©2016 Couchbase Inc. 23 Using JSON to Store Data CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Table: Customer { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2542-5847-3949", "expiry" : "2018-12" } ] } Customer DocumentKey: CBL2015 CustomerID Type Cardnum Expiry CBL2015 visa 5827… 2019-03 CBL2015 master 6274… 2018-12 Table: Billing  Value evolution  Simply add additional array element or update a value.
  24. 24. ©2016 Couchbase Inc. 24 Using JSON to Store Data CustomerID ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith Table: Connections { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2542-5847-3949", "expiry" : "2018-12" } ], "Connections" : [ { "ConnId" : "XYZ987", "Name" : "Joe Smith" }, { "ConnId" : ”SKR007", "Name" : ”Sam Smith" } } Customer DocumentKey: CBL2015  Structure evolution  Simply add new key-value pairs  No downtime to add new KV pairs  Applications can validate data  Structure evolution over time.  Relations via Reference
  25. 25. ©2016 Couchbase Inc. 25 Using JSON to Store Data { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } DocumentKey: CBL2015 CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Customer ID Type Cardnum Expiry CBL2015 visa 5827… 2019-03 CBL2015 maste r 6274… 2018-12 CustomerID ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith CustomerID item amt CBL2015 mac 2823.52 CBL2015 ipad2 623.52 CustomerID ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith Contacts Customer Billing ConnectionsPurchases
  26. 26. ©2016 Couchbase Inc. 26 Models for Representing Data Data Concern Relational Model JSON Document Model (NoSQL) Rich Structure  Multiple flat tables  Constant assembly / disassembly  Documents  No assembly required! Relationships  Represented  Queried (SQL)  Represented  Queried? Not until now… Value Evolution  Data can be updated  Data can be updated Structure Evolution  Uniform and rigid  Manual change (disruptive)  Flexible  Dynamic change
  27. 27. NoSQL: Landscape
  28. 28. ©2016 Couchbase Inc. 28 NoSQL: Landscape Document • Couchbase • MongoDB • DynamoDB • DocumentDB Graph • OrientDB • Neo4J • DEX • GraphBase Key-Value • Riak • BerkeleyDB • Redis • … Wide Column • Hbase • Cassandra • Hypertable
  29. 29. SQL for NoSQL
  30. 30. SQL for NoSQL: Motivation
  31. 31. ©2016 Couchbase Inc. 31 Business Use Cases Drive Applications How many new customers we got last month? Process Order Checkout Search stores for the shoe customer is looking for? Generate a list of shipment due today Load the new inventory data Retrieve the customer order Merge the customer lists
  32. 32. ©2015 Couchbase Inc. 32 ResultSet
  33. 33. ©2016 Couchbase Inc. 33 NoSQL Data Access  NoSQL systems provide specializedAPIs  Key-Value get and set  Script based query APIs  Limited declarative query  Richer query is needed
  34. 34. ©2015 Couchbase Inc. 34 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER • NoSQL systems provide specializedAPIs • Key-Value get and set • Each task requires custom built program • Should test & maintain it
  35. 35. ©2016 Couchbase Inc. 35 Client Side Implementation of Query is Inadequate Find High-Value Customers with Orders > $10000 Query customer objects from database • Complex codes and logic • Inefficient processing on client side For each customer object Find all the order objects for the customer Calculate the total amount for each order Sum up the grand total amount for all orders If grand total amount > $10000, Extract customer data Add customer to the high-value customer list Sort the high-value customer list LOOPING OVER MILLIONS OF CUSTOMERS IN APPLICATION!!!
  36. 36. ©2016 Couchbase Inc. 36 Why SQL for NoSQL?
  37. 37. SQL for NoSQL: Considerations
  38. 38. ©2016 Couchbase Inc. 38 Considerations  Flexible Schema – Cannot rely on predefined schema – columns, data types, data comparison  Nested Objects – support scalars, objects and array – SQL operators on nested objects  Work with distributed data store  Query Performance Optimization – Exploit data store performance – Design right kinds of indices – Optimizer { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "address" : { "Street" : "10, Downing Street", "City" : "San Francico”, “State” : “California”, “zip” :94401 } }
  39. 39. ©2016 Couchbase Inc. 39 Industry Is Extending SQL to Query JSON  JSON support in SQL – IBM Informix – Oracle – PostgreSQL – MySQL  Microsoft: SQL for DocumentDB  UC San Diego: SQL++  Couchbase: N1QL
  40. 40. N1QL: Bringing SQL to NoSQL
  41. 41. ©2016 Couchbase Inc. 41 N1QL: Agenda Enterprises Using N1QL Approach Features Indexing Architecture Ecosystem
  42. 42. N1QL: Enterprises Using N1QL
  43. 43. ©2016 Couchbase Inc. 43 Enterprises using Couchbase 43 CommunicationsTechnology Travel & Hospitality Media & Entertainment E-Commerce & Digital Advertising Retail & Apparel Games & GamingFinance & Business Services
  44. 44. N1QL: Approach
  45. 45. ©2015 Couchbase Inc. 45 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER • NoSQL systems provide specializedAPIs • Key-Value get and set • Each task requires custom built program • Should test & maintain it
  46. 46. ©2015 Couchbase Inc. 46 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER
  47. 47. ©2016 Couchbase Inc. 47 Considerations: SQL for NoSQL  Flexible Schema – Cannot rely on predefined schema – columns, data types, data comparison  Nested Objects – support scalars, objects and array – SQL operators on nested objects  Work with distributed data store  Query Performance Optimization – Exploit data store performance – Design right kinds of indices – Optimizer { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "address" : { "Street" : "10, Downing Street", "City" : "San Francico”, “State” : “California”, “zip” :94401 } }
  48. 48. ©2016 Couchbase Inc. 48 N1QL: Approach  Flexible Schema – Rely on JSON data interpretation – Define 4-value predicate logic – True, False, NULL, MISSING  Nested Objects – Key name becomes column reference – Use dot-notation and array[] reference  SQL operators on nested objects – select, join, project operators – nest and unnest for arrays & objects  Query Performance Optimization SELECT c.name, c.address.zip, c.phone[0] FROM customer c WHERE c.address.zip = 94587 AND ANY s IN c.status SATISFIES s = ‘Premium’ END AND purchases IS NOT MISSING;
  49. 49. ©2016 Couchbase Inc. 49 N1QL: Goal Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  50. 50. N1QL: Features
  51. 51. ©2016 Couchbase Inc. 51 N1QL: SELECT Statement SELECT customers.id, customers.NAME.lastname, customers.NAME.firstname Sum(orderline.amount) FROM orders UNNEST orders.lineitems AS orderline JOIN customers ON KEYS orders.custid WHERE customers.state = 'NY' GROUP BY customers.id, customers.NAME.lastname HAVING sum(orderline.amount) > 10000 ORDER BY sum(orderline.amount) DESC • Dotted sub-document reference • Names are CASE-SENSITIVE • UNNEST to flatten the arrays JOINS with Document KEY of customers
  52. 52. ©2016 Couchbase Inc. 52 N1QL: Examples SELECT d.C_ZIP, SUM(ORDLINE.OL_QUANTITY) AS TOTALQTY FROM CUSTOMER d UNNEST ORDERS as CUSTORDERS UNNEST CUSTORDERS.ORDER_LINE AS ORDLINE WHERE d.C_STATE = ”NY” GROUP BY d.C_ZIP ORDER BY TOTALQTY DESC; INSERT INTO CUSTOMER("PQR847", {"C_ID":4723, "Name":"Joe"}); UPDATE CUSTOMER c SET c.STATE=“CA”, c.C_ZIP = 94501 WHERE c.ID = 4723;
  53. 53. ©2016 Couchbase Inc. 53 N1QL: Composable SELECT Statement SELECT * FROM ( SELECT a, b, c FROM us_cust WHERE x = 1 ORDER BY x LIMIT 100 OFFSET 0 UNION ALL SELECT a, b, c FROM canada_cust WHERE y = 2 ORDER BY x LIMIT 100 OFFSET 0) AS newtab LEFT OUTER JOIN contacts ON KEYS newtab.c.contactid ORDER BY a, b, c LIMIT 10 OFFSET 0
  54. 54. ©2016 Couchbase Inc. 54 N1QL: SELECT Statement Highlights  Querying across relationships – JOINs – Subqueries  Aggregation – MIN, MAX – SUM, COUNT, AVG,ARRAY_AGG [ DISTINCT ]  Combining result sets using set operators – UNION, UNIONALL, INTERSECT, EXCEPT
  55. 55. ©2016 Couchbase Inc. 55 N1QL : Query Operators [ 1 of 2 ]  USE KEYS … – Direct primary key lookup bypassing index scans – Ideal for hash-distributed datastore – Available in SELECT, UPDATE, DELETE  JOIN … ON KEYS … – Nested loop JOIN using key relationships – Ideal for hash-distributed datastore – Current implementation supports INNER and LEFT OUTER joins
  56. 56. ©2016 Couchbase Inc. 56 N1QL : Query Operators [ 2 of 2 ]  NEST – Special JOIN that embeds external child documents under their parent – Ideal for JSON encapsulation  UNNEST – Flattening JOIN that surfaces nested objects as top-level documents – Ideal for decomposing JSON hierarchies JOIN, NEST, and UNNEST can be chained in any combination
  57. 57. ©2016 Couchbase Inc. 57 N1QL : Expressions for JSON ©2014 Couchbase, Inc. 57 Ranging over collections • WHERE ANY c IN children SATISFIES c.age > 10 END • WHERE EVERY r IN ratings SATISFIES r > 3 END Mapping with filtering • ARRAY c.name FOR c IN children WHEN c.age > 10 END Deep traversal, SET, and UNSET • WHERE ANY node WITHIN request SATISFIES node.type = “xyz” END • UPDATE doc UNSET c.field1 FOR c WITHIN doc END DynamicConstruction • SELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object • SELECT [ a, b ] FROM … // Dynamic array Nested traversal • SELECT x.y.z, a[0] FROM a.b.c … IS [ NOT ] MISSING • WHERE name IS MISSING
  58. 58. ©2016 Couchbase Inc. 58 N1QL : DataTypes from JSON N1QL supports all JSON data types  Numbers  Strings  Booleans  Null  Arrays  Objects
  59. 59. ©2016 Couchbase Inc. 59 N1QL : DataType Handling Non-JSON data types – MISSING – Binary Data type handling  Date functions for string and numeric encodings  Total ordering across all data types – Well defined semantics for ORDER BY and comparison operators  Defined expression semantics for all input data types – No type mismatch errors
  60. 60. ©2016 Couchbase Inc. 60 N1QL: Data Modification Statements  UPDATE … SET … WHERE …  DELETE FROM … WHERE …  INSERT INTO … ( KEY,VALUE ) VALUES …  INSERT INTO … ( KEY …,VALUE … ) SELECT …  MERGE INTO … USING … ON … WHEN [ NOT ] MATCHEDTHEN … Note: Couchbase provides per-document atomicity.
  61. 61. ©2016 Couchbase Inc. 61 N1QL: Data Modification Statements INSERT INTO ORDERS (KEY, VALUE) VALUES ("1.ABC.X382", {"O_ID":482, "O_D_ID":3, "O_W_ID":4}); UPDATE ORDERS SET O_CARRIER_ID = ”ABC987” WHERE O_ID = 482 AND O_D_ID = 3 AND O_W_ID = 4 DELETE FROM NEW_ORDER WHERE NO_D_ID = 291 AND NO_W_ID = 3482 AND NO_O_ID = 2483 JSON literals can be used in any expression
  62. 62. N1QL: Indexing
  63. 63. ©2016 Couchbase Inc. 63 Index Statements  CREATE INDEX ON …  DROP INDEX …  EXPLAIN … Highlights  Composite Index  Array Index  Functional index  Partial Index
  64. 64. ©2016 Couchbase Inc. 64 Index Overview: Secondary Index  Secondary Index can be created on any combination of attribute names.  CREATE INDEX idx_cust_cardnum customer(ccInfo.cardNumber, postalcode)  Useful in speeding up the queries.  Need to have matching indices with right key- ordering  (ccInfo.cardExpiry, postalCode)  (type, state, lastName, firstName) "customer": { "ccInfo": { "cardExpiry": "2015-11-11", "cardNumber”:"1212-232-1234", "cardType": "americanexpress” }, "customerId": "customer534", "dateAdded": "2014-04-06", "dateLastActive”:"2014-05-02”, "emailAddress”:”iles@kertz.name", "firstName": "Mckayla", "lastName": "Brown", "phoneNumber": "1-533-290-6403", "postalCode": "92341", "state": "VT", "type": "customer" } Document key: “customer534”
  65. 65. N1QL: Architecture
  66. 66. ©2016 Couchbase Inc. 66 Couchbase Server Cluster Service Deployment 66 STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Managed Cache Storage Data Servic e STORAGE Couchbase Server 2 Managed Cache Cluster ManagerCluster Manager Data Servic e STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Servic e STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Query Servic e STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Query Servic e STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Index Servic e Managed Cache Storage Managed Cache Storage Storage STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Index Servic e Storage Managed Cache Managed Cache SDK JDBC
  67. 67. ©2016 Couchbase Inc. 67 N1QL: Query Execution Flow Clients 1. Submit the query over RESTAPI 8. Query result 2. Parse, Analyze, create Plan 7. Evaluate: Documents to results 3. Scan Request; index filters 6. Fetch the documents Index Servic e Query Service Data Servic e 4. Get qualified doc keys 5. Fetch Request, doc keys SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  68. 68. ©2016 Couchbase Inc. 68 N1QL: Inside the Query Service Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  69. 69. N1QL: Ecosystem
  70. 70. ©2016 Couchbase Inc. 70 Enterprise Ecosystem via JDBC/ODBC JDBC ODBC SDK
  71. 71. Bringing SQL to NoSQL: Summary
  72. 72. ©2015 Couchbase Inc. 72 Find High-Value Customers with Orders > $10000 With SQL Without SQL Query customer objects from database • Proven and expressive query language • Leverage SQL skills and ecosystem • Extended for JSON • Complex codes and logic • Inefficient processing on client side For each customer object Find all the order objects for the customer Calculate the total amount for each order Sum up the grand total amount for all orders If grand total amount > $10000, Extract customer data Add customer to the high-value customer list Sort the high-value customer list SELECT Customers.ID, Customers.Name, SUM(OrderLine.Amount) FROM Orders UNNEST Orders.LineItems AS OrderLine JOIN Customers ON KEYS Orders.CustID GROUP BY Customers.ID, Customers.Name HAVING SUM(OrderLine.Amount) > 10000 ORDER BY SUM(OrderLine.Amount) DESC LOOPING OVER MILLIONS OF CUSTOMERS IN APPLICATION!!!
  73. 73. ©2016 Couchbase Inc. 73 Bringing SQL to NoSQL Query Features SQL on RDBMS SQL on NoSQL Statements  SELECT, INSERT, UPDATE, DELETE, MERGE  SELECT, INSERT, UPDATE, DELETE, MERGE Query Operations  Select, Join, Project, Subqueries  Strict Schema  StrictType checking  Select, Join, Project, Subqueries  Nest & Unnest  Look Ma! NoType Mismatch Errors!  JSON keys act as columns Schema  Predetermined Columns  Fully addressable JSON  Flexible document structure DataTypes  SQL Data types  Conversion Functions  JSON Data types  Conversion Functions Query Processing  INPUT: Sets ofTuples  OUPUT: Set ofTuples  INPUT: Sets of JSON  OUTPUT: Set of JSON
  74. 74. ©2016 Couchbase Inc. 74 Try SQL for JSON N1QL is available Couchbase 4.x and above –http://query.couchbase.com –Online Interactive Tutorial Ask us questions – Couchbase forums, Stack Overflow, @N1QL
  75. 75. Thank you Q & A Gerald Sangudi | @sangudi | Chief Architect | Couchbase Keshav Murthy| @rkeshavmurthy | Director | Couchbase @N1QL | query.couchbase.com

×