Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introducing N1QL: Query Without Compromise – Couchbase Connect 2015

1,324 views

Published on

This session introduces Couchbase’s query language for JSON, N1QL, and sets the stage for the rich selection of N1QL-related sessions at Couchbase Connect 2015. N1QL extends the querying power of SQL with the modeling flexibility of JSON. In this session, Couchbase’s Chief Architect of Query, Gerald Sangudi will give you an introduction to the N1QL language, architecture, and ecosystem. Plus, hear the benefits of N1QL for developers and for enterprises.

Published in: Technology
  • Be the first to comment

Introducing N1QL: Query Without Compromise – Couchbase Connect 2015

  1. 1. INTRODUCING N1QL: QUERY WITHOUT COMPROMISE Gerald Sangudi | @sangudi Chief Architect, Query | Couchbase Team @N1QL
  2. 2. ©2015 Couchbase Inc. 2 Agenda  The Nature of Data  Why Bring SQL to NoSQL  The N1QL Language  The Query Service  Benefits of N1QL for yourApplication and Enterprise  Community and Participation  Getting Started  Q & A
  3. 3. The Nature of Data
  4. 4. ©2015 Couchbase Inc. 4 Data Comes from the Real World
  5. 5. ©2015 Couchbase Inc. 5 Properties of Real-World Data  Rich structure  Attributes, Sub-structure  Relationships  To other data  Value evolution  Data is updated  Structure evolution  Data is reshaped Person Name DOB Billing Connections Purchases
  6. 6. So Why Bring SQL to NoSQL?
  7. 7. ©2015 Couchbase Inc. 7 Models for Representing Data Data Concern Relational Model JSON Document Model (NoSQL) Rich Structure  Multiple flat tables  Constant assembly / disassembly  Documents  No assembly required! Relationships  Represented  Queried (SQL)  Represented  Queried? Not until now… Value Evolution  Data can be updated  Data can be updated Structure Evolution  Uniform and rigid  Manual change (disruptive)  Flexible  Dynamic change
  8. 8. ©2015 Couchbase Inc. 8 For the Best of Both, Bring SQL to NoSQL  From JSON  Rich structure, no assembly  Schema flexibility  From SQL  General, proven, and pervasive query capabilities  Querying across relationships  The Result:  N1QL: SQL for JSON
  9. 9. The N1QL Language
  10. 10. ©2015 Couchbase Inc. 10 The N1QL Language Goal Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  11. 11. N1QL Features from SQL
  12. 12. ©2015 Couchbase Inc. 12 SELECT Statement SELECT [ DISTINCT ] … FROM … JOIN … WHERE … GROUP BY … HAVING … ORDER BY … LIMIT … OFFSET … ( UNION | INTERSECT | EXCEPT ) [ ALL ] …
  13. 13. ©2015 Couchbase Inc. 13 SELECT Statement Highlights  Querying across relationships  JOINs  Subqueries  Aggregation  MIN, MAX  ( SUM,COUNT,AVG, ARRAY_AGG ) [ DISTINCT ]  Combining result sets using set operators  ( UNION, INTERSECT, EXCEPT ) [ DISTINCT ]
  14. 14. ©2015 Couchbase Inc. 14 Data Modification Statements  UPDATE … SET …WHERE …  DELETE FROM …WHERE …  INSERT INTO … ( KEY,VALUE ) VALUES …  INSERT INTO … ( KEY …,VALUE … ) SELECT …  MERGE INTO … USING … ON … WHEN [ NOT ] MATCHEDTHEN … Note: Couchbase Server provides per-document atomicity.
  15. 15. ©2015 Couchbase Inc. 15 Index Statements  CREATE INDEX ON …  DROP INDEX …  EXPLAIN … Highlights  Functional indexes  on any data expression  Partial indexes
  16. 16. ©2015 Couchbase Inc. 16 N1QL Expressions from SQL ©2014 Couchbase, Inc. 16 Literals • Primitives [ 0, ‘hello’,TRUE ] • NULL Operators • Arithmetic [ +, -, *, /, % ] • Logical [AND,OR, NOT ] • Comparison [ <, <=, =, !=, >=, >, BETWEEN, IS NULL ] • Pattern matching [ LIKE ] • Conditional [ CASE ] Scalar functions • Numeric [ trigonometric, ROUND,TRUNC, … ] • String [ UPPER, LOWER,TRIM, SUBSTR, … ] • Date [ string and numeric dates, NOW, date arithmetic, … ] Aggregate functions • MIN, MAX, SUM, AVG,COUNT [ DISTINCT ] Subqueries • Subqueries are full expressions
  17. 17. N1QL Extensions to SQL
  18. 18. ©2015 Couchbase Inc. 18 N1QL Query Operators [ 1 of 2 ]  USE KEYS …  Direct primary key lookup bypassing index scans  Ideal for hash-distributed datastore  Available in SELECT, UPDATE, DELETE  JOIN … ON KEYS …  Nested loop JOIN using key relationships  Ideal for hash-distributed datastore
  19. 19. ©2015 Couchbase Inc. 19 N1QL Query Operators [ 2 of 2 ]  NEST  Special JOIN that embeds external child documents under their parent  Ideal for JSON encapsulation  UNNEST  Flattening JOIN that surfaces nested objects as top-level documents  Ideal for decomposing JSON hierarchies JOIN, NEST, and UNNEST can be chained in any combination
  20. 20. ©2015 Couchbase Inc. 20 N1QL Expressions for JSON ©2014 Couchbase, Inc. 20 Ranging over collections • WHERE ANY c IN children SATISFIES c.age > 10 END • WHERE EVERY r IN ratings SATISFIES r > 3 END Mapping with filtering • ARRAY c.name FOR c IN children WHEN c.age > 10 END Deep traversal, SET, and UNSET • WHERE ANY node WITHIN request SATISFIES node.type = “xyz” END • UPDATE doc UNSET c.field1 FOR c WITHIN doc END DynamicConstruction • SELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object • SELECT [ a, b ] FROM … // Dynamic array Nested traversal • SELECT x.y.z, a[0] FROM a.b.c … IS [ NOT ] MISSING • WHERE name IS MISSING
  21. 21. ©2015 Couchbase Inc. 21 N1QL DataTypes from JSON N1QL supports all JSON data types  Numbers  Strings  Booleans  Null  Arrays  Objects
  22. 22. ©2015 Couchbase Inc. 22 N1QL DataType Handling Non-JSON data types  MISSING  Binary Data type handling  Date functions for string and numeric encodings  Total ordering across all data types  Well defined semantics for ORDER BY and comparison operators  Defined expression semantics for all input data types  No type mismatch errors
  23. 23. The Query Service in Couchbase Server 4.0
  24. 24. ©2015 Couchbase Inc. 24 Topology of Couchbase Server 4.0 Client SDK App Index CB Node ODBC / JDBC App Query Manager Data Index CB Node Query Manager Data Index CB Node Query Manager Data Index CB Node Query Manager Data Index CB Node Query Manager Data Index CB Node Query Manager Data Multi-dimensional scaling Query throughput scales with nodes Multiple connectivity options ODBC / JDBC BI Tool
  25. 25. ©2015 Couchbase Inc. 25 Query Engine Query Processor Inside a Query Engine Service Listener Parser Optimizer Data Stores Execution Engine Couchbase Server Auth Data Indexers GSI Views Others… 8093/18903 File system Data Service Index Service ...... Cluster Manager Bucket #1 Bucket #2 Index#2Index#1 Admin Listener
  26. 26. ©2015 Couchbase Inc. 26 Query Request Processing Index Services Clients Data Services (1) Request (7) Response Query Services (3) Plan: AST to execution plan (6) Evaluate: Documents to results (4) Scan: Attributes to keys (5) Fetch: Keys to documents - Prepared statements - Streaming, no cursors (2) Parse: Query to AST
  27. 27. ©2015 Couchbase Inc. 27 Query Execution Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e Data-parallel — Query latency scales with cores In-memory, streamed Pluggable architecture — datastore, indexer, client, …
  28. 28. Benefits of N1QL for your Application and Enterprise
  29. 29. ©2015 Couchbase Inc. 29 Benefits for your Application and Enterprise  Model your Data Cleanly  Model once, query  Use both relationships and embedding  Query your Data with Flexibility  Query across structure, relationships, and datasets  Query across change and heterogeneity  Develop Rich Applications with Agility  Deliver features that transform, combine, and aggregate data  Use your favorite clients, frameworks, and interfaces  Integrate your Applications and Data  Leverage ecosystem: Connectivity, Analytics, Cloud and Packaged Apps
  30. 30. Community and Participation
  31. 31. ©2015 Couchbase Inc. 31 Community and Participation  Enterprise and Community Beta Programs  Community Beta  Work with CB engineers and influence the product  Showcase your use case  Community Ecosystem  Build complementary tools, products, and drivers  Build and integrate via our open APIs
  32. 32. ©2015 Couchbase Inc. 32 Community and Participation in Action Couchbase N1QL Query Runner  byWarren Postma, Beta customer  30-minute quick start with Python’sTkinter GUI package
  33. 33. Getting Started
  34. 34. ©2015 Couchbase Inc. 34 Getting Started Test drive Couchbase Server 4.0 Beta Visit query.couchbase.com Play with the online tutorial Ask us questions  Couchbase forums, Stack Overflow, @N1QL
  35. 35. N1QL Sessions at Couchbase Connect ‘15
  36. 36. ©2015 Couchbase Inc. 36 N1QL Customer Sessions Wed June 3 Thurs June 4 9:30am Keynote 9:45am Keynote 3:45pm Developer 1:45pm Developer
  37. 37. ©2015 Couchbase Inc. 37 N1QL Partner Sessions Wed June 3 Thurs June 4 1:00pm Big Data 1:45pm Big Data 3:45pm Big Data 3:45pm Big Data 4:30pm Big Data
  38. 38. ©2015 Couchbase Inc. 38 Wed June 3 Thurs June 4 1:45, 3:45pm Architecture 1:00pm Developer 2:30pm Architecture 1:45pm Architecture 4:30pm Developer 4:30pm Big Data Sampling of N1QL Sessions by Couchbase Indexing: Intro & Deep Dive N1QL: Internals & Power Features N1QL in Dev SDKs: Java, .NET, Node.js Migrating: MySQL to N1QL Tuning Query Performance Big Data Query Landscape
  39. 39. Q & A
  40. 40. Thank you. Gerald Sangudi, Couchbase @sangudi
  41. 41. ©2015 Couchbase Inc. 41

×