SlideShare a Scribd company logo
1 of 50
©2016 Couchbase Inc.
Understand Query Optimizer to Tune Queries
SitaramVemulapalli | Senior Software Engineer,Couchbase R&D | sitaram.vemulapalli@couchbase.com
Keshav Murthy | Director, Couchbase R&D | keshav@couchbase.com
1
©2016 Couchbase Inc.©2016 Couchbase Inc.
Agenda
• Brief Introduction to N1QL
• Understand Query Plan
• How Query Plan Generated
• Query + Indexing features
• Q&A
2
©2016 Couchbase Inc. 3
Introduction to N1QL
©2016 Couchbase Inc.©2016 Couchbase Inc.
SQL
4
ResultSet
Input: Relations Output: Relation
©2016 Couchbase Inc.©2016 Couchbase Inc.
NoSQL
5
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
LoyaltyInfo ResultDocuments
Orders
CUSTOMER
Built Manually; Expensive
Input: JSON Documents Output: JSON Documents
©2016 Couchbase Inc.©2016 Couchbase Inc.
N1QL Changes the Possibilities for NoSQL
©2016 Couchbase Inc.©2016 Couchbase Inc.
NoSQL
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
LoyaltyInfo ResultDocuments
Orders
CUSTOMER
Input: JSON Documents Output: JSON Documents
©2016 Couchbase Inc.©2016 Couchbase Inc.
Goal of N1QL: SQL for JSON
Give developers and enterprises an
expressive, powerful, and complete language
for querying, transforming, and manipulating
JSON data.
©2016 Couchbase Inc.©2016 Couchbase Inc. 9
SELECT Customers.ID,
Customers.Name,
SUM(OrderLine.Amount)
FROM Orders UNNEST
Orders.LineItems AS OrderLine
JOIN Customers ON KEYS Orders.CustID
GROUP BY Customers.ID, Customers.Name
HAVING SUM(OrderLine.Amount) > 10000
ORDER BY SUM(OrderLine.Amount) DESC
LIMIT 100 ;
©2016 Couchbase Inc. 10
Understanding Query Plan
©2016 Couchbase Inc.©2016 Couchbase Inc.
N1QL: Query Execution Flow
Clients
1. Submit the query over REST API 8. Query result
2. Parse, Analyze, create Plan 7. Evaluate: Filter, Join, Aggregate, Sort, Paginate
3. Scan Request;
index filters
6. Fetch the documents
Index
Service
Query
Service
Data
Service
4. Get qualified doc keys
5. Fetch Request,
doc keys
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Inside a Query Service
Client
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Service
Data
Service
©2016 Couchbase Inc.©2016 Couchbase Inc.
Inside a Query Service
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Service
Data
Service
Prepare
Phase
Projection
Phase
Run
Phase
Scan
Phase
Fetch
Phase
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggregate
Scan
• Each query can be executed in several ways
• Create the query execution plan
• Access path for each keyspace referenced
• Decide on the filters to push down
• Determine Join order and join method
• Create the execution tree
• For each keyspace reference:
• Look at the available indices
• Match the filters in the query with index condition and keys
• Choose one or more indices for each keyspace
• Decide if a covering index can be used
• Covering index is preferred
• Create index filters and post scan, post join filters
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Covering Index Scan
• Predicate pushdown to Indexer
• Pushdown limit to IndexScan (4.5)
• Use Index Order (4.5)
CREATE INDEX ts_c_id ON
`travel-sample` (country, id)
WHERE type = "airline";
EXPLAIN SELECT country, id
FROM `travel-sample` t
WHERE type = "airline" AND
country = "United States" AND
id BETWEEN 10 AND 1000
ORDER BY country, id
OFFSET 5
LIMIT 10;
{
"#operator": "IndexScan",
"covers": [
"cover ((`t`.`country`))",
"cover ((`t`.`id`))",
"cover ((meta(`t`).`id`))"
],
"filter_covers": {
"cover ((`t`.`type`))": "airline"
},
"index": "ts_c_id",
"index_id": "ccd0f0c297114733",
"keyspace": "travel-sample",
"limit": "(5 + 10)",
"namespace": "default",
"spans": [
{
"Range": {
"High": [
""United States"",
"1000"
],
"Inclusion": 3,
"Low": [
""United States"",
"10"
]
}
}
],
"using": "gsi"
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• NonCovering Index Scan
• Predicate pushdown to Indexer
• Pushdown limit to IndexScan (4.5)
• Use Index order (4.5)
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT country, id, name
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND id BETWEEN 10 AND
1000
ORDER BY country, id
OFFSET 5
LIMIT 10;
{
"#operator": "IndexScan",
"index": "ts_c_id",
"index_id": "ccd0f0c297114733",
"keyspace": "travel-sample",
"limit": "(5 + 10)",
"namespace": "default",
"spans": [
{
"Range": {
"High": [
""United States"",
"1000"
],
"Inclusion": 3,
"Low": [
""United States"",
"10"
]
}
}
],
"using": "gsi"
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
N1QL: Covering Index and Covered Query
Clients
1. Submit the query over REST API 6. Query result
2. Parse, Analyze, create Plan 5. Evaluate: Filter, Join, Aggregate, Sort, Paginate
3. Scan Request; index
filters
Index
Service
Query
Service
4. Get qualified doc keys
create index i1 on CUSTOMER(c_id, c_max, c_first, c_last)
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• NonCovering Query
• Fetch (Only for Non Covering Scan)
• Filter
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT country, id, name
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND id BETWEEN 10 AND
1000
ORDER BY country, id
OFFSET 5
LIMIT 10;
{
"#operator": "Fetch",
"keyspace": "travel-sample",
"namespace": "default"
}
{
"#operator": "Filter",
"condition": "((((`t`.`type`) =
"airline") and ((`t`.`country`) =
"United States")) and ((`t`.`id`) between
10 and 1000))"
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• IntersectScan
• For non covered queries only
CREATE INDEX ts_c ON `travel-
sample`(country) WHERE type = "airline"
;
CREATE INDEX ts_n ON `travel-
sample`(name) WHERE type = "airline" ;
EXPLAIN SELECT country, name
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND name = "US
Airways";
{"#operator": "IntersectScan",
"scans": [
{ "#operator": "IndexScan",
"index": "ts_c",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [ ""United States”"],
"Inclusion": 3,
"Low": [ ""United States””]
}
}
]
},
{ "#operator": "IndexScan",
"index": "ts_n",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [""US Airways”” ],
"Inclusion": 3,
"Low": [""US Airways”” ]
}
}
]
}
]
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Array Index Key Scan (4.5)
CREATE INDEX ts_aix1 ON `travel-
sample` (DISTINCT ARRAY v.day FOR
v IN schedule END) WHERE type =
"route";
SELECT META(t).id
FROM `travel-sample` t
WHERE type = "route" AND ANY v IN
schedule SATISFIES v.day IN [1,2]
END;
{ "#operator": "DistinctScan",
"scan": {
"#operator": "IndexScan",
"index": "ts_aix1",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
"1"
],
"Inclusion": 3,
"Low": [
"1"
]
}
},
{
"Range": {
"High": [
"2"
],
"Inclusion": 3,
"Low": [
"2"
]
}
}
]
}
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• IndexCountScan (4.5)
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT COUNT(1) AS count
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND id BETWEEN 10 AND
1000;
{ "#operator": "IndexCountScan",
"covers": [
"cover ((`t`.`country`))",
"cover ((`t`.`id`))",
"cover ((meta(`t`).`id`))"
],
"index": "ts_c_id",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
""United States"",
"1000"
],
"Inclusion": 3,
"Low": [
""United States"",
"10"
]
}
}
]
}
{ "#operator": "IndexCountProject",
"result_terms": [
{
"as": "count",
"expr": "count(1)"
}
]
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• JOIN, LEFT JOIN.
• NEST, LEFT NEST will have
operator NEST.
• LEFT side keyspace of JOIN can
be covered.
• Non-covered query.
CREATE INDEX ts_rix1 ON `travel-
sample` (DISTINCT ARRAY v.day FOR v
IN schedule END, airlineid, schedule)
WHERE type = "route";
EXPLAIN SELECT r, a
FROM `travel-sample` r
LEFT JOIN `travel-sample` a
ON KEYS r.airlineid
WHERE r.type = "route" AND ANY v IN
r.schedule SATISFIES v.day = 1 END
LIMIT 1;
"scan": {
"#operator": "IndexScan",
"index": "ts_rix1",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
"1"
],
"Inclusion": 3,
"Low": [
"1"
]
}
}
]
}
{
"#operator": "Join",
"as": "a",
"keyspace": "travel-sample",
"namespace": "default",
"on_keys": "(`r`.`airlineid`)",
"outer": true
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• JOIN LEFT side covered query.
CREATE INDEX ts_rix1 ON `travel-
sample` (DISTINCT ARRAY v.day FOR v
IN schedule END, airlineid, schedule)
WHERE type = "route";
EXPLAIN SELECT r.schedule, META(r).id
FROM `travel-sample` r
LEFT JOIN `travel-sample` a
ON KEYS r.airlineid
WHERE r.type = "route" AND ANY v IN
r.schedule SATISFIES v.day = 1 END
LIMIT 1;
"scan": {
"#operator": "IndexScan",
"covers": [
"cover ((distinct (array (`v`.`day`) for
`v` in (`r`.`schedule`) end)))",
"cover ((`r`.`airlineid`))",
"cover ((`r`.`schedule`))",
"cover ((meta(`r`).`id`))"
],
"filter_covers": {
"cover ((`r`.`type`))": "route"
}
"index": "ts_rix1",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
"1"
],
"Inclusion": 3,
"Low": [
"1"
]
}
}
]
}
{
"#operator": "Join",
"as": "a",
"keyspace": "travel-sample",
"namespace": "default",
"on_keys": ”cover ((`r`.`airlineid`))",
"outer": true
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
How does this work?
CREATE INDEX brewery_state ON `beer-sample`(state);
CREATE INDEX beer_brewery_id ON `beer-sample`(brewery_id);
SELECT brewery.name as breweryname, ARRAY_AGG(beer.name) beerlist
FROM `beer-sample` brewery
LEFT OUTER JOIN
`beer-sample` beer
ON KEY beer.brewery_id FOR brewery
WHERE brewery.state = "California"
GROUP BY brewery.name;
brewerybrewery_state
[“California”,
“21st_sfo_ale”]
beer_brewery_id
(brewery_id =
“21st_sfo_ale”)
beer
[“21st_sfo_ale”
, ”pale_ale”]
(state = ‘California’)
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Index JOIN, LEFT Index JOIN.
• Index NEST, LEFT Index NEST will have
operator NEST.
• LEFT side keyspace of JOIN can be
covered.
• In 4.6 RIGHT side keyspace of Index JOIN
can be covered. NOT for Index NEST.
CREATE INDEX ts_c ON `travel-
sample`(country) WHERE type =
"airline" ;
CREATE INDEX ts_aid ON `travel-
sample`(airlineid, airline) WHERE
type = "route" ;
EXPLAIN SELECT r, a
FROM `travel-sample` a
JOIN `travel-sample` r
ON KEY r.airlineid FOR a
WHERE r.type = "route" AND a.type =
"airline" AND a.country = "United
States"
LIMIT 1;
{
"#operator": "IndexScan",
"index": "ts_c",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
""United States""
],
"Inclusion": 3,
"Low": [
""United States""
]
}
}
]
}
{
"#operator": "IndexJoin",
"as": "r",
"for": "a",
"keyspace": "travel-sample",
"on_key": "(`r`.`airlineid`)",
"scan": {
"index": "ts_aid"
}
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Covered Index Join (4.5)
CREATE INDEX ts_c ON `travel-
sample`(country) WHERE type =
"airline" ;
CREATE INDEX ts_aid ON `travel-
sample`(airlineid, airline) WHERE
type = "route" ;
EXPLAIN SELECT a.country, r.airline
FROM `travel-sample` a
JOIN `travel-sample` r
ON KEY r.airlineid FOR a
WHERE r.type = "route" AND a.type =
"airline" AND a.country = "United
States";
{
"#operator": "IndexScan",
"covers": [
"cover ((`a`.`country`))",
"cover ((meta(`a`).`id`))"
],
"filter_covers": {
"cover ((`a`.`type`))": "airline"
},
"index": "ts_c",
"keyspace": "travel-sample",
"spans": [
{
"Range": {
"High": [
""United States""
],
"Inclusion": 3,
"Low": [
""United States""
]
}
}
]
}
{
"#operator": "IndexJoin",
"as": "r",
"for": "a",
"keyspace": "travel-sample",
"on_key": "(`r`.`airlineid`)",
"scan": {
"index": "ts_aid"
}
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• LEFT & RIGHT Covered Index Join (4.6)
CREATE INDEX ts_c ON `travel-
sample`(country) WHERE type =
"airline" ;
CREATE INDEX ts_aid ON `travel-
sample`(airlineid, airline) WHERE
type = "route" ;
EXPLAIN SELECT a.country, r.airline
FROM `travel-sample` a
JOIN `travel-sample` r
ON KEY r.airlineid FOR a
WHERE r.type = "route" AND a.type =
"airline" AND a.country = "United
States";
{"#operator": "IndexScan",
"covers": [
"cover ((`a`.`country`))",
"cover ((meta(`a`).`id`))"
],
"filter_covers": {
"cover ((`a`.`type`))": "airline"
},
"index": "ts_c”
………
}
{
"#operator": "IndexJoin",
"as": "r",
"for": "a",
"keyspace": "travel-sample",
"on_key": "cover ((`r`.`airlineid`))",
"scan": {
"covers": [
"cover ((`r`.`airlineid`))",
"cover ((`r`.`airline`))",
"cover ((meta(`r`).`id`))"
],
"filter_covers": {
"cover ((`r`.`type`))": "route"
},
"index": "ts_aid"
}
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggregate
Scan
• Aggregate
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT country, count(1) count
FROM `travel-sample` t
WHERE type = "airline"
GROUP BY country;
{
"#operator": "InitialGroup",
"aggregates": [
"count(1)"
],
"group_keys": [
"(`t`.`country`)"
]
}
{
"#operator": "FinalGroup",
"aggregates": [
"count(1)"
],
"group_keys": [
"(`t`.`country`)"
]
}
{
"#operator": "IntermediateGroup",
"aggregates": [
"count(1)"
],
"group_keys": [
"(`t`.`country`)"
]
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Sort
• Offset
• Limit
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT country, id, name
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND id BETWEEN 10 AND
1000
ORDER BY country, id DESC
OFFSET 5
LIMIT 10;
{
"#operator": "Order",
"limit": "10",
"offset": "5",
"sort_terms": [
{
"expr": "(`t`.`country`)"
},
{
"desc": true,
"expr": "(`t`.`id`)"
}
]
}
{
"#operator": "Offset",
"expr": "5"
}
{
"#operator": "Limit",
"expr": "10"
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Understanding Query Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggreg
ate
Scan
• Project
CREATE INDEX ts_c_id ON `travel-
sample`(country, id) WHERE type =
"airline";
EXPLAIN SELECT country, id, name
FROM `travel-sample` t
WHERE type = "airline" AND country =
"United States" AND id BETWEEN 10 AND
1000
ORDER BY country, id DESC
OFFSET 5
LIMIT 10;
{
"#operator": "InitialProject",
"result_terms": [
{
"expr": "(`t`.`country`)"
},
{
"expr": "(`t`.`id`)"
},
{
"expr": "(`t`.`name`)"
}
]
}
{
"#operator": "FinalProject"
}
©2016 Couchbase Inc. 31
How Query Plan Generated
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
• Rule based optimization
• The index chosen by N1QL will satisfy the query
• Each query block has is optimized separately & stitched.
32
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
• Index Selection Criteria
• If the query contains a USE KEYS clause, no index scan or full / primary scan is performed.
The input document keys are taken directly from the USE KEYS clause.
• If there is an explicit or implicit query predicate:
• Covering secondary scan;
• Regular secondary scan -- longest satisfying keys, intersect scan;
• UNNEST scan;
• Regular primary scan
• If the query does not contain a predicate
• covering primary scan;
• regular primary scan.
• If a primary scan is selected, and there is no primary index available, the query
errors out.
33
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
• Covering Primary Scan
• A covering primary scan is a primary scan that does not perform a subsequent document fetch. It is
used for queries that need a full / primary scan and only reference META().id.
• Regular Primary Scan
• A regular primary scan also performs a subsequent document fetch. It is used for queries that need a
full / primary scan and reference some document data other than META().id.
34
SELECT META(t).id FROM `travel-sample` t;
SELECT META(t).cas FROM `travel-sample` t;
SELECT * FROM `travel-sample` t;
SELECT t.type FROM `travel-sample` t;
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
Preliminaries :
• Online indexes
• Only online indexes are considered
• Preferred indexes
• USE INDEX hint is provided the indexes in that list are only considered
• Satisfying Index condition
• Partial / filtered indexes that index condition is super set of query predicate are
considered
• Satisfying Index keys
• Indexes whose leading keys satisfy query predicate are considered
• Longest satisfying index keys
• Redundancy is eliminated båy keeping longest satisfying index keys in same order.
• Index with satisfying keys (a,b,c) is retained over index with satisfying (a,b)
35
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
Covering Secondary Scan
• Each satisfied index with most number of index keys is examined for query coverage
• Shortest covering index will be used.
Regular Secondary Scan
• Indexes in with most number of matching index keys are used
• When more than one index are qualified, IntersectScan is used.
• To avoid IntersectScan provide hint with USE INDEX.
36
CREATE INDEX ts_name ON `travel-sample`(country, name) WHERE type = "airline";
SELECT country, name, type, META().id
FROM `travel-sample`
WHERE type = "airline" AND country = "United States";
SELECT country, name, type, id
FROM `travel-sample`
WHERE type = "airline" AND country = "United States";
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
UNNEST Scan
• Only array indexes are considered. And only queries with UNNEST clauses are considered
IndexCountScan
• Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered
• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and
argument to COUNT needs to be constant or leading key
37
CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline";
SELECT COUNT(1) AS count
FROM `travel-sample`
WHERE type = "airline" AND country = "United States";
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query Plan Generation
• In order to satisfy anAND predicate, an index only needs to satisfy any term in the AND
predicate.
• In order to satisfy an OR predicate, an index must satisfy every term in the OR predicate.
38
©2016 Couchbase Inc. 39
Query Plan Generation
Index Satisfies these predicates Index doesn’t satisfy these predicates
CREATE INDEX idx_dept ON employee( dept_id, hire_date, base_comp );
hire_date >= "2010" AND dept_id = "xyz"
hire_date >= "2010" AND dept_id = "xyz" AND
last_name = "Smith"
dept_id = "abc" AND base_comp > 50000
dept_id = "abc" AND bonus > 2000
dept_id = "abc" OR dept_id = "xyz"
(hire_date >= "2010" AND dept_id >= "xyz")
OR (hire_date >= "2010" AND dept_id = "abc")
dept_id = "abc" OR (hire_date >= "2010" AND
dept_id >= "xyz")
(hire_date >= "2010" AND dept_id >= "xyz")
OR (dept_id = "abc" AND bonus > 2000)
hire_date >= "2010" AND base_comp > 50000
hire_date >= "2010" AND bonus > 2000
bonus > 2000 AND base_comp > 50000
bonus > 2000 AND last_name = "Smith"
dept_id = "abc" OR hire_date >= "2010"
dept_id = "abc" OR dept_id = "xyz" OR
base_comp > 50000
dept_id = "abc" OR dept_id = "xyz" OR
bonus > 2000
hire_date > "2010" OR base_comp > 50000
©2016 Couchbase Inc. 40
Query OptimizationTo
Exploit Index Features
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
• N1QL Query engine can exploit the following index features.
• By creating right index and query that takes advantage performs better.
• Pushing predicates hint to Index
• Index Filters
• Using Index Order
• Pushing LIMIT to Index
• Pushing Offset as LIMIT (limit = limit + offset) to Index
• Top n Heap for Order
• Index Count (GSI Index only)
• Pushing MIN to Index
• Duplicate Indexes and Load balancing
• USE INDEX
41
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
USE INDEX
• Queries can specify which index to use with USE INDEX clause
• Multiple indexes qualify for query and one performs better than others
• To avoid Intersect Scan
• If there are many indexes on data source and queries are ad-hoc USE INDEX can reduce prepare time
Pushing predicates
• Predicates are pushed to indexer when possible
• These can be examined as spans in Index section of the EXPLAIN plan
• The spans can be exact or superset and never be subset of the predicates
• If the spans are exact the applying query predicates again will not eliminate the document
Index Filters
• If Index filter has equal on expression, that expression not required to include in the index keys.This allows keep
the index LEAN . Queries can exploit index filter condition for covered queries.
42
CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;
SELECT type, country FROM `travel-sample`
WHERE type = "airline" AND country = "United States";
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
Using Index Order
• Query ORDER BY list matches with index keys list order left to right with single range span
• Query doesn’t have any JOINs, GROUP BY or other clauses that can change the order produced by
indexer
• The query can exploit index order and avoid expensive sort and fetching lot of data unnecessarily in
some cases
43
CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ;
SELECT country, id FROM `travel-sample`
WHERE type = "airline" AND country = "United States”
ORDER BY id; --4.6 Uses Index order without any change
SELECT country, id FROM `travel-sample`
WHERE type = "airline" AND country = "United States”
ORDER BY country, id; --4.5 Uses Index order
CREATE INDEX ts_c_nid ON `travel-sample`(country, -id) WHERE type = "airline" ;
SELECT country, -(-id) FROM `travel-sample`
WHERE type = "airline" AND country = "United States”
ORDER BY country, -id; -- id prodcued in DESC order
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
Push LIMIT to Index
• Pushing the LIMIT is hint to indexer when to stop
• Pushing LIMIT can be done only when the predicates are pushed to indexer, spans are exact and
matches index order
• In case multiple spans LIMIT can be pushed to each span
44
CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ;
SELECT country, id FROM `travel-sample`
WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000
LIMIT 10; --LIMIT can be pushed
SELECT country, id FROM `travel-sample`
WHERE type = "airline" AND country IN ["United States", "United Kingdom"] AND id BETWEEN 0
AND 1000
LIMIT 10; --LIMIT can be pushed
CREATE INDEX ts_c_id ON `travel-sample`(id, country) WHERE type = "airline" ;
SELECT country, id FROM `travel-sample`
WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000
LIMIT 10; --LIMIT can’t be pushed. Indexer produces {"id":11, "country":"France"}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
Push Offset
• If LIMIT is present offset is added to limit and pushed limit to Indexer
Top-n Heap for Order
• Query requires order and has LIMIT less than 64K it usesTop-n Heap to eliminate unwanted documents before
order.
Index Count
• Count is done by indexer
• Only GSI index supports Index count
• Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered
• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and
argument to COUNT needs to be constant or leading key
45
CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline";
SELECT COUNT(1) AS count
FROM `travel-sample`
WHERE type = "airline" AND country = "United States";
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
MIN
• Queries with single projection of MIN aggregate, NO JOIN’s, GROUP BY is considered.
• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and
argument to MIN needs to be leading key.
• If the required conditions are satisfied limit 1 pushed to indexer.
46
CREATE INDEX ts_c ON `travel-sample`(country) WHERE
type = "airline";
SELECT MIN(country)
FROM `travel-sample` t
WHERE type = "airline" AND country > "United
States";
{
"#operator": "IndexScan",
"covers": [
"cover ((`t`.`country`))",
"cover ((meta(`t`).`id`))"
],
"filter_covers": {
"cover ((`t`.`type`))": "airline"
},
"index": "ts_c",
"keyspace": "travel-sample",
"limit": "1",
"spans": [
{
"Range": {
"Inclusion": 0,
"Low": [
""United States""
]
}
}
]
}
©2016 Couchbase Inc.©2016 Couchbase Inc.
Query OptimizationTo Exploit Index Features
Duplicate Indexes and Load balancing
• Allows duplicate indexes (i.e. different name but identical schema) on same or different Indexer services
• Query Service will choose one of the index during prepare time
• During execution Indexer client does load balancing and uses one of the indexer to service query
47
©2016 Couchbase Inc. 48
Q&A
©2016 Couchbase Inc. 49
Keshav Murthy
Director
Contact information
SitaramVemulapalli
Sr. Software Engineer
©2016 Couchbase Inc.
ThankYou!
50

More Related Content

What's hot

MongoDB Launchpad 2016: What’s New in the 3.4 Server
MongoDB Launchpad 2016: What’s New in the 3.4 ServerMongoDB Launchpad 2016: What’s New in the 3.4 Server
MongoDB Launchpad 2016: What’s New in the 3.4 ServerMongoDB
 
Indexing Strategies to Help You Scale
Indexing Strategies to Help You ScaleIndexing Strategies to Help You Scale
Indexing Strategies to Help You ScaleMongoDB
 
Back to Basics Webinar 3: Schema Design Thinking in Documents
 Back to Basics Webinar 3: Schema Design Thinking in Documents Back to Basics Webinar 3: Schema Design Thinking in Documents
Back to Basics Webinar 3: Schema Design Thinking in DocumentsMongoDB
 
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQL
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQLMongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQL
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQLMongoDB
 
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB Atlas
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB AtlasMongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB Atlas
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB AtlasMongoDB
 
Managing Social Content with MongoDB
Managing Social Content with MongoDBManaging Social Content with MongoDB
Managing Social Content with MongoDBMongoDB
 
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
 
Jumpstart: Introduction to MongoDB
Jumpstart: Introduction to MongoDBJumpstart: Introduction to MongoDB
Jumpstart: Introduction to MongoDBMongoDB
 
Modeling Data in MongoDB
Modeling Data in MongoDBModeling Data in MongoDB
Modeling Data in MongoDBlehresman
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLKeshav Murthy
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Javaantoinegirbal
 
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...MongoDB
 
Webinar: Schema Design
Webinar: Schema DesignWebinar: Schema Design
Webinar: Schema DesignMongoDB
 
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
 
Online | MongoDB Atlas on GCP Workshop
Online | MongoDB Atlas on GCP Workshop Online | MongoDB Atlas on GCP Workshop
Online | MongoDB Atlas on GCP Workshop Natasha Wilson
 
Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Matthew Groves
 

What's hot (20)

MongoDB Launchpad 2016: What’s New in the 3.4 Server
MongoDB Launchpad 2016: What’s New in the 3.4 ServerMongoDB Launchpad 2016: What’s New in the 3.4 Server
MongoDB Launchpad 2016: What’s New in the 3.4 Server
 
Indexing Strategies to Help You Scale
Indexing Strategies to Help You ScaleIndexing Strategies to Help You Scale
Indexing Strategies to Help You Scale
 
Back to Basics Webinar 3: Schema Design Thinking in Documents
 Back to Basics Webinar 3: Schema Design Thinking in Documents Back to Basics Webinar 3: Schema Design Thinking in Documents
Back to Basics Webinar 3: Schema Design Thinking in Documents
 
MongoDB Schema Design Tips & Tricks
MongoDB Schema Design Tips & TricksMongoDB Schema Design Tips & Tricks
MongoDB Schema Design Tips & Tricks
 
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQL
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQLMongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQL
MongoDB .local Munich 2019: Managing a Heterogeneous Stack with MongoDB & SQL
 
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB Atlas
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB AtlasMongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB Atlas
MongoDB Europe 2016 - MongoDB 3.4 preview and introduction to MongoDB Atlas
 
Managing Social Content with MongoDB
Managing Social Content with MongoDBManaging Social Content with MongoDB
Managing Social Content with MongoDB
 
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...
MongoDB .local Chicago 2019: Still Haven't Found What You Are Looking For? Us...
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
 
Jumpstart: Introduction to MongoDB
Jumpstart: Introduction to MongoDBJumpstart: Introduction to MongoDB
Jumpstart: Introduction to MongoDB
 
MongoDB + Spring
MongoDB + SpringMongoDB + Spring
MongoDB + Spring
 
Modeling Data in MongoDB
Modeling Data in MongoDBModeling Data in MongoDB
Modeling Data in MongoDB
 
MongoDB and Schema Design
MongoDB and Schema DesignMongoDB and Schema Design
MongoDB and Schema Design
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Java
 
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to...
 
Webinar: Schema Design
Webinar: Schema DesignWebinar: Schema Design
Webinar: Schema Design
 
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Munich 2019: MongoDB Atlas Data Lake Technical Deep Dive
 
Online | MongoDB Atlas on GCP Workshop
Online | MongoDB Atlas on GCP Workshop Online | MongoDB Atlas on GCP Workshop
Online | MongoDB Atlas on GCP Workshop
 
Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017
 

Viewers also liked

“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase
“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase
“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, CouchbaseLucidworks
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Keshav Murthy
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseCecile Le Pape
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingKeshav Murthy
 
Query in Couchbase. N1QL: SQL for JSON
Query in Couchbase.  N1QL: SQL for JSONQuery in Couchbase.  N1QL: SQL for JSON
Query in Couchbase. N1QL: SQL for JSONKeshav Murthy
 
Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Keshav Murthy
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Cecile Le Pape
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedKorea Sdec
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Keshav Murthy
 

Viewers also liked (11)

“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase
“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase
“N1QL” a Rich Query Language for Couchbase: Presented by Don Pinto, Couchbase
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5
 
Drilling on JSON
Drilling on JSONDrilling on JSON
Drilling on JSON
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and Indexing
 
Query in Couchbase. N1QL: SQL for JSON
Query in Couchbase.  N1QL: SQL for JSONQuery in Couchbase.  N1QL: SQL for JSON
Query in Couchbase. N1QL: SQL for JSON
 
Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.
 
Couchbase Day
Couchbase DayCouchbase Day
Couchbase Day
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speed
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data.
 

Similar to Understanding N1QL Optimizer to Tune Queries

MongoDB Stich Overview
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich OverviewMongoDB
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0Keshav Murthy
 
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBIntroducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBMongoDB
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5Keshav Murthy
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliKeshav Murthy
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorKeshav Murthy
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Keshav Murthy
 
No sq ls-biggest-lie_sql-never-went-away_martin-esmann
No sq ls-biggest-lie_sql-never-went-away_martin-esmannNo sq ls-biggest-lie_sql-never-went-away_martin-esmann
No sq ls-biggest-lie_sql-never-went-away_martin-esmannMartin Esmann
 
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...Amazon Web Services
 
NoSQL's biggest lie: SQL never went away - Martin Esmann
NoSQL's biggest lie: SQL never went away - Martin EsmannNoSQL's biggest lie: SQL never went away - Martin Esmann
NoSQL's biggest lie: SQL never went away - Martin Esmanndistributed matters
 
Aws summit devops 云端多环境自动化运维和部署
Aws summit devops   云端多环境自动化运维和部署Aws summit devops   云端多环境自动化运维和部署
Aws summit devops 云端多环境自动化运维和部署Leon Li
 
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017Amazon Web Services
 
Querying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and CouchbaseQuerying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and CouchbaseBrant Burnett
 
AWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAmazon Web Services
 
ELK - What's new and showcases
ELK - What's new and showcasesELK - What's new and showcases
ELK - What's new and showcasesAndrii Gakhov
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revisedMongoDB
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessMongoDB
 
Bridging Structured and Unstructred Data with Apache Hadoop and Vertica
Bridging Structured and Unstructred Data with Apache Hadoop and VerticaBridging Structured and Unstructred Data with Apache Hadoop and Vertica
Bridging Structured and Unstructred Data with Apache Hadoop and VerticaSteve Watt
 

Similar to Understanding N1QL Optimizer to Tune Queries (20)

MongoDB Stich Overview
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich Overview
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0
 
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBIntroducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index Advisor
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch Introduction
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
 
No sq ls-biggest-lie_sql-never-went-away_martin-esmann
No sq ls-biggest-lie_sql-never-went-away_martin-esmannNo sq ls-biggest-lie_sql-never-went-away_martin-esmann
No sq ls-biggest-lie_sql-never-went-away_martin-esmann
 
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...
0 to 60 with AWS AppSync: Rapid Development Techniques for Mobile APIs (MOB32...
 
NoSQL's biggest lie: SQL never went away - Martin Esmann
NoSQL's biggest lie: SQL never went away - Martin EsmannNoSQL's biggest lie: SQL never went away - Martin Esmann
NoSQL's biggest lie: SQL never went away - Martin Esmann
 
Aws summit devops 云端多环境自动化运维和部署
Aws summit devops   云端多环境自动化运维和部署Aws summit devops   云端多环境自动化运维和部署
Aws summit devops 云端多环境自动化运维和部署
 
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017
Cloud Adoption in Regulated Financial Services - SID328 - re:Invent 2017
 
Querying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and CouchbaseQuerying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and Couchbase
 
Into The Box 2018 cbelasticsearch
Into The Box 2018   cbelasticsearchInto The Box 2018   cbelasticsearch
Into The Box 2018 cbelasticsearch
 
AWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAWS CloudFormation Best Practices
AWS CloudFormation Best Practices
 
ELK - What's new and showcases
ELK - What's new and showcasesELK - What's new and showcases
ELK - What's new and showcases
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revised
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational Awareness
 
Bridging Structured and Unstructred Data with Apache Hadoop and Vertica
Bridging Structured and Unstructred Data with Apache Hadoop and VerticaBridging Structured and Unstructred Data with Apache Hadoop and Vertica
Bridging Structured and Unstructred Data with Apache Hadoop and Vertica
 

More from Keshav Murthy

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0Keshav Murthy
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...Keshav Murthy
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresKeshav Murthy
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Keshav Murthy
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber Keshav Murthy
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersKeshav Murthy
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications Keshav Murthy
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONKeshav Murthy
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLKeshav Murthy
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixKeshav Murthy
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherKeshav Murthy
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Keshav Murthy
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013Keshav Murthy
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveKeshav Murthy
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Keshav Murthy
 

More from Keshav Murthy (15)

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing features
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developers
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSON
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QL
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on Informix
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all together
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep dive
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
 

Recently uploaded

VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...SUHANI PANDEY
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...amitlee9823
 
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...amitlee9823
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...amitlee9823
 
hybrid Seed Production In Chilli & Capsicum.pptx
hybrid Seed Production In Chilli & Capsicum.pptxhybrid Seed Production In Chilli & Capsicum.pptx
hybrid Seed Production In Chilli & Capsicum.pptx9to5mart
 
Capstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramCapstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramMoniSankarHazra
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823
 
Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsJoseMangaJr1
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...ZurliaSoop
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...amitlee9823
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Delhi Call girls
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Pooja Nehwal
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNKTimothy Spann
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Standamitlee9823
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...amitlee9823
 
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...amitlee9823
 
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...amitlee9823
 

Recently uploaded (20)

VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
 
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
 
hybrid Seed Production In Chilli & Capsicum.pptx
hybrid Seed Production In Chilli & Capsicum.pptxhybrid Seed Production In Chilli & Capsicum.pptx
hybrid Seed Production In Chilli & Capsicum.pptx
 
Capstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramCapstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics Program
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter Lessons
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
 
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
 

Understanding N1QL Optimizer to Tune Queries

  • 1. ©2016 Couchbase Inc. Understand Query Optimizer to Tune Queries SitaramVemulapalli | Senior Software Engineer,Couchbase R&D | sitaram.vemulapalli@couchbase.com Keshav Murthy | Director, Couchbase R&D | keshav@couchbase.com 1
  • 2. ©2016 Couchbase Inc.©2016 Couchbase Inc. Agenda • Brief Introduction to N1QL • Understand Query Plan • How Query Plan Generated • Query + Indexing features • Q&A 2
  • 3. ©2016 Couchbase Inc. 3 Introduction to N1QL
  • 4. ©2016 Couchbase Inc.©2016 Couchbase Inc. SQL 4 ResultSet Input: Relations Output: Relation
  • 5. ©2016 Couchbase Inc.©2016 Couchbase Inc. NoSQL 5 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER Built Manually; Expensive Input: JSON Documents Output: JSON Documents
  • 6. ©2016 Couchbase Inc.©2016 Couchbase Inc. N1QL Changes the Possibilities for NoSQL
  • 7. ©2016 Couchbase Inc.©2016 Couchbase Inc. NoSQL { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER Input: JSON Documents Output: JSON Documents
  • 8. ©2016 Couchbase Inc.©2016 Couchbase Inc. Goal of N1QL: SQL for JSON Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  • 9. ©2016 Couchbase Inc.©2016 Couchbase Inc. 9 SELECT Customers.ID, Customers.Name, SUM(OrderLine.Amount) FROM Orders UNNEST Orders.LineItems AS OrderLine JOIN Customers ON KEYS Orders.CustID GROUP BY Customers.ID, Customers.Name HAVING SUM(OrderLine.Amount) > 10000 ORDER BY SUM(OrderLine.Amount) DESC LIMIT 100 ;
  • 10. ©2016 Couchbase Inc. 10 Understanding Query Plan
  • 11. ©2016 Couchbase Inc.©2016 Couchbase Inc. N1QL: Query Execution Flow Clients 1. Submit the query over REST API 8. Query result 2. Parse, Analyze, create Plan 7. Evaluate: Filter, Join, Aggregate, Sort, Paginate 3. Scan Request; index filters 6. Fetch the documents Index Service Query Service Data Service 4. Get qualified doc keys 5. Fetch Request, doc keys SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 12. ©2016 Couchbase Inc.©2016 Couchbase Inc. Inside a Query Service Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Service Data Service
  • 13. ©2016 Couchbase Inc.©2016 Couchbase Inc. Inside a Query Service FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Service Data Service Prepare Phase Projection Phase Run Phase Scan Phase Fetch Phase
  • 14. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggregate Scan • Each query can be executed in several ways • Create the query execution plan • Access path for each keyspace referenced • Decide on the filters to push down • Determine Join order and join method • Create the execution tree • For each keyspace reference: • Look at the available indices • Match the filters in the query with index condition and keys • Choose one or more indices for each keyspace • Decide if a covering index can be used • Covering index is preferred • Create index filters and post scan, post join filters
  • 15. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Covering Index Scan • Predicate pushdown to Indexer • Pushdown limit to IndexScan (4.5) • Use Index Order (4.5) CREATE INDEX ts_c_id ON `travel-sample` (country, id) WHERE type = "airline"; EXPLAIN SELECT country, id FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000 ORDER BY country, id OFFSET 5 LIMIT 10; { "#operator": "IndexScan", "covers": [ "cover ((`t`.`country`))", "cover ((`t`.`id`))", "cover ((meta(`t`).`id`))" ], "filter_covers": { "cover ((`t`.`type`))": "airline" }, "index": "ts_c_id", "index_id": "ccd0f0c297114733", "keyspace": "travel-sample", "limit": "(5 + 10)", "namespace": "default", "spans": [ { "Range": { "High": [ ""United States"", "1000" ], "Inclusion": 3, "Low": [ ""United States"", "10" ] } } ], "using": "gsi" }
  • 16. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • NonCovering Index Scan • Predicate pushdown to Indexer • Pushdown limit to IndexScan (4.5) • Use Index order (4.5) CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT country, id, name FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000 ORDER BY country, id OFFSET 5 LIMIT 10; { "#operator": "IndexScan", "index": "ts_c_id", "index_id": "ccd0f0c297114733", "keyspace": "travel-sample", "limit": "(5 + 10)", "namespace": "default", "spans": [ { "Range": { "High": [ ""United States"", "1000" ], "Inclusion": 3, "Low": [ ""United States"", "10" ] } } ], "using": "gsi" }
  • 17. ©2016 Couchbase Inc.©2016 Couchbase Inc. N1QL: Covering Index and Covered Query Clients 1. Submit the query over REST API 6. Query result 2. Parse, Analyze, create Plan 5. Evaluate: Filter, Join, Aggregate, Sort, Paginate 3. Scan Request; index filters Index Service Query Service 4. Get qualified doc keys create index i1 on CUSTOMER(c_id, c_max, c_first, c_last) SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 18. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • NonCovering Query • Fetch (Only for Non Covering Scan) • Filter CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT country, id, name FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000 ORDER BY country, id OFFSET 5 LIMIT 10; { "#operator": "Fetch", "keyspace": "travel-sample", "namespace": "default" } { "#operator": "Filter", "condition": "((((`t`.`type`) = "airline") and ((`t`.`country`) = "United States")) and ((`t`.`id`) between 10 and 1000))" }
  • 19. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • IntersectScan • For non covered queries only CREATE INDEX ts_c ON `travel- sample`(country) WHERE type = "airline" ; CREATE INDEX ts_n ON `travel- sample`(name) WHERE type = "airline" ; EXPLAIN SELECT country, name FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND name = "US Airways"; {"#operator": "IntersectScan", "scans": [ { "#operator": "IndexScan", "index": "ts_c", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ ""United States”"], "Inclusion": 3, "Low": [ ""United States””] } } ] }, { "#operator": "IndexScan", "index": "ts_n", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [""US Airways”” ], "Inclusion": 3, "Low": [""US Airways”” ] } } ] } ] }
  • 20. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Array Index Key Scan (4.5) CREATE INDEX ts_aix1 ON `travel- sample` (DISTINCT ARRAY v.day FOR v IN schedule END) WHERE type = "route"; SELECT META(t).id FROM `travel-sample` t WHERE type = "route" AND ANY v IN schedule SATISFIES v.day IN [1,2] END; { "#operator": "DistinctScan", "scan": { "#operator": "IndexScan", "index": "ts_aix1", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ "1" ], "Inclusion": 3, "Low": [ "1" ] } }, { "Range": { "High": [ "2" ], "Inclusion": 3, "Low": [ "2" ] } } ] } }
  • 21. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • IndexCountScan (4.5) CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT COUNT(1) AS count FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000; { "#operator": "IndexCountScan", "covers": [ "cover ((`t`.`country`))", "cover ((`t`.`id`))", "cover ((meta(`t`).`id`))" ], "index": "ts_c_id", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ ""United States"", "1000" ], "Inclusion": 3, "Low": [ ""United States"", "10" ] } } ] } { "#operator": "IndexCountProject", "result_terms": [ { "as": "count", "expr": "count(1)" } ] }
  • 22. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • JOIN, LEFT JOIN. • NEST, LEFT NEST will have operator NEST. • LEFT side keyspace of JOIN can be covered. • Non-covered query. CREATE INDEX ts_rix1 ON `travel- sample` (DISTINCT ARRAY v.day FOR v IN schedule END, airlineid, schedule) WHERE type = "route"; EXPLAIN SELECT r, a FROM `travel-sample` r LEFT JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.type = "route" AND ANY v IN r.schedule SATISFIES v.day = 1 END LIMIT 1; "scan": { "#operator": "IndexScan", "index": "ts_rix1", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ "1" ], "Inclusion": 3, "Low": [ "1" ] } } ] } { "#operator": "Join", "as": "a", "keyspace": "travel-sample", "namespace": "default", "on_keys": "(`r`.`airlineid`)", "outer": true }
  • 23. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • JOIN LEFT side covered query. CREATE INDEX ts_rix1 ON `travel- sample` (DISTINCT ARRAY v.day FOR v IN schedule END, airlineid, schedule) WHERE type = "route"; EXPLAIN SELECT r.schedule, META(r).id FROM `travel-sample` r LEFT JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.type = "route" AND ANY v IN r.schedule SATISFIES v.day = 1 END LIMIT 1; "scan": { "#operator": "IndexScan", "covers": [ "cover ((distinct (array (`v`.`day`) for `v` in (`r`.`schedule`) end)))", "cover ((`r`.`airlineid`))", "cover ((`r`.`schedule`))", "cover ((meta(`r`).`id`))" ], "filter_covers": { "cover ((`r`.`type`))": "route" } "index": "ts_rix1", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ "1" ], "Inclusion": 3, "Low": [ "1" ] } } ] } { "#operator": "Join", "as": "a", "keyspace": "travel-sample", "namespace": "default", "on_keys": ”cover ((`r`.`airlineid`))", "outer": true }
  • 24. ©2016 Couchbase Inc.©2016 Couchbase Inc. How does this work? CREATE INDEX brewery_state ON `beer-sample`(state); CREATE INDEX beer_brewery_id ON `beer-sample`(brewery_id); SELECT brewery.name as breweryname, ARRAY_AGG(beer.name) beerlist FROM `beer-sample` brewery LEFT OUTER JOIN `beer-sample` beer ON KEY beer.brewery_id FOR brewery WHERE brewery.state = "California" GROUP BY brewery.name; brewerybrewery_state [“California”, “21st_sfo_ale”] beer_brewery_id (brewery_id = “21st_sfo_ale”) beer [“21st_sfo_ale” , ”pale_ale”] (state = ‘California’)
  • 25. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Index JOIN, LEFT Index JOIN. • Index NEST, LEFT Index NEST will have operator NEST. • LEFT side keyspace of JOIN can be covered. • In 4.6 RIGHT side keyspace of Index JOIN can be covered. NOT for Index NEST. CREATE INDEX ts_c ON `travel- sample`(country) WHERE type = "airline" ; CREATE INDEX ts_aid ON `travel- sample`(airlineid, airline) WHERE type = "route" ; EXPLAIN SELECT r, a FROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR a WHERE r.type = "route" AND a.type = "airline" AND a.country = "United States" LIMIT 1; { "#operator": "IndexScan", "index": "ts_c", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ ""United States"" ], "Inclusion": 3, "Low": [ ""United States"" ] } } ] } { "#operator": "IndexJoin", "as": "r", "for": "a", "keyspace": "travel-sample", "on_key": "(`r`.`airlineid`)", "scan": { "index": "ts_aid" } }
  • 26. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Covered Index Join (4.5) CREATE INDEX ts_c ON `travel- sample`(country) WHERE type = "airline" ; CREATE INDEX ts_aid ON `travel- sample`(airlineid, airline) WHERE type = "route" ; EXPLAIN SELECT a.country, r.airline FROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR a WHERE r.type = "route" AND a.type = "airline" AND a.country = "United States"; { "#operator": "IndexScan", "covers": [ "cover ((`a`.`country`))", "cover ((meta(`a`).`id`))" ], "filter_covers": { "cover ((`a`.`type`))": "airline" }, "index": "ts_c", "keyspace": "travel-sample", "spans": [ { "Range": { "High": [ ""United States"" ], "Inclusion": 3, "Low": [ ""United States"" ] } } ] } { "#operator": "IndexJoin", "as": "r", "for": "a", "keyspace": "travel-sample", "on_key": "(`r`.`airlineid`)", "scan": { "index": "ts_aid" } }
  • 27. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • LEFT & RIGHT Covered Index Join (4.6) CREATE INDEX ts_c ON `travel- sample`(country) WHERE type = "airline" ; CREATE INDEX ts_aid ON `travel- sample`(airlineid, airline) WHERE type = "route" ; EXPLAIN SELECT a.country, r.airline FROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR a WHERE r.type = "route" AND a.type = "airline" AND a.country = "United States"; {"#operator": "IndexScan", "covers": [ "cover ((`a`.`country`))", "cover ((meta(`a`).`id`))" ], "filter_covers": { "cover ((`a`.`type`))": "airline" }, "index": "ts_c” ……… } { "#operator": "IndexJoin", "as": "r", "for": "a", "keyspace": "travel-sample", "on_key": "cover ((`r`.`airlineid`))", "scan": { "covers": [ "cover ((`r`.`airlineid`))", "cover ((`r`.`airline`))", "cover ((meta(`r`).`id`))" ], "filter_covers": { "cover ((`r`.`type`))": "route" }, "index": "ts_aid" } }
  • 28. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggregate Scan • Aggregate CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT country, count(1) count FROM `travel-sample` t WHERE type = "airline" GROUP BY country; { "#operator": "InitialGroup", "aggregates": [ "count(1)" ], "group_keys": [ "(`t`.`country`)" ] } { "#operator": "FinalGroup", "aggregates": [ "count(1)" ], "group_keys": [ "(`t`.`country`)" ] } { "#operator": "IntermediateGroup", "aggregates": [ "count(1)" ], "group_keys": [ "(`t`.`country`)" ] }
  • 29. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Sort • Offset • Limit CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT country, id, name FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000 ORDER BY country, id DESC OFFSET 5 LIMIT 10; { "#operator": "Order", "limit": "10", "offset": "5", "sort_terms": [ { "expr": "(`t`.`country`)" }, { "desc": true, "expr": "(`t`.`id`)" } ] } { "#operator": "Offset", "expr": "5" } { "#operator": "Limit", "expr": "10" }
  • 30. ©2016 Couchbase Inc.©2016 Couchbase Inc. Understanding Query Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggreg ate Scan • Project CREATE INDEX ts_c_id ON `travel- sample`(country, id) WHERE type = "airline"; EXPLAIN SELECT country, id, name FROM `travel-sample` t WHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000 ORDER BY country, id DESC OFFSET 5 LIMIT 10; { "#operator": "InitialProject", "result_terms": [ { "expr": "(`t`.`country`)" }, { "expr": "(`t`.`id`)" }, { "expr": "(`t`.`name`)" } ] } { "#operator": "FinalProject" }
  • 31. ©2016 Couchbase Inc. 31 How Query Plan Generated
  • 32. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation • Rule based optimization • The index chosen by N1QL will satisfy the query • Each query block has is optimized separately & stitched. 32
  • 33. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation • Index Selection Criteria • If the query contains a USE KEYS clause, no index scan or full / primary scan is performed. The input document keys are taken directly from the USE KEYS clause. • If there is an explicit or implicit query predicate: • Covering secondary scan; • Regular secondary scan -- longest satisfying keys, intersect scan; • UNNEST scan; • Regular primary scan • If the query does not contain a predicate • covering primary scan; • regular primary scan. • If a primary scan is selected, and there is no primary index available, the query errors out. 33
  • 34. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation • Covering Primary Scan • A covering primary scan is a primary scan that does not perform a subsequent document fetch. It is used for queries that need a full / primary scan and only reference META().id. • Regular Primary Scan • A regular primary scan also performs a subsequent document fetch. It is used for queries that need a full / primary scan and reference some document data other than META().id. 34 SELECT META(t).id FROM `travel-sample` t; SELECT META(t).cas FROM `travel-sample` t; SELECT * FROM `travel-sample` t; SELECT t.type FROM `travel-sample` t;
  • 35. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation Preliminaries : • Online indexes • Only online indexes are considered • Preferred indexes • USE INDEX hint is provided the indexes in that list are only considered • Satisfying Index condition • Partial / filtered indexes that index condition is super set of query predicate are considered • Satisfying Index keys • Indexes whose leading keys satisfy query predicate are considered • Longest satisfying index keys • Redundancy is eliminated båy keeping longest satisfying index keys in same order. • Index with satisfying keys (a,b,c) is retained over index with satisfying (a,b) 35
  • 36. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation Covering Secondary Scan • Each satisfied index with most number of index keys is examined for query coverage • Shortest covering index will be used. Regular Secondary Scan • Indexes in with most number of matching index keys are used • When more than one index are qualified, IntersectScan is used. • To avoid IntersectScan provide hint with USE INDEX. 36 CREATE INDEX ts_name ON `travel-sample`(country, name) WHERE type = "airline"; SELECT country, name, type, META().id FROM `travel-sample` WHERE type = "airline" AND country = "United States"; SELECT country, name, type, id FROM `travel-sample` WHERE type = "airline" AND country = "United States";
  • 37. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation UNNEST Scan • Only array indexes are considered. And only queries with UNNEST clauses are considered IndexCountScan • Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered • Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to COUNT needs to be constant or leading key 37 CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline"; SELECT COUNT(1) AS count FROM `travel-sample` WHERE type = "airline" AND country = "United States";
  • 38. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query Plan Generation • In order to satisfy anAND predicate, an index only needs to satisfy any term in the AND predicate. • In order to satisfy an OR predicate, an index must satisfy every term in the OR predicate. 38
  • 39. ©2016 Couchbase Inc. 39 Query Plan Generation Index Satisfies these predicates Index doesn’t satisfy these predicates CREATE INDEX idx_dept ON employee( dept_id, hire_date, base_comp ); hire_date >= "2010" AND dept_id = "xyz" hire_date >= "2010" AND dept_id = "xyz" AND last_name = "Smith" dept_id = "abc" AND base_comp > 50000 dept_id = "abc" AND bonus > 2000 dept_id = "abc" OR dept_id = "xyz" (hire_date >= "2010" AND dept_id >= "xyz") OR (hire_date >= "2010" AND dept_id = "abc") dept_id = "abc" OR (hire_date >= "2010" AND dept_id >= "xyz") (hire_date >= "2010" AND dept_id >= "xyz") OR (dept_id = "abc" AND bonus > 2000) hire_date >= "2010" AND base_comp > 50000 hire_date >= "2010" AND bonus > 2000 bonus > 2000 AND base_comp > 50000 bonus > 2000 AND last_name = "Smith" dept_id = "abc" OR hire_date >= "2010" dept_id = "abc" OR dept_id = "xyz" OR base_comp > 50000 dept_id = "abc" OR dept_id = "xyz" OR bonus > 2000 hire_date > "2010" OR base_comp > 50000
  • 40. ©2016 Couchbase Inc. 40 Query OptimizationTo Exploit Index Features
  • 41. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features • N1QL Query engine can exploit the following index features. • By creating right index and query that takes advantage performs better. • Pushing predicates hint to Index • Index Filters • Using Index Order • Pushing LIMIT to Index • Pushing Offset as LIMIT (limit = limit + offset) to Index • Top n Heap for Order • Index Count (GSI Index only) • Pushing MIN to Index • Duplicate Indexes and Load balancing • USE INDEX 41
  • 42. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features USE INDEX • Queries can specify which index to use with USE INDEX clause • Multiple indexes qualify for query and one performs better than others • To avoid Intersect Scan • If there are many indexes on data source and queries are ad-hoc USE INDEX can reduce prepare time Pushing predicates • Predicates are pushed to indexer when possible • These can be examined as spans in Index section of the EXPLAIN plan • The spans can be exact or superset and never be subset of the predicates • If the spans are exact the applying query predicates again will not eliminate the document Index Filters • If Index filter has equal on expression, that expression not required to include in the index keys.This allows keep the index LEAN . Queries can exploit index filter condition for covered queries. 42 CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ; SELECT type, country FROM `travel-sample` WHERE type = "airline" AND country = "United States";
  • 43. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features Using Index Order • Query ORDER BY list matches with index keys list order left to right with single range span • Query doesn’t have any JOINs, GROUP BY or other clauses that can change the order produced by indexer • The query can exploit index order and avoid expensive sort and fetching lot of data unnecessarily in some cases 43 CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ; SELECT country, id FROM `travel-sample` WHERE type = "airline" AND country = "United States” ORDER BY id; --4.6 Uses Index order without any change SELECT country, id FROM `travel-sample` WHERE type = "airline" AND country = "United States” ORDER BY country, id; --4.5 Uses Index order CREATE INDEX ts_c_nid ON `travel-sample`(country, -id) WHERE type = "airline" ; SELECT country, -(-id) FROM `travel-sample` WHERE type = "airline" AND country = "United States” ORDER BY country, -id; -- id prodcued in DESC order
  • 44. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features Push LIMIT to Index • Pushing the LIMIT is hint to indexer when to stop • Pushing LIMIT can be done only when the predicates are pushed to indexer, spans are exact and matches index order • In case multiple spans LIMIT can be pushed to each span 44 CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ; SELECT country, id FROM `travel-sample` WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000 LIMIT 10; --LIMIT can be pushed SELECT country, id FROM `travel-sample` WHERE type = "airline" AND country IN ["United States", "United Kingdom"] AND id BETWEEN 0 AND 1000 LIMIT 10; --LIMIT can be pushed CREATE INDEX ts_c_id ON `travel-sample`(id, country) WHERE type = "airline" ; SELECT country, id FROM `travel-sample` WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000 LIMIT 10; --LIMIT can’t be pushed. Indexer produces {"id":11, "country":"France"}
  • 45. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features Push Offset • If LIMIT is present offset is added to limit and pushed limit to Indexer Top-n Heap for Order • Query requires order and has LIMIT less than 64K it usesTop-n Heap to eliminate unwanted documents before order. Index Count • Count is done by indexer • Only GSI index supports Index count • Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered • Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to COUNT needs to be constant or leading key 45 CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline"; SELECT COUNT(1) AS count FROM `travel-sample` WHERE type = "airline" AND country = "United States";
  • 46. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features MIN • Queries with single projection of MIN aggregate, NO JOIN’s, GROUP BY is considered. • Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to MIN needs to be leading key. • If the required conditions are satisfied limit 1 pushed to indexer. 46 CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline"; SELECT MIN(country) FROM `travel-sample` t WHERE type = "airline" AND country > "United States"; { "#operator": "IndexScan", "covers": [ "cover ((`t`.`country`))", "cover ((meta(`t`).`id`))" ], "filter_covers": { "cover ((`t`.`type`))": "airline" }, "index": "ts_c", "keyspace": "travel-sample", "limit": "1", "spans": [ { "Range": { "Inclusion": 0, "Low": [ ""United States"" ] } } ] }
  • 47. ©2016 Couchbase Inc.©2016 Couchbase Inc. Query OptimizationTo Exploit Index Features Duplicate Indexes and Load balancing • Allows duplicate indexes (i.e. different name but identical schema) on same or different Indexer services • Query Service will choose one of the index during prepare time • During execution Indexer client does load balancing and uses one of the indexer to service query 47
  • 49. ©2016 Couchbase Inc. 49 Keshav Murthy Director Contact information SitaramVemulapalli Sr. Software Engineer

Editor's Notes

  1. Data-parallel — Query latency scales up with cores Memory-bound
  2. Data-parallel — Query latency scales up with cores Memory-bound
  3. Step 1. Scan brewery using index on brewery_state. Predicate brewery.state = ”California” Step 2. Using the meta(brewery).id as the matching value, scan brewery_id to get the meta(beer).id Step 3. If you do find the matching document, you have meta(beer).id, fetch the beer document to do the JOIN Step 4. Do the JOIN and project necessary projections. Step 5. Do the GROUP BY Step 6. Do the aggregation & projection.