NoSQL's biggest lie: SQL never went away - Martin Esmann

NoSQL's biggest lie: SQL never went away!
Martin Esmann
Developer Advocate, Couchbase
1

Let’s do something unspectacular
2

©2014 Couchbase Inc.
A query
3
SELECT * FROM `travel-sample`
WHERE type = ‘airline’
AND country = ‘United Kingdom’;

A query
4

Did I say unspectacular?
5

That was JSON
6

We put the SQL back into NoSQL
8

But first, let’s take a step back
9

NoSQL?
10
Polyglot
persistence
is
“…using
multiple
data

storage
technologies,
chosen
based
upon
the

way
data
is
being
used
by
individual

applications.
Why
store
binary
images
in

relational
database,
when
there
are
better

storage
systems?”

Martin
Fowler
and
Pramod Sadalage

Types of NoSQL database
11
By
Schumi4ever
(Own
work)
[CC
BY-‐SA
3.0

(http://creativecommons.org/licenses/by-‐sa/3.0)],
via
Wikimedia
Commons

Key value
12
Email: martin@couchbase.comemail:email: {
“personal”: “martin.esmann@hotmail.com”,
“work”: “martin@couchbase.com”
}

London
matthew@couchbase.com
james@couchbase.com
laura@couchbase.com
tom@couchbase.com
david@couchbase.com
greg@couchbase.com
Document
13
Developer Advocacy
james@couchbase.com
laura@couchbase.com
laurent@couchbase.com
martin@couchbase.com
matt@couchbase.com
nic@couchbase.com
will@couchbase.com
martin@couchbase.com:
{
"city": ”Copenhagen",
"glasses": false,
"team": "Developer Advocacy",
"music": ”Electronic!"
}

Document
14
London
james@couchbase.com
laura@couchbase.com
Developer Advocacy
james@couchbase.com
laura@couchbase.com
London and Developer Advocacy
james@couchbase.com
laura@couchbase.com

And the others
15

There's always a trade-‐off
17
• Offload
from
some
other
data
store
(i.e.
caching)
• Computation
offload
• Speed
• Scalability
• Availability
• Flexibility
in
what
you
store
• Query
flexibility

Where Couchbase comes in
18

Couchbase Server 4.0
19
High
availability

cache
Key-‐value

store
Document

database N1QL SQL-‐like
query

for
JSON

A user profile as JSON
21

How do we query that?
22

Querying the JSON profile
23
• Look-‐up
documents:
i.e.
manual
secondary
indexing
(2i)
• Couchbase views:
i.e.
automated
secondary
indexing
(2i)
• N1QL

Manual 2i
24

Automatic 2i: views
25

N1QL
26

SELECT
27
SELECT
...

SELECT
28
SELECT 1 + 1;
{

"requestID":
"3ccebac7-‐341a-‐4c31-‐a2c5-‐b46aaed54356",

"signature":
{

"$1":
"number"

},

"results":
[

{

"$1":
2

}

],

"status":
"success",

"metrics":
{

"elapsedTime":
"31.826219ms",

"executionTime":
"29.800616ms",

"resultCount":
1,

"resultSize":
31

}
}

SELECT COUNT
29
SELECT COUNT(*) FROM `default`
WHERE office = "London";
{
"requestID": "6e733000-ac83-44ba-95a7-9b012e9c553d",
"signature": {
"$1": "number"
},
"results": [
{
"$1": 6
}
],
"status": "success",
"metrics": {
"elapsedTime": "18.603124ms",
"executionTime": "18.327696ms",
"resultCount": 1,
"resultSize": 31
}
}

SELECT email FROM `default`
WHERE office = "London";
SELECT
30

WHERE office = "London"
AND team = "Developer Advocacy";
SELECT
31

WHERE office = "London"
AND team != "Developer Advocacy";
SELECT
32

SELECT conferences[0].name
AS event_name FROM `default`;
ARRAY ELEMENTS
34

SELECT DISTINCT
conferences[0].name
AS event_name FROM `default`;
DISTINCT ARRAY ELEMENTS
35

SELECT DISTINCT
conferences[0].name
AS event_name FROM `default`
WHERE conferences
IS NOT MISSING;
REMOVE MISSING ITEMS
36

SELECT email AS person,
conferences[0].name AS event
FROM `default`
WHERE ANY event in conferences
SATISFIES event.name =
"Droidcon Sweden" END;
WHO IS GOING TO DROIDCON SWEDEN?
37

What's going on underneath?
38

EXPLAIN SELECT email AS person,
conferences[0].name AS event
FROM `default`
WHERE ANY event in conferences
SATISFIES event.name =
"Droidcon Sweden" END;
EXPLAIN
39

Is N1QL read-‐only?
40

Updating and deleting
41
• DELETE:
provide
the
key
to
delete
the
document
• INSERT:
provide
a
key
and
some
JSON
to
create
a
new
document
• UPSERT:
as
INSERT
but
will
overwrite
existing
docs
• UPDATE:
change
individual
values
inside
existing
docs

UPDATE
42

DELETE
43

A larger data-‐set: travel-‐sample
44

TRAVEL SAMPLE DATA
45

travel-‐sample
46

CREATE PRIMARY INDEX
47
CREATE PRIMARY INDEX
ON `travel-sample`
USING GSI;

SELECT
48
WHERE type = "airline";

SELECT
49

SELECT
50
WHERE type = "airline"
AND country = "United States";

SELECT
51

CREATE INDEX
53
CREATE INDEX airline
ON `travel-sample`(type)
WHERE type = "airline"
USING GSI;

JOINs
55
• Retrieve
data
from
two
documents
in
a
single
SELECT
• Join
within
a
keyspace/bucket
• Join
across
keyspaces/buckets

A SIMPLE JOIN
56
SELECT * FROM `travel-sample` r
JOIN `travel-sample` a
ON KEYS r.airlineid
WHERE r.sourceairport="LHR"
AND r.destinationairport = "SFO";

WHO FLIES LHR-‐>SFO?
57
SELECT DISTINCT a.name FROM
`travel-sample` r
ON KEYS r.airlineid
AND r.destinationairport = "SFO";

UNNEST
58
• Breaks
out
nested
JSON
from
the
results

SOMETHING USEFUL
59
SELECT a.name, s.flight, s.utc, r.sourceairport,
r.destinationairport, r.equipment
FROM `travel-sample` r
UNNEST r.schedule s
ON KEYS r.airlineid
AND r.destinationairport = "SFO"
AND s.day=1
ORDER BY s.utc;

N1QL and views
61
N1QL Views
Ad-hoc querying Predictable queries
JSON in and JSON out Number crunching
Large growth clusters Multi-dimensional/geospatial
queries

Multi-‐dimensional scaling
62

Scaling out
63
Horizontal scaling
▪ Partitions a dataset onto one or more homogenous nodes
▪ Each node runs the same mixed workloads
▪ Re-‐partition dataset with additional hardware capacity

What is Multi-‐Dimensional Scalability?
MDS is the architecture that enables independent scaling of data, query, and
indexing workloads.
64

65
Isolated Service for minimized interference
▪ Independent “zones” for query, index, and data services
Minimize indexing and query overhead on core key-‐value
operations.

66
Independent scalability for the best computational capacity
per service
Heavier
indexing
(index
more
fields):

scale
up
index
service
nodes.
More
RAM
for
query
processing:
scale
up
query
service
nodes.

Geospatial views
68
• Experimental
in
3.0,
now
GA
in
4.0
• Performance
and
stability
improvements
• GeoJSON output
• Bounding-‐box
and
range
queries
on

multiple
dimensions

What else is in Couchbase Server 4.0?
69

Other new things in Couchbase Server 4.0
70
• ForestDB:
GSIs
are
stored
with
a
new
storage
engine
• Filtered
XDCR:
more
efficient
cross-‐data
centre
replication
• Security:
LDAP,
admin
auditing

Developers?
SDK support for N1QL!
71

.NET SDK: Connection
72
// Option 1: Create Cluster and open bucket
var cluster = new Cluster(config);
var bucket = cluster.OpenBucket(bucketName);
// Option 2: ClusterHelper and open bucket (Singleton, thread safe bucket instance
ClusterHelper.Initialize(config);
var cluster = ClusterHelper.Get();
var bucket = ClusterHelper.GetBucket(bucketName);
// Close connection
cluster.CloseBucket(bucket);
cluster.Dispose();

.NET SDK: Client Configuration
73
// Cluster Configuration
var config = new ClientConfiguration
{
Servers = new List<Uri> { new Uri("http://10.211.55.2:8091/pools"),},
UseSsl = false,
BucketConfigs = new Dictionary<string, BucketConfiguration>
{
{
"travel-‐sample",
new BucketConfiguration
{
BucketName = "travel-‐sample",
UseSsl = false,
Password = "",
PoolConfiguration = new PoolConfiguration
{
MaxSize = 10,
MinSize = 5
}
}
}
}
};

.NET SDK: Basic Operations
74
// Create document (dynamic)
var doc = new Document<dynamic>
{
Id = "doc1",
Content = new
{
Id = "doc1",
Title = "My Document",
Type = "basic",
Pages = 3
}
};
// update or create document
var upsertResult = await bucket.UpsertAsync<dynamic>(doc);
// Get document
var getResult = bucket.GetDocument<dynamic>("key1");
// Delete document
var deleteResult = await bucket.RemoveAsync<dynamic>(doc);
// Check if doc/key exsists
var exsists = await bucket.ExistsAsync("key1");

.NET SDK: N1QL Query (raw)
75
// Create Query Request. Raw API request
var query = QueryRequest.Create("SELECT COUNT(*) FROM `travel-‐sample` WHERE type = 'airline'");
// Execute Query.
var response = await bucket.QueryAsync<dynamic>(query);
// Convert result to string for easy console print out.
var result = JsonConvert.SerializeObject(response, Formatting.Indented);

.NET SDK: N1QL Query (Linq2Couchbase)
76
// Lambda syntax
var result = ClusterHelper.GetBucket("beer-‐sample")
.Queryable<Beer>()
.Where(a => a.Type == "beer")
.Select(a => a)
.Join(
ClusterHelper.GetBucket("beer-‐sample")
.Queryable<Brewery>()
.Where(airline => airline.Type == "brewery"),
innerKey => innerKey.BreweryId,
outerKey => N1Ql.Key( outerKey),
(inner,outer) => new { Inner = inner, Outer = outer})
.Take(1)
.ToList();

.NET SDK: N1QL Query (Linq2Couchbase)
77
// Query syntax
var query =
from beer in QueryFactory.Queryable<Beer>(mockBucket.Object)
join breweryGroup in QueryFactory.Queryable<Brewery>(mockBucket.Object)
on beer.BreweryId equals N1Ql.Key(breweryGroup) into bg
from brewery in bg.DefaultIfEmpty()
select new { beer.Name, beer.Abv, BreweryName = brewery.Name };

https://github.com/couchbaselabs/N1QL-‐Intro-‐dotNET-‐
Supplement-‐Demo-‐code

Couchbase Developer Portal
developer.couchbase.com
80

Forums
http://forums.couchbase.com
81

NoSQL's biggest lie: SQL never went away - Martin Esmann

More Related Content

What's hot

Similar to NoSQL's biggest lie: SQL never went away - Martin Esmann

More from distributed matters

Recently uploaded

NoSQL's biggest lie: SQL never went away - Martin Esmann