Couchbase: Introducción y Demo end-to-end https://www.meetup.com/Madrid-NoSQL/events/234346360/

1. Introducción a Couchbase
Manuel Hurtado
Solutions Engineer

2. What is Couchbase?

3. ©2015 Couchbase Inc. 3
Couchbase Server Defined
§ Couchbase Server 4.X is the first NoSQL Database to enable
you to develop with agility and operate at any scale.
Managed Cache Key-Value Store Document Database Embedded Database Sync Management

4. ©2015 Couchbase Inc. 4
The Power of the Flexible JSON Schema
Ability to store data in multiple ways
• Denormalized single document, as opposed to normalizing data across multiple table
• Dynamic Schema to add new values when needed

5. ©2015 Couchbase Inc. 5
Document Fundamentals -­‐ Composition
§ Similar to primary keys in relational databases
§ Documents are partitioned based on the
document ID
§ ID based document lookup is extremely fast
§ Must be unique
§ JSON
§ Binary -­‐ integers, strings, booleans
§ Common binary values include serialized objects,
compressed XML, compressed text, encrypted
values
Document ID or Key
Value
§ CAS Value (unique identifier for concurrency)
§ TTL
§ Flags (optional client library metadata)
§ Revision #
Metadata
Document
Document ID /
Key
MetadataValue

6. ©2015 Couchbase Inc. 6
Storing and Retrieving Documents

7. ©2015 Couchbase Inc. 7
Couchbase Data Access
• Everything is layered on top of Key Value
• A Document store is a special case of Key-
Value
• Views provide aggregation and real-time
analytics through incremental map-reduce
• Global Secondary Indexes provide low
latency/high throughput indexes
• N1QL is a language that provides a powerful
and expressive way of accessing documents

8. ©2015 Couchbase Inc. 8
Choose how to build your apps, and where to run them
Couchbase supports a wide range of frameworks, languages, platforms, and infrastructure choices

9. ©2015 Couchbase Inc. 9
Docker Support
§ Deployment Scenarios
§ Single host, single container
§ Single host multiple containers
§ Multiple hosts, single container on each host
§ Multiple hosts, multiple containers per host
§ https://hub.docker.com/_/couchbase/

10. ©2015 Couchbase Inc. 10
Top Use Cases
Profile Management Personalization 360o Customer View Internet of Things
Content Management Catalog Real Time Big Data Digital Communication
Mobile Applications
HighAvailabilityCaching

11. ©2015 Couchbase Inc. 11
World of Operational Databases
11
Graph
Stores
Neo4j
RDBMS/Transactional
Stores
Oracle, DB2, SQLServer,
MySql, Postgres
Caching
Stores
Coherence
, Redis,
Gemfire
Full-­‐text
Stores
Lucene, SOLR,
ElasticSearch
Key/Value
Stores
Hbase,
Cassandra,
Riak, Dynamo
Document Stores
Marklogic, mongoDB,
Cloudant,
DocumentDB
Couchbase

12. Couchbase Architecture
Single Node
12

13. ©2015 Couchbase Inc. 13
Couchbase Architecture
• Data Service – builds and
maintains Distributed secondary
indexes (MapReduce Views)
• Indexing Engine – builds and
maintains Global Secondary
Indexes
• Query Engine – plans,
coordinates, and executes queries
against either Global or
Distributed indexes
• Cluster Manager – configuration,
heartbeat, statistics, RESTful
Management interface

14. ©2015 Couchbase Inc. 14
Couchbase Read Operation
APPLICATION SERVER
MANAGED CACHE
DISK
DISK
QUEUE
REPLICATION
QUEUE
DOC 1
GET
DOC 1
DOC 1

15. ©2015 Couchbase Inc. 15
APPLICATION SERVER
MANAGED CACHE
DISK
DISK
QUEUE
REPLICATION
QUEUE
Write Operation
DOC 1
DOC 1DOC 1

16. ©2015 Couchbase Inc. 16
Auto sharding – Bucket and vBuckets
vB
Data buckets
vB
1 ….. 1024
Virtual buckets
§ A bucket is a logical, unique key space
§ Multiple buckets can exist within a single cluster of nodes
§ Each bucket has active and replica data sets (1, 2 or 3 extra copies)
§ Each data set has 1024 Virtual Buckets (vBuckets)
§ Each vBucket contains 1/1024th portion of the data set
§ vBuckets do not have a fixed physical server location
§ Mapping between the vBuckets and physical servers is called the
cluster map
§ Document IDs (keys) always get hashed to the same vbucket
§ Couchbase SDK’s lookup the vbucket -­‐> server mapping

17. ©2015 Couchbase Inc. 17
Cluster Map
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK

18. ©2015 Couchbase Inc. 18
Cluster Map
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK

19. ©2015 Couchbase Inc. 19
Cluster Map – 2 nodes added
Couchbase Cluster
CLUSTER MAP
Hashing Algorithm
CRC32
vBucket2
vBucket3
vBucket4
vBucket5
vBucket6
vBucket7
vBucket1024
vBucket1
...
Couchbase SDK

20. ©2015 Couchbase Inc. 20
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
Basic Operation
SHARD
5
SHARD
2
SHARD
9
SHARD SHARD SHARD
SHARD
4
SHARD
7
SHARD
8
SHARD SHARD SHARD
SHARD
1
SHARD
3
SHARD
6
SHARD SHARD SHARD
SHARD
4
SHARD
1
SHARD
8
SHARD SHARD SHARD
SHARD
6
SHARD
3
SHARD
2
SHARD SHARD SHARD
SHARD
7
SHARD
9
SHARD
5
SHARD SHARD SHARD

21. ©2015 Couchbase Inc. 21
Add Nodes to Cluster
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD
5
SHARD
2
SHARD SHARD
SHARD
4
SHARD SHARD
SHARD
1
SHARD
3
SHARD SHARD
SHARD
4
SHARD
1
SHARD
8
SHARD SHARD SHARD
SHARD
6
SHARD
3
SHARD
2
SHARD SHARD SHARD
SHARD
7
SHARD
9
SHARD
5
SHARD SHARD SHARD
SHARD
7
SHARD
SHARD
6
SHARD
SHARD
8
SHARD
9
SHARD
READ/WRITE/UPDATE

22. ©2015 Couchbase Inc. 22
Fail Over Node
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD
5
SHARD
2
SHARD SHARD
SHARD
4
SHARD SHARD
SHARD
1
SHARD
3
SHARD SHARD
SHARD
4
SHARD
1
SHARD
8
SHARD SHARD
SHARDSHARD
6
SHARD
2
SHARD SHARD SHARD
SHARD
7
SHARD
9
SHARD
5
SHARD SHARD
SHARD
SHARD
7
SHARD
SHARD
6
SHARDSHARD
8
SHARD
9
SHARD
SHARD
3
SHARD
1
SHARD
3
SHARD

23. ©2015 Couchbase Inc. 23
Modern Architecture – Multi-Dimensional Scaling
MDS is the architecture that enables independent scaling of
data, query, and indexing workloads while being managed as
one cluster.

24. ©2015 Couchbase Inc. 24
Modern Architecture – Multi-Dimensional Scaling

25. ©2015 Couchbase Inc. 25
Modern Architecture – Multi-Dimensional Scaling

26. ©2015 Couchbase Inc. 26
Demo: Install Couchbase
§ Install node 1
§ Add node 2
§ Rebalance
§ Admin UI
beer-­‐sample travel-­‐sample
node 1
192.168.56.101
node 2
192.168.56.102
Cluster Madrid prod

27. Cross Datacenter Replication
(XDCR)
27

28. ©2015 Couchbase Inc. 28
XDCR: Cross Data Center Replication
§ Application can access both clusters (master – master)
§ Scales out linearly
§ Within a cluster operations are strongly consistent and you can read-­‐
your-­‐own-­‐write. Between clusters mutations are eventually consistent.

29. ©2015 Couchbase Inc. 29
Memory-­‐to-­‐memory, topology aware replication
New York
San Francisco
SF Server Cluster
Couchbase Server 1
MEMORY DISK
Couchbase Server 2
MEMORY DISK
Couchbase Server 3
MEMORY DISK
NYC Server Cluster
Couchbase Server 1
MEMORY DISK
Couchbase Server 2
MEMORY DISK
Couchbase Server 3
MEMORY DISK
Couchbase Server 4
MEMORY DISK
USE CASES
§ Access by Localization
§ Active-­‐Active
§ Active-­‐Passive
§ Disaster Recovery
§ BI environment
§ Test environment

30. ©2015 Couchbase Inc. 30
Demo: XDCR: Replicate data between clusters
§ Install node 3 / Cluster backup
§ Enable XDCR between clusters
§ Replicate airlines

31. Developing an Application
31

32. ©2015 Couchbase Inc. 32
Languages and Interfaces for Couchbase
§ Official SDKs
– Java
– .NET
– Node.js
– Python
§ For each of these we have
– Full Document support
– Interoperability
– Common Programming Model
Others: Erlang, Perl, TCL, Clojure, Scala
– PHP
– C
– Go
– Ruby
JDBC and ODBC

33. ©2015 Couchbase Inc. 33
Interacting with Data
Views
N1QL
Query
Key Value Store
JSON Documents

34. Document Modeling

35. ©2015 Couchbase Inc. 35
Data Modeling
§ An ecommerce order in relational world

36. ©2015 Couchbase Inc. 36
Data Modeling
§ Embed vs Refer

37. ©2015 Couchbase Inc. 37
Data Modeling
§ Embed vs Refer

38. ©2015 Couchbase Inc. 38
Reference Documents for Lookups
38
§ Lookup documents which reference the
primary one
§ PROS
§ High Performance
§ Extremely scalable
§ Immediately consistent
§ CONS
§ Needs to be maintained by the application
§ GSI is a viable alternative
§ PROS
§ No need for Referential integrity
§ Good performance
§ CONS
§ Not immediately consistent -­‐ but can optional wait
for index updates (READ YOUR OWN WRITES)

39. ©2015 Couchbase Inc. 39
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "matthew@couchbase.com",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ],
“productsViewed”: [8, 33, 99, 100]
}

40. ©2015 Couchbase Inc. 40
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "matthew@couchbase.com",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{
"productsList": [
8, 33, 99, 100
]
}

41. ©2015 Couchbase Inc. 41
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "matthew@couchbase.com",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{
"productsList": [
8, 33, 99, 100
]
}
p::8
{
id": 1,
"name": "T-shirt",
"description": "Red Couchbase shirt",
"quantityInStock": 99,
"image": "tshirt.jpg”
}

42. ©2015 Couchbase Inc. 42
Compound keys: example
u::1001
{
"name": "Matthew Revell",
"email": "matthew@couchbase.com",
"address": "11-21 Paul Street",
"city": "London",
"postCode": "EC2A 4JU",
"telephone": "44-20-3837-9130",
"orders": [ 1, 9, 698, 32 ]
}
u::1001::productsviewed
{
"productsList": [
8, 33, 99, 100
]
}
p::8
{
id": 1,
"name": "T-shirt",
"description": "Red Couchbase shirt",
"quantityInStock": 99
}
p::8::img
“http://someurl.com/tshirt.jpg”

43. ©2015 Couchbase Inc. 43
Demo: Code examples: CRUD
Sample code for:
§ Create
§ Read
§ Update
§ Delete

44. Couchbase Query (N1QL)
N1QL Introduction

45. ©2015 Couchbase Inc. 45
Why SQL for JSON?
JSON Documents Provide
RichStructure
(Noassembly)
StructureEvolution
(Flexibleschema,easytochange)
SQL Provides
Generalqueryfunctionality
Queryacrossrelationships
Why N1QL?
Mostdevelopersalreadyknow
SQL
Noneedforcomplexquery
frameworks

46. ©2015 Couchbase Inc. 46
SELECT
§ Standard SELECT pipeline
– SELECT, FROM, WHERE, GROUP BY, ORDER BY, LIMIT, OFFSET
§ Queries across relationships
– JOINs
– Subqueries
– NEST — a JOIN that embeds child objects within their parent
– UNNEST — a JOIN that surfaces nested objects as top-­‐level data
§ Aggregation
§ Set operators
– UNION, INTERSECT, EXCEPT

47. ©2015 Couchbase Inc. 47
Indexing
§ CREATE / DROP INDEX
§ Two types of indexes
– View indexes
– GSI indexes (global secondary indexes)
§ Can index any data expression
– Nested / complex expressions
– Computed expressions
§ EXPLAIN

48. ©2015 Couchbase Inc. 48
Data writes
§ UPDATE … WHERE …
– Partial updates; deep updates
§ DELETE … WHERE …
– Deeply nested conditions
§ INSERT … VALUES …; INSERT … SELECT …
– Bulk insert; transfer and transformation
§ MERGE
– INSERT or UPDATE; ETL support

49. ©2015 Couchbase Inc. 49
Document-­‐oriented Extensions
§ Nested
– Paths—user.profile.email, children[0], children[0:2]
– NEST, UNNEST
– Ranging—EVERY child.age > 5 FOR child IN children END
– Transformations—SELECT {“name”: first_name || last_name}
§ Non-­‐uniform
– IS MISSING
– Type checking and conversion
§ Distributed
– Direct lookup—USE KEYS
– Efficient joins—ON KEYS

50. ©2015 Couchbase Inc. 50
Expressions
©2014 Couchbase, Inc.
50
Expressions
Literals
q Primitives [ 0, ‘hello’, TRUE ]
q NULL
Operators
q Arithmetic [ +, -­‐, *, /, % ]
q Logical [ AND, OR, NOT ]
q Comparison [ <, <=, =, !=, >=, >, BETWEEN, IS NULL ]
q Pattern matching [ LIKE ]
q Conditional [ CASE ]
Scalar functions
q Numeric [ trigonometric, ROUND, TRUNC, … ]
q String [ UPPER, LOWER, TRIM, SUBSTR, … ]
q Date [ string and numeric dates, NOW, date arithmetic, … ]
Aggregate functions q MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]
Subqueries q Subqueries are full expressions

51. ©2015 Couchbase Inc. 51
Document-­‐Oriented Expression Features
©2014 Couchbase, Inc.
51
Features
Ranging over collections
WHERE ANY c IN children SATISFIES c.age > 10 END
WHERE EVERY r IN ratings SATISFIES r > 3 END
Mapping with filtering ARRAY c.name FOR c IN children WHEN c.age > 10 END
Deep traversal and update
WHERE ANY node WITHIN request SATISFIES node.type = “xyz” END
UPDATE doc SET c.name = Title(c.name) FOR c WITHIN doc END
Dynamic Construction
SELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object
SELECT [ a, b ] FROM … // Dynamic array
Nested sourcing SELECT * FROM a.b.c … // Select from nested objects

52. ©2015 Couchbase Inc. 52
Demo: N1QL samples
N1QL Queries from
§ Web UI
§ Command line cbq
§ REST

53. Summary
53

54. ©2015 Couchbase Inc. 54
Develop with Agility
With Couchbase Server, you can Develop Applications with Agility. Here’s why:
Easier, Faster Development Flexible Data Modeling Powerful Querying
SQL Integration & Migration Big Data Integration Mobile / IoT

55. ©2015 Couchbase Inc. 55
Operate at Any Scale
With Couchbase Server, you can deploy, manage, and scale apps without limits. Here’s why:
Elastic Scalability Consistent High
Performance
Always-on Availability
Multi-Data Center Deployment Simple, Powerful
Administration
Enterprise Grade Security

56. Q&A
56

57. ©2015 Couchbase Inc. 57
Additional Resources
57
§ Couchbase Labs: https://github.com/couchbaselabs
§ Sample Applications:
§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=try
§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=beer
§ General Docs: http://docs.couchbase.com
§ Query Portal: http://query.couchbase.com
§ Blog: http://blog.couchbase.com
§ Forum: http://forums.couchbase.com

58. ©2015 Couchbase Inc. 58
beer-­‐sample travel-­‐sample
node 1
192.168.56.101
node 2
192.168.56.102
Cluster Madrid prod
airlines
node 3
192.168.56.1013
Cluster Backup
XDCR
airlines

59. ©2015 Couchbase Inc. 59
airlines
node 3
192.168.56.1013
Cluster Backup