Developing with Couchbase – Made
Easy
Karthik Babu Sekar
Solutions	Architect,	World	Wide	Technical	Field	Operations	@	Couchbase
- LinkedIn - https://www.linkedin.com/in/karthikbabusekar
- Twitter- https://twitter.com/SKBhere
- E-mail – karthik@couchbase.com
Change is
happening:
the way we
engage has
evolved
Technology is Further Driving Digital Innovation
Internet
of
Things
Big
Data
Mobile
Cloud
Personalized Feature-rich Quick Anywhere Anytime
Customers now expect exceptional experiences
Agility to adapt to changes in demand
Responsive, context-aware customer experiences
Multi-channel, multi-device engagement
Ability to support a large and growing number of
users
What businesses need to succeed in this new world
But	these	requirements	cannot	be	met	just	at	the	application	tier:	
they	need	to	be	applied	to	the	database
©2017 Couchbase Inc. 7
History	of	Couchbase
The Rise of the System of Engagement Database
8©2017 Couchbase Inc.
System of
Record
1980’
s
CRM ERP ECM
System of
Record
1995
CACHE
CRM ERP ECM
NoSQLSystem of
Record
2005
CACHE
CRM ERP ECM
TODAY
NoSQL
System of
Engagement
MOBILEQUERY
INDEX
SEARCH REPLICATION
ANALYTICS
DATA CACHE
Database requirements to power engaging experiences
9
Adaptable.
Flexible schema
Responsive.
Consistent
High-Performance
Scale as needed.
Able to handle any demand
(operations, queries, data)
Secure.
Device to cloud,
auditable, pluggable
Global.
Data locality,
Geo-distributed,
Multi-Data Center
Always on.
FaultTolerant. Able to
continue operations
under failure conditions
Mobile experience.
Offline, sync’d,
secure, low latency,
personalized
Intelligent.
FullText Search, Real
Time Analytics, Big
Data enabled
©2017 Couchbase Inc.
Couchbase Data Platform
Unified Administration
Unified Programming Interface
Data Query Index SearchMobileReplication Analytics
{N1QL}
The System of Engagement Database
Developing with Couchbase
Couchbase supports a wide range of frameworks, languages,
platforms and infrastructure choices.
Frameworks Languages Platforms Infrastructure
MOBILE MOBILE
©2017 Couchbase Inc. 12
Languages and Interfaces for Couchbase
§ Official SDKs
– Java
– .NET
– Node.js
– Python
§ For each of these we have
– Full Document support
– Interoperability
– Common Programming Model
Others: Erlang, Perl, TCL, Clojure, Scala
- PHP
- C
- Go
- Ruby
JDBC and ODBC
Couchbase	Server	Architecture
Overview
©2017 Couchbase Inc. 14
Service-Based Architecture
§ Multi-Dimensional Scalability (MDS) – Optionally Scale each service
independently:
– Data
– Index
– Query
MDS is the architecture
that enables independent
scaling of data, query, and
indexing workloads while
being managed as one
cluster.
©2017 Couchbase Inc. 15
Full Cluster Architecture
15
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 2
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
Manager
Cluster
Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service
©2017 Couchbase Inc. 16
Couchbase Data Access
• Everything is built on top of Key Value
• A Document store is a special case of Key-Value
• Views provide aggregation and real-time analytics
through incremental map-reduce
• Global Secondary Indexes provide low latency/high
throughput indexes
• N1QL is a language that provides a powerful and
expressive way of accessing documents
©2017 Couchbase Inc. 17
Cluster
Bucket
CRUD
View
Query
N1QL
Query
Interacting with Data
©2017 Couchbase Inc. 18
Couchbase SDKs
§ What does it mean to be a Couchbase SDK?
Cluster
Bucket
CRUD
Functional
Give the application developer a concurrent
API for basic (k-v) or document management
API
get()
insert()
replace()
upsert()
remove()
©2017 Couchbase Inc. 19
The Document
§ Documents are integral to the SDKs.
§ All SDKs support JSON format
§ In addition: Serialized objects, Unquoted Strings, Binary pass-through
§ A Document contains:
19
Property Description
ID The bucket-unique identifier
Content The value that is stored
Expiry An expiration time
CAS Check-and-Set identifier
©2017 Couchbase Inc. 20
Bucket API – Modifying Documents
§ insert() the document if it does not exist
§ replace() the document if it does exist
§ upsert() the document (insert or replace)
§ remove() the document
§ append() data to the document
§ prepend() data to the document
§ counter() for increment/decrement type operations
©2017 Couchbase Inc. 21
Bucket API – Retrieving Documents
§ get() the document
§ getFromReplica() if the active is not available
§ getAndTouch() to load the document and reset the expiry
§ getAndLock() to load the document and write-lock it
©2017 Couchbase Inc. 22
Unified API – DML, CAS Example
1. Two Clients retrieve the same document "XYZ"
2. Client A retrieves it first.
3. Client B then retrieves XYZ. Both clients will have the same CAS value for document XYZ
4. Client B tries to perform an update to document XYZ. The update succeeds as the CAS value
was unchanged from when Client B initially retrieved the document. Once the update
succeeds, the CAS value for XYZ changes.
5. Client A then tries to perform an update on XYZ immediately after Client B. The update will
fail as Client A's CAS value is out of date. When Client B updated XYZ, the CAS value
changed.
©2014 Couchbase, Inc.
©2017 Couchbase Inc. 23
Durability Requirements – Mutations
§ By default, a write operation is successful when the data has been written to the
memory of the node with the active vBucket.
– Eventually (but pretty quickly), the data will also be written to persistent storage and to replica
vBuckets
§ persistTo – write operation is successful when data has been written to persistent
storage
– Can be set to require writes to persistent storage for replicas, too
– Using persistTo will slow performance
§ replicateTo – write operation is successful when data has been written to both the
active vBucket and replica vBuckets
– If more than one replica, can set the number of replicas that must be written for success
©2017 Couchbase Inc. 24
Bucket API – Querying
§ query() is possible for
§ Views
§ N1QL
§ Streams N response rows as they arrive from the server
§ Loads more than one Document based on Criteria
§ Typically used to satisfy secondary and advanced querying use cases
©2017 Couchbase Inc. 25©2016 Couchbase Inc. 25
Bootstrapping your application - Service List
25
• Dynamic Distributed Services
• Dynamic Configuration Updates – No additional
work from the developer
• FaultTolerant/Durable Connectivity (we are paranoid)
Client Connectivity Characteristics:
©2017 Couchbase Inc. 26
Key/Value
§ Fastest way to access & mutate documents
§ Access by Document ID only
§ Includes subdocument manipulation
§ Efficient binary protocol
Latency Flexibility
©2017 Couchbase Inc. 27
Key/Value – API DML Methods (Create/Delete/Update)
• insert-Insert a document or binary key/value. Fails if the item exists.
• upsert-Stores a document or binary key/value to the bucket, or updates if a
document exists.
• replace-Replaces a document or binary key/value in a bucket. Fails if the
item doesn’t exist.
• remove-Deletes an item from the bucket. Fails if the item doesn’t exist
• append/prepend-Appends or prepends in place the value of a binary k/v
item. Does NOT work with documents
• touch-Updates the ttl of a documet.
• getAndTouch-Retrieves a document or binary key/value and updates the
expiry of the item at the same time.
• counter-Increments or decrements a key's numeric value.
©2017 Couchbase Inc. 28
Key/Value – API DML Methods (Retrieval)
• get-Retrieves a document or binary key/value.
• getAndLock-Lock the document or binary key/value on the server and
retrieve it. When an document is locked, its CAS changes and
subsequent operations on the document (without providing the current
CAS) will fail until the lock is no longer held.
• getReplica-Get a document binary key/value from a replica server in
your cluster.
• unlock-Unlock a previously locked document or binary key/value on
within a bucket.
©2017 Couchbase Inc. 29
Key/Value – Durability Methods
§ By default, a write operation is successful when the data has been written to the
memory of the node with the active vBucket.
– Eventually (but pretty quickly), the data will also be written to persistent storage and to replica vBuckets
persistTo – A write operation is
successful when data has been
written to persistent storage.
• Can be set to require writes to persistent
storage for replicas, too
• Using persistTo will slow performance
replicateTo – A write operation is
successful when data has been written
to both the active vBucket and replica
vBuckets.
• If more than one replica, can set the number of
replicas that must be written to for success
©2017 Couchbase Inc. 30
Key/Value - Sub-Document Operations
§ Document Mutations:
• Atomic Operate on individual
fields
• Identical syntax behavior to
regular bucket methods
(upsert, insert, get, replace)
• Support for JSON fragments.
• Support for Arrays with
uniqueness guarantees and
ordinal placement
(front/back)
©2017 Couchbase Inc. 31
Views
§ Distributed Map-Reduce
§ Great for Aggregations (fast)
§ Spatial Support
Latency Flexibility
©2017 Couchbase Inc. 32
N1QL
§ Very flexible, but get your index right
§ Global Index instead of Map/Reduce
§ Similar to SQL
Latency Flexibility
©2017 Couchbase Inc. 33
Full Text Search
§ Very Flexible, but more concepts to learn initially
§ Inverted Index
§ Still Developer Preview
§ Similar to Elasticsearch
Latency Flexibility
©2017 Couchbase Inc. 34
Show me some code ...
ü Create connection
ü Open bucket
ü Create JSON object
ü Create doc from object, assign
key, and "upsert"
• if exists, update
• if not, insert
ü Get the document by key,
and display
// Connect to localhost
Cluster cluster = CouchbaseCluster.create();
// Open a bucket connection
Bucket bucket = cluster.openBucket("customers");
// Create a document
JsonObject user = JsonObject.empty()
.put("first", "Walter")
.put("last", "White")
.put("job", "chemistry teacher")
.put("age", 50);
// Store a document
JsonDocument stored =
bucket.upsert(JsonDocument.create("walter", user));
// Get the document
JsonDocument walter = bucket.get("walter");
System.out.println("Found: " + walter.getString(“first”));("first"));
§ Connect, create, and retrieve
a document (Java)
DEMO
Questions?

Couchbase Chennai Meetup: Developing with Couchbase- made easy

  • 1.
    Developing with Couchbase– Made Easy Karthik Babu Sekar Solutions Architect, World Wide Technical Field Operations @ Couchbase - LinkedIn - https://www.linkedin.com/in/karthikbabusekar - Twitter- https://twitter.com/SKBhere - E-mail – karthik@couchbase.com
  • 2.
    Change is happening: the waywe engage has evolved
  • 3.
    Technology is FurtherDriving Digital Innovation Internet of Things Big Data Mobile Cloud
  • 4.
    Personalized Feature-rich QuickAnywhere Anytime Customers now expect exceptional experiences
  • 5.
    Agility to adaptto changes in demand Responsive, context-aware customer experiences Multi-channel, multi-device engagement Ability to support a large and growing number of users What businesses need to succeed in this new world
  • 6.
  • 7.
    ©2017 Couchbase Inc.7 History of Couchbase
  • 8.
    The Rise ofthe System of Engagement Database 8©2017 Couchbase Inc. System of Record 1980’ s CRM ERP ECM System of Record 1995 CACHE CRM ERP ECM NoSQLSystem of Record 2005 CACHE CRM ERP ECM TODAY NoSQL System of Engagement MOBILEQUERY INDEX SEARCH REPLICATION ANALYTICS DATA CACHE
  • 9.
    Database requirements topower engaging experiences 9 Adaptable. Flexible schema Responsive. Consistent High-Performance Scale as needed. Able to handle any demand (operations, queries, data) Secure. Device to cloud, auditable, pluggable Global. Data locality, Geo-distributed, Multi-Data Center Always on. FaultTolerant. Able to continue operations under failure conditions Mobile experience. Offline, sync’d, secure, low latency, personalized Intelligent. FullText Search, Real Time Analytics, Big Data enabled ©2017 Couchbase Inc.
  • 10.
    Couchbase Data Platform UnifiedAdministration Unified Programming Interface Data Query Index SearchMobileReplication Analytics {N1QL} The System of Engagement Database
  • 11.
    Developing with Couchbase Couchbasesupports a wide range of frameworks, languages, platforms and infrastructure choices. Frameworks Languages Platforms Infrastructure MOBILE MOBILE
  • 12.
    ©2017 Couchbase Inc.12 Languages and Interfaces for Couchbase § Official SDKs – Java – .NET – Node.js – Python § For each of these we have – Full Document support – Interoperability – Common Programming Model Others: Erlang, Perl, TCL, Clojure, Scala - PHP - C - Go - Ruby JDBC and ODBC
  • 13.
  • 14.
    ©2017 Couchbase Inc.14 Service-Based Architecture § Multi-Dimensional Scalability (MDS) – Optionally Scale each service independently: – Data – Index – Query MDS is the architecture that enables independent scaling of data, query, and indexing workloads while being managed as one cluster.
  • 15.
    ©2017 Couchbase Inc.15 Full Cluster Architecture 15 STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
  • 16.
    ©2017 Couchbase Inc.16 Couchbase Data Access • Everything is built on top of Key Value • A Document store is a special case of Key-Value • Views provide aggregation and real-time analytics through incremental map-reduce • Global Secondary Indexes provide low latency/high throughput indexes • N1QL is a language that provides a powerful and expressive way of accessing documents
  • 17.
    ©2017 Couchbase Inc.17 Cluster Bucket CRUD View Query N1QL Query Interacting with Data
  • 18.
    ©2017 Couchbase Inc.18 Couchbase SDKs § What does it mean to be a Couchbase SDK? Cluster Bucket CRUD Functional Give the application developer a concurrent API for basic (k-v) or document management API get() insert() replace() upsert() remove()
  • 19.
    ©2017 Couchbase Inc.19 The Document § Documents are integral to the SDKs. § All SDKs support JSON format § In addition: Serialized objects, Unquoted Strings, Binary pass-through § A Document contains: 19 Property Description ID The bucket-unique identifier Content The value that is stored Expiry An expiration time CAS Check-and-Set identifier
  • 20.
    ©2017 Couchbase Inc.20 Bucket API – Modifying Documents § insert() the document if it does not exist § replace() the document if it does exist § upsert() the document (insert or replace) § remove() the document § append() data to the document § prepend() data to the document § counter() for increment/decrement type operations
  • 21.
    ©2017 Couchbase Inc.21 Bucket API – Retrieving Documents § get() the document § getFromReplica() if the active is not available § getAndTouch() to load the document and reset the expiry § getAndLock() to load the document and write-lock it
  • 22.
    ©2017 Couchbase Inc.22 Unified API – DML, CAS Example 1. Two Clients retrieve the same document "XYZ" 2. Client A retrieves it first. 3. Client B then retrieves XYZ. Both clients will have the same CAS value for document XYZ 4. Client B tries to perform an update to document XYZ. The update succeeds as the CAS value was unchanged from when Client B initially retrieved the document. Once the update succeeds, the CAS value for XYZ changes. 5. Client A then tries to perform an update on XYZ immediately after Client B. The update will fail as Client A's CAS value is out of date. When Client B updated XYZ, the CAS value changed. ©2014 Couchbase, Inc.
  • 23.
    ©2017 Couchbase Inc.23 Durability Requirements – Mutations § By default, a write operation is successful when the data has been written to the memory of the node with the active vBucket. – Eventually (but pretty quickly), the data will also be written to persistent storage and to replica vBuckets § persistTo – write operation is successful when data has been written to persistent storage – Can be set to require writes to persistent storage for replicas, too – Using persistTo will slow performance § replicateTo – write operation is successful when data has been written to both the active vBucket and replica vBuckets – If more than one replica, can set the number of replicas that must be written for success
  • 24.
    ©2017 Couchbase Inc.24 Bucket API – Querying § query() is possible for § Views § N1QL § Streams N response rows as they arrive from the server § Loads more than one Document based on Criteria § Typically used to satisfy secondary and advanced querying use cases
  • 25.
    ©2017 Couchbase Inc.25©2016 Couchbase Inc. 25 Bootstrapping your application - Service List 25 • Dynamic Distributed Services • Dynamic Configuration Updates – No additional work from the developer • FaultTolerant/Durable Connectivity (we are paranoid) Client Connectivity Characteristics:
  • 26.
    ©2017 Couchbase Inc.26 Key/Value § Fastest way to access & mutate documents § Access by Document ID only § Includes subdocument manipulation § Efficient binary protocol Latency Flexibility
  • 27.
    ©2017 Couchbase Inc.27 Key/Value – API DML Methods (Create/Delete/Update) • insert-Insert a document or binary key/value. Fails if the item exists. • upsert-Stores a document or binary key/value to the bucket, or updates if a document exists. • replace-Replaces a document or binary key/value in a bucket. Fails if the item doesn’t exist. • remove-Deletes an item from the bucket. Fails if the item doesn’t exist • append/prepend-Appends or prepends in place the value of a binary k/v item. Does NOT work with documents • touch-Updates the ttl of a documet. • getAndTouch-Retrieves a document or binary key/value and updates the expiry of the item at the same time. • counter-Increments or decrements a key's numeric value.
  • 28.
    ©2017 Couchbase Inc.28 Key/Value – API DML Methods (Retrieval) • get-Retrieves a document or binary key/value. • getAndLock-Lock the document or binary key/value on the server and retrieve it. When an document is locked, its CAS changes and subsequent operations on the document (without providing the current CAS) will fail until the lock is no longer held. • getReplica-Get a document binary key/value from a replica server in your cluster. • unlock-Unlock a previously locked document or binary key/value on within a bucket.
  • 29.
    ©2017 Couchbase Inc.29 Key/Value – Durability Methods § By default, a write operation is successful when the data has been written to the memory of the node with the active vBucket. – Eventually (but pretty quickly), the data will also be written to persistent storage and to replica vBuckets persistTo – A write operation is successful when data has been written to persistent storage. • Can be set to require writes to persistent storage for replicas, too • Using persistTo will slow performance replicateTo – A write operation is successful when data has been written to both the active vBucket and replica vBuckets. • If more than one replica, can set the number of replicas that must be written to for success
  • 30.
    ©2017 Couchbase Inc.30 Key/Value - Sub-Document Operations § Document Mutations: • Atomic Operate on individual fields • Identical syntax behavior to regular bucket methods (upsert, insert, get, replace) • Support for JSON fragments. • Support for Arrays with uniqueness guarantees and ordinal placement (front/back)
  • 31.
    ©2017 Couchbase Inc.31 Views § Distributed Map-Reduce § Great for Aggregations (fast) § Spatial Support Latency Flexibility
  • 32.
    ©2017 Couchbase Inc.32 N1QL § Very flexible, but get your index right § Global Index instead of Map/Reduce § Similar to SQL Latency Flexibility
  • 33.
    ©2017 Couchbase Inc.33 Full Text Search § Very Flexible, but more concepts to learn initially § Inverted Index § Still Developer Preview § Similar to Elasticsearch Latency Flexibility
  • 34.
    ©2017 Couchbase Inc.34 Show me some code ... ü Create connection ü Open bucket ü Create JSON object ü Create doc from object, assign key, and "upsert" • if exists, update • if not, insert ü Get the document by key, and display // Connect to localhost Cluster cluster = CouchbaseCluster.create(); // Open a bucket connection Bucket bucket = cluster.openBucket("customers"); // Create a document JsonObject user = JsonObject.empty() .put("first", "Walter") .put("last", "White") .put("job", "chemistry teacher") .put("age", 50); // Store a document JsonDocument stored = bucket.upsert(JsonDocument.create("walter", user)); // Get the document JsonDocument walter = bucket.get("walter"); System.out.println("Found: " + walter.getString(“first”));("first")); § Connect, create, and retrieve a document (Java)
  • 35.
  • 36.