This document provides an overview of a Couchbase meetup presentation on Couchbase architecture and scalability. The presentation covers the history of Couchbase, its key capabilities as a data management solution, architecture including services for data, indexing and querying, common use cases, what makes Couchbase unique, always-on availability features, administration and management, performance and scalability including auto sharding and cross data center replication, and examples of using its query language N1QL.

1. Couchbase Sydney
Meetup
Couchbase Architecture and Scalability
Karthik Babu Sekar
Solutions Architect, WW Field Operations @ Couchbase
- LinkedIn - https://www.linkedin.com/in/karthikbabusekar
- Twitter- https://twitter.com/SKBhere
- E-mail – karthik@couchbase.com

2. ©2016 Couchbase Inc.©2015 Couchbase Inc.
History of Couchbase
2

3. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase provides a complete Data Management solution
3
High availability
cache
Key-value
store
Document
database
Embedded database Sync
management
Multi-purpose capabilities support a broad range of apps and use cases
Enterprises often start with cache, then broaden usage to other apps and use cases

4. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase meets today’s & tomorrow’s requirements
4
Flexible data model
Consistent performance at scale
High availability
Easy, affordable scalability
24x365

5. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Common Use-Cases
5
360 Degree
Customer View
Profile
Management
Catalog Fraud
Detection
Content
Management
Internet of
Things
Digital
Communication
Real Time
Big Data
Mobile
Applications
Personalization

6. ©2016 Couchbase Inc.©2015 Couchbase Inc.
What makes Couchbase unique?
6
Performance &
scalability leader
Sub millisecond latency
with high throughput;
memory-centric
architecture
Multi-
purpose
Enterprise class
Web Console
Easy to deploy &
manage; integrated
Admin Console, single-
click cluster expansion &
rebalance
Cache, key value store,
document database, and
local/mobile database in
single platform
Always-on
availability
Data replication across
nodes, clusters, and
data centers
Enterprises choose Couchbase for several key advantages
24x365

7. ©2016 Couchbase Inc.©2015 Couchbase Inc.
§ Consolidated cache and
database
§ Tune memory required based
on application requirements
Multi-purpose database supports many uses
7
7
Tunable built-in
cache
Flexible schemas
with JSON
Couchbase Lite
§ Represent data with varying
schemas using JSON on the
server or on the device
§ Index and query data with
Javascript views
§ Light weight embedded DB for
always available apps
§ Sync Gateway syncs data
seamlessly with Couchbase
Server

8. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase delivers always-on availability
8
High
Availability
Disaster
Recovery
Backup &
Restore
§ In-memory replication with
manual or automatic fail over
§ Rack-zone awareness to
minimize data unavailability
§ Memory-to-memory cross
cluster replication across data
centers or geos
§ Active-active topology with bi-
directional setup
§ Full backup or Incremental
backup with online restore
§ Delta node catch-ups for faster
recovery after failures
24x365

9. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Simplified administration for exceptional ease of use
Online upgrades and
operations
Built-in enterprise
class admin console
Restful APIs
§ Online software, hardware and
DB upgrades
§ Indexing, compaction,
rebalance, backup & restore
§ Perform all administrative tasks
with the click of a button
§ Monitor status of the system
visual at cluster level, database
level, server level
§ All admin operations available
via UI, REST APIs or CLI
commands
§ Integrate third party monitoring
tools easily using REST

10. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase leads in performance and scalability
Auto Sharding Memory-memory
XDCR
Multi Dimensional
Scaling
§ No manual sharding
§ Database manages data
movement to scale out – not
the user
§ Market’s only memory-to-
memory database replication
across clusters and geos
§ Provides disaster recover /
data locality
§ Hugely simplifies management
of clusters
§ Easy to scale clusters by adding
any number of nodes

11. Service-Based Architecture
Multi-Dimensional Scalability (MDS) – Optionally Scale each service independently:
• Data
• Index
• Query
MDS is the architecture
that enables independent
scaling of data, query, and
indexing workloads while
being managed as one
cluster.

12. ©2015 Couchbase Inc.
Full Cluster Architecture
12
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 2
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service

13. ©2015 Couchbase Inc.
Full Cluster Architecture
13
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 2
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster Manager
Managed
Cache
Storage
Data Service
Index Service
Query Service

14. Couchbase Architecture: Single
Node
14

15. High level architecture - Single Node
Couchbase Server nodes
are identical
Two core components
ü Cluster Manager
ü Data Manager
Three independent services
ü Data Service
ü Index Service
ü Query Service
Couchbase Server Node
Data Manager Cluster Manager
Management
REST API
Web UI
Erlang / OTP
Node /
Cluster
Coordination
Data
Service
Storage
Managed
Cache
Get/Set
Map
Reduce
Views
Query
Service
Query
Engine
Query Plan
Cache
Storage
Index
Service
Indexing
Engine
Managed
Cache
Storage

16. Cluster Manager
Management and coordination
Responsible for
ü UI / REST interface
ü Configuration / Administration
ü Coordinating cluster rebalancing
ü Process monitoring
ü Statistics
ü No data processing
Erlang/OTP code base
ü Developed by Ericsson
ü Telco-switch grade performance
Couchbase Server Node
Cluster Manager
Management
REST API
Web UI
Erlang / OTP
Node /
Cluster
Coordination
Data Manager
Data
Service
Storage
Managed
Cache
Get/Set
Map-
Reduce
Views
Query
Service
Query
Engine
Query Plan
Cache
Storage
Index
Service
Indexing
Engine
Managed
Cache
Storage
:8091 Admin Console/API

17. Node Manager
Copyright © 2015 Couchbase, Inc.
Three supported workloads
Data Service
ü Get/Set of individual documents
ü Distributed secondary indexes (Views)
Index Service
ü Global secondary indexes
ü Index maintenance and provision
Query Service
ü N1QL (SQL superset for JSON)
ü Execution plans, index coordination
C / C++ and Go code bases
ü Very fast and memory efficient
even with large memory footprint Couchbase Server Node
Cluster Manager
Management
REST API
Web UI
Erlang / OTP
Node /
Cluster
Coordination
Data Manager
Data
Service
Storage
Managed
Cache
Get/Set
Map-
Reduce
Views
Query
Service
Query
Engine
Query Plan
Cache
Storage
Index
Service
Indexing
Engine
Managed
Cache
Storage

18. Single node type / code base
ü All services available on every node
ü Services enabled independently
Allows cluster to be configured for
ü Varying hardware capacities per node
ü Application workload tuning
Couchbase Server Node
Cluster Manager
Management
REST API
Web UI
Erlang / OTP
Node /
Cluster
Coordination
Data Manager
Query
Service
Data
Service
Index
Service
Index
Service
Indexing
Engine
Managed
Cache
Storage
Query
Service
Query
Engine
Query Plan
Cache
Storage
Data
Service
Storage
Managed
Cache
Get/Set
Map-
Reduce
Views
Multi-Dimensional Scalability
Copyright © 2015 Couchbase, Inc.

19. ©2015 Couchbase Inc. 19
Couchbase Data Access
• Everything is built on top of Key Value
• A Document store is a special case of Key-Value
• Views provide aggregation and real-time analytics
through incremental map-reduce
• Global Secondary Indexes provide low latency/high
throughput indexes
• N1QL is a language that provides a powerful and
expressive way of accessing documents

20. Key Differentiators - N1QL
N1QL is a Next-generation NoSQL query language for JSON.
20
§ JOINS
§ Powerful Extensions for JSON
§ ODBC/JDBC drivers available

21. Introduction to N1QL
SQL query language for documents
• SELECT * FROM WHERE/LIKE/JOIN/GROUP/etc, CREATE INDEX
• Extended for JSON to support nested and hierarchical data structures
• Support for map-reduce views and Global Secondary Indexes
• Query (DQL), Manipulation (DML), Description (DDL)
• ODBC/JDBC drivers

22. SQL 4 documents - examples
2222
Listing products in a category Finding the most popular products in a
category

23. Global Secondary Indexes
New Storage Options for Global Secondary Index
1. Memory-Optimized Global Secondary Indexes (MOI)
• Optimized for Memory: small memory footprint, optimized for lowest latency queries
• Faster Indexing: fresh indexes under heavy mutations with lock free index maintenance
• Predictable low latency with Query and Key Based Operations, even in presence of large
number of indexes
1. Standard Global Secondary Indexes
• Optimized for Storage + Memory: efficient spill to disk with ForestDB as the storage
engine
• Advanced IO Performance: new circular writes that minimize compaction overhead of
append only writes

24. ©2016 Couchbase Inc.
Mobile
Search
Cache
OLAP
Data
Warehouse
Hadoop
RDBMS

25. ©2016 Couchbase Inc.
Mobile
Search
Cache
OLAP
Data
Warehouse
Hadoop
RDBMS

26. ©2016 Couchbase Inc.
Search
OLAP
Data
Warehouse
Hadoop
{N1QL}

27. ©2016 Couchbase Inc.
Search
OLAP
Data
Warehouse
Hadoop
{N1QL}

28. ©2016 Couchbase Inc.
OLAP
Data
Warehouse
Hadoop
{N1QL}

29. ©2016 Couchbase Inc.
INTRODUCING SEARCH
Structured Query AND Search
One platform.
|
Search Feeling lucky?

30. ©2016 Couchbase Inc.
OLAP
Data
Warehouse
Hadoop
{N1QL
}

31. ©2016 Couchbase Inc.
OLAP
Data
Warehouse
Hadoop
{N1QL
}

32. ©2016 Couchbase Inc.
Data
Warehouse
Hadoop
{N1QL
}

33. ©2016 Couchbase Inc.
Data
Warehouse
Hadoop
{N1QL
}

34. ©2016 Couchbase Inc.
INTRODUCING REALTIME ANALYTICS
You may also like these products
Thank you for your order!
Order #: 3423943
Paid Partner Content
Operations & Analytics.
One platform.

35. ©2016 Couchbase Inc.
OLAP
Data
Warehouse
Hadoop
{N1QL
}

36. ©2016 Couchbase Inc.
Data
Warehouse
Hadoop
{N1QL
}

37. ©2016 Couchbase Inc.
Data
Warehouse
Hadoop
{N1QL
}

38. ©2016 Couchbase Inc.
INTRODUCING REALTIME ANALYTICS
You may also like these products
Thank you for your order!
Order #: 3423943
Paid Partner Content
Operations & Analytics.
One platform.

39. ©2016 Couchbase Inc.
Data
Warehouse
Hadoop
THE PLATFORM FOR YOUR DIGITAL APPLICATIONS

40. ©2016 Couchbase Inc.
THE DATA PLATFORM FOR THE DIGITAL ECONOMY
Unified Adminstration
Unified Programming Interface
Data Query Index SearchMobileReplication Analytics
{N1QL
}

41. XDCR: Cross Data Center Replication
Application can access both clusters (master – master)
Scales out linearly
Different from intra-cluster replication (“CP” versus “AP”)

42. XDCR: Flexible topologies
One-one, one-many, many-one
Differently sized and resourced clusters supported

43. Couchbase Architecture: Mobile
43

44. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase Mobile Overview

45. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Always Available and Fast
45

46. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Native Support
46

47. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase Mobile Solution
47

48. Key Take-aways
Couchbase Mobile = Couchbase Lite + Couchbase Sync Gateway + Couchbase Server
Couchbase Lite: Only NoSQL database for mobile devices (phones, tables, embedded
systems)
Couchbase Sync Gateway: Offline/Online synchronization
48

49. ©2016 Couchbase Inc.©2015 Couchbase Inc.
Backup and Restore with Couchbase
§ Zer0-downtime backup and restore
§ Built-in utilities: cbbackup / cbrestore
§ Full, differential and cumulative backup available per-bucket.
§ Restore from any point, to any bucket or topology

50. Demo & Q&A