• Share
  • Email
  • Embed
  • Like
  • Private Content
Couchbase_TLV_2014_Introduction to couchbase
 

Couchbase_TLV_2014_Introduction to couchbase

on

  • 928 views

 

Statistics

Views

Total Views
928
Views on SlideShare
471
Embed Views
457

Actions

Likes
1
Downloads
36
Comments
0

1 Embed 457

http://www.couchbase.com 457

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • All nodes are equal, single node type, easy to scale your cluster. No single point of failoverEvery node manages some active data and some replica data. Data is distributed across the clsuter and hence the load is also uniformly distributed using auto sharding. We have a fixed number of shards that a key get hashed to. 1024 shards, distributed across the cluster. Replication within the cluster for high availability. Number of replicas are configurable with upto 3 replicas. With auto-failiover or manual failover, replica information is immediately promoted to active Add multiple nodes at a time to grow and shrink your cluster.
  • Todo: Icons for availabilty and security--Ease of administration - Better connection managementMore are things like CCP- Faster client bootstrap timeMore reliable behaviour during reliable and failoverOver time all client releases will do this …
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then quickly Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then quickly Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A get request comes in from the application .2.  Couchbase Server sends the data from the caching layer at very low latency to the reader (if the data is not already in the working set and cached, it gets read from disk into cache first)
  • Now, as you fill up memory (click), some data that has already been written to diskwill be ejected from RAM to make room for new data. (click)Couchbase supports holding much more data than you have RAM available. It’s important to size the RAM capacity appropriately for your working set: the portion of data your application is working with at any given point in time and needs very low latency, high throughput access to. In some applications this is the entire data set, in others it is much smaller. As RAM fills up, we use a “not recently used” algorithm to determine the best data to be ejected from cache.
  • Should a read now come in for one of those documents that has been ejected (click), it is copied back from disk into RAM and sent back to the application. The document then remains in RAM as long as there is space and it is being accessed.
  • Bulletize the text. Make sure the builds work.
  • Overview of what this feature is
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk

Couchbase_TLV_2014_Introduction to couchbase Couchbase_TLV_2014_Introduction to couchbase Presentation Transcript

  • Introduction to Couchbase Server Ilam Siva Senior Product Manager
  • Couchbase Server NoSQL Document Database
  • Couchbase Open Source Project • Leading NoSQL database project focused on distributed database technology and surrounding ecosystem • Supports both key-value and document-oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged software in both enterprise and community editions. Couchbase Open Source Project
  • In this session • Overview of Couchbase Server features • What’s new in Couchbase Server 2.5 (HOT OFF THE SHELF!) • Architectural Overview and Couchbase Operations • Live Demo with a peak into new features
  • Couchbase Server Easy Scalability Grow cluster without application changes, without downtime with a single click Always On 24x365 No downtime for software upgrades, hardware maintenance, etc. Consistent High Performance Consistent sub-millisecond read and write response times with consistent high throughput JSON JSON JSO JSON JSON N Flexible Data Model JSON document model with no fixed schema.
  • Core Couchbase Server Features Built-in clustering – All nodes equal Append-only storage layer Data replication with auto-failover Online compaction Zero-downtime maintenance Monitoring and admin API & UI Built-in managed cached SDK for a variety of languages
  • New New in 2.2 2.1 introduced in 2.1 and 2.2  Multi-threaded persistence engine  New XDCR protocol based on memcached  Optimistic XDCR  Read-only admin user  CBHealthcheck – Cluster health check tool  Automated and optimized purge management  Hostname management  CBRecovery Data recovery tool from remote clusters  Rebalance progress indicators  Non-root, non-sudo install
  • Coming Soon!!! Couchbase 2.5! AVAILABILITY AND RELIABILITY Rack awareness SECURITY Secure cross data center replication EASE OF ADMINISTRATION Better client connection management
  • Couchbase Server Architecture 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 New Persistence Layer vBucket state and replication manager Node health monitor Rebalance orchestrator storage interface Global singleton supervisor Data Manager Configuration manager Couchbase EP Engine Process monitor Memcached Heartbeat Moxi REST management API/Web UI Query Engine 8092 Cluster Manager http on each node one per cluster Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199
  • Couchbase Server Architecture Query Engine Query API 11210 / 11211 8091 Admin Console Data access ports http Object-managed Cache Erlang /OTP 8092 REST management API/Web UI Replication, Rebalance, Shard State Manager Multi-threaded Persistence Engine Data Manager Cluster Manager
  • Couchbase Operations
  • Auto sharding and Cluster Map Hash function (KEY) Physical servers Logical Partitions Cluster Map vB1 vB2 vB3 vB4 A A B B vB5 vB7 C D vB6 vB8 D C More scalability required Add node
  • Single node – Couchbase Write Operation App Server Doc 1 3 Replication Queue 3 Managed Cache Doc 1 Disk Couchbase Server Node Disk Queue To other node 2
  • Single node – Couchbase Update Operation App Server Doc 1 3 Replication Queue 3 Managed Cache Doc 1 Disk Doc 1 Couchbase Server Node Disk Queue To other node 2
  • Single node – Couchbase Read Operation App Server Get Doc 1 3 Replication Queue 3 Managed Cache Doc 1 Disk Doc 1 Couchbase Server Node Disk Queue To other node 2
  • Cache Ejection To other node Replication Queue Doc 2 6 5 4 3 Managed Cache Doc 1 Disk Doc 1 Doc 6 Doc 5 Doc 4 Doc 3 Doc 2 Couchbase Server Node Disk Queue App Server
  • Cache Miss To other node Replication Queue Get Doc 1 Managed Cache Doc 6 Doc 4 Doc 1 Doc Doc 2 Doc 5 3 Disk Doc 1 Doc 6 Doc 5 Doc 4 Doc 3 Doc 2 Couchbase Server Node Disk Queue App Server
  • Basic Operation App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Read/write/update SERVER 1 SERVER 2 SERVER 3 Active Active Active Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 2 Doc Doc 7 Doc Doc 3 Doc Doc 9 Doc Doc 8 Doc Doc 6 Doc Replica Replica Replica • Docs distributed evenly across servers • Each server stores both active and replica docs - Only one server active at a time • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 1 Doc Doc 3 Doc Doc 9 Doc • App reads, writes, updates docs Doc 8 Doc Doc 2 Doc Doc 5 Doc • Multiple app servers can access same document at same time Couchbase Server Cluster User Configured Replica Count = 1 - App never needs to know
  • Add Nodes to Cluster App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Read/write/update Read/write/update SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 Active Active Active Active Active Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 7 Doc Doc 3 Doc Doc 9 Doc Doc 8 Doc Doc 6 • Docs automatically rebalance across cluster Doc Doc 2 Doc Replica Replica Replica Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 8 Doc Doc 2 Doc Doc 5 Doc Couchbase Server Cluster User Configured Replica Count = 1 • Two servers added with one-click operation - Even distribution of docs - Minimum doc movement • Cluster map updated Replica Replica • App database calls now distributed over larger number of servers
  • Fail Over Node App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 Active Active Active Active Active Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 9 Doc Doc 2 Doc Doc 7 Doc Doc 3 Doc Doc 8 Doc Doc 1 Doc 6 Doc • Requests to Server 3 fail Doc • Cluster detects server failed Doc 3 Replica Replica Replica Replica Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 5 Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 2 Couchbase Server Cluster User Configured Replica Count = 1 Doc • App servers accessing docs Replica Doc 8 Doc Doc – Promotes replicas of docs to active – Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow
  • Demo Time
  • Indexing and Querying – The basics • Define materialized views on JSON documents and then query across the data set • Using views you can define • Primary indexes • Simple secondary indexes (most common use case) • Complex secondary, tertiary and composite indexes • Aggregations (reduction) • Indexes are eventually indexed • Queries are eventually consistent • Built using Map/Reduce technology • Map and Reduce functions are written in Javascript
  • Indexing and Querying APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Query SERVER 1 SERVER 2 ACTIVE ACTIVE SERVER 3 ACTIVE • Indexing work is distributed amongst nodes Doc 5 Doc Doc 5 Doc Doc 5 Doc • Large data set possible Doc 2 Doc Doc 2 Doc Doc 2 Doc • Parallelize the effort Doc 9 Doc Doc 9 Doc Doc 9 Doc REPLICA REPLICA REPLICA Doc 4 Doc Doc 4 Doc Doc 4 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 8 Doc Doc 8 Doc Doc 8 Doc COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 • Each node has index for data stored on it • Queries combine the results from required nodes
  • Cross Data Center Replication – The basics • Replicate your Couchbase data across clusters • Clusters may be spread across geos • Configured on a per-bucket (per-database) basis • Supports unidirectional and bidirectional operation • Application can read and write from both clusters - Active – Active replication • Replication throughput scales out linearly • Different from intra-cluster replication
  • Cross data center replication – Data flow 2 Doc 1 App Server Couchbase Server Node 3 2 Managed Cache Replication Queue Doc 1 Disk Queue To other node 3 Disk Doc 1 XDCR Engine To other cluster
  • Cross Data Center Replication (XDCR) Optimistic replication Couchbase Server – San Francisco SERVER 1 SERVER 2 SERVER 3 Per replication Tunable Parameters Couchbase Server – New York SERVER 1 SERVER 2 Optimized protocol based on memcached Reliability and performance at scale SERVER 3
  • Demo Time
  • A brief look into 2.5!
  • Rack awareness in Couchbase Server
  • Secure Cross Data Center Replication
  • Why is secure cross data center replication important ? • More and more sensitive data is getting stored in NoSQL databases • Certain industries have security and compliance regulations to follow - Healthcare (HIPAA) Financial services (PCI) Public education (FERPA) • Keep sensitive information across the internet encrypted so that only the intended recipient can understand it.
  • A look into the future!
  • Learn Couchbase Query Language more about N1QL at query.couchbase.com N1QL Read “Nickel” Our next generation query language for JSON In Dev Preview
  • Couchbase Server www.couchbase.com/download
  • Thank you! ilam@couchbase.com Download Couchbase Server http://www.couchbase.com/download