Download to read offline
![14
Relational Models
Database
// order (structure / dictionary / object / map /etc)
order['order_id'] = 3492843;
order['order_date'] = "2016-07-14T18:27:22.586Z";
order['products'] = [{
'product_id' = 78323,
'quantity' = 2,
'price' = 39.99,
'sub_total' = 79.98,
}];
order['user_id'] = 123;
order['billing_address_1'] = "1302 Pleasant Ridge Rd";
order['billing_address_2'] = "";
order['billing_city'] = "Greensboro";
order['billing_region_code'] = "NC"; order['billing_postal_code'] = "27409";
order['billing_country_code'] = "US";
order['shipping_address_1'] = "1302 Pleasant Ridge Rd";
order['shipping_address_2'] = "";
order['shipping_city'] = "Greensboro";
order['shipping_region_code'] = "NC";
order['shipping_postal_code'] = "27409";
order['shipping_country_code'] = "US";
order['card_number'] = "3337151609084503";
order['expiration_month'] = 11;
order['expiration_year'] = 2019;
Code](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-14-2048.jpg)
![15
Non-Relational Models
// order (structure / dictionary / object / map /etc)
order['order_id'] = 3492843;
order['order_date'] = "2016-07-14T18:27:22.586Z";
order['products'] = [{
'product_id' = 78323,
'quantity' = 2,
'price' = 39.99,
'sub_total' = 79.98,
}];
order['user_id'] = 123;
order['billing_address_1'] = "1302 Pleasant Ridge Rd";
order['billing_address_2'] = "";
order['billing_city'] = "Greensboro";
order['billing_region_code'] = "NC"; order['billing_postal_code'] = "27409";
order['billing_country_code'] = "US";
order['shipping_address_1'] = "1302 Pleasant Ridge Rd";
order['shipping_address_2'] = "";
order['shipping_city'] = "Greensboro";
order['shipping_region_code'] = "NC";
order['shipping_postal_code'] = "27409";
order['shipping_country_code'] = "US";
order['card_number'] = "3337151609084503";
order['expiration_month'] = 11;
order['expiration_year'] = 2019;
Database (JSON)Code
{
"order_id": 3492843,
"order_date": "2016-07-14T18:27:22.586Z",
"products": [{
'product_id': 78323,
'quantity': 2,
'price': 39.99,
'sub_total': 79.98,
}];
"user_id": 123,
"billing_address_1": "1302 Pleasant Ridge Rd",
"billing_address_2": "",
"billing_city": "Greensboro",
"billing_region_code": "NC",
"billing_postal_code": "27409",
"billing_country_code": "US",
"shipping_address_1": "1302 Pleasant Ridge Rd",
"shipping_address_2": "",
"shipping_city": "Greensboro",
"shipping_region_code": "NC",
"shippping_postal_code": "27409",
"shippping_country_code": "US",
"card_number": "3337151609084503",
"expiration_month": 11,
"expiration_year": 2019
}](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-15-2048.jpg)
![51
Connecting to Couchbase
22
public boolean function onApplicationStart(){
application['couchbase'] = new cfcouchbase.CouchbaseClient({
servers: "127.0.0.1",
bucketName: "beer-sample"
});
return true;
}
public boolean function onApplicationEnd(){
application.couchbase.shutdown( 10 );
return true;
}](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-51-2048.jpg)
![52
Writing Documents
22
var document = {
"id": "512_brewing_company",
"name": "(512) Brewing Company",
"city": "Austin",
"state": "Texas",
"code": "78745",
"country": "United States",
"phone": "512.707.2337",
"website": "http://512brewing.com/",
"type": "brewery",
"updated": "2010-07-22 20:00:20",
"description": "(512) Brewing Company is a microbrewery located in the
heart of Austin that brews for the community using as many local,
domestic and organic ingredients as possible.",
"address": [
"407 Radam, F200"
],
"geo": {
"accuracy": "ROOFTOP",
"lat": 30.2234,
"lon": -97.7697
}
};
application.couchbase.upsert
id=document.id,
value=document
);
Define a Document Save the Document](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-52-2048.jpg)
![N1QL Operators
59
Type Support
Arithmetic + - * / % -val
Collection ANY EVERY ARRAY FIRST EXISTS IN WITHIN
Comparison = == != <> > >= < <=
(NOT) BETWEEN (NOT) LIKE IS (NOT) NULL
IS (NOT) MISSING IS (NOT) VALUED
Conditional CASE expression WHEN value THEN expression
Construction Array [ value, value, ... ]
Object { key:value, key:value, ... }
Logical AND OR NOT
String ||](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-59-2048.jpg)
![N1QL Operators
60
ARRAY_AGG(EXP)
ARRAY_AGG(DISTINCT EXP)
AVG(EXP)
AVG(DISTINCT EXP)
COUNT(*)
COUNT(EXP)
COUNT(DISTINCT EXP)
MAX(EXP)
MIN(EXP)
SUM(EXP)
SUM(DISTINCT EXP)
Aggregate Functions
Object Functions
OBJECT_LENGTH(EXP)
OBJECT_NAMES(EXP)
OBJECT_PAIRS(EXP)
OBJECT_VALUES(EXP)
Conditionals - Unknowns
IFMISSING(EXP1, EXP2, …)
IFMISSINGORNULL(EXP1, EXP2, …)
IFNULL(EXP1, EXP2, …)
MISSINGIF(EXP1, EXP2)
NULLIF(EXP1, EXP2)
Conditionals - Numbers
IFINF(EXP1, EXP2, …)
IFNAN(EXP1, EXP2, …)
IFNANORINF(EXP1, EXP2, …)
NANIF(EXP1, EXP2)
NEGINFIF(EXP1, EXP2)
POSINFIF(EXP1, EXP2)
Comparison Functions
GREATEST(EXP1, EXP2)
LEAST(EXP1, EXP2)
Meta and UUID Functions
BASE64(EXP)
BASE64_ENCODE(EXP)
BASE64_DECODE(EXP)
META(EXP)
UUID()
Number Functions
ABS(EXP)
ACOS(EXP)
ASIN(EXP)
ATAN(EXP)
ATAN2(EXP1, EXP2)
CEIL(EXP)
COS(EXP)
DEGREES(EXP)
E(EXP)
EXP(EXP)
LN(EXP)
LOG(EXP)
FLOOR(EXP)
PI(EXP)
POWER(EXP1, EXP2)
RADIANS(EXP)
RANDOM([ EXP ])
ROUND(EXP [, DIGITS])
SIGN(EXP)
SIN(EXP)
SQRT(EXP)
TAN(EXP)
TRUNC(EXP [, DIGITS])
Type Checking Functions
ISARRAY(EXP)
ISATOM(EXP)
ISBOOLEAN(EXP)
ISNUMBER(EXP)
ISOBJECT(EXP1, EXP2)
ISSTRING(EXP)
TYPE(EXP)
Type Conversion
Functions
TOARRAY(EXP)
TOATOM(EXP)
TOBOOLEAN(EXP)
TONUMBER(EXP)
TOOBJECT(EXP)
TOSTRING(EXP)](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-60-2048.jpg)
![N1QL Operators
61
ARRAY_APPEND(EXP, VAL)
ARRAY_AVG(EXP)
ARRAY_CONCAT(EXP1, EXP2)
ARRAY_CONTAINS(EXP, VAL)
ARRAY_COUNT(EXP)
ARRAY_DISTINCT(EXP)
ARRAY_IFNULL(EXP)
ARRAY_LENGTH(EXP)
ARRAY_MAX(EXP)
ARRAY_MIN(EXP)
ARRAY_POSITION(EXP, VAL)
ARRAY_PREPEND(VAL, EXP)
ARRAY_PUT(EXP, VAL)
ARRAY_RANGE(START, END [,STEP])
ARRAY_REMOVE(EXP, VAL)
ARRAY_REPEAT(VAL, N)
ARRAY_REPLACE(EXP, VAL1, VAL2 [,N])
ARRAY_REVERSE(EXP)
ARRAY_SORT(EXP)
ARRAY_SUM(EXP)
Array Functions Date Functions
CLOCK_MILLIS()
CLOCK_STR ([FMT ])
DATE_ADD_MILLIS(EXP, N, PART)
DATE_ADD_STR(EXP, N,PART)
DATE_DIFF_MILLIS(EXP1, EXP2, PART)
DATE_DIFF_STR(EXP1, EXP2, PART)
DATE_PART_MILLIS(EXP, PART)
DATE_PART_STR(EXP, PART)
DATE_TRUNC_MILLIS(EXP, PART)
DATE_TRUNC_STR(EXP, PART)
MILLIS(EXP)
STR_TO_MILLIS(EXP)
MILLIS_TO_STR(EXP [, FMT ])
MILLIS_TO_UTC(EXP [, FMT ])
MILLIS_TO_ZONE_NAME(EX, TZ[,FMT])
NOW_MILLIS()
NOW_STR([ FMT ])
STR_TO_MILLIS(EXP)
MILLIS(EXP)
STR_TO_UTC(EXP)
STR_TO_ZONE_NAME(EXP, TZ_NAME)
Number Functions
CONTAINS(EXP, SUBSTRING)
INITCAP(EXP )
TITLE(EXP)
LENGTH(EXP)
LOWER(EXP)
LTRIM(EXP [,CHARACTERS ])
POSITION(EXP, SUBSTRING)
REPEAT(EXP, N)
REPLACE(EXP, SBSTR, REPL [, N ])
RTRIM(EXP, [,CHARACTERS ])
SPLIT(EXP [, SEP ])
SUBSTR(EXP, POS[, LEN ])
TRIM(EXP [, CHARACTERS ])
UPPER(EXP)
Pattern Matching Functions
REGEXP_CONTAINS(EXP, PATTERN)
REGEXP_LIKE(EXP, PATTERN)
REGEXP_POSITION(EXP, PATTERN)
REGEXP_REPLACE(EXP, PTRN, REPL [, N ])
JSON Functions
DECODE_JSON(EXP)
ENCODE_JSON(EXP)
ENCODED_SIZE(EXP)
POLY_LENGTH(EXP)](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-61-2048.jpg)
![62
Retrieving Documents
22
couchbase.n1qlQuery(
statement="
SELECT airport_id,
airport_name, airport_type,
municipality, geo, timezone,
airport_iata, airport_icao,
FROM `flight-data`
WHERE iso_country = $1
AND iso_region = $2
AND doc_type = $3
ORDER BY airport_name ASC
LIMIT 2
",
parameters=[
"US",
"US-ME",
"airport"
]
);
CF N1QL Query](https://image.slidesharecdn.com/couchbase-overview-161108182736/75/CFCamp-2016-Couchbase-Overview-62-2048.jpg)
More Related Content
Similar to CFCamp 2016 - Couchbase Overview
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
CFCamp 2016 - Couchbase Overview
- 1. Couchbase Overview Aaron Benton "Agood programmer is someone who looks both ways before crossing a one-way street." - Doug LinderCredit: Couchbase
- 4. 4 Agenda What is NoSQL? Whatis Couchbase? Couchbase Architecture CFCouchbase Queries Demo (if time permits)
- 5.
- 6. 6 What is NoSQL? Non-Relational ClusterFriendly Generally Open-Source 21st Century Schema-Less
- 7. 7 Scaling Scale Vertically (RDBMS) Addresources to a single node in a system Enhance the server (more CPU, more RAM, etc) High availability, difficult to implement Scale Horizontally (NoSQL) Add more nodes to a system More servers, distributing load High Availability, easy to implement
- 8.
- 9. Any type ofdata Flexible Application Managed Change is easy Known Models Fixed Fields Data Types Database Managed Change can be difficult 9 Schemas Relational Non-Relational
- 10. 10 Types of NoSQLDatabases Key-Value Redis Riak Memcached Document Couchbase CouchDB MongoDB Column-Family Cassandra Base BigTable Graph Neo4J Giraph OrientDB
- 11. 11 In Development… Objects areassembled as a whole: Cart Order Product Profile Saving these Objects requires: Deconstructing Multiple Rows Multiple Tables
- 12. 12 Impedance Mismatch "The object-relationalimpedance mismatch is a set of conceptual and technical difficulties that are often encountered when a relational database management system (RDBMS) is being used by a program written in an object-oriented programming language or style, particularly when objects or class definitions are mapped in a straightforward way to database tables or relational schema." - Wikipedia
- 13.
- 14. 14 Relational Models Database // order(structure / dictionary / object / map /etc) order['order_id'] = 3492843; order['order_date'] = "2016-07-14T18:27:22.586Z"; order['products'] = [{ 'product_id' = 78323, 'quantity' = 2, 'price' = 39.99, 'sub_total' = 79.98, }]; order['user_id'] = 123; order['billing_address_1'] = "1302 Pleasant Ridge Rd"; order['billing_address_2'] = ""; order['billing_city'] = "Greensboro"; order['billing_region_code'] = "NC"; order['billing_postal_code'] = "27409"; order['billing_country_code'] = "US"; order['shipping_address_1'] = "1302 Pleasant Ridge Rd"; order['shipping_address_2'] = ""; order['shipping_city'] = "Greensboro"; order['shipping_region_code'] = "NC"; order['shipping_postal_code'] = "27409"; order['shipping_country_code'] = "US"; order['card_number'] = "3337151609084503"; order['expiration_month'] = 11; order['expiration_year'] = 2019; Code
- 15. 15 Non-Relational Models // order(structure / dictionary / object / map /etc) order['order_id'] = 3492843; order['order_date'] = "2016-07-14T18:27:22.586Z"; order['products'] = [{ 'product_id' = 78323, 'quantity' = 2, 'price' = 39.99, 'sub_total' = 79.98, }]; order['user_id'] = 123; order['billing_address_1'] = "1302 Pleasant Ridge Rd"; order['billing_address_2'] = ""; order['billing_city'] = "Greensboro"; order['billing_region_code'] = "NC"; order['billing_postal_code'] = "27409"; order['billing_country_code'] = "US"; order['shipping_address_1'] = "1302 Pleasant Ridge Rd"; order['shipping_address_2'] = ""; order['shipping_city'] = "Greensboro"; order['shipping_region_code'] = "NC"; order['shipping_postal_code'] = "27409"; order['shipping_country_code'] = "US"; order['card_number'] = "3337151609084503"; order['expiration_month'] = 11; order['expiration_year'] = 2019; Database (JSON)Code { "order_id": 3492843, "order_date": "2016-07-14T18:27:22.586Z", "products": [{ 'product_id': 78323, 'quantity': 2, 'price': 39.99, 'sub_total': 79.98, }]; "user_id": 123, "billing_address_1": "1302 Pleasant Ridge Rd", "billing_address_2": "", "billing_city": "Greensboro", "billing_region_code": "NC", "billing_postal_code": "27409", "billing_country_code": "US", "shipping_address_1": "1302 Pleasant Ridge Rd", "shipping_address_2": "", "shipping_city": "Greensboro", "shipping_region_code": "NC", "shippping_postal_code": "27409", "shippping_country_code": "US", "card_number": "3337151609084503", "expiration_month": 11, "expiration_year": 2019 }
- 16. BASEACID 16 Transaction Processing Atomicity Consistency Isolation Durability Basically Available Soft-State EventualConsistency Relational Non-Relational
- 17.
- 18. 18 Only NoSQL? Is SQLgoing away? No NoSQL databases are just another tool
- 19.
- 20.
- 21. 21 High availability cache Key-value store Document database Embedded database Sync management Couchbase ServerCouchbase Lite Couchbase Sync Gateway Data management for a broad range of use cases
- 22. 22 Couchbase Tenants Flexible datamodel Consistent performance at scale High availability Easy, affordable scalability 24x365
- 23.
- 24.
- 25.
- 26. 26 Couchbase Server Node Single-nodetype means easier administration and scaling Single installation Two major components/processes: Data manager cluster manager Data manager: C/C++ Layer consolidation of caching and persistence Cluster manager: Erlang/OTP Administration UI’s Out-of-band for data requests
- 27. 27 Couchbase Read Operation APPLICATIONSERVER MANAGED CACHE DISK DISK QUEUE REPLICATION QUEUE DOC 1 GET DOC 1 DOC 1 Single-node type means easier administration and scaling Reads out of cache are extremely fast No other process/system to communicate with Data connection is a TCP-binary protocol DOC 1
- 28. 28 APPLICATION SERVER MANAGED CACHE DISK DISK QUEUE REPLICATION QUEUE CouchbaseWrite Operation DOC 1 DOC 1DOC 1 Single-node type means easier administration and scaling Writes are async by default Application gets acknowledgement when successfully in RAM and can trade- off waiting for replication or persistence per-write Replication to 1, 2 or 3 other nodes Replication is RAM-based so extremely fast Off-node replication is primary level of High Availability Disk written to as fast as possible – no waiting
- 29. 29 Couchbase Cache Ejection APPLICATIONSERVER MANAGED CACHE DISK DISK QUEUE REPLICATION QUEUE DOC 1 DOC 2DOC 3DOC 4DOC 5 DOC 1 DOC 2 DOC 3 DOC 4 DOC 5 Single-node type means easier administration and scaling Layer consolidation means read through and write through cache Couchbase automatically removes data that has already been persisted from RAM
- 30. 30 APPLICATION SERVER MANAGED CACHE DISK DISK QUEUE REPLICATION QUEUE DOC1 Couchbase Cache Miss DOC 2 DOC 3 DOC 4 DOC 5 DOC 2 DOC 3 DOC 4 DOC 5 GET DOC 1 DOC 1 DOC 1 Single-node type means easier administration and scaling Layer consolidation means 1 single interface for App to talk to and get its data back as fast as possible Separation of cache and disk allows for fastest access out of RAM while pulling data from disk in parallel
- 31.
- 32.
- 33. Architecture Couchbase Cluster: Nodeand SDK Interaction
- 34. 34 Auto sharding –Bucket and vBuckets vB Data buckets vB 1 ….. 1024 Virtual buckets A bucket is a logical, unique key space Multiple buckets can exist within a single cluster of nodes Each bucket has active and replica data sets (1, 2 or 3 extra copies) Each data set has 1024 Virtual Buckets (vBuckets) Each vBucket contains 1/1024th portion of the data set vBuckets do not have a fixed physical server location Mapping between the vBuckets and physical servers is called the cluster map Document IDs (keys) always get hashed to the same vbucket Couchbase SDK’s lookup the vbucket -> server mapping
- 35. 35 ACTIVE ACTIVE ACTIVE REPLICAREPLICA REPLICA Couchbase Server 1 Couchbase Server 2 Couchbase Server 3 Basic Operation SHARD 5 SHARD 2 SHARD 9 SHARD SHARD SHARD SHARD 4 SHARD 7 SHARD 8 SHARD SHARD SHARD SHARD 1 SHARD 3 SHARD 6 SHARD SHARD SHARD SHARD 4 SHARD 1 SHARD 8 SHARD SHARD SHARD SHARD 6 SHARD 3 SHARD 2 SHARD SHARD SHARD SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Application has single logical connection to cluster (client object) Data is automatically sharded resulting in even document data distribution across cluster Each vbucket replicated 1, 2 or 3 times (“peer-to- peer” replication) Docs are automatically hashed by the client to a shard Cluster map provides location of which server a shard is on Every read/write/update/delete goes to same node for a given key Strongly consistent data access (“read your own writes”) A single Couchbase node can achieve 100k’s ops/sec so no need to scale reads
- 36.
- 37.
- 38. 38 Cluster Map –2 nodes added
- 39. 39 Rebalance ACTIVE ACTIVE ACTIVE REPLICAREPLICA REPLICA Couchbase Server 1 Couchbase Server 2 Couchbase Server 3 ACTIVE ACTIVE REPLICA REPLICA Couchbase Server 4 Couchbase Server 5 SHARD 5 SHARD 2 SHARD SHARD SHARD 4 SHARD SHARD SHARD 1 SHARD 3 SHARD SHARD SHARD 4 SHARD 1 SHARD 8 SHARD SHARD SHARD SHARD 6 SHARD 3 SHARD 2 SHARD SHARD SHARD SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD SHARD 7 SHARD SHARD 6 SHARD SHARD 8 SHARD 9 SHARD READ/WRITE/UPDATE Application has single logical connection to cluster (client object) Multiple nodes added or removed at once One-click operation Incremental movement of active and replica vbuckets and data Client library updated via cluster map Fully online operation, no downtime or loss of performance
- 40. 40 Fail Over Node ACTIVEACTIVE ACTIVE REPLICA REPLICA REPLICA Couchbase Server 1 Couchbase Server 2 Couchbase Server 3 ACTIVE ACTIVE REPLICA REPLICA Couchbase Server 4 Couchbase Server 5 SHARD 5 SHARD 2 SHARD SHARD SHARD 4 SHARD SHARD SHARD 1 SHARD 3 SHARD SHARD SHARD 4 SHARD 1 SHARD 8 SHARD SHARD SHARDSHARD 6 SHARD 2 SHARD SHARD SHARD SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD SHARD 7 SHARD SHARD 6 SHARDSHARD 8 SHARD 9 SHARD SHARD 3 SHARD 1 SHARD 3 SHARD Application has single logical connection to cluster (client object) When node goes down, some requests will fail Failover is either automatic or manual Client library is automatically updated via cluster map Replicas not recreated to preserve stability Best practice to replace node and rebalance
- 41. 41 XDCR Cross Datacenter Replication Replicationto other clusters Bi-Directional Uni-directional Filtered Replication
- 42.
- 43. 43 MongoDB Replication: Master -Slave Primaries and Secondaries Not all writes are local Need for 3rd Party Cache No Mobile Solution Complex Topology Database + Collections + BSON
- 44. Cluster with 2replicas / backups Distributed Load 44 Couchbase vs MongoDB
- 45.
- 46.
- 47.
- 48. 48 Documents are integralto the SDKs. All SDK’s support JSON format In addition: Serialized objects, Unquoted Strings, Binary pass-through A Document contains: Couchbase SDK 22 Property Description ID The bucket-unique identifier Content The value that is stored Expiry An expiration time CAS Check-and-Set identifier
- 49. CF Couchbase 2.0Brings Java SDK (2.2.5) N1QL Support w/ GSI Indexes Replica Reads Document Locking Prepared Statements Design Document Management Expanded Config And More… 1.* 2.0 add insert incr decr counter delete remove newQuery n1qlQuery viewQuery set upsert setMulti upsertMulti setWithCAS replaceWithCAS Method Changes
- 50. 50 Other Couchbase SDKs OfficialSDKs Java .NET Node.js Python For each of these there is: Full Document support Interoperability Common yet idiomatic Programming Model Others: Erlang, Perl, TCL, Clojure, Scala Also fully REST accessible PHP C / C++ Go Ruby JDBC and ODBC
- 51. 51 Connecting to Couchbase 22 publicboolean function onApplicationStart(){ application['couchbase'] = new cfcouchbase.CouchbaseClient({ servers: "127.0.0.1", bucketName: "beer-sample" }); return true; } public boolean function onApplicationEnd(){ application.couchbase.shutdown( 10 ); return true; }
- 52. 52 Writing Documents 22 var document= { "id": "512_brewing_company", "name": "(512) Brewing Company", "city": "Austin", "state": "Texas", "code": "78745", "country": "United States", "phone": "512.707.2337", "website": "http://512brewing.com/", "type": "brewery", "updated": "2010-07-22 20:00:20", "description": "(512) Brewing Company is a microbrewery located in the heart of Austin that brews for the community using as many local, domestic and organic ingredients as possible.", "address": [ "407 Radam, F200" ], "geo": { "accuracy": "ROOFTOP", "lat": 30.2234, "lon": -97.7697 } }; application.couchbase.upsert id=document.id, value=document ); Define a Document Save the Document
- 53.
- 54. 54 Counter Operations 22 // incrementthe user counter next_id = couchbase.counter( id="user_counter", value=1, defaultValue=0 ); // write the user record couchbase.insert("user_" & next_id, { "user_id": next_id, "name": "John Smith", "email": "john.smith@email.com" }); Increment // decrement the tickets counter available_tix = couchbase.counter( id="tickets_counter", value=-1, defaultValue=1000 ); // if there are no more tickets // available redirect the user if (available_tix < 1) { cflocation( url="sold-out.cfm", addtoken=false ); } ... Decrement
- 55.
- 56. 56 Map / ReduceViews In Couchbase, Map-Reduce is specifically used to create Indexes Map functions are applied to JSON documents and their output or "emit" data is stored in an index
- 57. 57 Querying SELECT first_name, last_name,children FROM users looks like SQL…WHERE EVERY child IN children SATISFIES child.age > 10 END
- 58. 58 N1QL Examples INSERT INTOecommerce ( KEY, VALUE ) VALUES ("user_1021", { "user_id": 1021, "doc_type": "user", "first_name": "John", "last_name": "Smith", "email": "john.smith@mail.com" }) UPDATE ecommerce USE KEYS "user_1021" SET email = "jsmith@gmail.com", gender = "M", token = UUID() RETURNING token UPSERT INTO ecommerce ( KEY, VALUE ) VALUES ("user_1021", { "user_id": 1021, "doc_type": "user", "first_name": "John", "last_name": "Smith", "email": "john.smith@mail.com" }) DELETE FROM ecommerce USE KEYS "user_1021"
- 59. N1QL Operators 59 Type Support Arithmetic+ - * / % -val Collection ANY EVERY ARRAY FIRST EXISTS IN WITHIN Comparison = == != <> > >= < <= (NOT) BETWEEN (NOT) LIKE IS (NOT) NULL IS (NOT) MISSING IS (NOT) VALUED Conditional CASE expression WHEN value THEN expression Construction Array [ value, value, ... ] Object { key:value, key:value, ... } Logical AND OR NOT String ||
- 60. N1QL Operators 60 ARRAY_AGG(EXP) ARRAY_AGG(DISTINCT EXP) AVG(EXP) AVG(DISTINCT EXP) COUNT(*) COUNT(EXP) COUNT(DISTINCT EXP) MAX(EXP) MIN(EXP) SUM(EXP) SUM(DISTINCT EXP) Aggregate Functions Object Functions OBJECT_LENGTH(EXP) OBJECT_NAMES(EXP) OBJECT_PAIRS(EXP) OBJECT_VALUES(EXP) Conditionals - Unknowns IFMISSING(EXP1, EXP2, …) IFMISSINGORNULL(EXP1, EXP2, …) IFNULL(EXP1, EXP2, …) MISSINGIF(EXP1, EXP2) NULLIF(EXP1, EXP2) Conditionals - Numbers IFINF(EXP1, EXP2, …) IFNAN(EXP1, EXP2, …) IFNANORINF(EXP1, EXP2, …) NANIF(EXP1, EXP2) NEGINFIF(EXP1, EXP2) POSINFIF(EXP1, EXP2) Comparison Functions GREATEST(EXP1, EXP2) LEAST(EXP1, EXP2) Meta and UUID Functions BASE64(EXP) BASE64_ENCODE(EXP) BASE64_DECODE(EXP) META(EXP) UUID() Number Functions ABS(EXP) ACOS(EXP) ASIN(EXP) ATAN(EXP) ATAN2(EXP1, EXP2) CEIL(EXP) COS(EXP) DEGREES(EXP) E(EXP) EXP(EXP) LN(EXP) LOG(EXP) FLOOR(EXP) PI(EXP) POWER(EXP1, EXP2) RADIANS(EXP) RANDOM([ EXP ]) ROUND(EXP [, DIGITS]) SIGN(EXP) SIN(EXP) SQRT(EXP) TAN(EXP) TRUNC(EXP [, DIGITS]) Type Checking Functions ISARRAY(EXP) ISATOM(EXP) ISBOOLEAN(EXP) ISNUMBER(EXP) ISOBJECT(EXP1, EXP2) ISSTRING(EXP) TYPE(EXP) Type Conversion Functions TOARRAY(EXP) TOATOM(EXP) TOBOOLEAN(EXP) TONUMBER(EXP) TOOBJECT(EXP) TOSTRING(EXP)
- 61. N1QL Operators 61 ARRAY_APPEND(EXP,VAL) ARRAY_AVG(EXP) ARRAY_CONCAT(EXP1, EXP2) ARRAY_CONTAINS(EXP, VAL) ARRAY_COUNT(EXP) ARRAY_DISTINCT(EXP) ARRAY_IFNULL(EXP) ARRAY_LENGTH(EXP) ARRAY_MAX(EXP) ARRAY_MIN(EXP) ARRAY_POSITION(EXP, VAL) ARRAY_PREPEND(VAL, EXP) ARRAY_PUT(EXP, VAL) ARRAY_RANGE(START, END [,STEP]) ARRAY_REMOVE(EXP, VAL) ARRAY_REPEAT(VAL, N) ARRAY_REPLACE(EXP, VAL1, VAL2 [,N]) ARRAY_REVERSE(EXP) ARRAY_SORT(EXP) ARRAY_SUM(EXP) Array Functions Date Functions CLOCK_MILLIS() CLOCK_STR ([FMT ]) DATE_ADD_MILLIS(EXP, N, PART) DATE_ADD_STR(EXP, N,PART) DATE_DIFF_MILLIS(EXP1, EXP2, PART) DATE_DIFF_STR(EXP1, EXP2, PART) DATE_PART_MILLIS(EXP, PART) DATE_PART_STR(EXP, PART) DATE_TRUNC_MILLIS(EXP, PART) DATE_TRUNC_STR(EXP, PART) MILLIS(EXP) STR_TO_MILLIS(EXP) MILLIS_TO_STR(EXP [, FMT ]) MILLIS_TO_UTC(EXP [, FMT ]) MILLIS_TO_ZONE_NAME(EX, TZ[,FMT]) NOW_MILLIS() NOW_STR([ FMT ]) STR_TO_MILLIS(EXP) MILLIS(EXP) STR_TO_UTC(EXP) STR_TO_ZONE_NAME(EXP, TZ_NAME) Number Functions CONTAINS(EXP, SUBSTRING) INITCAP(EXP ) TITLE(EXP) LENGTH(EXP) LOWER(EXP) LTRIM(EXP [,CHARACTERS ]) POSITION(EXP, SUBSTRING) REPEAT(EXP, N) REPLACE(EXP, SBSTR, REPL [, N ]) RTRIM(EXP, [,CHARACTERS ]) SPLIT(EXP [, SEP ]) SUBSTR(EXP, POS[, LEN ]) TRIM(EXP [, CHARACTERS ]) UPPER(EXP) Pattern Matching Functions REGEXP_CONTAINS(EXP, PATTERN) REGEXP_LIKE(EXP, PATTERN) REGEXP_POSITION(EXP, PATTERN) REGEXP_REPLACE(EXP, PTRN, REPL [, N ]) JSON Functions DECODE_JSON(EXP) ENCODE_JSON(EXP) ENCODED_SIZE(EXP) POLY_LENGTH(EXP)
- 62. 62 Retrieving Documents 22 couchbase.n1qlQuery( statement=" SELECT airport_id, airport_name,airport_type, municipality, geo, timezone, airport_iata, airport_icao, FROM `flight-data` WHERE iso_country = $1 AND iso_region = $2 AND doc_type = $3 ORDER BY airport_name ASC LIMIT 2 ", parameters=[ "US", "US-ME", "airport" ] ); CF N1QL Query
- 63.
- 64.
- 65.
- 66.
- 67.
Editor's Notes
- #3 About Me Who we are What we do How we use Couchbase + Sync Gateway
- #21 Membase was developed by several leaders of the memcached project who founded a company, NorthScale, to develop a key-value storage engine CouchOne was founded by CouchDB creator Damien Katz and provided database solutions powered by the Apache CouchDB database project Membase and CouchOne joined forces in Feb. 2011 to create Couchbase, the first and only provider of a comprehensive, end-to-end family of NoSQL database projects
- #22 Couchbase is an open source NoSQL Data Management platform. It specializes in operational data management to run the full range of online scenarios – web, mobile, and Internet of Things It is open source, and available in community and enterprise editions. It is built upon an integrated object-managed cache, which is both memory centric and distributed. That’s the core. All three of these together pretty well cover the web, mobile, and IOT use cases Couchbase was designed for. KEY POINT: COUCHBASE PROVIDES A SET OF MULTI-PURPOSE, CORE CAPABILITIES THAT SUPPORT A BROAD RANGE OF APPLICATIONS AND USE CASES, ALL IN A SINGLE DATA MANAGEMENT PLATFORM. Couchbase provides a set of technology capabilities to support a broad range of applications and use cases: High Availability Cache: Couchbase provides an integrated managed object cache, so you can start out using Couchbase as a high availability cache on top of your existing relational database. For example, you can use Couchbase as a session store in front of your relational database, if your relational DB is struggling to keep up with the load required for online interactive applications. Key-Value Store: Many customers start with Couchbase as a cache and then broaden their usage to other capabilities, like using Couchbase as a Key-Value Store for things like Profile Management. Document Database: From there, you can grow into using Couchbase as a Document Database, where you can do more with capabilities like indexing and Cross Data Center Replication. Embedded Database: Couchbase also provides an embedded database called Couchbase Lite. It’s a purpose-built database for the device, so you can build applications that are always available and always work, whether offline or online. Sync Management: Finally, as part of our solution for mobile applications, we provide Couchbase Sync Gateway, which automatically synchronizes data on the device with Couchbase Server in the cloud so your developer doesn’t have to write code to manage the complex sync process. Starting with cache and then expanding to other capabilities is often a good way to learn the technology and get comfortable with Couchbase for a wider set of use cases. Couchbase Open Source Projects (Apache 2.0 Public License) Community Editions Enterprise Editions NoSQL Database (Couchbase Server) Document Database + Key / Value Store + Distributed Cache Cross Data Center Replication Mobile Database (Couchbase Lite) iOS, Android, Java, and .NET Mobile Synchronization (Couchbase Sync Gateway)
- #23 Couchbase Server is a NoSQL document database for interactive web applications. It has a flexible data model, is easily scalable, provides consistent high performance and is “always-on,” meaning it can serve application data 24 hours, 7 days a week. KEY POINTS: COUCHBASE DELIVERS ALL THE CAPABILITIES NEEDED TO MEET TODAY’S REQUIREMENTS FOR PERFORMANCE, SCALABILITY, AVAILABILITY, AND DATA MODEL FLEXIBILITY. THESE TRANSLATE INTO MAJOR BENEFITS FOR YOUR BUSINESS. Couchbase was purpose-built to solve today’s requirements for enterprise-class, mission-critical, web and mobile applications. Specifically, Couchbase delivers the following capabilities: Fast performance at scale -- submillisecond latency to enable highly responsive applications, for millions or even hundreds of millions of users. Easy, affordable scalability – Couchbase is a distributed database that scales out on commodity hardware with push button simplicity. We make it very easy to add or remove capacity on demand with no system downtime. On premises, in the cloud, wherever you want. High availability – Couchbase automatically replicates your data across your servers, clusters, and data centers, so it’s always available, 24x7. And Couchbase doesn’t require any downtime to maintain. Flexible data model – Couchbase gives you complete flexibility to handle any kind of data, and to change your data model on the fly to accommodate new data attributes or new data types. It’s the kind of flexibility that developers love, because it gets rid of the rigid schemas that slow them down. So developers can build applications faster and easier. All this adds up to powerful benefits for your enterprise: Faster development & time to market Better business agility Improved customer experience Increased loyalty and revenue Lower IT costs and increased efficiency
- #24 Couchbase Server is a NoSQL document database for interactive web applications. It has a flexible data model, is easily scalable, provides consistent high performance and is “always-on,” meaning it can serve application data 24 hours, 7 days a week. KEY POINTS: COUCHBASE DELIVERS ALL THE CAPABILITIES NEEDED TO MEET TODAY’S REQUIREMENTS FOR PERFORMANCE, SCALABILITY, AVAILABILITY, AND DATA MODEL FLEXIBILITY. THESE TRANSLATE INTO MAJOR BENEFITS FOR YOUR BUSINESS. Couchbase was purpose-built to solve today’s requirements for enterprise-class, mission-critical, web and mobile applications. Specifically, Couchbase delivers the following capabilities: Fast performance at scale -- submillisecond latency to enable highly responsive applications, for millions or even hundreds of millions of users. Easy, affordable scalability – Couchbase is a distributed database that scales out on commodity hardware with push button simplicity. We make it very easy to add or remove capacity on demand with no system downtime. On premises, in the cloud, wherever you want. High availability – Couchbase automatically replicates your data across your servers, clusters, and data centers, so it’s always available, 24x7. And Couchbase doesn’t require any downtime to maintain. Flexible data model – Couchbase gives you complete flexibility to handle any kind of data, and to change your data model on the fly to accommodate new data attributes or new data types. It’s the kind of flexibility that developers love, because it gets rid of the rigid schemas that slow them down. So developers can build applications faster and easier. All this adds up to powerful benefits for your enterprise: Faster development & time to market Better business agility Improved customer experience Increased loyalty and revenue Lower IT costs and increased efficiency
- #27 Each Couchbase node is exactly the same. All nodes are broken down into two components: A data manager (on the left) and a cluster manager (on the right). It’s important to realize that these are separate processes within the system specifically designed so that a node can continue serving its data even in the face of cluster problems like network disruption. The data manager is written in C and C++ and is responsible both for the object caching layer, persistence layer and querying engine. It is based off of memcached and so provides a number of benefits; -The very low lock contention of memcached allows for extremely high throughput and low latencies both to a small set of documents (or just one) as well as across millions of documents -Being compatible with the memcached protocol means we are not only a drop-in replacement, but inherit support for automatic item expiration (TTL), atomic incrementer. -We’ve increased the maximum object size to 20mb, but still recommend keeping them much smaller -Support for both binary objects as well as natively supporting JSON documents -All of the metadata for the documents and their keys is kept in RAM at all times. While this does add a bit of overhead per item, it also allows for extremely fast “miss” speeds which are critical to the operation of some applications….we don’t have to scan a disk to know when we don’t have some data. The cluster manager is based on Erlang/OTP which was developed by Ericsson to deal with managing hundreds or even thousands of distributed telco switches. This component is responsible for configuration, administration, process monitoring, statistics gathering and the UI and REST interface. Note that there is no data manipulation done through this interface.
- #30 Now, as you fill up memory (click), some data that has already been written to disk will be ejected from RAM to make room for new data. (click) Couchbase supports holding much more data than you have RAM available. It’s important to size the RAM capacity appropriately for your working set: the portion of data your application is working with at any given point in time and needs very low latency, high throughput access to. In some applications this is the entire data set, in others it is much smaller. As RAM fills up, we use a “not recently used” algorithm to determine the best data to be ejected from cache.
- #31 Should a read now come in for one of those documents that has been ejected (click), it is copied back from disk into RAM and sent back to the application. The document then remains in RAM as long as there is space and it is being accessed.
- #32 1 Dimensional Deployment
- #37 The application makes a call for a key called NYC MQ1 We run the key through the crc 32 function and the result of that hash function is that it points to vbucket3 Which in turn points to couchbase server number 1
- #38 We now run a different key through through the has and we now come up with differnet vbucket, vbucket 4 and that points to server 3
- #39 We now run a different key through through the has and we now come up with differnet vbucket, vbucket 4 and that points to server 3
- #44 http://blog.couchbase.com/2016/february/moving-from-mongodb-to-couchbase-server http://blog.couchbase.com/2015/november/server-utilization-couchbase-vs-mongodb https://mhurtadoblog.wordpress.com/2015/10/21/setting-up-a-production-ready-cluster-with-mongodb-and-couchbase/ http://www.couchbase.com/have-you-outgrown-mongodb
- #53 Automatic object serialization
- #57 Views are part of the Data service