This document introduces Couchbase 4.5 and Couchbase Mobile 1.2 and discusses several use cases for using Couchbase as a NoSQL database solution. It summarizes five common use cases: 1) high-availability caching to speed up database operations, 2) using Couchbase as a session store, 3) creating a globally distributed user profile store, 4) aggregating data from various sources, and 5) storing and accessing content and metadata.
3. Application objects
Popular search query results
Session information
Heavily accessed web landing
pages
High-Availability Caching
Speed up RDBMS
Consistently low response times
for document / key lookups
High-availability 24x7x365
Replacement for entire caching tier
Data cached in Couchbase? Application characteristic
Use Case 1
http://www.Look.PopularSearchWuerycom
Look Something Search
WEB % of clicks % of clicks
something 56.3 28
DoSomething.com 13.4 25.08
SomethingFishy.org 9.8 14.68
Popular
5. Session Store
Extremely fast access to session data
using unique session ID
Easy scalability to handle fast growing
number of users and user-generated
data
Always-on functionality for global
user base
Application characteristic
Use Case 2
Session values or Cookies
(stored as key-value pairs)
Examples include: items in a shopping
cart, flights selected, search results,
etc.
Data stored in Couchbase?
7. http://www.ProfileStore.com
e enim nec felis rhoncus, ac volutpat magna blandit. Nunc facilisis turpis eget dolor mollis, id tincidunt dui mattis. Nunc sodales elementum turpis, vel interdum ante congue
quis. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.Aliquam erat volutpat. Nullam suscipit diam nec tortor pharetra, vitae
adipiscing dolor pretium. Integer ac porta tortor. Vestibulum imperdiet quam laoreet nisl scelerisque, a tempus tortor tincidunt. Mauris suscipit dui ac urna dignissim, vitae
aliquet velit convallis. Phasellus lobortis felis eu magna vulputate dapibus. Ut ornareut quam a vulputat
ullam et dui odio. Nulla pharetra, velit ac convallis semper, dolor turpis porta nunc, in egestas mauris leo a nisi. Pellentesque fringilla sagittis magna vitae imperdiet. Mauris ac
leo ut tellus aliquet interdum. Interdum et malesuada fames ac ante ipsum primis in faucibus. Nunc cursus odio sit amet elit mollis, et sollicitudin lacus accumsan.Nulla facilisi.
Fusce et vehicula sem. Curabitur interdum vestibulum nulla id accumsan.Integer ut tortor in ligula semper vehicula. Vestibulum ut nibh ultrices, venenatis metus at, adipiscing
ipsum. Donec quis consequat lectus.
Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Donec a diam tempus, aliquet ipsum eu, vestibulum sapien. Donec eleifend lectus sit
amet luctus facilisis.Morbi porttitor, orci sit amet placerat tempus, nisi justo dictum augue, ac dignissim elit enim eget dolor. Praesent pulvinar ipsum arcu,eu posuere eros
luctus nec. Vestibulum odio eros, ultrices non metus sit amet, tristique malesuada augue. Pellentesque lacinia dolor nec diam eleifend mollis. Vestibulum sit amet ultrices diam.
Aliquam lacinia accumsan eros id hendrerit. Cras placeratlaoreet urna scelerisque rutrum. Duis ornare mi ac augue varius, sit amet accumsan leo lacinia. Vivamus nec egestas
neque. Quisque interdum enim molestie urn.
turpis eget dolor mollis, id tincidunt dui mattis. Nunc
sodales elementum turpis, vel interdum ante congue quis.
Pellentesque habitant morbi tristique senectus et netus et
malesuada
Welcome back Laura!
You have 3 items in your shopping cart
waiting for you.
LOGIN
ID:
PASS:
Globally Distributed User Profile Store
Extremely fast access to individual profiles
Always online system as multiple
applications access user profiles
Flexibility to add and update user
attributes
Easy scalability to handle fast growing
number of users
User profile with unique ID
User setting / preferences
User’s network
User application state
Data stored in Couchbase? Application characteristic
Use Case 3
Laura930
********
8. Data Aggregation
Flexibility to store any kind of content
Flexibility to handle schema changes
Full-text Search across data set
High speed data ingestion
Scales horizontally as more content gets
added to the system
Social media feeds: Twitter, Facebook,
LinkedIn
Blogs, news, press articles
Data service feeds: Hoovers, Reuters
Data form other systems
Data stored in Couchbase? Application characteristic
Use Case 4
in
F
t
NEWS
Blog
9. Use Case 5
Content and Metadata
Nature, Field, Summer, Farm, Sky, Environment, Landscaped, Gr
ass, Green,Blue, Oilseed,
Rape, Agriculture, Scenics, Land, Spring, Non-Urban
Scene,Environmental, Conservation, Sun, Meadow, Horizon,
Season, Cloud, Landscapes, Travel Locations, Pasture, Cultivated
Land, Stratoshpere, cloudy day, Oliseed Rape, Rural
Scene, Vibrant Color, No People, Beauty In Nature,Gold, Color
Image, Beauty, Idyllic, Multicolored, Yellow, Colors, Cloudscape,
Outdoors, Plant, Sunlight, Horizon Over Land
Content and metadata store
10. Content and Metadata Store
Flexibility to store any kind of content
Fast access to content metadata (most
accessed objects) and content
Full-text Search across data set
Scales horizontally as more content gets
added to the system
Content metadata
Content: Articles, text
Landing pages for website
Digital content: eBooks, magazine,
research material
Data stored in NoSQL? Application characteristic
Use Case 5
http://www.LandingPage.com
ebook
Mag
11. Macro Trends Driving NoSQL Technology
NoSQL
+ +
More Data More Users Interactive Apps
12. Why The Digital Economy Needed A New Database Solution
Question: What are the biggest problems with Relational Database that are driving adoption of
NoSQL?
LACK OF
FLEXIBILITY/ HAS
RIGID SCHEMAS
INABILITY
TO SCALE OUT
PERFORMANCE
CHALLENGES
49%
69%
50%
47%
44%
COST
ALL OF THE ABOVE
35%
29%
16%
12%
13. Agile Development
Hotel Descriptions
Reviews
User Profiles
Reviews points
to users
Hotels points
to reviews
{
“ID”: 1,
“NAME”: “Fairmont
San Francisco”,
…}
{
“REVIEW_ID”: 1,
“REVIEW”: “Loved
Hotel…”,
…}
{
“REVIEW_ID”: 2,
“REVIEW”: “Nice,
but …”,
…}
{
“USER_ID”: 1,
“DISPLAY”: “Ted’s
Trip…”,
…}
{
“USER_ID”: 2,
“DISPLAY”:
“WhatWhat …”,
…}
Must support flexible schemas to make development agile
14. Must Dynamically Scale Apps to Support Millions of Users
Scalability
RDBMS Scales Up
Get a bigger, more complex server
Users
Application Scales Out
Just add more commodity web servers
Users
System Cost
Application Performance
System Cost
Application Performance
Won’t
scale
beyond
this point
16. Apps Must Now Stay Online 24 x 365
Availability
PERFORMANCE
JSON
JSON
JSON
JSONJSON
24/7
http://www.mypage.com
turpis eget dolor mollis, id tincidunt dui mattis.
Nunc sodales elementum turpis, vel interdum
ante congue quis. Pellentesque habitant morbi
tristique senectus et netus et malesuada
Well, this is embarrassing.
We are having some difficulties and we apologies for the inconvenience.
18. NoSQL Considerations
Accessing data
– No standards exist yet
– Typically via SDKs or over HTTP
– Check if the programing language of your choice
is supported.
App Server
App Server
App Server
Consistency
– Consistent only at the document level
– Most documents stores currently don’t support multi-
document transactions
– Analyze your application needs
Availability
– Each node stores active and replica data
(Couchbase)
– Each node is either a master or slave (MongoDB)
19. NoSQL considerations
Operations
– Monitoring the system
– Backup and restore the system
– Upgrades and maintenance
– Support
App Server
App Server
Client
Ease of Scaling
– Ease of adding and reducing capacity
– Single node type
– App availability on topology changes
Indexing and Querying
– Secondary indexes
– Aggregates Grouping
– Basic querying / Ad hoc querying
20. 3rd party or user defined structure (Twitter feeds)
Support for unlimited data growth (Viral apps)
Data with non-homogenous structure
Need to quickly and often change data structure
Variable length documents
Sparse data records
Hierarchical data
Where is NoSQL a good fit?
KEY POINT: THE TREND TO NOSQL WILL ACCELERATE AS APPLICATIONS ARE GOING TO NEED TO SUPPORT AN EXPLODING NUMBER OF USERS, NUMBER OF DEVICES AND AMOUNT OF TRAFFIC
Relational databases were designed to support applications with perhaps hundreds or thousands of users – not hundreds of millions, or even a billion in some cases – and a limited, structured set of data – not terabytes of semi-structured data
Relational databases have been around for a long time – 30+ years, and they do some things really well.
But they were designed at a very different time, for a different set of requirements – and they struggle to meet the new requirements being driven by the web, mobile, and big data trends:
Can’t deliver highly responsive experience that consumers demand online
Can’t affordably and easily scale to support millions of user. Relational scales up rather than out – which means you need to add expensive hardware to scale up, rather than scale out on reasonably sized, standard hardware.
Can’t manage massive volumes of data at high speed
Can’t handle lots of different data types, including unstructured and semi-structured data
Relationnal :
- Hundreds or thousands of inter-related tables
Handles structured data well, unstructured data poorly
Rigid schema requires migrations that can take weeks, months
Impedance mismatch with developers
NoSQL:
Aggregates & denormalizes data into single document
Handles structured & unstructured data equally well
Inferred schema requires no migration
JSON rapidly being adopted
Relationnal:
Centralized, scale up architecture with big, expensive servers
Manual sharding at app level struggles to support “web scale”
High software costs & TCO
NoSQL
Distributed, scale-out architecture with cluster of low-cost, commodity servers
Auto-sharding at database level to support Big Data, Big Users
Open source & lower TCO
Relational:
Architecture based on “speed of disk”
Requires joins across hundreds or thousands of tables
High throughput requires very expensive hardware
NoSQL (Couchbase)
Architecture based on “speed to memory”
Faster access to aggregated, de-normalized objects
High throughput at low costs with cluster of commodity servers
Relational:
Relational systems use clustering as an afterthought
Must take database down for “maintenance windows”
Struggle to support XDCR replication across many DCs
NoSQL:
Clustered systems with intra-cluster replication for availability
Designed for online software upgrades & maintenance
Native master-master XDCR for higher availability
KEY POINTS – WITH THE TOOLS AND FUNCTIONALITY COUCHBASE BRING TO BEAR FOR THE PROSPECT, IT IS IS EASY TO DEVELOP APPLICATIONS BE THEY WEB, MOBILE OR IOT LIKE NEVER BEFORE, BUT STILL PERFORM VERY WELL AND SCALE OUT WITH THE EASE COUCHBASE IS KNOWN FOR. THIS IS WHAT MAKES COUCHBASE SPECIAL.
- Really try and drive home the Develop with Agility line and try and define what it means to the customer’s needs.
- Operate at any scale is key to be able to explain that we can grow and change with the prospects needs, but still remain easy to to scale, always on and high performance.
Later slides all drive home the points on this slide, so do not spend more than two minutes giving a high overview oft this and what makes Couchbase special.
KEY POINT: COUCHBASE HAS YOU COVERED FOR YOUR GENERAL PURPOSE DB NEEDS. FROM CACHING TO KV STORE, TO JSON DOCUMENT STORE, TO MOBILE APPS. NO OTHER NOSQL DB VENDOR HAS THIS BREADTH AND DEPTH OF TECHNOLOGY
The purpose of this slide is to discuss the high level concepts of Couchbase, and if the SE wants to discuss what parts of Couchbase make up each concept. It is not to go over specific technologies like N1QL, ODBC, etc
Key Point: Major enterprises across numerous industries are adopting couchbase NoSQL technology to support their data management needs. NoSQL is not just for large Internet companies – Its for everyone now.
So who are some of the companies that are actually adopting Couchbase?
As this slide shows, major enterprises across many different industries are adopting Couchbase NoSQL solution.
You see many innovative Internet companies here – like eBay, LinkedIn, Orbitz, and PayPal.
They were the early adopters of NoSQL. But it’s clear that NoSQL is now being adopted by a broad range of companies in many other industries:
Consumer electronics and technology companies like Apple and Cisco
Retail companies like Walmart and Tesco
Financial Services companies like VISA and Wells Fargo
Telcos like Verizon, AT&T and Vodafone
What’s also interesting is that we’re seeing the use of NoSQL expand inside many of these companies. Orbitz, the online travel company, is a great example – they started using Couchbase to store their hotel rate data, and now they use Couchbase in many other ways.
So the big takeaway is that an increasing number of companies across industries are seeing the value of NoSQL for a growing number of use cases.
KEY POINT: YOU HAVE THE OPTION TO REPRESENT DATA QUITE DIFFERENTLY USING JSON AS OPPOSED TO A RELATIONAL DATABASE.
- Where in relational databases you might have to have multiple tables to best represent your data, in JSON you can model your data like an object might already be in your programming language of choice. No ORM (Object Relational Model) needed.
You can do relationships in Couchbase, but they are different than in a relational database and outside of the scope of an intro call normally.
Make sure to stress that normalization is still something that can be done in Couchbase where it makes sense for the application, but this diagram is something that helps people coming from relational understand what is possible for JSON.
KEY POINT: Applications communicate directly to the services they need to fulfil the aplication request and the application does not need to be topology aware as the SDK has that already.
Single node type, services defined dynamically
One node acts the same as 100, just the services are spread out in the cluster
Query service accesses Index and Data to formulate response
All query and document access is topology aware and dynamically scalable
Develop with one node, deploy against multiple production nodes
The Couchbase SDK handles knowing about where in the cluster it needs to go to satisfy whatever the application is requesting, be I CRUD or cluster management.
KEY POINT: N1QL is a marriage of the strengths of the JSON flexible schema with the power and familiarity of SQL.
KEY POINT: N1QL FULLY SUPPORTS JOIN ACROSS DOCUMENTS IN A BUCKET, but it has required criteria to work corrextly
JOINs only work on IDs of one object against a value that is that ID in the JSON.
KEY POINT:
KEY POINT:
KEY POINT: GROUND THE USER IN HOW OBJECTs RELATE TO BUCKETS AND THOSE ARE SPREAD ACROSS THE CLUSTER; AS WELL AS HOW THINGS IN THE CLUSTER ARE STACKED PHYSICALLY.
Talk to the audience about how documents move in the application at a high level and the relation between data buckets and how they are spread evenly across the cluster.
Remember that vBuckets are not in this diagram, but that is on purpose. That comes later and might confuse people at this point. Going over this slide now sets you up for the vBucket discussion later in the presentation.
Convey what a bucket is. That it is a logical key space, with its own set of server resources, queues, etc.
Make sure to stress that one can have multiple buckets, but to not just create them like you would tables or schemas in a relational database
An example of when to split data into different buckets is using Views across different object types. For example, if you have JSON documents and base64 encoded XML documents in the same bucket, a view or index will have to interact with that object even though it will never need it. So it would be better to put the XML into another bucket so the views and indexes are only looking at the JSON data they actually will be indexing.
KEY POINT: EACH COUCHBASE NODE HAS THE SAME SOFTWARE AND THESE ARE THE COMPONENTS. WHETHER THEY TURN THEM ALL ON DEPENDS ON THEIR FUNCTIONAL AND SCALLING NEEDS.
Data Node is the heart of the database. Everything interacts with this service to read and write data. It includes a high performance managed cache
Index Nodes receive a DCP stream of all changes from the data nodes, but for the N1QL query execution flow are only accessed by the Query Service.
Query Service – Holds no data, but accesses indexes on the Indexing Services as well as Views and Data on the Data Service. It exposes a REST API to interface with it and then communicates with the other service as needed to
Cluster Manager – Contains the WebUI, REST API and Performs the background tasks of the cluster such as cluster orchestration.
Add lots of notes
Add lots of notes
Add lots of notes
Add lots of notes
Add lots of notes
Add lots of notes
Key Points: MDS provides the ability for cluster administrator to separate the various workloads and tailor the cluster architecture for the application’s scalability and performance needs. This while providing the functionality needed by by the application.
Pictured is a possible architecture where all the cluster has been split with the Query and Index Service on different nodes of the cluster.
This allows you to not only isolate the workload, but you could have the nodes with the other services with different server configs.
For example, the query service needs CPU and RAM and has no storage, but the Index service needs Ram and fast disks. So you could configure these nodes different from the Data Service nodes to tailor performance even further.
This is what allows Couchbase to scale with querying.
The indexing and query services are eventually consistent by default, but in code can be made to fully
Key Points: MDS provides the ability for cluster administrator to separate the various workloads and tailor the cluster architecture for the application’s scalability and performance needs. This while providing the functionality needed by by the application.
Pictured is a possible architecture where all the cluster has been split with the Query and Index Service on different nodes of the cluster.
This allows you to not only isolate the workload, but you could have the nodes with the other services with different server configs.
For example, the query service needs CPU and RAM and has no storage, but the Index service needs Ram and fast disks. So you could configure these nodes different from the Data Service nodes to tailor performance even further.
This is what allows Couchbase to scale with querying.
The indexing and query services are eventually consistent by default, but in code can be made to fully
Key Points: MDS provides the ability for cluster administrator to separate the various workloads and tailor the cluster architecture for the application’s scalability and performance needs. This while providing the functionality needed by by the application.
Pictured is a possible architecture where all the cluster has been split with the Query and Index Service on different nodes of the cluster.
This allows you to not only isolate the workload, but you could have the nodes with the other services with different server configs.
For example, the query service needs CPU and RAM and has no storage, but the Index service needs Ram and fast disks. So you could configure these nodes different from the Data Service nodes to tailor performance even further.
This is what allows Couchbase to scale with querying.
The indexing and query services are eventually consistent by default, but in code can be made to fully
KEY POINT: COUCHBASE ENABLES YOU TO EASILY MEET YOUR HA AND DR REQUIREMENTS WITH MEMORY TO MEMORY REPLICATION BETWEEN TO SEPARATE CLUSTERS.
Cross–Data Center Replication to provide an easy yet powerful way to replicate data from one cluster to another for increased high availability, disaster recovery, and geographic load balancing.
In Couchbase Server 4.0, we’ve added filtering capabilities to Cross–Data Center Replication (XDCR) in order to significantly reduce the amount of data replicated across data centers.
- XDCR Filtering achieves this reduction by replicating only data relevant to the destination.
- Couchbase customers no longer need to create many different buckets just to segment data for Cross–Data Center Replication.
KEY POINT: COUCHBASE ENABLES YOU TO EASILY MEET YOUR HA AND DR REQUIREMENTS WITH MEMORY TO MEMORY REPLICATION BETWEEN TO SEPARATE CLUSTERS.
Cross–Data Center Replication to provide an easy yet powerful way to replicate data from one cluster to another for increased high availability, disaster recovery, and geographic load balancing.
In Couchbase Server 4.0, we’ve added filtering capabilities to Cross–Data Center Replication (XDCR) in order to significantly reduce the amount of data replicated across data centers.
- XDCR Filtering achieves this reduction by replicating only data relevant to the destination.
- Couchbase customers no longer need to create many different buckets just to segment data for Cross–Data Center Replication.
KEY POINT: THERE ARE THREE CORE COMPONENTS TO THE COUCHBASE MOBILE SOLUTION: 1) COUCHBASE LITE, A LIGHTWEIGHT MOBILE DATABASE THAT RUNS ON THE DEVICE, 2) SYNC GATEWAY, WHICH SECURELY SYNCS DATA BETWEEN YOUR PRIVATE CLOUD AND YOUR PUBLIC CLOUD, AND 3) COUCHBASE SERVER AS THE PRIMARY DATA STORE FOR YOUR APPLICATION
Couchbase Lite is a NoSQL mobile database, which means it runs in process with your application, and it has a very small footprint.
There are four major elements of Couchbase Lite:
1. It is a document oriented database
2. It provides a Map Reduce query engine
3. It provides a suite of event notifications
4. It provides sync (multi master replication)
Sync Gateway handles the boundary between your private data center and your public cloud. When I say private data center, that can still be in a public cloud like EC2 or Azure where you’ve got a number of VMs running. It handles the same cross cutting concerns from your application that you would have in any web-based application: things like Authentication, Authorization, and unique to Sync Gateway: it provides a method of data orchestration. It also only syncs the information that users cares about, and we need a way to programmatically define what that user cares about. And that’s another thing that Sync Gateway does for you.
Couchbase Server serves as the primary data store for your application data.