Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
THE COUCHBASE &
CONFLUENT BRIDGE
FROM RELATIONAL TO NOSQL
July 14, 2020
Jeff Morris, VP Product & Solutions Marketing
BIG DATA
ARCHITECTURE
EVOLUTION1
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 3
Big Data
Architecture
Evolution
Big Data version 1 focused on
• ingest
• archive
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 4
Big Data
Architecture
Evolution
Big Data version 2 focused on
• collate
• analyze
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 5
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 6
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix
• ... engage
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 7
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix
• ... engage
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 8
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 9
Data
Processing
Platform
• Critical connectivity
• Analysis
• Streaming
HDFS
DBMS
Mobile
Other
Platforms
Data
Processing
Platform
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 10
Stream
Data
Platform
• Critical connectivity
• Analysis
• Streaming
• Distributed
• High throughput
• Wider engagement
HDFS
DBMS
Mobile
Other
Platforms
Stream
Data
Platform
KAFKA AS A
STREAMING BUS
2
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 12
Couchbase
& Kafka
• Source
• Sink
• Custom Filter
• Apache Kafka
• Confluent Platform
Kafka
?
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 13
How Organizations Handle Data Flow: A Giant Mess
Data
Warehouse
Hadoop
NoSQL
Oracle
SFDC
Logging
Bloomberg
… any sink/source
… and more
OLTP
ActiveMQ
App App
Caches
OLTP OLTPAppAppApp
Web Custom Apps Microservices Monitoring Analytics
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 14
Kafka Connect
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 15
The Couchbase Kafka Connect Plug-In
Reads Data
Connect Protocol
stream from
Couchbase and
publishes to
Kafka topic
Consumes Kafka topic
and writes records to
Couchbase
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 16
Apache Kafka™: A Distributed Streaming Platform
Apache Kafka
Data
Warehouse
HadoopOracle
SFDC
Twitter
Bloomberg
… any sink/source … any sink/source
… and more
Web Custom Apps Microservices Monitoring Analytics
REAL WORLD
USE CASE
3
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 18
Applications Across Industries
Healthcare & Pharma
Patient Monitoring, Pharma
Substance control, Patient
Relapse, Lab Results Alerts
Banking & Capital Markets
Fraud Detection, Trade Data Capture,
Customer 360
Retail
Inventory Management, Product
Catalog, A/B Testing, Proactive
Alerts
Telecommunications
Personalized Ads, Customer 360,
Network Integrity
Automotive
Connected Car, Manufacturing
Data Processing
Travel & Leisure
Visitor Segmentation,
Fraud Detection
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 19
Couchbase Data Sharing Use Cases
Catalog and
Inventory Management
Catalogs
• Deliver relevant product
content and a real-time
view of inventory
• Scale to millions of
products and requests
for the latest information
• Serves highly engaged
online audiences
Profile and
Session Management
Personalization
• Create custom experiences in
real time based on aggregate
data from multiple
• Aggregate customer data,
recommendations, user
profiles, session, history data
Customer 360
Single View
• Deliver a consistent, single
view of your data with one
platform
• Improve customer experience
and operational visibility
Digital
Transformations
Offload
• Create transformative digital
experiences by offloading
mainframe, RDBMS systems
• Reduce costs and improve
productivity and agility
Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 20
Confluent
Platform
Enterprise
Streaming based
on Apache
Kafka™
Database
Changes
Log
Events
loT Data
Web
Events
…
CRM
Data
Warehouse
Database
Hadoop
Data
Integration
…
Monitoring
Analytics
Custom
Apps
Transformations
Real-time
Applications
…
Apache Open
Source
Confluent Open
Source
Confluent
Enterprise
Confluent Platform
Apache Kafka™
Data Compatibility
Monitoring & Administration
Operations
Clients Connectors
Complete Open Trusted Enterprise Grade
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 21
Bank Reduces OpEx by $25M/year via Mainframe Offload
Date Amount
1/27/2017 $4.56
1/22/2017 $32.14
Transaction Data
Vendor Description
Starbucks Coffee
Walmart Blu-Ray
Transaction Description
Schema
Website
Microservices
Match data and
description
Client profiles
Lookup client
profiles
Mainframe MIPS = $$
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 22
Ingest, Process, Load, and Serve Data at a Global Scale
Couchbase
…
Couchbase
…
Kafka cluster
Applications
Other data
stores
Kafka cluster
Kafka Streams
(Data Enrichment and Transformation)
Kafka Connect
(Connectors to Extract and Load data)
Confluent
Replicator
Confluent
Replicator
Custom
Replication
Custom
Replication
Raw event
data
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 23
Couchbase’s NoEQUAL Architectural Differentiation
Developer agility & versatility
• Multi-model: Key Value & JSON documents
• Multi-mode: Memory-first, ACID, operational & analytic workloads
• Programmable: schema flexibility + SQL in N1QL + stack-based SDKs
Performance at any scale
• No hassle scale out – shared-nothing, asynchronous, elastic architecture
• Built-in replication (XDCR)
• Always-on, globally distributed, edge-to-cloud
Easy to manage
• Workload isolation with multi-dimensional scaling
• Automatic cluster rebalancing
• Location and deployment agnostic
• Kubernetes & microservices-friendly
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 24
Sample QA SetupSample Dev Setup
Elastic Scaling Architecture
Sample Production DeploymentNODE 1
Query
Global Index
Data
Analytics
Full Text
Cluster Manager
NODE 2
Eventing
NODE 1 NODE 13
Cluster Manager
Data Full Text AnalyticsGlobal Index Query Eventing
NODE 1
Global IndexQuery
Full Text
Analytics
Data
Cluster Manager
NODE 4
Eventing
Flexible cluster topology adjusts with growing demand
DEMO
4
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 26 26
Customers spend
more time
interacting than
transacting…
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 27
Couchbase Behind Today’s Business-Critical Applications
Application
Customers
Infrastructure
ManageabilityPerformance at Scale
Performance
Flight availability,
booking, pricing
analytics, etc.
15M
ops / second
<2.5ms
response time
Open source identity
management
1B
full-trip, web
authentications
per day
13.6k transactions
per second
Customer 360 single
view, unified notes
210M
documents
100K
users
Developer Agility
Caching & session
store for single view
2M+
reads/sec.
10M
queries/sec.
Real-time pricing,
product catalog,
inventory management
10M+
unique SKUs
35K
requests/sec.
Security + Availability
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 28
A Proven Enterprise Solution Chosen by Industry Leaders
3 of the Top 10
eCommerce
Companies
6 of the Top 10
Broadcast
Companies
Retail & E-
Commerce
3 of the Top 3
GDS
Companies
Travel &
Hospitality
Telecom
6 of the Top 10
Online Casino
Gaming Companies
Gaming
3 of the Top 3
Credit Reporting
Companies
Financial
Services
3 Fortune 500
Healthcare
Companies
Healthcare
Media &
Entertainment
2 of the Top 2
IoT
Platforms
Industrial IoT
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2019. All rights reserved.
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Thank You

Couchbase Cloud No Equal (Rick Jacobs, Couchbase) Kafka Summit 2020

  • 1.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. THE COUCHBASE & CONFLUENT BRIDGE FROM RELATIONAL TO NOSQL July 14, 2020 Jeff Morris, VP Product & Solutions Marketing
  • 2.
  • 3.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 3 Big Data Architecture Evolution Big Data version 1 focused on • ingest • archive
  • 4.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 4 Big Data Architecture Evolution Big Data version 2 focused on • collate • analyze
  • 5.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 5 Big Data Architecture Evolution Big Data version 3 ... • stream • remix
  • 6.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 6 Big Data Architecture Evolution Big Data version 3 ... • stream • remix • ... engage
  • 7.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 7 Big Data Architecture Evolution Big Data version 3 ... • stream • remix • ... engage
  • 8.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 8
  • 9.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 9 Data Processing Platform • Critical connectivity • Analysis • Streaming HDFS DBMS Mobile Other Platforms Data Processing Platform
  • 10.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 10 Stream Data Platform • Critical connectivity • Analysis • Streaming • Distributed • High throughput • Wider engagement HDFS DBMS Mobile Other Platforms Stream Data Platform
  • 11.
  • 12.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 12 Couchbase & Kafka • Source • Sink • Custom Filter • Apache Kafka • Confluent Platform Kafka ?
  • 13.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 13 How Organizations Handle Data Flow: A Giant Mess Data Warehouse Hadoop NoSQL Oracle SFDC Logging Bloomberg … any sink/source … and more OLTP ActiveMQ App App Caches OLTP OLTPAppAppApp Web Custom Apps Microservices Monitoring Analytics
  • 14.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 14 Kafka Connect
  • 15.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 15 The Couchbase Kafka Connect Plug-In Reads Data Connect Protocol stream from Couchbase and publishes to Kafka topic Consumes Kafka topic and writes records to Couchbase
  • 16.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 16 Apache Kafka™: A Distributed Streaming Platform Apache Kafka Data Warehouse HadoopOracle SFDC Twitter Bloomberg … any sink/source … any sink/source … and more Web Custom Apps Microservices Monitoring Analytics
  • 17.
  • 18.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 18 Applications Across Industries Healthcare & Pharma Patient Monitoring, Pharma Substance control, Patient Relapse, Lab Results Alerts Banking & Capital Markets Fraud Detection, Trade Data Capture, Customer 360 Retail Inventory Management, Product Catalog, A/B Testing, Proactive Alerts Telecommunications Personalized Ads, Customer 360, Network Integrity Automotive Connected Car, Manufacturing Data Processing Travel & Leisure Visitor Segmentation, Fraud Detection
  • 19.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 19 Couchbase Data Sharing Use Cases Catalog and Inventory Management Catalogs • Deliver relevant product content and a real-time view of inventory • Scale to millions of products and requests for the latest information • Serves highly engaged online audiences Profile and Session Management Personalization • Create custom experiences in real time based on aggregate data from multiple • Aggregate customer data, recommendations, user profiles, session, history data Customer 360 Single View • Deliver a consistent, single view of your data with one platform • Improve customer experience and operational visibility Digital Transformations Offload • Create transformative digital experiences by offloading mainframe, RDBMS systems • Reduce costs and improve productivity and agility
  • 20.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 20 Confluent Platform Enterprise Streaming based on Apache Kafka™ Database Changes Log Events loT Data Web Events … CRM Data Warehouse Database Hadoop Data Integration … Monitoring Analytics Custom Apps Transformations Real-time Applications … Apache Open Source Confluent Open Source Confluent Enterprise Confluent Platform Apache Kafka™ Data Compatibility Monitoring & Administration Operations Clients Connectors Complete Open Trusted Enterprise Grade
  • 21.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 21 Bank Reduces OpEx by $25M/year via Mainframe Offload Date Amount 1/27/2017 $4.56 1/22/2017 $32.14 Transaction Data Vendor Description Starbucks Coffee Walmart Blu-Ray Transaction Description Schema Website Microservices Match data and description Client profiles Lookup client profiles Mainframe MIPS = $$
  • 22.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 22 Ingest, Process, Load, and Serve Data at a Global Scale Couchbase … Couchbase … Kafka cluster Applications Other data stores Kafka cluster Kafka Streams (Data Enrichment and Transformation) Kafka Connect (Connectors to Extract and Load data) Confluent Replicator Confluent Replicator Custom Replication Custom Replication Raw event data
  • 23.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 23 Couchbase’s NoEQUAL Architectural Differentiation Developer agility & versatility • Multi-model: Key Value & JSON documents • Multi-mode: Memory-first, ACID, operational & analytic workloads • Programmable: schema flexibility + SQL in N1QL + stack-based SDKs Performance at any scale • No hassle scale out – shared-nothing, asynchronous, elastic architecture • Built-in replication (XDCR) • Always-on, globally distributed, edge-to-cloud Easy to manage • Workload isolation with multi-dimensional scaling • Automatic cluster rebalancing • Location and deployment agnostic • Kubernetes & microservices-friendly
  • 24.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 24 Sample QA SetupSample Dev Setup Elastic Scaling Architecture Sample Production DeploymentNODE 1 Query Global Index Data Analytics Full Text Cluster Manager NODE 2 Eventing NODE 1 NODE 13 Cluster Manager Data Full Text AnalyticsGlobal Index Query Eventing NODE 1 Global IndexQuery Full Text Analytics Data Cluster Manager NODE 4 Eventing Flexible cluster topology adjusts with growing demand
  • 25.
  • 26.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 26 26 Customers spend more time interacting than transacting…
  • 27.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 27 Couchbase Behind Today’s Business-Critical Applications Application Customers Infrastructure ManageabilityPerformance at Scale Performance Flight availability, booking, pricing analytics, etc. 15M ops / second <2.5ms response time Open source identity management 1B full-trip, web authentications per day 13.6k transactions per second Customer 360 single view, unified notes 210M documents 100K users Developer Agility Caching & session store for single view 2M+ reads/sec. 10M queries/sec. Real-time pricing, product catalog, inventory management 10M+ unique SKUs 35K requests/sec. Security + Availability
  • 28.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 28 A Proven Enterprise Solution Chosen by Industry Leaders 3 of the Top 10 eCommerce Companies 6 of the Top 10 Broadcast Companies Retail & E- Commerce 3 of the Top 3 GDS Companies Travel & Hospitality Telecom 6 of the Top 10 Online Casino Gaming Companies Gaming 3 of the Top 3 Credit Reporting Companies Financial Services 3 Fortune 500 Healthcare Companies Healthcare Media & Entertainment 2 of the Top 2 IoT Platforms Industrial IoT Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2019. All rights reserved.
  • 29.
    Confidential and Proprietary.Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. Thank You