Data in Motion Tour 2023 Keynote

1. Reinventing Kafka in
the Data Streaming Era
Jun Rao, co-founder @ Confluent

2. Loyalty Rewards
Curbside Pickup
Trending Now
Popular on Netflix
Top Picks for Joshua
Created by the founders of
Confluent while at LinkedIn
Apache Kafka has ushered in the
data streaming era…
>70%
of the Fortune 500
>100,000+
Organizations
>41,000
Kafka Meetup Attendees
>200
Global Meetup Groups
>750
Kafka Improvement Proposals (KIPs)
>12,000
Jiras for Apache Kafka
>32,000
Stack Overflow Questions
Real-time Trades
Ride ETA
Personalized Recommendations

6. The need for a cloud-native, data streaming platform
Connecting all your apps, systems and data into a central nervous system

7. Self-managing Kafka
comes with cost and
complexity
● Cluster sizing
● Cluster provisioning
● Architecture planning
● Upgrades / patches
● Source / sink connectors
● Monitoring / reporting
● Security / reliability
management
● Mirroring
● Expansion planning
● Load rebalancing
● Utilization optimization
● Cluster migrations
● Performance enhancements
● Zookeeper management
● Multi-tenancy
Each new project adds complexity…
Client 1
…
Client n
Networking spend for
increased throughput
LB
Storage
Disks
Broker 1 Broker 2 Broker n
Excess, underutilized
storage/compute
Disk n
…
…
Example of a typical Kafka deployment in the cloud
Resources to manage
infrastructure failures

8. PUTTING KAFKA IN THE CLOUD…
ISN’T JUST PUTTING KAFKA IN THE CLOUD.

9. Confluent Cloud
COMPLETE EVERYWHERE
CLOUD-NATIVE

10. Kora: the Cloud-Native Engine for Apache Kafka
Serverless
● Elastic scaling up & down from
0 to GBps
● Auto capacity mgmt, load
balancing, and upgrades
Infinite Storage
● Store data cost- effectively at
any scale without growing
compute
Resilience
● Multi-AZ and multi-region
replication
● Durability self-validation
High Availability
● 99.99% SLA
● Multi-region / AZ availability across
cloud providers
● Patches deployed in Confluent
Cloud before Apache Kafka
Network Flexibility
● Public, VPC, and Private Link
● Seamlessly link across clouds
and on-prem with Cluster
Linking

11. How Kafka Stores Data
Application Application Application Application Application
Cost
Elasticity
Broker Broker Broker

12. Infinite Storage with Confluent Cloud
Broker Broker
Application Application Application Application
Cost
efficiency
Improved
Elasticity
Broker
local
hotset
remote
local
hotset
remote
local
hotset
remote
Object Storage
Broker
local
hotset
remote

13. Benefits from Cloud Native
30XELASTICITY
50
40
30
20
10
0
OSS Kafka Confluent Cloud
Hours required to scale 3 brokers to 4, replication factor of 3,
30-day retention, 100 MBps throughput, 10GBps network
100X RESILIENCY -X STORAGE
0
22.5 45 67.5 90
Confluent
Cloud
OSS Kafka
Other Kafka
Service
99.99%2
99.9%
99%1
10X 100X
Minimum downtime commitment by
Kafka service based on SLA
Infinite Storage
AWS GA
Infinite Storage
GCP GA
Infinite Storage
Azure GA
Time
Hours required to scale 3 brokers to 4, replication factor of 3,
30-day retention, 100 MBps throughput, 10GBps network
AVG Storage
per Cluster
AWS
GCP
AZURE

14. Performance benefits too!
Large performance gains are realized too,
with an open to path to even more.
CCloud (Next-gen
Replication Engine)
CCloud (Present)
Apache Kafka

15. No more spending on excess, wasted resources - storage
and compute costs scale up and down with demand
Solving all aspects of the Kafka infra costs with networking
optimizations that solve 90% of the Kafka TCO problem
Common
Breakdown
Of Kafka Infra Costs
(1GBps, 3x fanout)
Networking: 90.8% Compute: 3.0% Storage: 6.2%
Confluent has the lowest TCO for Apache Kafka
Our fully managed SaaS offering solves all aspects of the TCO challenge for Apache Kafka

16. COMPLETE EVERYWHERE
CLOUD-NATIVE
Confluent Cloud

17. 70+ FULLY MANAGED CONNECTORS
Amazon S3

18. Custom Connectors
ANNOUNCING

19. You need to trust and understand the data moving
through your data streaming platform
Schemas Data Lineage
Metadata Integrity & Validation

20. Email Address →
→
→
Does it contain @ and domain?
Is the right digits given the country?
Is it the right length and format given the country?
Data Quality Rules in Action
Online Retail Example
Postal Code
Phone Number

21. val fraudulentPayments: KStream[String, Payment] = builder
.stream[String, Payment](“payments-kafka-topic”)
.filter((_ ,payment) => payment.fraudProbability > 0.8)
fraudulentPayments.to(“fraudulent-payments-topic”)
CREATE STREAM fraudulent_payments AS
SELECT * FROM payments
WHERE fraudProbability > 0.8;
STANDALONE APPS FRAMEWORKS

22. SPRING 2023 FALL 2023 WINTER 2023 2024
We plan to GA our SQL Flink service
in Winter 2023
SQL Early Access SQL Public Preview SQL General Availability Java + Python

23. Provision a new cluster
*Must be present to win. Raffle will occur during afternoon reception

24. ANNOUNCING STREAM SHARING
Easily share real-time data
without delays in a few
clicks from Confluent to any
Kafka client
Safely share and protect your data
with robust authenticated sharing,
access management, and layered
encryption controls
Trust the quality and compatibility
of shared data by enforcing
consistent schemas across users,
teams, and organizations

25. Confluent Cloud
COMPLETE EVERYWHERE
CLOUD-NATIVE

26. Any Cloud. Any Region.
Everywhere You Want To Be

27. Why Confluent is the world’s most trusted
data streaming platform
Focus & Expertise
Only company focused on data in motion:
● Founded in 2014 by the Original Creators of
Apache Kafka
● Over 80% of Kafka commits are by
Confluent employees
● Advised on thousands of real-world Kafka
deployments across a wide range of
patterns & industries
Building and supporting a world class product:
● >9 million engineering hours spent
building building our product
● We internally manage >15,000 clusters (and
counting) in Confluent Cloud
● Over 1 million cumulative hours of Kafka
expertise within Confluent support &
services
Execution at Scale

28. Start your free trial of
Confluent Cloud & get $400 in
credits
Get started with Confluent Cloud is easy!
confluent.io/confluent-cloud/tryfree/
Request a personalized TCO
assessment for your current
Kafka needs or future Data
Streaming Platform

29. Thank You

30. September 26-27, 2023 | San Jose, CA