The document discusses best practices for debugging client applications, particularly in relation to Kafka producers, consumers, and Kafka Streams applications. It highlights various educational sessions and certifications offered by Confluent to enhance skills in using Apache Kafka, alongside details about the benefits of adopting Confluent Cloud for data streaming. Additionally, it covers a migration journey and the challenges associated with managing Kafka on-premises versus using Confluent's managed services.

1. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?
Starting soon…
STARTING SOOOOON..
STARTING SOON…
S
Starting Soon…
STARTING SOON..

2. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?
Starting soon…
STARTING SOOOOON..
STARTING SOON…
S
Starting Soon…

3. Tech Talk Q4: Synthesis Software
Technologies
The Chronicles of Kafka:
A Migration Journey
Confluent Cloud Free Trial
New signups receive $400
to spend during their first 30 days.

4. Our Partner Technical Sales Enablement offering
Scheduled sessions On-demand
Join us for these live sessions
where our experts will guide you
through sessions of different level
and will be available to answer your
questions. Some examples of
sessions are below:
● Confluent 101: for new starters
● Workshops
● Path to production series
Learn the basics with a guided
experience, at your own pace with our
learning paths on-demand. You will
also find an always growing repository
of more advanced presentations to
dig-deeper. Some examples are below:
● Confluent 10
● Confluent Use Cases
● Positioning Confluent Value
● Confluent Cloud Networking
● … and many more
AskTheExpert /
Workshops
For selected partners, we’ll offer
additional support to:
● Technical Sales workshop
● JIT coaching on spotlight
opportunity
● Build CoE inside partners by
getting people with similar
interest together
● Solution discovery
● Tech Talk
● Q&A

5. Certifications + Accreditations
5
1. Certified Developer for Apache Kafka®
Qualifications: Completion of On-Demand/ Instructor-Led Course
Course Alignment: Confluent Developer Skills for Building Apache
Kafka® Duration: 60 multiple choice questions in 90 minutes
4. Fundamentals Accredited
Know the Basics
Prework Apache and Quiz
Demonstrates Kafka and Confluent Solution awareness
5. Technical Sales Associate Accredited
Confluent Positioning
Technical Sales Bootcamp Pre-work and Quiz
Enablement to equip sales and pre-sales to articulate the
Confluent value proposition
6. Technical Sales Professional Accredited
Hands on, Coaching Sales Pitch. A deeper dive into important
Confluent IP and differentiators in a way that allows attendees to
demonstrate their knowledge of the product, its value, and how to
demo it.
2. Certified Administrator for Apache Kafka® (Operations)
Qualifications: Completion of On-Demand/ Instructor-Led Course
Course Alignment: “Apache Kafka®” Administration by
Confluent Kafka® Duration: 60 multiple choice questions in 90
minutes
3. PS Delivery Qualification COMING SOON
Delivery-focused enablement, prescriptive delivery kits and work
on a real customer use case with Confluent SME’s. Our subject
matter experts will review the architecture and solution and
provide real-time feedback as well as recommendations for
additional enablement or shadowing as needed.
NEW

6. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?

7. Goal
Partners Tech Talks are webinars where subject matter experts from a Partner talk about a
specific use case or project. The goal of Tech Talks is to provide best practices and
applications insights, along with inspiration, and help you stay up to date about innovations
in confluent ecosystem.

8. Why Confluent Cloud
8

9. Today’s customers expect data in motion
Initiate an action
Instant confirmation
Source: https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-latency-cost-them-1-in-sales
~100 ms
in latency
can cost
you…
… 20% of
digital
traffic
… $400M in
revenue
9

10. Finance & Banking Insurance Telecom
Travel & Retail
10 OUT
OF 10 8 OUT
OF 8
Fortune 500 Companies
Using Apache Kafka
70+%
Transportation Energy & Utilities Entertainment
Technology
8 OUT
OF 10 9 OUT
OF 10
10 OUT
OF 10
10 OUT
OF 10
10 OUT
OF 10 8 OUT
OF 10
Kafka is the de facto standard for Data in Motion

11. Operationalizing Kafka on your own is difficult
Kafka is hard in experimentation. It gets harder (and riskier) as you add
mission-critical data and use cases.
“We are in the business of selling and renting clothes. We are not
in the business of managing an event streaming platform… If we
had to manage everything ourselves, I would’ve had to hire at
least 10 more people to keep the systems up and running."
● Architecture planning
● Cluster sizing
● Cluster provisioning
● Broker settings
● Zookeeper management
● Partition placement & data
durability
● Source/sink connectors
development & maintenance
● Monitoring & reporting tools
setup
● Software patches and upgrades
● Security controls and
integrations
● Failover design & planning
● Mirroring & geo-replication
● Streaming data governance
● Load rebalancing & monitoring
● Expansion planning & execution
● Utilization optimization &
visibility
● Cluster migrations
● Infrastructure & performance
upgrades / enhancements
I N V E S T M E N T & T I M E
V
A
L
U
E
1
2
3
4
5
Experimentation
/ Early Interest
Central Nervous
System
Mission critical,
disparate LOBs
Identify a
Project
Mission-critical,
connected LOBs
Key challenges:
Operational burden & resources
Manage and scale platform to support
ever-growing demand
Security & governance
Ensure streaming data is as safe & secure as
data-at-rest as Kafka usage scales
Real-time connectivity & processing
Leverage valuable legacy data to power
modern, cloud-based apps & experiences
Global availability
Maintain high availability across environments
with minimal downtime

12. Business Risk
● Increasing risk of costly downtime and security breaches
as Kafka spans more use cases, apps / data systems, teams,
and environments
● Valuable resources diverted to address unplanned
downtime and breaches
Unplanned downtime
and breaches
Substantial hidden costs incl. lost
revenue, reputation damage, fines /
audits, reduced CSAT, and data loss
Infra Spend
● Over-provisioning required to support fluctuating demand
● Inability to scale storage w/o growing compute
● Performance degradations with manual configs,
re-balancing, and upgrades / patching
What does this all mean for your business?
Unpacking the direct and indirect costs of self-supporting Open Source Kafka
FTE Costs
It’s hard because... Which results in...
● Costly time & resources spent building / maintaining
low level infra tools vs. differentiating the business
● Increasingly difficult to hire and retain Kafka talent
(job listings & compensation +40% YoY)
Delayed time-to-value
2+ years to reach production at scale
with ~$3-5M+ in ongoing platform
development & operations costs
Increased Total Cost
of Ownership
Hundreds of thousands of $$$’s
spent annually on underutilized &
sub-optimal infra

13. Confluent Cloud
Cloud-native data streaming platform built by the founders of Apache Kafka®
Everywhere
Connect your data in real
time with a platform that
spans from on-prem to
cloud and across clouds
Complete
Go above & beyond Kafka
with all the essential tools
for a complete data
streaming platform
Cloud-Native
The 10x Apache Kafka®
service: elastic, resilient
and performant, powered
by the Kora Engine
Stream confidently on the world’s most trusted data streaming platform built by the founders of
Apache Kafka©, with resilience, security, compliance, and privacy built-in by default. 13

14. “Before Confluent, when we had broker outages that
required rebuilds, it could take up to three days of
developer time to resolve. Now, Confluent takes care
of everything for us, so our developers can focus on
building new features and applications.”
Elastic scalability: Scale and shrink to handle GBps+
workloads and petabyte storage 10x faster and easier
without any operational burden
Guaranteed resiliency: Leave Kafka reliability worries
behind with the industries most comprehensive 99.99%
uptime SLA and built-in data durability
Low latency: Speed up real-time analytics and
customer experiences with predictable low latency,
sustained over time
Cloud-Native
The 10x Apache Kafka® service: elastic,
resilient and performant, powered by the
Kora Engine
Kora Engine: The Apache Kafka® engine
built for the cloud

15. Complete
Go above & beyond Kafka with all the
essential tools for a complete data
streaming platform
Connectors & Stream Processing: Connect to
and from any app / system and process your
data streams in-flight
Reduce TCO and architectural complexity with our
portfolio of 120+ pre-built connectors and stream
processing powered by ksqlDB, all available fully
managed and built-in with Confluent Cloud
Stream Designer: Quickly build and deploy
streaming apps & pipelines
Rapidly build, test, and deploy streaming data
pipelines with Stream Designer, extensible with SQL,
while reducing the need to write boilerplate code
Security & Governance: Secure, discover, and
organize your data streams
Build trust and put your data streams to work with
enterprise-grade security and the only Stream
Governance suite for data in motion
“BHG is a fast-moving company, and Confluent is
quickly becoming not only a central highway for
our data with their vast connector portfolio, but a
streaming transformation engine as well for a vast
number of use cases… We are making Confluent the
true backbone of BHG, including leveraging 20+
Confluent connectors across both modern,
cloud-based technologies & legacy systems, to help
integrate our critical apps & data systems together.”
15
Connectors
Security
Data
Governance
Stream
Processing
Monitoring
Global
Resilience
Stream
Designer

16. Everywhere
Connect your data in real time with a
platform that spans from on-prem to
cloud and across clouds
Run Anywhere: Deploy across any environment
Provision Confluent as a fully managed service on AWS,
Azure, and Google Cloud across 60+ regions w/
Confluent Cloud, or on-premises w/ Confluent Platform
Unified: Unify data across hybrid and
multi-clouds
Provide consistent, self-service access to real-time data
across all your environments with Cluster Linking and
globally connected clusters that perfectly mirror data
Consistent: Learn one platform for all
environments
Remove the burden of learning new tools for each
environment with a consistent experience spanning
across cloud, on-prem, and hybrid / multicloud
“Our transformation to a cloud-native, agile
company required a large-scale migration from
open source Apache Kafka. With Confluent, we now
support real-time data sharing across all of our
environments, and see a clear path forward for our
hybrid cloud roadmap.”
16

17. …enabling you to deploy, operate, and scale in
minutes instead of months
Kafka re-architected to be truly Cloud-Native
17
Months Minutes
Weeks
Open Source
Apache Kafka
In-house development and
maintenance without support
Self-managed
Kafka Services
Manual operations with basic
tooling and/or support
Confluent
Cloud
Fully managed, elastic,
and automated product
capabilities with zero overhead

18. Confluent Cloud for
Apache Flink®
Simple, Serverless Stream Processing
Easily build high-quality,
reusable data streams with
the industry’s only cloud-
native, serverless Flink service
Effortlessly filter, join, and enrich your
data streams with Flink, the de facto
standard for stream processing
Enable high-performance and efficient
stream processing at any scale, without
the complexities of infrastructure
management
Experience Kafka and Flink as a
unified platform, with fully integrated
monitoring, security, and governance
Available for preview in select regions – see the docs for regional availability

19. C O N F I D E N T I A L
Process data streams in-flight to maximize actionability,
fidelity, and portability
Blob storage
3rd party app
Databases Data Warehouse
Database
SaaS app
Low latency apps
and data pipelines
Consistent, reusable
data products
Optimized resource
utilization

20. Build streaming
data pipelines to
inform real-time
decision making
Create new enriched and curated
streams of higher value using:
● Data transformations
● Streaming joins, temporal joins, lookup
joins, and versioned joins
● Fan out queries, multi-cluster queries
20
t1, 21.5 USD
t3, 55 EUR
t5, 35.3 EUR
t0, EUR:USD=1.00
t2,
EUR:USD=1.05
t4: EUR:USD=1.10
t1, 21.5 USD
t3, 57.75 USD
t5, 38.83 USD
Currency rate
Orders
STREAMING DATA PIPELINES

21. How to: Migrate to Confluent Cloud

22. Migration Overview
Kafka Clients &
Streaming Apps
Kafka Platform
Managed
Self-Managed
Migrate to Confluent Cloud involves moving the underlying Kafka Platform, Kafka client & Streaming
Applications.
Depending on the situation, some Kafka Clients could transition into managed world and some Kafka Clients
might remain as self-managed. One of the key decision is driven by whether private network is required or
not.
Connector
Apps
(Producer/Consumer)
ksqlDB
Schema
Connector
ksqlDB
Schema
Confluent Platform Confluent Cloud
Repoint
Self-Managed
Applications to
Confluent Cloud

23. 23
Real-time
connectivity
& processing
Security
& governance
Operational
burden &
resources
Apache Kafka Confluent
Tedious configuration. Manual & resource-
intensive day-to-day operations to deploy,
manage, and scale clusters.
Custom-built security tools & integrations that
creates tech & operational debt. Lacks
governance for streaming data.
Instant cluster provisioning and scaling.
Automated capacity mgmt, load balancing, and
upgrades.
Battle-tested and enterprise-grade security &
governance tools ready out-of-the-box. (e.g., RBAC,
Audit Logs, Encryption, Stream Governance)
Siloed solutions and custom integrations,
escalating complexity & costs to manage and
maintain platform as it scales.
Enterprise-wide data fabric across apps & data
systems with a rich, pre-built connector
ecosystem and stream processing.
Global
availability
Resource-intensive failover design & operations,
exponentially increasing in complexity & risk as
volume and variety of environments grows.
Highly available and consistent access to data
across all of your environments, spanning all
clouds & on-prem.
“Confluent has become a lynchpin to harness Apache Kafka for improved operational
transparency and timely data-driven decisions. We are delighted with the improvements we have
seen in the monitoring of our business flows, speeding up lending approvals, and providing better and
more timely fraud analytics, and look forward to expanding services to leverage the platform to
simplify, automate, and drive business decisions.”
These capabilities make us the perfect partner to
address the key challenges with Open Source Kafka

24. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?
Starting soon…
STARTING SOOOOON..
STARTING SOON…
S
Starting Soon…

25. The Chronicles of
Kafka: A Migration
Story
Bhavesh Sooka and Jonathan Lew

26. Who we are
Kafkateer
Bhavesh Sooka
Kafkateer
Jonathan Lew
Who we are

27. Our
Goal
Our Goal
For the audience to learn
something from our
migration experience
and take it back to their
migration journey.

28. Contents
•Prologue
•Chronicles
•Epilogue

29. Gamified Fitness
VDP & V1
- 19 clusters
- 40 Markets Globally
using 8 AWS Regions
- 27 million monthly
users
- 20 Device
Manufacturers
- 10 million
workouts/day
- 124m messages
in a day
- 1000s of
microservices

30. Timeline
Prologue
2015 – V1:
Kafka 0.10 on
EC2
2019 – V1: Upgrade
Apache Kafka 0.10
-> CP 5.0.1
2021: - VDP:
Tibco EMS -> CP
6.0.1
2014 – V1:
Kafka on K8s
2015 – VDP: Tibco
EMS (Event
Messaging Service)
2020 – V1 Upgrade
(x2) CP 5.0.1 ->
5.4.1 -> CP 6.0.1
2021 – The
Migration: Enter your
speakers
1
2
3 5 7
6
4
7 Years

31. Timeline
Nov 2021 – Feb 2022:
ARB x2
June – Sept: Execution
4 years: Lead
up
Feb - June: Planning Oct: Hyper Care
4
5
3
2
1
Migration Timeline
11 Months

32. help!
VS
Winner!
ARB

33. Planning
Planning took 4
Months
Planning Outcomes
- Design decisions
- Configuration
- Secret Storage
- RBAC
- Networking
- Automation
- Deployment plan

34. Planning:
Design Decisions
Two DD’s we want to
mention in specific are:
Code and config split
Vault Module

35. Planning
Planning: Networking
Private Link TG
W
VPC
Peering

36. SSO enabled and LDAP
connectivity
Planning: RBAC
Split into 2 problems to
Solve
People Applications

37. Planning: Automation
Topic creation
Cluster and Networking

38. Planning: Deployment Day
Goal: 100% Uptime During Migration
• Move Producer
• Drain
• Move Consumer
• Repeat

39. Line Manager
• Decision Maker
• Autonomy
• On our Side
Operational Requests
SNAG: CIO
- Flight plan
- Commercials
Business Decisions/Red Tape

40. This was child’s
play!!!!!
4 Environments
=
4 Months
1 month of Hyper
Care
Execution

41. • Agile vs Waterfall
• Stakeholders buy in and trust
• Contextualized planning
• Team dynamic
• Business vs IT
• Industry Experts in Kafka
Learnings

42. Happy
Thank You
Confluent, Jack and
Dan
Vitality
Group/Vitality
Global
Synthesis
You

43. Happy
And so, they lived happily ever after…
#TagYourIt

44. Q&A
Q&A