Kafka summit apac session

1. Cloud-Native Kafka,
Integrations, and Kubernetes
Christina Lin
Senior Principal Product Marketing Manager
@Christina_wm
Evan Shortiss
Principal Technical Marketing Manager
@evanshortiss
1

2. 2
Next-generation of messaging - designed for retaining,
integrating and streaming large amounts of data with low
overhead
INTRODUCTION
The impact of Apache Kafka
Proven open source technology for real-time information processing
Delivers real-time data applications - eliminates the
delays of batch data processing and ELT applications for
better customer experiences
A fundamental building block - simplifies the delivery
of data-driven, microservices based applications

3. INTRODUCTION
3
Kafka in use today
Digital experiences
Delivers real-time experiences with immediate
access to information and response time
Microservices
applications
Loosely couples microservices so
development teams can remain agile
Streaming ETL
Modernize applications driven by batch data
for real-time performance
Real-time analytics
Ingest data from multiple sources for better
business insights
Edge & hybrid scenarios
Collect data from diverse and disparate
devices and systems

4. Making the case for a
Cloud-Native, Managed
Kafka Platform
4

5. 5
OPENSHIFT STREAMS FOR APACHE KAFKA
Why use a managed Apache Kafka service?
Reduced Complexity
Brokers
Load
Balancers
VMs & Networking
Applications
Components
Events

6. 6
OPENSHIFT STREAMS FOR APACHE KAFKA
Why use a managed Apache Kafka service?
Reduced Complexity
Brokers
Load
Balancers
VMs & Networking
Applications
Components
Events

7. 7
OPENSHIFT STREAMS FOR APACHE KAFKA
Why use a managed Apache Kafka service?
Reduced Complexity
Applications
Events
Cloud-Native Managed Kafka
Topic A
Topic B
Topic C

8. RED HAT MANAGEMENT
8
Hosted & managed service offering
Red Hat cloud services are managed and operated by Red Hat’s Site Reliability Engineers
▸ SREs serve as the cloud provider account owner and cluster
administrator owning the 99.95% SLA
▸ Responsible for the 24x7 support for all managed and hosted
environments
■ Including for building, installing, upgrading, managing
and maintaining every cluster
▸ SRE teams are distributed across 3 regions: APAC, EMEA
and Americas
▸ The team ensures open communications channels
centralized around the dedicated customer portal
Americas EMEA APAC

9. CONFIDENTIAL designator
Red Hat OpenShift Streams for Apache Kafka
Complete solution for stream-based applications
PRODUCT OVERVIEW
9
Metrics
&
monitoring
Configuration
mgmt
Hosted & managed (99.95% SLA)
Kafka cluster
Broker(s)
Topic(s)
Streamlined developer experience: a curated solution
with a developer-first, consistent experience
Delivered as a service, managed by Red Hat SRE -
24x7 global support and a 99.95% service-level
agreement (SLA)
Real-time, streaming data broker - Dedicated Apache
Kafka cluster deliver as a service in the cloud and
location of choice
▸ Access to Kafka brokers, topics, and partitions
▸ Managed ZooKeeper
▸ Metrics and monitoring
▸ Integrated identity & access management
STREAMLINED DEVELOPER EXPERIENCE
UI service binding API
CLI

10. CONFIDENTIAL designator
PRODUCT OVERVIEW
The value of Red Hat OpenShift Streams for Apache Kafka
10
Faster application
velocity
Unified experience across all
clouds
Kafka ecosystem for
streams-based applications
Begin developing
immediately and
continuously respond to
change
Seamlessly connects
applications across public
and private clouds
Delivers a curated set of
cloud services to simplify
delivery of stream-based
applications

11. 11
OPENSHIFT STREAMS FOR APACHE KAFKA
What’s right for me?
Choose wisely.

12. 12
INTRODUCTION
Red Hat’s commitment to Kafka-based products
A history of innovation and success
Red Hat brings Kafka to
Kubernetes and launches Red Hat
AMQ Streams (Red Hat
Integration)
Ecosystem to deliver
event-driven solutions
Customer success
Kafka innovation since
2018

13. GETTING STARTED
13
Development Preview *NEW*
Hosted and managed Kafka service
for stream-based applications
● Spin up your own Kafka cluster
● Create your topics and its partitions
● Connect your producers and consumers
● Get started with the quick starts
● Integrate your apps to the service
Managed Kafka cluster
● Access for 48 hours
● Limited number of topics & brokers
Time and resource limited
● Go to: red.ht/TryKafka
● Create your own Red Hat account
● Sign-in to try the service
Sign-up

14. Kamelet
Connect, Stream for on
Kubernetes
14

15. 15
Kamelet - Connect, Stream to Kafka on Kubernetes

16. 16

17. 17
Efficient, High throughput
And only focus on receiving, store
and replicate data.
I don’t want to know what’s in it...
Connect
Connect and talks in client and
Kafka protocol.
Converter
Serialize or deserialize with
various data format and
validating it’s schema. Transform
Making changes to into data
content. With additional data,
filtering or masking it.
Error Handle
What to do when things go
wrong? Where to place the
problem data?

18. 18
Apache Camel
Red Hat Integration
APP
▸ Comprehensive connectors
▸ Microservices orchestration
with EIP
……….
▸ Data Transformation
HL7
FHIR
AVRO
Protobuf
Zip
CSV
▸ Data Transformation with Tools
Kamelet - Connect, Stream to Kafka on Kubernetes

19. 19
Efficient, High throughput
And only focus on receiving, store
and replicate data.
I don’t want to know what’s in it...
Connect
Connect and talks in client and
Kafka protocol.
Converter
Serialize or deserialize with
various data format and
validating it’s schema. Transform
Making changes to into data
content. With additional data,
filtering or masking it.
Error Handle
What to do when things go
wrong? Where to place the
problem data?
Kamelet - Connect, Stream to Kafka on Kubernetes
300+ Connectors
Built-in transform
data formats with
simple configuration
EIP, split, filter, even
customization of
processes
Dead letter queue,
catching exceptions

20. Kamelet - Connect, Stream to Kafka on Kubernetes
Deploy
Scaling
Monitoring
Managing
Repository

21. 21
Troubleshooting
SpringBoot Apps
SpringBoot Apps
Containerize
Docker
Linux Container
Application Server
JavaEE
OSGi
Bundle
WAR/JAR
Steep learning Curve
Kamelet - Connect, Stream to Kafka on Kubernetes
? ?
?
? Service
Deployment
DeploymentConfig
ConfigMap
SpringBoot Apps
Java Main
Quakus

22. KAMELET
apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
name: telegram-text-source
...
spec:
definition:
title: "Telegram Text Source"
required:
- botToken
properties:
botToken:
title: Token
...
flow:
from:
uri: telegram:bots
parameters:
authorizationToken: "#property:botToken"
steps:
- convert-body-to:
type: "java.lang.String"
type-class: "java.lang.String"
charset: "UTF8"
- filter:
simple: "${body} != null"
- log: "${body}"
- to: "kamelet:sink"
Serverless
OpenShift
Kamelet - Connect, Stream to Kafka on Kubernetes
Connect
Converter
Transform
Kamelet
Kamelet
Kamelet
Kamelet
Kamelet
Kamelet
Developers
Repository

23. 23
KAMELET
Kamelet - Connect, Stream to Kafka on Kubernetes
OpenShift/Kubernetes
YAML:
Source:Kamelet
Sink: Kafka
YAML:
Source: Kafka
Sink: Kamelet
DEFINE
DEPLOY
SCALE
Monitor, Manage

24. Demo
24
Kamelet - Connect, Stream to Kafka on Kubernetes

25. Demo
25
Kamelet - Connect, Stream to Kafka on Kubernetes

26. 26
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams

27. 27
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams
CloudEvent
Channel
Filter Transform
Kamelet
AWS JP
Kamelet
AWS KO
Kamelet
CloudEvent
Channel

28. 28
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams
CloudEvent
Channel CloudEvent
Channel
Filter Transform
Kamelet
AWS JP
Kamelet
AWS KO
Kamelet
Telegram
Kamelet

29. 29
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams
CloudEvent
Channel CloudEvent
Channel
Filter Transform
Kamelet
AWS JP
Kamelet
AWS KO
Kamelet
Telegram
Kamelet
Translate-ZH
AWS RU
Kamelet
AWS ZH
Kamelet

30. 30
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams
CloudEvent
Channel CloudEvent
Channel
Filter Transform
Kamelet
AWS JP
Kamelet
AWS KO
Kamelet
Telegram
Kamelet
Translate-ZH
AWS RU
Kamelet
AWS ZH
Kamelet
Topic - Telegram
CloudEvent
Channel
Telegram
Kamelet
CloudEvent
Kamelet

31. 31
DEMO
SalesForce
Filter & Mask
Translate-JP
Telegram
Translate-KO
Translate-RU
CloudEvents
Topic - Salesforce
Salesforce
Kamelet
Campaign
Streams
CloudEvent
Channel CloudEvent
Channel
Filter Transform
Kamelet
AWS JP
Kamelet
AWS KO
Kamelet
Telegram
Kamelet
Translate-ZH
AWS RU
Kamelet
AWS ZH
Kamelet
Topic - Telegram
CloudEvent
Channel
Telegram
Kamelet
CloudEvent
Kamelet
Filter Transform
Kamelet

32. RED HAT MANAGEMENT
32
Hosted & managed service offering
Red Hat cloud services are managed and operated by Red Hat’s Site Reliability Engineers
▸ SREs serve as the cloud provider account owner and cluster
administrator owning the 99.95% SLA
▸ Responsible for the 24x7 support for all managed and hosted
environments
■ Including for building, installing, upgrading, managing
and maintaining every cluster
▸ SRE teams are distributed across 3 regions: APAC, EMEA
and Americas
▸ The team ensures open communications channels
centralized around the dedicated customer portal
Americas EMEA APAC

33. GETTING STARTED
33
Development Preview *NEW*
Hosted and managed Kafka service
for stream-based applications
● Spin up your own Kafka cluster
● Create your topics and its partitions
● Connect your producers and consumers
● Get started with the quick starts
● Integrate your apps to the service
Managed Kafka cluster
● Access for 48 hours
● Limited number of topics & brokers
Time and resource limited
● Go to: red.ht/TryKafka
● Create your own Red Hat account
● Sign-in to try the service
Sign-up