Apache Kafka streams, library to build streaming applications that also provide high scalability, parallel processing and fault tolerance, etc.

1. Presented By:
Meenakshi Goyal
Software Consultant
Knoldus Inc
Real-Time Streaming using
Apache Kafka

2. Lack of etiquette and manners is a huge turn off.
KnolX Etiquettes
Punctuality
Join the session 5 minutes prior to
the session start time. We start on
time and conclude on time!
Feedback
Make sure to submit a constructive
feedback for all sessions as it is
very helpful for the presenter.
Silent Mode
Keep your mobile devices in silent
mode, feel free to move out of
session in case you need to attend
an urgent call.
Avoid Disturbance
Avoid unwanted chit chat during
the session.

3. Our Agenda
Concept of Event Streams
01
Real-Time Streaming Architecture
02
Apache Kafka in Picture
03
Kafka Producer/ Consumer
04
Creating Stream Topology
05
Implementing Stream Topology
06

4. Concept of Event Streams
12643

5. LEARN NOW
c
Idea of Event
Stream
● Real-time streaming processing starts
with sequences of event.
● Events are everywhere.
● Events with Daily Examples -
○ Retail Business -
■ Orders, Shipments, Return
○ Banking -
■ Request, Stock, Trade transactions
○ Online Portal -
■ Click Event, Adds, Impression Click

6. Identifying and
Modeling Events.
Processing Event.
Stream.
Transporting
Event
3
2
1
4

7. Real-time Stream
Architecture

8. LEARN NOW
c
Data Integration
● Almost impossible to develop an
application that does everything in an
organization.
● Some Applications can be purchased
from 3rd party vendors as well.
● Data Integration becomes a difficult
task when it comes to different data
sources.

9. LEARN NOW
c
Design
Considerations
● Time Sensitivity.
● Decoupling.
● Data Format evolution.
● Reliability.
● Scalability.

10. Apache Kafka

11. LEARN NOW
c
What is Apache
Kafka
● Distributed Streaming Platform.
● Client library for building applications
and microservices and unbounded
data.
● Interact with the clusters to process a
stream of data.
● Data is represented in it is as key-value
records

12. How does it Look Like
Creating a Stream Kafka Server Processing Stream

13. Kafka
Producer/Consumer

14. Creating Stream
Topology

15. LEARN NOW
c
Stream
Topology
● The Flow to create Kafka Streams
Application -
○ Create Configuration.
○ Create Streams Topolgy.
○ Start the Stream.
○ Add Shutdown Hook.
● Step-by-Step computational logic of a
stream processing application.
● KStream - Abstraction of a stream of Kafka
message records.

16. Implementing
Stream Topology

17. Thank You !