The document discusses Apache Kafka’s Streams API, designed for building scalable, fault-tolerant real-time applications. It emphasizes the flexibility of app deployment and highlights key functionalities like exactly-once processing, event-time processing, and integrated database joins. The main goal is to enable developers to focus on creating applications rather than managing infrastructure.

1. R ET HINKING
Stream Processing
With Apache Kafka

2. 2
1.0 Enterprise
Ready
0.10 Data Processing
(Streams API)
0.11 Exactly-once
Semantics
Kafka the Streaming Data Platform
2013 2014 2015 2016 2017 2018
0.8 Intra-cluster
replication
0.9 Data Integration
(Connect API)

3. 3
Your System, a Hotel
APP

4. 4
Your System, a Hotel
Microservices
APP
APP
APPSecurity
Guard
ReceptionistRoom
Service

5. 5
Your System, a Hotel
Distributed Apps
APP
Security Guard
Security Team

6. 6
As developers, we want
to build APPS not INFRASTRUCTURE

7. 7
We want our apps to be:
Scalable Elastic Fault-tolerant
Stateful Distributed

8. 8
Where do I put my compute?

9. 9
Where do I put my state?

10. 10
The actual question is
Where is my code?

11. 11
the KAFKA STREAMS API is a
JAVA API to
BUILD REAL-TIME APPLICATIONS to
POWER THE BUSINESS

12. 12
App
Streams
API
Not running
inside brokers!

13. 13
Brokers?
Nope!
App
Streams
API
App
Streams
API
App
Streams
API
Same app, many instances

14. 14
Before
DashboardProcessing Cluster
Your Job
Shared Database

15. 15
After
Dashboard
APP
Streams
API

16. 16
this means you can
DEPLOY your app ANYWHERE using
WHATEVER TECHNOLOGY YOU WANT

17. 17
Things Kafka Streams Does
Runs
everywhere
Clustering
done for you
Exactly-once
processing
Event-time
processing
Integrated
database
Joins, windowing,
aggregation
S/M/L/XL/XXL/XXXL
sizes

18. 18
First, some
API CONCEPTS

19. 19
STREAMS are
EVERYWHERE

20. 20
TABLES are EVERYWHERE

21. 21
STREAMS <-> TABLES

22. 22
// Example: reading data from Kafka
KStream<byte[], String> textLines = builder.stream("textlines-topic", Consumed.with(
Serdes.ByteArray(), Serdes.String()));
// Example: transforming data
KStream<byte[], String> upperCasedLines= rawRatings.mapValues(String::toUpperCase));
KStream

23. 23
// Example: aggregating data
KTable<String, Long> wordCounts = textLines
.flatMapValues(textLine -> Arrays.asList(textLine.toLowerCase().split("W+")))
.groupBy((key, word) -> word)
.count();
KTable

24. 24
DEMO

25. 25
Remember, we want to build
APPS not
INFRASTRUCTURE

26. 26
the KAFKA STREAMS API is a
JAVA API to
BUILD REAL-TIME APPLICATIONS to
POWER THE BUSINESS

27. 27
Things Kafka Streams Does
Runs
everywhere
Clustering
done for you
Exactly-once
processing
Event-time
processing
Integrated
database
Joins, windowing,
aggregation
S/M/L/XL/XXL/XXXL
sizes

28. THANK YOU!