Watch this talk here: https://www.confluent.io/online-talks/building-event-driven-applications-apache-kafka-and-confluent-platform Apache Kafka® has become the de facto technology for real-time event streaming. Confluent Platform, developed by the creators of Apache Kafka, is an event-streaming platform that enables the ingest and processing of massive amounts of data in real time. In this session, we will cover the easiest ways to start developing event-driven applications with Apache Kafka using Confluent Platform. We will also demo a contextual event-driven application built using our ecosystem of connectors, REST proxy, and a variety of native clients. View now to learn: -How to create Apache Kafka topics in minutes and process event streams in real time -Check the health of an Apache Kafka broker using Confluent Control Center -The latest enhancements to Confluent Platform that make it easier to run Apache Kafka at scale -How to use KSQL, streaming SQL for Apache Kafka, to process event streams in real time using simple SQL queries

1. @gamussa | #eventdriven | @ConfluentINc
Building Event-Driven
Apps with
Apache Kafka &
Confluent Platform

2. @gamussa | #eventdriven | @ConfluentINc
2

3. @gamussa | #eventdriven | @ConfluentINc
3
I build highly scalable
Hello World
apps @kennybastani

4. Photoby KhaiSzeOng on Unsplash

5. Photoby RainierRidao on Unsplash

6. Photoby RohitTandon on Unsplash

7. Photoby TheodoreMoore on Unsplash

8. Photoby CristianGrecu on Unsplash

9. @gamussa | #eventdriven | @ConfluentINc
9
Normal
Applications
(i.e., monoliths)

10. @gamussa | #eventdriven | @ConfluentINc
10
Monoliths
are hard to
think about

11. @gamussa | #eventdriven | @ConfluentINc
11
Monoliths
are hard to
change

12. Refactoring to
microservices

13. @gamussa | #eventdriven | @ConfluentINc
13
Re-
integration?

14. @gamussa | #eventdriven | @ConfluentINc
14
Re-
integration

15. Photoby DevaDarshan from Pexels

16. @gamussa | #eventdriven | @ConfluentINc
16
There are no good
ways to integrate
microservices

17. @gamussa | #eventdriven | @ConfluentINc
17
There are no-good
ways to integrate
microservices

18. @gamussa | #eventdriven | @ConfluentINc
18
Filesystem

19. @gamussa | #eventdriven | @ConfluentINc
19

20. @gamussa | #eventdriven | @ConfluentINc
20
Database

21. @gamussa | #eventdriven | @ConfluentINc
21

22. @gamussa | #eventdriven | @ConfluentINc
22
Integrating Microservices
through the database
● “I have a database and I know how to use it.”
● Eventually causes services to co-mingle.
● Violation the «bounded context»
● Great to use inside a service boundary!
● Terrible for sharing data or negotiating
change.

23. @gamussa | #eventdriven | @ConfluentINc
23
RPC

24. @gamussa | #eventdriven | @ConfluentINc
24

25. @gamussa | #eventdriven | @ConfluentINc
25
Integrating
microservices via RPC
● Avoids problems of database integration
● Feels natural
● Aligns with the request/response paradigm
● Problem: cascading failures
● Question: how do you debug this system?
● Answer: you build a log.
🤔

26. @gamussa | #eventdriven | @ConfluentINc
26
Log4j type
of log? 🤔

27. @gamussa | #eventdriven | @ConfluentINc
27
Events?

28. @gamussa | #eventdriven | @ConfluentINc
28
What’s an
event?

29. @gamussa | #eventdriven | @ConfluentINc
A shared narrative
describing the
evolution of the business
over time
29

30. @gamussa | #eventdriven | @ConfluentINc
A combination of:
Notification
State transfer
30

31. @gamussa | #eventdriven | @ConfluentINc
Also,
events
are immutable.
31

32. @gamussa | #eventdriven | @ConfluentINc
32
Core abstraction
● DB - table
● Hadoop - file
● Kafka - ?

33. @gamussa | #eventdriven | @ConfluentINc
33
LOG

34. @gamussa | #eventdriven | @ConfluentINc
34
A Rich
Microservices
Application

35. @gamussa | #eventdriven | @ConfluentINc
35

36. @gamussa | #eventdriven | @ConfluentINc
36
Ratings Service operating on the event stream
Order
Requested
ratings
raw-rating
Ratings
Service
KStreams API
used to count and
sum ratings as
they arrive
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings

37. @gamussa | #eventdriven | @ConfluentINc
37
Schema Management
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
KStreams API
used to count and
sum ratings as
they arrive

38. @gamussa | #eventdriven | @ConfluentINc
38
Access legacy relation data
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
KStreams API
used to count and
sum ratings as
they arrive

39. @gamussa | #eventdriven | @ConfluentINc
39
Materialize tables inside the app
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Lookup table
created inside the
Rating Service
Movies

40. @gamussa | #eventdriven | @ConfluentINc
40
Create a new table, persist it to the log
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Movies
rated-movies
rated-movies
Writable
table
created for
rated
movies

41. @gamussa | #eventdriven | @ConfluentINc
41
Create a new table, persist it to the log
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Movies
rated-movies
rated-movies
TRANSACTION

42. @gamussa | #eventdriven | @ConfluentINc
42
Hydrate a materialized view
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
rated-movies
an embedded
database
CDC

43. @gamussa | #eventdriven | @ConfluentINc
43
What’s new
in Cp 5.2?

44. @gamussa | #eventdriven | @ConfluentINc
44
Message browser
enhancements
Why
Makes reading messages easier for application
development & troubleshooting
Features
● Feed auto-load, scroll, pause, and resume
● Seek to specific offsets (per partition)
● Card & table formats
● Message metadata

45. @gamussa | #eventdriven | @ConfluentINc
45
KSQL UI enhancements
Why
Improves KSQL usability & overall UX
Features
● Feed pause & resume
● Card & table formats
● Clearer query status
● Improved error messaging
● Output metadata

46. @gamussa | #eventdriven | @ConfluentINc
46
Schema management
Why
Enables schema editing & policy management
Features
● Create and edit schemas
● Validate schemas against compatibility
policy
● Manage compatibility policy

47. @gamussa | #eventdriven | @ConfluentINc
47
Dynamic broker
configuration
Why
Helps operators manage their brokers
Features
● Change broker configs that don’t require
restarts

48. @gamussa | #eventdriven | @ConfluentINc
48
License management
Why
Makes managing the subscription license easier and
prevents unexpected expirations
Features
● License checking
● License submission
● Pending expiration alerts (3mo out, 1mo out, weekly
for the last month, daily for the last week)
● Expired license notice

49. @gamussa | #eventdriven | @ConfluentINc
49
Multi-cluster KSQL
Why
Runs queries on 1+ KSQL clusters
Features
● Many-to-1 mapping for KSQL-to-Kafka
Configuration
● confluent.controlcenter.ksql.<name of cluster>.url =
<list of urls>
● confluent.controlcenter.ksql.<name of
cluster>.advertised.url = <list of urls>

50. @gamussa | #eventdriven | @ConfluentINc
50
Multi-cluster Connect
Why
Run, monitor, and manage connectors on 1+ Connect
clusters
Features
● Many-to-1 mapping for Connect-to-Kafka
Configuration
● confluent.controlcenter.connect.<name>.
cluster = <list of urls>

51. @gamussa | #eventdriven | @ConfluentINc
51
My personal favorite

52. @gamussa | #eventdriven | @ConfluentINc
52
Kafka-topics use AdminClient API

53. CP 5.2 is Free
forever on
a single Kafka
broker

54. 54
Stay in touch
cnfl.io/meetupscnfl.io/slackcnfl.io/blog

55. 55