Event-driven architectures look great on paper (or a whiteboard), but it’s not always clear how to implement them. The transformation from request/response based microservices to event-driven can seem daunting, but we don’t have to stop the world and change everything. Instead, we can transition an existing system gradually. In this talk, we’ll work through such a transition, using Apache Kafka and Python. We’ll learn how to introduce Kafka into an architecture and then gradually use it to make our application more efficient, less coupled, and much easier to evolve.

1. with Python and Apache Kafka
Event-driven Microservices
Dave Klein
dklein@confluent.io twitter.com/daveklein

2. What do we mean by Event-driven?
https:/
/martinfowler.com/articles/201701-event-driven.html
https:/
/www.youtube.com/watch?v=STKCRSUsyP0

3. What do we mean by Event-driven?
1. Event Noti
fi
cation
2. Event-carried State Transfer
3. Event Sourcing
4. CQRS
https:/
/martinfowler.com/articles/201701-event-driven.html
https:/
/www.youtube.com/watch?v=STKCRSUsyP0

4. Noti
fi
cation
Order Placed
Temperature Read
State
{item:123, price: 29.95,
qty: 2}
{temp:29, unit: F,
time: 1606949836369}
Event

5. Kafka
The Diminutive Guide

6. "key": "a1",
"value": {
"eventType": “added-to-cart",
"title": "Kafka Streams in Action”,
"author": "Bill Bejeck",
"price": 44.99
}

7. 1 2 3 4 5 6 7 8 9 10 11
Topic (log)

8. 1 2 3 4 5 6 7 8 9 10 11
Producer

9. 1 2 3 4 5 6 7 8 9 10 11
Producer

10. 1 2 3 4 5 6 7
8 9 10 11
Producer
Consumer

11. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer

12. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer

13. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer
Committed Offset: 3

14. Consumer
1 2 3 4 5 6 7 8 9 10 11
Producer
Committed Offset: 3

15. Consumer
1 2 3 4 5 6 7 8 9 10 11
Producer
Committed Offset: 3

16. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer
Committed Offset: 3

17. Schema Registry
Producer
Consumer
1 2 3 4 5 6 7 8 9 10

18. Using Apache Kafka with Python
kafka-python
KafkaProducer
KafkaConsumer
https:/
/pypi.org/project/kafka-python

19. Using Apache Kafka with Python
con
fl
uent-kafka-python
Producer
Consumer
AvroProducer
AvroConsumer
AdminClient
https:/
/pypi.org/project/con
fl
uent-kafka

20. Back to Event-driven Microservices

21. Potential Current State

25. How do we get there?

26. Find Events

27. https:/
/videos.con
fl
uent.io/watch/uw9bRczCiYMoHTcGJYtjau
Event Modeling

28. Find Events in Existing Systems
What causes a service to start working?
What info does a service need to do its work?
What action does a service perform?
What information results from that action?

29. Next De
fi
ne Your Topics
One topic per event type?
One topic per domain?
Somewhere in-between?

30. Don’t forget the schema!
A schema for every event type
Schemas will change.
Think about compatibility

31. Oh, and one more thing…
Event-driven does not mean No Databases

32. Demo
Build a random pizza generator
With Python and Kafka

33. Pizza Sauce Cheese Meats Veggies
https:/
/github.com/con
fl
uentinc/demo-scene/tree/master/python-microservices

34. https:/
/cn
fl
.io/book-bundle
https:/
/developer.con
fl
uent.io/learn-kafka
https:/
/developer.con
fl
uent.io/get-started/python
dklein@con
fl
uent.io — twitter.com/daveklein
https:/
/github.com/con
fl
uentinc/demo-scene/tree/master/python-microservices