Les architectures orientées événements (ou EDA) sont perçues comme des entités magiques qui transforment instantanément vos systèmes en systèmes « temps réel » ! MAIS, en y réfléchissant, ne sont-ils pas déjà « temps réel » ? Je veux dire par là qu’ajouter un article dans un panier est pratiquement instantané dans la plupart des boutiques en ligne.
En fait, une EDA résout un ensemble de problèmes totalement différents et, en faisant appel à Apache Kafka, nous allons suivre la voie de l’évolution (ou de la révolution).
Les microservices sont faciles à prendre en main, mais une fois que c’est fait, on butte toujours sur les mêmes problèmes : l’accès aux données, la cohérence et les échecs (cela vous parle ? ).
La solution ? Les modèles, les modèles, rien que les modèles… Vous avez déjà dû entendre parler des notions de « Event Notification », d’« Event-carried State Transfer » ou même d’« Event Sourcing », mais comment les utiliser pour résoudre vos problèmes ? Et, plus important, comment peut-on utiliser Apache Kafka pour tirer parti de ces modèles ?
C’est ce que allons découvrir.
Introduction: name, company
Myth: Event-driven architectures are all about being real-time / ETL
There are specific use cases for real-time data such as fraud detection ( atm example ), sensors reading ( smoke alarms )
EDA solves a different kind of problems, something that is much more powerfull
Theory is good, practice is even better: Apache Kafka
Start with the beginning
I think that everyone knows about monoliths
It’s a simple but powerfull pattern
All code is encapsulated into a single running process
Usually, a database is attached to it ( hundreds of tables )
Monoliths are great but, there is a but
All problems condenses -> scalability
Application scalability: the app must server more and more requests, increase instance
Team scalability: 20-30 teams working on the same code repository; a lot time wasted due to communication
- Some very smart persons come up with the idea of splitting up the monolith
Service oriented architecture
Code is split into multiple modules
It was hard to properly define the borders of each service
Domain-driven microservices
Domain -> sub domains -> bounded contexts -> aggregates -> cool stuff
Each microservice should have ownership over data
Different databases: relational, nosql: document-based, key value stores
Start monolith -> solves
In software engineering we don’t have good or right, bad or wrong, we have tradeoffs.
New challenge
It is really hard to access the data you need
Data is encapsulated by each microservice
One customer makes a request -> cascade of other requests to aggregate data
Challenges
Coupling
Event-driven architectures to the rescue
Haven’t told what event-driven architectures are
- change of paradigm from data to events
Microservices are typically data driven: each microservice encapsulates data
Events on the other hand promote a concept called data liberation
events is the centerpiece
definition
- simplify
-make it unidirectional
-transpose it event-driven
New concept: event stream
Event stream -> middleware to facilitate transport of events
2 types of applications: producers and consumers
Create and send events
Retrieve and process events
Protocols and tools
Anything that you can asynchronously subcribe to
Examples
Reactive events: once consumed, they can never be brought back;
Persistsed events: if something fails during consuming process, can replay
Event streams as topics in Kafka
4 Key observations
Inversion of control -> spring example; lis a good thing; REST-based producer pushes data, event-driven consumers has to retrieve it; producer only pushes it to the event stream
Fan out -> multiple consumers can retrieve events from the same event stream; data is accesible; it is there to be taken
1 producer type per event stream -> difference between event stream and a topic; a topic can persist anything; data structure must be in place
Producers dont know consumers: from application perspective; architecture perspective everything is well known
Event-driven Architectures solves the data access problem
Software engineering is about tradeoffs
Syncronous APIs -> we got used to Strong Conistency
Microservices have strong consistency
Not all use cases can be made event-driven
Typical system is a mix between event-driven and synchronous APIs
Talk later about why eventual consistency matters
What is an event?
A message is just data exchanged between 2 services
Data must have a structure
A message may have differnet forms
Patterns of EDA
2 types of events
Action that has happened or has been triggered
It must be a meaningfull action to the busines
Cusomer Logins to our mobile application -> customer takes a photoTalk about e-commerce example; suggestions carousel
Great example -> No response
Very similar; Difference nuance compared to event notification
It introduces coupling
Let me say that again
Data is scattered across multiple services
Workaround: Copying parts of data from one database to another
Challenges:
-how to keep it in sync?
How to make sure we have the latest versions of the data?
Distributed systems needs another
Most powerfull pattern, but also very demanding
It all comes down to an event log
Event log
State is called materialized view
Eventual consistency -> not all consumers retrieve the data at the same time
Can always discard the entire database and rebuild it
Barely scratched the surface: there are a lot of things to learn
Change Data Capture, streaming pattern
Thank you for joining this essions
Feel free to get in touch with me