Download to read offline
Diving into event-sourcing and event-driven architectures
- 1. DIVING INTO EVENTDRIVEN ARCHITECTURES BCN ON RAILS
- 2. UNAI ESTEIBAR SOFTWARE ENGINEER @XING @UESTEIBAR ON THE INTERNETS
- 3. TEXT THIS TALK ▸ Introductionto event-sourcing and event-driven architectures. ▸ Our experience on developing an event-sourced app on rails. ▸ Not rails specific. ▸ Oversimplified explanations. ▸ Gifs. ▸ Bad jokes.
- 4. WE NEED TOBUILD A STATISTICS APP TO GATHER AND ANALYSE USERS ACTIVITY ON THE PLATFORM. Big boss LONG STORY SHORT
- 5. SO… WE’RE GONNASYNC EVERYTHING THROUGH REST ENDPOINTS EVERY FEW MINUTES, RIGHT? THAT SHOULD DO THE JOB… Famous last words FIRST DISCUSSIONS
- 7. I HEARD ABOUTTHIS EVENT SOURCING THING… SOUNDS FANCY Excited developers
- 9.
- 12. ▸ You canquery more complex things. Like visits in a certain time range. ▸ You can build histograms easily. ▸ Data is easier to analyse (useful for providing dashboards and stats to Product). GOODIES
- 13. NOT SO GOODIES ▸You can end up with *a lot of events*. ▸ I mean *a lot*. ▸ Onboarding can be complicated.
- 14. BUT IF YOUEND UP WITH A LOT OF EVENTS… WHAT ABOUT PERFORMANCE? Anyone, anywhere FIRST DISCUSSIONS
- 15.
- 16.
- 17.
- 18. WHAT’S THIS? SEPARATE WRITEAND READ MODELS TRADITIONALLY, DATABASES ARE: ▸ Fast to write, slow to read. ▸ Slow to write, fast to read. CQRS ALLOWS US TO BE FAST TO WRITE AND FAST TO READ.
- 19. ▸ Save yourwrite model (an event) to a fast to write database. ▸ Build your projection (read model) in a way that can make querying easier. (avoiding to do joins, for example). ▸ Write this projection to a fast to read database. HOW?
- 20. ▸ Querying theprojection is really fast. No more joins. ▸ You can have multiple projections based on the same write model to accommodate different queries. ▸ If your projection needs more data, you can recreate it based on your write model. ▸ If your projection is corrupt, you can recreate it as well. ▸ Since the bigger part of your heavy processing is happening in a worker, it’s much easier to scale. GOODIES
- 21. ▸ Data won’tbe immediately consistent, but eventually consistent. COSTS
- 22. BUT YOU HAVEDUPLICATED DATA… WHAT HAPPENED TO HAVING A SINGLE SOURCE OF TRUTH? You, right now NOT SO GOODIES
- 23. YOU STILL HAVEA SINGLE SOURCE OF TRUTH. YOUR PROJECTION IS BUILT BASED ON YOUR ORIGINAL DATA (THE WRITE MODEL) AND CAN (AND SHOULD) BE RECONSTITUTED AS NECESSARY.
- 24.
- 25. PROFILES APP GET /profile/1 WORKER DB WORKER Record projection (readmodel) Record event (write model) Publish profile.visited DB Publish profile.visited.registered
- 26.
- 27. THINGS TO KEEPIN MIND ▸ Payload structure of the published event. ▸ Possibility of losing messages. ▸ Transactionality.
- 28. PAYLOAD STRUCTURE ▸ Asextensible as possible. ▸ Avoid versioning as much as you can. ▸ If you need to version, there’s different strategies: ▸ Deal with the different versions everywhere in the code. ▸ Upcast the old events on first read. ▸ Upcast the entire event store right away.
- 29. DON’T MISS ANEVENT: PUBLISHING Publish profile.visited APP WORKER Risk of losing the message APP PUBLISHED EVENTS STORE WORKERPublish profile.visited As a cron job
- 30. DON’T MISS ANEVENT: CONSUMING Consume profile.visited WORKER DEAD LETTER ▸ Try to keep dead letter as independent from your app as possible. DB
- 31.
- 32. EVENT-BASED APPROACH APP A BC D WORKER X10000 WORKER WORKER WORKERWORKERWORKERWORKER DB
- 34. ▸ Scaling iseasy and painless. ▸ Dependencies between apps are (almost) gone. ▸ We avoid the domino effect. GOODIES NOT SO GOODIES ▸ Adds more complexity to your architecture.
- 35. TIPS TO DOTHIS IN THE REAL WORLD ▸ Make sure the entire team is on board and have a common understanding of these techniques. ▸ Explain it to your product team. They will figure out ways to exploit this that you would never think of. ▸ Your recovery strategy should be painless. One script should be enough to rebuild your projections. ▸ Stick to the strategy. If you need a new data source, make it so it comes into your app as an event (e.g. recreating events for historical data).
- 36. LINKS SOME NICE READSTO GO DEEPER ▸ Versioning: http://files.movereem.nl/2017saner- eventsourcing.pdf ▸ Software Architecture Patterns by Mark Richards (Chapter 2: Event-Driven Architecture) ▸ Exploring CQRS and Event Sourcing by Microsoft (https:// www.microsoft.com/en-us/download/details.aspx? id=34774) ▸ Implementing Domain-Driven Design by Vaughn Vernon