Slides for the `Diving into event-sourcing and event-driven architectures` talk in the barcelona on rails meetup @ badi

1. DIVING INTO EVENT DRIVEN
ARCHITECTURES
BCN ON RAILS

2. UNAI ESTEIBAR
SOFTWARE ENGINEER
@ XING
@UESTEIBAR ON THE INTERNETS

3. TEXT
THIS TALK
▸ Introduction to event-sourcing and event-driven architectures.
▸ Our experience on developing an event-sourced app on rails.
▸ Not rails specific.
▸ Oversimplified explanations.
▸ Gifs.
▸ Bad jokes.

4. WE NEED TO BUILD A STATISTICS APP
TO GATHER AND ANALYSE USERS
ACTIVITY ON THE PLATFORM.
Big boss
LONG STORY SHORT

5. SO… WE’RE GONNA SYNC EVERYTHING
THROUGH REST ENDPOINTS EVERY FEW
MINUTES, RIGHT? THAT SHOULD DO THE
JOB…
Famous last words
FIRST DISCUSSIONS

7. I HEARD ABOUT THIS EVENT
SOURCING THING… SOUNDS
FANCY
Excited developers

9. IN A NUTSHELL

12. ▸ You can query more complex things. Like visits in a certain
time range.
▸ You can build histograms easily.
▸ Data is easier to analyse (useful for providing dashboards
and stats to Product).
GOODIES

13. NOT SO GOODIES
▸ You can end up with *a lot of events*.
▸ I mean *a lot*.
▸ Onboarding can be complicated.

14. BUT IF YOU END UP WITH A LOT
OF EVENTS… WHAT ABOUT
PERFORMANCE?
Anyone, anywhere
FIRST DISCUSSIONS

15. WELCOME TO
CQRS

16. COMMAND
QUERY
RESPONSIBILITY
SEGREGATION

17. WARNING
OVERSIMPLIFIED
EXPLANATION

18. WHAT’S THIS?
SEPARATE WRITE AND READ MODELS
TRADITIONALLY, DATABASES ARE:
▸ Fast to write, slow to read.
▸ Slow to write, fast to read.
CQRS ALLOWS US TO BE FAST TO WRITE AND FAST TO READ.

19. ▸ Save your write model (an event) to a fast to write
database.
▸ Build your projection (read model) in a way that can make
querying easier. (avoiding to do joins, for example).
▸ Write this projection to a fast to read database.
HOW?

20. ▸ Querying the projection is really fast. No more joins.
▸ You can have multiple projections based on the same write
model to accommodate different queries.
▸ If your projection needs more data, you can recreate it
based on your write model.
▸ If your projection is corrupt, you can recreate it as well.
▸ Since the bigger part of your heavy processing is
happening in a worker, it’s much easier to scale.
GOODIES

21. ▸ Data won’t be immediately consistent, but eventually
consistent.
COSTS

22. BUT YOU HAVE DUPLICATED DATA…
WHAT HAPPENED TO HAVING A
SINGLE SOURCE OF TRUTH?
You, right now
NOT SO GOODIES

23. YOU STILL HAVE A SINGLE SOURCE OF TRUTH.
YOUR PROJECTION IS BUILT BASED ON YOUR
ORIGINAL DATA (THE WRITE MODEL) AND CAN (AND
SHOULD) BE RECONSTITUTED AS NECESSARY.

24. LIFECYCLE OF A
DOMAIN EVENT

25. PROFILES
APP
GET /profile/1
WORKER
DB
WORKER
Record projection
(read model)
Record event
(write model)
Publish
profile.visited
DB
Publish
profile.visited.registered

26. THINGS TO KEEP
IN MIND

27. THINGS TO KEEP IN MIND
▸ Payload structure of the published event.
▸ Possibility of losing messages.
▸ Transactionality.

28. PAYLOAD STRUCTURE
▸ As extensible as possible.
▸ Avoid versioning as much as you can.
▸ If you need to version, there’s different strategies:
▸ Deal with the different versions everywhere in the code.
▸ Upcast the old events on first read.
▸ Upcast the entire event store right away.

29. DON’T MISS AN EVENT: PUBLISHING
Publish
profile.visited
APP WORKER
Risk of losing the message
APP
PUBLISHED
EVENTS
STORE
WORKERPublish
profile.visited
As a cron job

30. DON’T MISS AN EVENT: CONSUMING
Consume
profile.visited
WORKER
DEAD LETTER
▸ Try to keep dead letter as independent from your app as
possible.
DB

31. ARCHITECTURAL
OVERVIEW

32. EVENT-BASED APPROACH
APP
A B C D
WORKER
X10000
WORKER WORKER WORKERWORKERWORKERWORKER
DB

34. ▸ Scaling is easy and painless.
▸ Dependencies between apps are (almost) gone.
▸ We avoid the domino effect.
GOODIES
NOT SO GOODIES
▸ Adds more complexity to your architecture.

35. TIPS TO DO THIS IN THE REAL WORLD
▸ Make sure the entire team is on board and have a common
understanding of these techniques.
▸ Explain it to your product team. They will figure out ways
to exploit this that you would never think of.
▸ Your recovery strategy should be painless. One script
should be enough to rebuild your projections.
▸ Stick to the strategy. If you need a new data source, make
it so it comes into your app as an event (e.g. recreating
events for historical data).

36. LINKS
SOME NICE READS TO GO DEEPER
▸ Versioning: http://files.movereem.nl/2017saner-
eventsourcing.pdf
▸ Software Architecture Patterns by Mark Richards (Chapter
2: Event-Driven Architecture)
▸ Exploring CQRS and Event Sourcing by Microsoft (https://
www.microsoft.com/en-us/download/details.aspx?
id=34774)
▸ Implementing Domain-Driven Design by Vaughn Vernon