Inari has undergone a deep transformation in terms of sofware architecture, moving from Uncle Bob's Clean Architecture to a more scalable CQRS + ES architecture. This architecture very easily gave our solution the perfect traceability and scalability that we required, and we wanted to share the lessons we learnt when refactoring our solution to CQRS + ES

1. Event sourcing and CQRS:
Lessons from the trenches
(Bonus: with Blockchain)
David Jiménez Martínez
davigetto@gmail.com
http://www.linkedin.com/in/davjim/
http://github.com/Rydra/
inari.io

2. ●
Assuming certain knowledge of what CQRS
and ES are
●
Based on our experience
●
“What you see and learn is what you
believe”
●
No zealotry. Software development is
more of an art, a craftmanship, than a
science. Your craft will be diferent than
my craft probably.
●
Feedback is a gift which improves your
craft
About this talk...

3. ●
Lead developer/Chief architect @ Inari
●
Created Melange, an OSS infrastructure
messaging library written in python to
create event-driven, distributed
architectures
●
(https://github.com/Rydra/melange)
– Used in 21Buttons (social network with
high concurrency), Inari (insurtech high-
traceability application supported by AI
and blockchain)
●
An ex .NET developer who found love in
Python and DDD
Who am I?

4. Quick review at concepts

5. ●
CQRS is an architectural pattern coined by Greg Young
– More info: https://martinfowler.com/bliki/CQRS.html
– In fact it is very simple: Commands (state-changing operations) are separated from
queries (retrieve-only operations). They can have a diferent technology stack and
patterns (and they might live in the same monolith if you see it ft, but the concept is
free from any specifc infrastructure implementation).
– My ELI5 explanation: (usually) you have at least two databases, one to write, and one
to read quickly/do searches (which gets updated some time after the other database
gets written).
●
(In the case of Inari, Blockchain and Postgres/Elasticsearch)
CQRS

6. CQRS
CQRS, a simplifed view

7. ●
Event sourcing is a technique in which, instead of persisting the current state of
an aggregate root, you persist the events that have happened for that aggregate.
●
This concept and its application has MASSIVE implications in terms of
architecture and implementation when refactoring (as we are going to see)
Event sourcing

8. Event sourcing

9. ●
We were following the Clean Architecture (the architecture proposed by Uncle
Bob)
●
ELI5: Make the use case/interactor the central part of your system and the main
entry point for your domain. From that point forward, and with the help of
dependency injection, you would coordinate your domain
entities/repositories/services, shielding your domain and business rules (the real
value of your application) from volatile details like framework, IO...
Clean architecture

10. Clean architecture
CLEAN ARCHITECTURE

11. Event sourcing implementation @ Inari

12. ●
TERRAIN SCOUT:
– Requirement: as an InsurTech, our solution requires high traceabililty and
auditability. Every event, every action needs to be recorded in our platform.
– Requirement: we need to provide an historic of everything that has happened in the
application, for each aggregate root.
– Requirement: Our tech stack is backed up by Blockchain. Blockchain is a
decentralized, IMMUTABLE database, very hard to hack or manipulate (once you
write there you cannot rewrite or delete data).
ES: The objectives of this war

13. STEWING SAUCE FOR EVENT SOURCING!
ES: The objectives of this war

14. ●
The promised land:
– It could bring us the high traceability and auditability that we need!
– It could help us defne clearly the events of our system (trust me, giving names and
defne the events of our domain was a big challenge). Refer to Event Storming.
– It could help us separate better the diferent aggregate roots of our domain.
– It could help us making migrations easier.
– It could favor the CQS we were looking for (Fine tune performance in both read side
and write side separately)
ES: The treasure we’re hunting

15. ES: The treasure we’re hunting

16. What’s the worst that could happen?

17. ●
We were following the Clean Architecture
●
Our interactors: e.g. GetProductInteractor, GetCompanyInteractor,
RejectProductInteractor, AddPaymentInteractor…
●
Our entities were classic OO classes with behaviour (and some events thrown
here and there). Repository pattern to abstract persistence.
●
Storage technologies: blockchain (where we were storing jsons with the data)
and postgres (where we would store the same data and make searches there).
What we were before...

18. What we were before...

19. What we were before...

20. What we were before...

21. We know our status quo, and we know where we want to be. Let’s design a
strategy!
What we were before...

22. ●
We started with the main, central aggregate of our application: the Product
– The aggregate root should orchestrate all behaviour. And each, every single
state-modifying method must result in one or several event objects
(logic, otherwise we would not be able to reconstitute this object again).
– Every event relevant to the aggregate should have a method to
reconstitute internal data from the event (called mutator)
– NOTE: No need to move every aggregate to be event sourced at once!
Moving to ES: The strategy

23. – An event-sourced aggregate needs to alter its constructor: it should only
receive a single parameter, the event stream (which is no more than an
array of events with a version number for optimistic concurrency)
– The constructor would then rehydrate the properties of the object
through the events
– Once the aggregate is in position, the repository should also change to be
able to store the events into the blockchain (our event store)
Moving to ES: The strategy

24. ●
Baby steps:
1) First, remodel internally the aggregate without touching the constructor (BIGGEST
step by far)
2) Then, transform blockchain into an event store and save the events into the it (while
still saving the data in the old way)
3) Remodel the constructor to accept an EventStream instead of a list of properties to
fll
4) Change callers
Moving to ES: The tactics

25. ●
Baby steps:
1) First, remodel internally the aggregate without touching the constructor
(BIGGEST step by far)
2) Then, save the events into the blockchain in the form of events while still saving the
data in the old way
3) Remodel the constructor to accept an EventStream instead of a list of properties to
fll.
4) Change callers
Moving to ES: The war

26. The war: Talk is cheap, show me the code

27. The war: Talk is cheap, show me the code

28. ●
It forced us, for every state-changing method, to think of a suitable event and
a name for it and make it explicit in our code
●
Great help in expanding our ubiquitous language with new terms and even
realize some design/requirement mistakes
●
The Product entity turned out to be a little more complex and less stream-
lined than before from our POV, (maybe because we are not yet used to that
thing, like a lot of things in life?)
Moving to ES: The war

29. ●
Baby steps:
1) First, remodel internally the aggregate without touching the constructor (BIGGEST
step by far)
2) Then, transform blockchain into an event store and save the events into the it
(while still saving the data in the old way)
3) Remodel the constructor to accept an EventStream instead of a list of properties to
fll.
4) Change callers
Moving to ES: The war

30. The war: Talk is cheap, show me the code

31. The war: Talk is cheap, show me the code

32. ●
Save the events into the blockchain in the form of events while still saving the
data in the old way
– We needed to make serializers for each event in the system (serialize the
events to json)
– Still save the data in the old format elsewhere (postgres + a huge horrible
json into another blockchain setup)
– The repository pattern helped us to mask the access to the event store (and
therefore not coupling too much the technology stack with our business
domain)
Moving to ES: The war

33. ●
Baby steps:
1) First, remodel internally the aggregate without touching the constructor (BIGGEST
step by far)
2) Then, transform blockchain into an event store and save the events into the it (while
still saving the data in the old way)
3) Remodel the constructor to accept an EventStream instead of a list of
properties to fll.
4) Change callers
Moving to ES: The tactics

34. The war: Talk is cheap, show me the code

35. ●
Remodel the constructor to accept an EventStream instead of a list of
properties to fll
– Changing the constructor was easy… the hard part was to change all the callers.
Relied on the Factory method pattern to solve and refactor this
– Unit/Integration testing the aggregate became a little bit diferent. When
constructing it, you build it in terms of the events that happened for it instead of the
value of the properties you want to supply. Once again, we relied on factory methods
to simplify building event sourced aggregates
– For unit testing we added assertions where we would check that a certain/s event/s
were raised
Moving to ES: The war

36. ●
Better domain and ubiquitous language thanks to be able to properly defne good
event names and defne new ones.
●
Better aggregate isolation
●
Free timeline/historic functionality for the aggregate
●
Forced us to think more in terms of Queries and Commands (CQRS). We are slowly
translating and isolating our interactors between commands (which execute
behaviours and write into blockchain) and queries (which we fne-tune and couple it a
bit more to our storage solution to ofer maximum performance)
●
The solution is a bit more complex, but nothing otherworldly once you have the
infrastructure in motion
●
Moving the Product aggregate was a success, so we can continue moving other
aggregates
Aftermath...

37. ●
Yeah, blockchain is immutable. You cannot delete data from it, you cannot
update data… YOU CANNOT DO MIGRATIONS ON IT!
●
What if you want to do a refactor in your source code (like changing property
names of events, create new ones, delete some of them, etc…?)
– We used Protocol Bufers (from Google). Protobuf ofers backwards compatibility
with message serialization (one part of what we needed).
– New data = create scripts to add new events to correct any desynchronizations.
– Yeah, those “correcting events” need to be considered by your aggregate.
●
Other complex techniques: Upcasting
Moving to ES: But… Postwar

38. ●
Performance
– If you have a lot of events for each aggregate… This could hurt performance!
– Snapshots
– For the time being we are not dealing with a huge amount of events per aggregate
(100 at most), so we are surviving the storm for now without snapshooting.
Moving to ES: But… Postwar

39. ●
Not really unless you have either big concurrency (e.g. a social network) or
high traceability and auditability requirements
●
Though it helps a ton in improving the ubiquitous language through the forced
extensive use of events
Would I recommend CQRS + ES?

40. ●
Microservices: one single, central event store vs one event store per service?
– “Should the book of your history be written in a single book or distributed in several
ones?”
– Possible implementations? Is there any silver bullet or are there any good solutions?
Next challenges for Inari

41. ●
Implementing Domain Driven Design (Vaughn Vernon)
●
Clean Architecture (Robert C. Martin)
●
Protocol Bufers vs Apache Thrift (https://es.slideshare.net/IgorAnishchenko/pb-vs-thrift-vs-avro)
●
Event sourcing versioning (Greg Young) (https://leanpub.com/esversioning)
●
Implementing Event sourcing in Python (
https://breadcrumbscollector.tech/implementing-event-sourcing-in-python-part-1-aggregates/)
●
Event sourcing in Python (https://hackernoon.com/1-year-of-event-sourcing-and-cqrs-fb9033ccd1c6)
●
Clean architecture in Python (https://speakerdeck.com/enforcer/clean-architecture-in-python?slide=30)
●
Melange, a messaging library written in Python to create event driven, distributed applications and
microservices (https://github.com/Rydra/melange)
Bibliography