Event Sourcing is becoming a popular persistence pattern. But when to trade simplicity of relational databases for scalability and high throughput? Analysing the rationale behind Event Sourcing and separating Commands from Queries, will help you make an informed decision aware of benefits and risks. Event Sourcing is becoming a popular persistence pattern for scalable applications, reactive architectures and microservices. The subject is rather new, and we have no well defined, “gang-of-four”-like design patterns yet. Nevertheless, most of the introductory materials imply a particular implementation style. This presentation explains Event Sourcing and CQRS without assuming a particular implementation, to give you the freedom of choosing the pattern most appropriate for your business case. It’s a story, following the much-loved format of a visual picture book. Starting from relational and normalised persistent model, through the issues you face when scaling and moving to asynchronous, distributed application. How to get around these limits with event or command sourcing, why you have to use CQRS and what are your options for queries. The journey ends with benefits, drawbacks and, most importantly, with lessons learned “at the coal face” from real projects.

1. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
From C to Q,
one Event at a time
1
Lorenzo Nicora
OpenCredo
Event Sourcing illustrated

2. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Lorenzo Nicora
@nicusX
lorenzo.nicora@opencredo.com
https://opencredo.com/author/lorenzo/
Senior Consultant @ OpenCredo
• Microservices
• Cloud
• Event-driven, Event Sourcing, Reactive
• Java, Spring, Akka…
2

3. 3
Concepts from DDD
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Aggregate
Event Sourcing,CQRS => DDD/
State: f(t)
Domain Event

4. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 4
Once upon a time…
Everything was Synchronous
Request <-> Response

5. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 5
Scaling out…
Updates —> Locks —> Contention!
<— Block! <—
-> Distributed
still Synchronous

6. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 6
Let’s go Asynchronous
-> Distributed & Message-driven
Request/Response ACID Transactions
• Distributed: Propagation takes time
• Async: No global order strictly guaranteed
Updates applied in the wrong order
=> Inconsistent state!

7. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 7
Command Sourcing
💡
• Append Only —> No Contention, No Update
• Build State from Commands history
K/V Store
Distributed

8. Command
“Submit this Order! (Please)”
-> A request (imperative sentence)
-> May fail, be rejected
-> May affect multiple Aggregates
8
Commands
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
✉
Rebuild Aggregate State
from Commands

9. 9
Events
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Event
“Order submitted”
-> Statement of facts (past tense)
-> Never fails
-> Can’t be changed
-> May be designed
to affect a single Aggregate
✉
Rebuild Aggregate State
from Events

10. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 10
Command > Event Sourcing
💡

11. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 11
Commands to Events
X
Y
Z
?

12. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 12
(Business) Consistency
(ACID) Transactions
Eventual (Business) Consistency
Guess —> Compensate—> Apologies

13. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 13
e.g. Saga
Stateful
Out of band
Corrective Action
(Command / Event)
Saga

14. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
–Greg Young
“Accountants don’t use pencils.
They use pens”
Corrective Actions
14

15. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 15
Benefits of Event Sourcing
History
(for free)
Rebuild State
at a point in Time

16. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 16
Benefits of Event Sourcing
Easier
Eventual Business Consistency
—> Corrective Events
Robust to data corruption
(bugs, malicious, fat fingers…)

17. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 17
Benefits of Event Sourcing
Horizontal Scalability
&
Low Latency writes
-> Distributed systems & data store
—> Append-only Log
—> Asynchronous processing

18. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
18
Commands to Events
Stateful: Events depends 0n current State
Stateless: Validate, split by Aggregate…
Depends on
Business
Domain
x Point of synchronisation
? Out-of-order commands (IoT, Mobile)

19. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 19
Thinking fast or slow
Stateless:
Think fast,
Write fast,
More thinking later
Stateful:
Think slow…(rebuild state),
Write fast,
Less thinking later

20. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
What about reads?
20

21. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 21
Retrieving the State
How do I retrieve the State?
“Get details of Order ‘AB123’”
❔
not very efficient, but…
…may work

22. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 22
Querying (Searching) the State
❓ How do query the State?
“Get all Orders delivered to ‘SE1 0NZ’”

23. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 23
CQRS
Command
Query
Responsibility
Segregation
💡
Separate
• Code
• muService
• Datastore
-> Update
-—> Retrieve

24. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 24
Not a new idea
Specialised
Downstream

25. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 25
CQRS and Event Sourcing
for low-latency writes
Event Sourcing => CQRS

26. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 26
Materialised Views
• In Memory
K/V Store
Graph DB
RDBMS
• Rebuildable
from Events
💡
a.k.a.
Read-optimised Views
Read Views
Projections…

27. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 27
Materialised Views
* Views are optimised for
specific query use cases
—> multiple Views from same Events
* May be updated asynchronously
+ low latency writes, scalability
- delayed
+ may reorder Events

28. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 28
Materialised Views
The Event Log
is your Source of Truth
* Easy to evolve or fix
—> change or fix logic;
rebuild view from events

29. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 29
Indexes
• Search Engines
K/V Stores
Graph DB
+ Optimised for querying (less for retrieving)
+ Reduced delay of State
💡

30. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 30
Hybrid solutions: e.g. Snapshots
+ Speed up rebuilding the State
+ Use recent Events to rebuild up-to-date
💡 Long delayed

31. Distributed + Asynchronous system
Distributed ACID Transactions (2PC)
31
No Global Current State
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
AS
S
A
B
Information propagates
at a finite speed

32. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 32
Lesson
from the Trenches

33. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 33
Lesson Learned #1
If you put data in…
…you will eventually
have to get them out!
The “Query” side
is not secondary

34. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 34
Lessons Learned #2
In old days:
normalising one DB
to support as many queries as possible
With CQRS (also No SQL in general)
multiple denormalised “data stores”
optimised for different queries

35. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 35
Lessons Learned #3
Centralised Event Store
= Event-sourced monolith
Each microservice owns its state
(Event Store)
Microservices:

36. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
+++ Summing up +++
36

37. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 37
ES/CQRS Optimal Use Cases
• High Volume, Low Latency writes
• Out-of-order Commands (IoT, Mobile)
Event Sourcing + CQRS 😋
Domain: stateless Command-> Event

38. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 38
ES/CQRS Drawbacks
x No “One-Size-Fits-All”
—> Multiple “Q” implementations
x Delayed reads
x No ACID Transactions
x Additional complexity (!!!)
🙁

39. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 39
ES/CQRS Benefits
+ No “One-Size-Fits-All”
—> “Q” optimised per use cases
+ Distributed systems (Microservices)
+ Eventual (Business) Consistency
+ History, Temporal queries
+ Robust to data corruption
😀

40. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
That’s all, Folks!
40

41. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
??? Questions ???
41
Thanks.
⏳