COLABORATIVE
EVENTSOURCING
INTRODUCTION
Bartosz Sypytkowski
▪ @Horusiath
▪ b.sypytkowski@gmail.com
▪ bartoszsypytkowski.com
 Eventsourcing everywhere
 Popular tools and challenges
 Multi-master replication: motivation
 How to…
AGENDA
EVENTSOURCING
EVENTSOURCING
QUICK RECALL
Event stream
E1 E2 E3 E4 E5 E6
EVENTSOURCING
QUICK RECALL
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
T5
Reader B
Writer
E7
EVENTSOURCING
QUICK RECALL
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
T5
Reader B
Writer
E7
Sn = fold(Sn-1, En)
EVENTSOURCING
1. (Usually) one logical writer
2. Many possible readers
3. State is constructed incrementally by
applying (totally ordered) stream of
events over the previous version of
state
4. Asynchronous
READERS &
WRITERS
COMPAR
ING
VECTOR
CLOCKS
EVENTS
1. Describe facts that already happened
2. Undeniable
3. Ordered
4. Side effect free (idempotent)
5. Consumed ≠ Deleted
EVENTSOURCING
SCALLING PRODUCTION WORKLOAD
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
T5
Reader B
Writer
E7
Snapshot store
T3
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
Reader B
Writer
E7
Snapshot store
T3
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
Reader B
Writer
E7
Snapshot store
T3T3
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
Writer
E7
Snapshot store
T3
Reader B
T3
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
Writer
E7
Snapshot store
T3
Reader B
T4
EVENTSOURCING
SNAPSHOTS
Event stream
E1 E2 E3 E4 E5 E6
S1
Reader A
Writer
E7
Snapshot store
T3
Reader B
T5
QUIZ: RELATIONS BETWEEN SNAPSHOT, WRITER,
READER AND EVENT STREAM?
EVENTSOURCING
MULTIPLE
STREAMS
EventLog
E1 E2 E3 E4 E5
S1
Reader A
T5
Reader B
Writer
D1 D2 D3
C1 C2 C3 C4 C5 C6
E7
E6
EVENTSOURCING
SCALLING OUT TO DISTRIBUTED SYSTEM
COMPAR
ING
VECTOR
CLOCKS
EVENT
REPLICATION
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
S3
S3
S2
REPLICATION
EVENTUAL CONSISTENCY
COMPAR
ING
VECTOR
CLOCKS
EVENTUAL
CONSISTENCY
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
read
read
S3
S3
S2
CAN WE EVEN ACHIEVE STRONG CONSISTENCY?
ATTEMPT #1
2-PHASE COMMIT
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2
E3
Follower A
E1 E2
Follower B
E1 E2
S2
S2
S2
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2 E3
Follower A
E1 E2
Follower B
E1 E2
S2
S2
S2
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
S2
S2
S2
E3
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
S2
S2
S2
E3
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
S3
S3
S2
E3
COMPAR
ING
VECTOR
CLOCKS
2-PHASE
COMMIT
Leader
E1 E2 E3
Follower A
E1 E2 E3
Follower B
E1 E2
S3
S3
S2
E3
ATTEMPT #2
REPLICATED WRITE/READ
REPLICATED
WRITE
Leader
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
REPLICATED
WRITE
Leader
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
E3
REPLICATED
WRITE
Leader
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S2
E3
REPLICATED
WRITE
Leader
E1 E2
S3
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S3
Follower
E1 E2
S3
E3
E3
E3
REPLICATED
READ
Leader
E1 E2
S3
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S3
Follower
E1 E2
S3
E3
E3
E3
REPLICATED
READ
Leader
E1 E2
S3
Follower
E1 E2
S2
Follower
E1 E2
S2
Follower
E1 E2
S3
Follower
E1 E2
S3
E3
E3
E3
REPLICATED
READ
Leader
E1 E2
S3
Follower
E1 E2
S3
Follower
E1 E2
S2
Follower
E1 E2
S3
Follower
E1 E2
S3
E3
E3
E3
E3
REPLICATED
READ
Leader
E1 E2
S3
Follower
E1 E2
S3
Follower
E1 E2
S2
Follower
E1 E2
S3
Follower
E1 E2
S3
E3
E3
E3
E3
CONSENSUS
BASED
REPLICATION
1. Latency
2. Geo-replication
3. Network partitions
4. Offline capabilities
5. Single writer bottleneck
PROBLEMS AND
AVOIDANCE
ROUND TRIP
TIMES
Source: https://speakerdeck.com/ept/local-first-software-returning-data-ownership-to-users
CAN WE GIVE UP ON TOTAL ORDER?
UNORDERED LOG
Sam
Alice
Bob
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
UNORDERED LOG
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
COMPAR
ING
VECTOR
CLOCKS
UNORDERED
LOG
1. It’s hard to detect conflicts and
potentially invalid states …
2. … and event harder to correct them
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Alice: Asian restaurant?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Alice: Asian restaurant?
CAUSALITY
VIOLATION
Disallowed!
ILLEGAL STATES CAN HAPPEN EVEN WITHOUT TIME
TRAVELING
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?CAUSALITY
VIOLATION
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?CAUSALITY
VIOLATION
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSALITY
VIOLATION
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Bob: Foodtruck? Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSALITY
VIOLATION
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Bob: Foodtruck? Sam: Where are we going
for lunch?
Bob: Foodtruck?
WTF?
CAUSALITY
VIOLATION
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSALITY
VIOLATION
CAUSAL ORDERING
PARTIAL LOG
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Keep original sender
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Deduplicate
CAUSAL ORDER
Sam
Alice
Bob
Sam Alice Bob
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
Sam: Where are we going
for lunch?
Bob: Foodtruck?
COMPAR
ING
VECTOR
CLOCKS
DEDUPLICATION
1. Unique ID for each event…
2. Timestamps for causality tracking
PROBLEM: WALL CLOCK TIME DOESN’T TELL US
ANYTHING ABOUT CAUSALITY
TRACKING CAUSAL ORDER
WITH VECTOR CLOCKS
VECTOR
CLOCK
Alice 2
Bob 3
Sam 1
Unique node ID Sequence Nuber
COMPAR
ING
VECTOR
CLOCKS
CAUSAL
ORDERING
Equals Greater than
Less than Concurrent
A 2
B 3
C 1
A 2
B 3
C 1
A 2
B 3
C 2
A 2
B 3
C 1
A 2
B 2
C 1
A 2
B 3
C 1
A 2
B 3
C 1
A 2
B 2
C 2
=
=
=
=
=
>
=
<
=
=
>
<
Greater than
A 2
B 3
C 2
A 2
B 3
C 1
=
=
>
VECTOR
CLOCK
MERGE
MAX ( , )
=
A 1
B 3
A 2
B 1
C 1
A 2
B 3
C 1
PEER TO PEER REPLICATION
PEER
REPLICATION
Alice Bob
{ A:2, B:1 } { A:0, B:3 }
B1 B1
A1
A2
B2
B3
PEER
REPLICATION
Alice Bob
{ A:2, B:1 } { A:0, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
PEER
REPLICATION
Alice Bob
{ A:2, B:1 } { A:0, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:0, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A: 0, B: 3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:0, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
Sam: Sure, these
are the events you
may be interested
in.
A1
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:1, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
A1
Sam: Sure, these
are the events you
may be interested
in.
A1
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:1, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
A1
Sam: Sure, these
are the events you
may be interested
in.
A1
Sure, these are the events you may be interested in.
A1 A2
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:1, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
A1
Sam: Sure, these
are the events you
may be interested
in.
A1
Sure, these are the events you may be interested in.
A1 A2
Ben: A1? I’ve seen it already.
PEER
REPLICATION
Alice Bob
{ A:2, B:3 } { A:1, B:3 }
B1 B1
A1
A2
B2
B3
Can you give me all events, which happened
since { A:2, B:1 } ?
Sure, these are the events you may be interested in.
B2 B3
B2
B3
Can you give me all events, which
happened since { A:0, B:3 } ?
A1
Sam: Sure, these
are the events you
may be interested
in.
A1
Sure, these are the events you may be interested in.
A1 A2
A2
Ben: A1? I’ve seen it already.
{
streamId: “chat/general”,
version: {A:1,B:1,C:2},
origin: A,
data: <user data>
}
STANDARDT EVENT
{
streamId: “chat/general”,
sequenceNr: 98300211,
data: <user data>
}
REPLICATED EVENT
* both models are simplified
VERSIONING STATE
INTERACTIVE MODEL (GIT-LIKE)
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S0
{A:0, B:0}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
commit(S1, {A:1, B:0})
Alice
S1
{A:1, B:0}
S0
{A:0, B:0}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
Bob
S1
{A:1, B:0}
S0
{A:0, B:0}
commit(S2, {A:0, B:1})S2
{A:0, B:1}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S1
{A:1, B:0}
S0
{A:0, B:0}
S2
{A:0, B:1}
pull(ALICE)
Bob
S2
{A:0, B:1}
S1
{A:1, B:0}
Manual
resolution
required
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S1
{A:1, B:0}
S0
{A:0, B:0}
S2
{A:0, B:1}
S2
{A:0, B:1}
S1
{A:1, B:0}
commit(S3, {A:1, B:2})
Bob
S3
{A:1, B:2}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S1
{A:1, B:0}
S0
{A:0, B:0}
S2
{A:0, B:1}
S2
{A:0, B:1}
S1
{A:1, B:0}
S3
{A:1, B:2}
commit(S4, {A:2, B:0})
Alice
S4
{A:2, B:0}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S1
{A:1, B:0}
S0
{A:0, B:0}
S2
{A:0, B:1}
S2
{A:0, B:1}
S1
{A:1, B:0}
S3
{A:1, B:2}
pull(Bob)
Alice
S4
{A:2, B:0}
S3
{A:1, B:2}
S4
{A:2, B:0}
COMPAR
ING
VECTOR
CLOCKS
INTERACTIVE
STATE
RESOLUTION
S1
{A:1, B:0}
S0
{A:0, B:0}
S2
{A:0, B:1}
S2
{A:0, B:1}
S1
{A:1, B:0}
S3
{A:1, B:2}
pull(Bob)
Alice
S4
{A:2, B:0}
S3
{A:1, B:2}
S4
{A:2, B:0}
Conflict
S1/S2 was
already
resolved
VERSIONING STATE
AUTOMATIC MODEL
“AUTOMATIC”: IN CASE OF CONFLICT IT WILL ALWAYS
PREFER ONE (AND THE SAME) OPERATION OVER
ANOTHER
ADD-WINS SET Alice
ADDING A VALUE
follow(@elon)
command
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
ADD-WINS SET Alice
ADDING A VALUE
followed(@elon)
commit
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
ADD-WINS SET Alice
ADDING A VALUE
followed(@elon)
commit
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
ADD
{ A:1 }
ADD-WINS SET Alice
ADDING A VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
E1
{ A:1 }
apply
replicate
E1
{ A:1 }
apply
replicate
ADD
{ A:1 }
apply
replicate
ADD-WINS SET Alice
ADDING A VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
ADD
{ A:1 }
apply
{
“Alice”: [{A:1}]
}
ADD-WINS SET Alice
ADDING A VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
unfollow(@elon)
command
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
unfollowed({A:1})
commit
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
unfollowed({A:1})
commit
REM
{ A:2 }
followed(@elon)
commit
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
E2
REM
apply
followed(@elon)
commit
{ A:2 }
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
[
“Alice”: [{A:1}]
]
{
“Alice”: [{A:1}]
}
E2
REM
apply
replicatereplicatereplicate
{ A:2 }
ADD
{ A:1,B:1 }
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
{
“Alice”: [{A:1}]
}
ADD
apply
REM
{ A:1,B:1 }
replicate
{ A:2 }
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
apply
{
“Alice”: [
{A:1},
{A:1, B1}
]
}
REM
{ A:2 }
ADD
{ A:1,B:1 }
apply
ADD-WINS SET
Alice
ADDING /
REMOVING
VALUE
@elon/followers
Server A
Event
Log
@elon/followers
Server B
Event
Log
{
“Alice”: [
{A:1, B1}
]
}
{
“Alice”: [
{A:1, B1}
]
}
NEXT
STEPS…
1. How to optimize reading events by
vector clock?
2. Transactions
3. Batching and compression of events
SUMMARY
 What eventsourcing is all about: https://martinfowler.com/eaaDev/EventSourcing.html
 Eventuate: http://rbmhtechnology.github.io/eventuate/
 CRDTs for collaborative text editing: https://www.slideshare.net/BartoszSypytkowski1/collaborative-text-
editing-132892964
REFERENCES
THANK YOU

Collaborative eventsourcing