Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemotion Rome 2019
The document discusses the risks and considerations surrounding event sourcing in software architecture, including performance, scalability, and eventual consistency challenges. It explores different levels of event sourcing and CQRS implementations, along with various alternatives and best practices. The presentation also emphasizes the importance of schema evolution and highlights the complexities of scaling event-driven systems.
Introduction to the event sourcing concept and potential issues involved, presented by Andrzej Ludwikowski.
About the presenter, Andrzej Ludwikowski, and affiliations with SoftwareMill.
Explanation of event sourcing via examples of database state and logs of state changes.
Historical overview of event sourcing, mentioning ancient practices and its emergence in modern applications.
Advantages of event sourcing including complete logging of state changes, improved debugging, performance, and scalability.
Introduction of Command Query Responsibility Segregation (CQRS) in the context of event sourcing.
Different levels of event sourcing implementation along with their advantages and disadvantages.
Discussion of alternatives to event sourcing, such as change data capture, and additional relevant concepts.
Different methods and frameworks for implementing event sourcing and relevant considerations.
Various methods of event storage suitable for event sourcing like RDBMS, MongoDB, Kafka, and more.Discussion on serialization formats for events focusing on binary, text formats, and their implications. Managing schema evolution's importance in serialized events, particularly with Avro and Protobuf.
Managing state updates, handling duplicates, and employing compensation actions in event-sourced applications.
Challenges in distributed systems such as split-brain scenarios and best practices for clustering.
Summary of key takeaways from the presentation on event sourcing and its related complexities.
What is EventSourcing?
DB
Order {
items=[itemA, itemB]
}
9.
What is EventSourcing?
DB
DB
Order {
items=[itemA, itemB]
}
ItemAdded(itemA)
ItemAdded(itemC)
ItemRemoved(itemC)
ItemAdded(itemB)
10.
History
● 9000 BC,Mesopotamian Clay Tablets,
e.g. for market transactions
11.
History
● 2005, EventSourcing
“Enterprise applications that use Event Sourcing
are rarer, but I have seen a few applications (or
parts of applications) that use it.”
12.
Why Event Sourcing?
●complete log of every state change
● debugging
● performance
● scalability
13.
ES and CQRS
CommandService
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
14.
ES and CQRSlevel 1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Transaction
15.
ES and CQRSlevel 1
● Entry-level, synchronous & transactional event sourcing
● slick-eventsourcing
16.
ES and CQRSlevel 1
+ easy to implement
+ easy to reason about
+ 0 eventual consistency
- performance
- scalability
17.
ES and CQRSlevel 2
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
Transaction
18.
ES and CQRSlevel 2
+/- performance
+/- scalability
- eventual consistency
- increased events DB load
- lags
19.
ES and CQRSlevel 3
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
Transaction
event
bus
20.
ES and CQRSlevel 3.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
21.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
22.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
At-least-once delivery
23.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
24.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
25.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
26.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
27.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
28.
ES and CQRSlevel 3.1.1
Command Service
Domain
Events
Client
Query Service
Data access
Commands Queries
Read
modelRead
modelRead
models
Updater
event
bus
Transaction
?
29.
ES and CQRSlevel 3.2
Events
Client
Query Service
Data access
Commands
Queries
Read
modelRead
modelRead
models
Updater
event
bus
Command Service
Domain
Command Service
Domain
Command Service
Domain
Transaction
`
Sharded
Cluster
30.
ES and CQRSlevel 3.x
+ performance
+ scalability
- eventual consistency
- complex implementation
31.
ES and CQRSalternatives
● Change Capture Data (CDC) logging instead of event bus?
● event bus instead of DB?
32.
Not covered butworth to check
● Command Sourcing
● Event Collaboration
akka-persistence-jdbc trap
val theTag= s"%$tag%"
sql"""
SELECT "#$ordering", "#$deleted", "#$persistenceId", "#$sequenceNumber",
"#$message", "#$tags"
FROM (
SELECT * FROM #$theTableName
WHERE "#$tags" LIKE $theTag
AND "#$ordering" > $theOffset
AND "#$ordering" <= $maxOffset
ORDER BY "#$ordering"
)
WHERE rownum <= $max"""
Cassandra perfect forES?
● partitioning by design
● replication by design
● leaderless (no single point of failure)
● optimised for writes (2 nodes = 100 000 tx/s)
● near-linear horizontal scaling
49.
ScyllaDB ?
● Cassandrawithout JVM
○ same protocol, SSTable compatibility
● C++ and Seastar lib
● up to 1,000,000 IOPS/node
● not fully supported by Akka Persistence
Plain text Binary
human-readabledeserialization required
precision issues (JSON IEEE 754, DoS) -
storage consumption compress
slow fast
55.
Plain text Binary
human-readabledeserialization required
precision issues (JSON IEEE 754, DoS) -
storage consumption compress
slow fast
poor schema evolution support full schema evolution support
Avro writer schemasource
● add schema to the payload
● custom solution
○ schema in /resources
○ schema in external storage (must be fault-tolerant)
○ Darwin project
● Schema Registry
ES with RODO/GDPR
●“right to forget” with:
○ data shredding (and/or deleting)
■ events, state, views, read models
○ retention policy
■ message brokers, backups, logs
○ data before RODO migration
109.
ES and CQRSlevel 3.2
Events
Client
Query Service
Data access
Commands
Queries
Read
modelRead
modelRead
models
Updater
event
bus
Command Service
Domain
Command Service
Domain
Command Service
Domain
Transaction
Sharding
Clustering
Cluster best practises
●remember about the split brain
● very good monitoring & alerting
● a lot of failover tests
● cluster also on dev/staging
● keep it as small as possible (code base, number of nodes, etc.)
117.
Summary
● carefully chooseES lib/framework
● understand event/command/state schema evolution
● eventual consistency is your friend
● scaling is complex
● database inside-out
● log-based processing mindset