Event Sourcing from the Trenches (DDD Europe 2020)

Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver

Command Handlers
Commands
Domain Model
Event Store
Events
App
Query Store
Data Access
Projectors
Events
Query HTTP API
Projections
Events
Command HTTP API
Great unit
of testing
Great unit
of testing
Auditability
comes for free
Can look at
the state in
the past
Can be scaled
independently
Autonomous
projections
Aligns well
with Event
Storming
Forces you to
understand your
domain thoroughly
No relational
table structure
anymore.
Can replay old
data against new
rules
More difficult to
envisage domain
relationships
Great for
replication

Application
Command
Service
Correct
Customer
Email Handler
Customer
Unit of
Work
Projector
Data
Access
Layer
Read
Database Write
Database
CorrectCustomerEmailCommand
HTTP API / In-process invocation
Get<Customer>(identity)
Correct Email
Event Store
Load(events)
Apply
Get changes
CustomerEmailCorrectedEvent
Submit changes
History
Dennis Doomen | @ddoomen | The Continuous Improver
NES
Aggregates.NET
SimpleDomain
NStore
EventStore
NEventStore
SQLStreamStore
NStore
NES
Marten
Handelier
Brighter
MediatR (*)
Projac
LiquidProjections (*)

Application
Domain
NoSQL / RDBMS
OR/M / DAL
Web UI, HTTP API,
etc
Lucene Index
Document Projector
Web UI, HTTP
API, etc
Web UI, HTTP API, etc
Domain
Commands
Events
Event StoreProjections
Projectors
Uses Event
Sourcing
Uses traditional
CRUD
architecture
Indexing-based
architecture
Subcribe
to
webhooks
Coarse-
grained
HTTP
requests.
Bus
Subscribe
Publish coarse-
grained event

Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Treat as a
message, not a
type
Order #456
Correct Shipping Address
Command Handler
Application
Customer Shipping Address
Corrected Handler
Order Redirected
Event.
Identity !=
natural key
Identify
partition key
Primitive types
only
Avoid terms like
Create, Update,
Delete, Change.
Avoid property
change events
Don’t use as
inter-domain
contracts
Don’t expose
outside the
domain

Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Order #456
Correct Shipping Address
Command Handler
Application
Order Redirected
Event

Command Service
Some Command
Handler
Customer #123
Event Store
Customer Created
Event
Get<Customer>(“123”)
Customer
Created Event
Converter
Customer Enrolled
Event
May split or
merge events
Can also run as
part of migration
May change the
identity
Unaffected Events

Event
Store
Projector Projector
RDBMS
Subscribe
Subscribe
Document
DB
Projector
RDBMS
Subscribe
Raw SQLNHibernate RavenDB
Lookup
Autonomous &
independent
Storage technique
optimized for
projection
No joining of
projections
Avoid reusing the
projections for
multiple purposes
Use aggressive
caching during
rebuilds.
Caching strategy
optimized for
projector
Owned by
projector
Stream-by-stream
projections during
rebuilds

var mapBuilder = new EventMapBuilder<MyContext, DocumentProjection, string>();
mapBuilder
.Map<WarrantAssignedEvent>()
.AsCreateOf(anEvent => anEvent.Number)
.Using((document, anEvent) =>
{
document.Type = "Warrant";
document.Kind = anEvent.Kind;
document.Country = anEvent.Country;
document.State = anEvent.InitialState;
});
mapBuilder
.Map<StateTransitionedEvent>()
.When(anEvent => anEvent.State != "Closed")
.AsUpdateOf(anEvent => anEvent.DocumentNumber)
.Using((document, anEvent) => document.State = anEvent.State);

Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
INSERT/UPDATE … WHERE …
Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
Unit of Work

private async Task<ExceptionResolution> OnException(ProjectionException exception, int attempts)
{
if (IsTransient(exception))
{
if (attempts < 3)
{
await Task.Delay(TimeSpan.FromSeconds(attempts ^ 2));
return ExceptionResolution.Retry;
}
return ExceptionResolution.Abort;
}
else
{
if (exception.TransactionBatch.Count > 1)
{
return ExceptionResolution.RetryIndividual;
}
// Mark projection as corrupt
return ExceptionResolution.Ignore;
}
}

Event Store
Projector
Subscription
Projection
Subscribes to events
from multiple
aggregates
Lookup

Event Store
Other Projector
Lookup Projector
Subscription
Subscription
Storage
(persistent)
Lookup
Event
Modifier
Modifies events
with lookup data
Receives
extended events

var stats = new ProjectionStats(() => DateTime.UtcNow);
stats.StoreProperty("CountByDocument", "some setting key", "some value");
stats.LogEvent("CountByDocument", "some significant thing that happened");
stats.TrackProgress("CountByDocument", currentCheckpoint);
float? transactionsPerSecond = stats.GetSpeed("CountByDocument");
TimeSpan? eta = stats.GetTimeToReach("CountByDocument", targetCheckpoint);
public void Configure(IAppBuilder builder)
{
builder.UseStatistics(stats);
}

IAppBuilder builder = ….
builder.UseStatistics(stats);
GET http://localhost/projectionStats/CountByDocument
{
"projectorId": "CountByDocument",
"lastCheckpoint": 1000,
"lastCheckpointUpdatedUtc": "2018-05-10T10:39:00Z",
"properties": [{
"key": "some setting key",
"value": "some value",
"lastUpdatedUtc": "2018-05-10T10:39:00Z"
}],
"eventsUrl": "http://localhost/projectionStats/CountByDocument/events"
}

Document #1
Created Event
Event Store
Graph
Projector
Document #1
Closed Event
(other events)
Projector with
active
projections
Archiving
Projector
Start archiving
Document #1
Marked As
Archivable Event
Mark all events
as archivable
Tracks dependencies
between documents
Deletes
projections
related to
Document #1
Can skip all
archivable events
during next
rebuild.

Event Store
Lucene
Projector
Document #1
Marked As
Archivable Event
Allows projectors
to clean up
Lucene Index
Take snapshot
Purge events
Tombstone
$tombstone
stream
Application
Document
Projector
Search.
Tracks deleted streams
for future references
Stream Tombstoned
Event
Search

Event Store
Projector
Application
Version 2
Owns schema
(e.g. FluentMigrator)
Version 1
X
Involves down-time until
projections are rebuild

Event Store
Projector
Application Application
Network Load Balancer
Event Store
Version 1 Version 2
events
Projection ProjectorProjection
bring off-line
Returns HTTP 503
(Service Unavailable)
Returns HTTP 503
(Service Unavailable)

Dennis Doomen
@ddoomen

• The Good, The Bad and the Ugly of Event
Sourcing
https://www.continuousimprover.com/search/label/event%20sourcing
• Effective Aggregate Design (Vaughn Vernon)
http://dddcommunity.org/library/vernon_2011/
• Liquid Projections
https://www.liquidprojections.net
• Distributed Event Sourcing (Slides)
https://www.slideshare.net/dennisdoomen/building-occasionally-
connected-applications-using-event-sourcing
• Data schema changes in an event sourced
system (paper)
https://files.movereem.nl/2017saner-eventsourcing.pdf

Event Sourcing from the Trenches (DDD Europe 2020)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Event Sourcing from the Trenches (DDD Europe 2020)

Similar to Event Sourcing from the Trenches (DDD Europe 2020) (20)

More from Dennis Doomen

More from Dennis Doomen (20)

Recently uploaded

Recently uploaded (20)

Event Sourcing from the Trenches (DDD Europe 2020)