This document provides an overview of using event sourcing with F# and functional programming principles. It discusses domain design using event storming, defining primitive types and events, implementing domain logic as pure functions, serialization, error handling, and using Cosmos DB for event storage and change feeds to build eventually consistent read models. The key aspects covered include defining aggregates, commands, and events as discriminated unions; using Railway Oriented Programming for error handling; serializing to and from DTOs; implementing handlers as pure functions that return aggregates and events; and processing events in Azure Functions using change feeds to update read models.
A CASE STUDY ON ONLINE TICKET BOOKING SYSTEM PROJECT.pdf
Event sourcing in the functional world (22 07-2021)
1. Event Sourcing in the
Functional World
A practical journey of using F#
and event sourcing
2. ● Why Event Sourcing?
● Domain Design: Event Storming
● Primitives, Events, Commands, and
Aggregates
● Domain Logic as Pure Functions
● Serialization and DTOs
● Error Handling: Railway-Oriented
Programming
● Cosmos DB: Event / Aggregate Store
● Strong(-er) Consistency at Scale
● Assembling the Pipeline: Dependency
Rejection
● Change Feed: Eventually Consistent Read
Models https://fsharpforfunandprofit.com/
3. Why Event
Sourcing?
• We need a reliable audit log
• We need to know the the
state of the system at a
given point in time
• We believe it helps us to build
more scalable systems*
(your mileage may vary)
“Event Sourcing -
Step by step in
F#”
https://medium.com/@dzoukr/event
-sourcing-step-by-step-in-f-
be808aa0ca18
“Scaling Event-
Sourcing at Jet”
https://medium.com/@eulerfx/scalin
g-event-sourcing-at-jet-
9c873cac33b8
4. Event
Storming
Form a shared mental
model, avoid
technical jargon
Focus on business
events, not data
structures
Discover Events,
Commands, and
Domain Aggregates
5. Primitives
The modelling
building blocks
Simple Wrappers
type FirstName = FirstName of string
type LastName = LastName of string
type MiddleName = MiddleName of string
Wrappers with guarded access (recommended)
type EmailAddress = private EmailAddress of string
[<RequireQualifiedAccess>]
module EmailAddress =
let create email =
if Regex.IsMatch(email, ".*?@(.*)")
then email |> EmailAddress |> Ok
else Error "Incorrect email format"
let value (EmailAddress e) = e
Composition
type ContactDetails = { Name: Name; Email: EmailAddress }
and Name =
{ FirstName: FirstName
MiddleName: MiddleName option
LastName: LastName }
6. Events
What happened
to the system?
Different event types are just DU cases
type VaccinationEvent =
| ContactRegistered of ContactDetails
| AppointmentCreated of VaccinationAppointment
| AppointmentCanceled of AppointmentId
| VaccineAdministered of AppointmentId
| ObservationComplete of AppointmentId * ObservationStatus
| SurveySubmitted of AppointmentId * SurveyResult
Metadata:
● Timestamp
● Aggregate Id
● Sequence No.
● Correlation Id
● Causation Id
● Grouping
attributes
and VaccinationAppointment =
{ Id: AppointmentId
Vaccine: VaccineType
Hub: VaccinationHub
Date: DateTime }
and AppointmentId = string
and VaccineType = Pfizer | Moderna | AstraZeneca
and ObservationStatus =
| NoAdverseReaction
| AdverseReaction of ReactionKind
and VaccinationHub = exn
and ReactionKind = exn
and SurveyResult = exn
type Event<'D, 'M> = { Id: Guid; Data: 'D; Metadata: 'M }
7. What caused
the change?
(Sometimes)
Who and why
requested the
change?
Commands Commands are intents, and reflect actions.
Naming: usually “VerbNoun”
type VaccinationCommand =
| RegisterContact of ContactDetails
| CreateAppointment of VaccinationAppointment
| CancelAppointment of AppointmentId
| AdministerVaccine of AppointmentId
* ObservationStatus option
| SubmitSurvey of AppointmentId * SurveyResult
Valid commands generate events
let toEvents c = function
| RegisterContact cd -> [ ContactRegistered cd ]
| CreateAppointment a -> [ AppointmentCreated a ]
| CancelAppointment appId -> [ AppointmentCanceled appId ]
| AdministerVaccine (appId, None) ->
[ VaccineAdministered appId ]
| AdministerVaccine (appId, Some status) ->
[ VaccineAdministered appId
ObservationComplete (appId, status) ]
| SubmitSurvey (appId, s) -> [ SurveySubmitted (appId, s) ]
Event.Metadata.CausationId = Command.Id
8. “Designing with
types: Making
illegal states
unrepresentable”
https://fsharpforfunandprofit.com/po
sts/designing-with-types-making-
illegal-states-unrepresentable/
Domain
Aggregates
Aggregates should have enough information to power
business rules
type VaccineRecipient =
{ Id: Guid
ContactDetails: ContactDetails
RegistrationDate: DateTime
State: VaccineRecipientState }
and VaccineRecipientState =
| Registered
| Booked of VaccinationAppointment nlist
| InProcess of VaccinationInProcess
| FullyVaccinated of VaccinationResult nlist
and VaccinationInProcess =
{ Administered: VaccinationResult nlist
Booked: VaccinationAppointment nlist }
and VaccinationResult = VaccinationAppointment
* ObservationStatus option
* SurveyResult option
and 'T nlist = NonEmptyList<'T>
and NonEmptyList<'T> = 'T list
9. Pure functions
Domain
Logic
Event sourcing in a nutshell
type Folder<'Aggregate, 'Event> =
'Aggregate -> 'Event list -> 'Aggregate
type Handler<'Aggregate, 'Command, 'Event> =
'Aggregate -> 'Command -> 'Aggregate * 'Event list
Add error handling, separate “create” and “update”
let create newId timestamp event =
match event with
| ContactRegistered c ->
{ Id = newId; ContactDetails = c;
RegistrationDate = timestamp; State = Registered } |> Ok
| _ -> Error "Aggregate doesn't exist"
let update aggregate event =
match aggregate.State, event with
| Registered, AppointmentCreated apt ->
{ aggregate with State = Booked [ apt ] } |> Ok
| Booked list, AppointmentCreated apt ->
{ aggregate with State = Booked (list @ [apt] ) } |> Ok
| _, _ -> "Exercise left to the reader" |> Error
10. “Serializing your
domain model”
https://fsharpforfunandprofit.
com/posts/serializing-your-
domain-model/
DTOs
DTOs reflect unvalidated input outside of our control
type NameDto =
{ FirstName: string
MiddleName: string // can be null :(
LastName: string }
module NameDto =
let toDomain (name: NameDto) =
({ FirstName = name.FirstName |> FirstName
MiddleName = name.MiddleName
|> Option.ofObj |> Option.map MiddleName
LastName = name.LastName |> LastName }: Name) |> Ok
You might need to apply versioning to your DTOs
type VaccinationEventDto =
| AppointmentCreated of VaccinationAppointment
| AppointmentCreatedV2 of VaccinationAppointment
* Confirmation
Events are stored forever, design them carefully!
11. Error
Handling
Error Handling is tedious
type ContactDetailsDto = { Name: NameDto; Email: string }
module ContactDetailsDto =
let toDomainTheBoringWay contact =
let nameResult = contact.Name |> NameDto.toDomain
match (nameResult) with
| Ok name ->
let emailResult = contact.Email |> EmailAddress.create
match emailResult with
| Ok email ->
({ Name = name; Email = email }: ContactDetails) |> Ok
| Error e -> Error e
| Error e -> Error e
“Railway Oriented
Programming”
https://fsharpforfunand
profit.com/rop/
FsToolkit.ErrorHandling helps
module ContactDetailsDto =
let toDomain contact = result {
let! name = contact.Name |> NameDto.toDomain
let! email = contact.Email |> EmailAddress.create
return ({ Name = name; Email = email }: ContactDetails)
}
12. There and back
again
Libraries:
FSharp.Json
https://github.com/vsaprono
v/FSharp.Json
Thoth.Json (works
with Fable!)
https://thoth-
org.github.io/Thoth.Json/
Serialization FSharp.Json can handle wrapper types and options
let record: Name =
{ FirstName = FirstName "John"
MiddleName = None
LastName = LastName "Smith" }
record |> Json.serialize |> printfn "%s"
{
"FirstName": "John",
"MiddleName": null,
"LastName": "Smith"
}
JSON -> DTO can fail
let tryParse<'Data> s =
try
Json.deserialize<'Data> s |> Ok
with
| x -> sprintf "Cannot deserialize: %s" x.Message |> Error
JSON -> Domain? Compose using ROP
let readContacts contactsString =
contactsString
|> tryParse<ContactDetailsDto>
|> Result.bind ContactDetailsDto.toDomain
13. CosmosDB:
Event /
Aggregate Store
Persistence
Define partition strategy in the code, not DB
let! response =
database.CreateContainerIfNotExistsAsync(
containerName, "/partitionKey")
module VaccinationRecipientKeyStrategy =
let toPartitionKey streamId =
$"VaccinationRecipient-{streamId}"
Partition boundaries determine consistency
Stream 1
(partition 1)
Event 1 Event 2 Event 3
Aggregate
Stream 2
(partition 2)
Event 1 Event 2
Aggregate
14. Defining Entities
in F#
Persistence Useful DB Wrappers
module Db =
type PartitionKey = PartitionKey of string
type Id = Id of string
type ETag = ETag of string
type EntityType = Event | AggregateRoot
Use JSON serializer specifics to map to DB fields
type PersistentEntity<'Payload> =
{ [<JsonField("partitionKey")>]
PartitionKey: Db.PartitionKey
[<JsonField("id")>] Id: Db.Id
[<JsonField("etag")>] ETag: Db.ETag option
Type: EntityType
Payload: 'Payload }
Concrete serialized types
module VaccinationPersistence =
type Event = PersistentEntity<VaccinationEvent>
type Aggregate = PersistentEntity<VaccineRecipient>
15. CosmosDB
Transactions
Persistence Use Batch API to add events + “upsert” the aggregate
module CosmosDb =
let createBatch (Db.PartitionKey key) (container: Container) =
container.CreateTransactionalBatch(PartitionKey(key))
let createAsString (payload: string)
(batch: TransactionalBatch) =
new MemoryStream(Encoding.UTF8.GetBytes(payload))
|> batch.CreateItemStream
let replaceAsString (Db.Id id) (payload: string)
(Db.ETag etag)
(batch: TransactionalBatch) =
let stream = new MemoryStream
(Encoding.UTF8.GetBytes(payload))
let options = TransactionalBatchItemRequestOptions()
options.IfMatchEtag <- etag
batch.ReplaceItemStream(id, stream, options)
let executeBatch (batch: TransactionalBatch) = taskResult {
let! response = batch.ExecuteAsync()
if (response.IsSuccessStatusCode) then
return! Ok ()
else
return! Error (response.StatusCode, response.ErrorMessage)
}
We use tasks for an easier
interop with Azure SDKs
Optimistic
concurrency
17. Using
CosmosDB to
build read
models
Change
Feed
You can use either:
• Azure Functions CosmosDBTrigger (simpler)
• or ChangeFeed Processor running in AppServices
(more control)
You will need to provision:
• A separate Lease collection in CosmosDB
• The target storage (DB, blobs, SQL DB, etc)
Leases ensure events will be processed in sync, but...
Events might come in several partitions at once!
let groups = docs
|> Seq.groupBy
(fun d ->
d.GetPropertyValue<string> "partitionKey")
for (pk, group) in groups do // process by partitions
You have to handle errors (make your own DLQ, etc)
You can reprocess events from the beginning!