Developing functional domain models with event sourcing (sbtb, sbtb2015)


Published on

Event sourcing persists each entity as a sequence of state changing event. An entity’s current state is derived by replaying the events. Event sourcing is a great way to implement event-driven micro services. When one service updates an entity, the new events are consumed by other services, which then update their own state. In this talk we describe how to implement business logic using a domain model that is based on event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach. You will learn how Domain Driven Design concepts such as bounded contexts and aggregates fit in with event-driven microservices.

Published in: Software

Developing functional domain models with event sourcing (sbtb, sbtb2015)

  1. 1. @crichardson Developing functional domain models with event sourcing Chris Richardson Author of POJOs in Action Founder of the original Founder of a microservices platform startup @crichardson
  2. 2. @crichardson Presentation goal How to design functional domain models using event sourcing?
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on microservices (
  5. 5. @crichardson About Chris Founder of a startup that is creating a platform that makes it easy for application developers write microservices (
  6. 6. @crichardson For more information
  7. 7. @crichardson Agenda Why event sourcing? Designing a domain model based on event sourcing Event sourcing and service design
  8. 8. @crichardson Tomcat Traditional monolithic architecture Browser/ Client WAR/EAR RDBMS Customers Accounts Transfers Banking UI develop test deploy Simple to Load balancer scale Spring MVC Spring Hibernate ... HTML REST/JSON ACID
  9. 9. @crichardson But that leads* to monolithic hell For large and/or complex applications…
  10. 10. @crichardson Today: use a microservice, polyglot architecture Banking UI Account Management Service MoneyTransfer Management Service Account Database MoneyTransfer Database Standalone services Sharded SQLNoSQL DB
  11. 11. @crichardson But now we have distributed data management problems
  12. 12. @crichardson Customer management Maintaining invariants across service boundaries Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open <= customer.creditLimit ?
  13. 13. @crichardson Maintaining invariants with (non- ACID) NoSQL databases Non-ACID NoSQL database Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open <= customer.creditLimit ?
  14. 14. @crichardson Use an event-driven architecture Services publish events when state changes Services subscribe to events and update their state Maintain eventual consistency across multiple aggregates (in multiple datastores) Synchronize replicated data
  15. 15. @crichardson Order Management Order id : 4567 total: 343 state = CREATED Customer Management Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = OPEN Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
  16. 16. @crichardson How to atomically update the database and publish events without 2PC? (dual write problem)
  17. 17. @crichardson Update the database and Publish events
  18. 18. @crichardson Event sourcing For each aggregate (aka. business entity): Identify (state-changing) domain events Define Event classes For example, Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEvent ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent
  19. 19. @crichardson Persists events NOT current state Account balance open(initial) debit(amount) credit(amount) AccountOpened Event table AccountCredited AccountDebited 101 450 Account table X 101 101 101 901 902 903 500 250 300
  20. 20. @crichardson Replay events to recreate state Account balance AccountOpenedEvent(balance) AccountDebitedEvent(amount) AccountCreditedEvent(amount) Events Periodically snapshot to avoid loading all events
  21. 21. @crichardson The present is a fold over history
  22. 22. @crichardson Event Aggregates: Command => Events AggregateCommand Event
  23. 23. @crichardson Request handling in an event-sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Account new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A (optimistic locking)
  24. 24. @crichardson Event Store publishes events - consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate CQRS View update() update() Microservice B send notifications …
  25. 25. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL-based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built-in, reliable audit log, temporal queries Preserved history More easily implement future requirements
  26. 26. @crichardson Drawbacks of event sourcing Weird and unfamiliar Events = a historical record of your bad design decisions Handling duplicate events can be tricky Application must handle eventually consistent data Event store only directly supports PK-based lookup => use Command Query Responsibility Segregation (CQRS) to handle queries
  27. 27. @crichardson Agenda Why event sourcing? Designing a domain model based on event sourcing Event sourcing and service design
  28. 28. @crichardson Use the familiar building blocks of DDD Entity Value object Services Repositories Aggregates
  29. 29. @crichardson Partition the domain model into Aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
  30. 30. Aggregate design Graph consisting of a root entity and one or more other entities and value objects Each core business entity = Aggregate: e.g. customer, Account, Order, Product, …. Reference other aggregate roots via primary key Often contains partial copy of other aggregates’ data Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  31. 31. @crichardson Modular business logic = flexible deployment Tomcat WAR/EAR Customer Order Tomcat WAR/EAR Customer Tomcat WAR/EAR Order OR
  32. 32. @crichardson Aggregate granularity is important Transaction = processing one command by one aggregate No opportunity to update multiple aggregates within a transaction If an update must be atomic (i.e. no compensating transaction) then it must be handled by a single aggregate e.g. scanning boarding pass at security checkpoint or when entering jetway
  33. 33. @crichardson Aggregate granularity Customer Product Order Customer Product Order Consistency Decomposability/ Scalability/ User experience
  34. 34. Designing domain events Records state changes for an aggregate Records key “business events” Part of the public API of the domain model ProductAddedToCart id : TimeUUID productId productName productPrice shoppingCartId Required by aggregate Enrichment: Required by consumers
  35. 35. @crichardson Designing commands Created by a service from incoming request Processed by an aggregate Immutable Contains value objects for Validating request Creating event Auditing user activity
  36. 36. @crichardson Hybrid OO/FP domain objects
  37. 37. @crichardson OO = State + Behavior creditLimit creditReservations Customer processCommand : PartialFunction[Command, Events] applyEvent : PartialFunction[Event, Account] State Behavior
  38. 38. @crichardson Aggregate traits Map Command to Events Apply event returning updated Aggregate Used by Event Store to reconstitute aggregate Event types are not precisely typed :-(
  39. 39. @crichardson Customer - command processing Check available credit
  40. 40. @crichardson Customer - applying events Immutable
  41. 41. @crichardson Event Store API trait EventStore { def save[T <: Aggregate[T]](entity: T, events: Seq[Event], assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]] def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion, entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]] def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) : Future[EntityWithIdAndVersion[T]] def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId) Future[Option[EntityWithIdAndVersion[T]]] def subscribe(subscriptionId: SubscriptionId): Future[AcknowledgableEventStream] }
  42. 42. @crichardson Unsatisfying design with imprecise typing “I hate OO”
  43. 43. @crichardson A detour to Haskell Oakland Advanced Scala Study Group
  44. 44. @crichardson Haskell event sourcing aggregate… Associated data family ~ abstract type members The operations
  45. 45. @crichardson …Haskell Aggregate
  46. 46. @crichardson Haskell TicTacToe aggregate Concrete type definitions Command Processing Apply Event
  47. 47. @crichardson Functional event sourcing in Scala
  48. 48. @crichardson “This section briefly explains GHC Haskell’s associated types and shows in detail how they can be encoded in Scala using type members …”
  49. 49. @crichardson FP = Separation of State and Behavior Customer creditLimit … CustomerAggregate processCommand(Account, Command) : Seq[Events] applyEvent(Account, Event) : Account State Behavior
  50. 50. @crichardson Aggregate type classes Used by Event Store to reconstitute aggregates Hardwired
  51. 51. @crichardson Customer Aggregate…. State Behavior
  52. 52. @crichardson …command processing…
  53. 53. @crichardson … applying events
  54. 54. @crichardson Using Lenses with aggregates
  55. 55. @crichardson FP-style Event Store API Aggregate type class
  56. 56. @crichardson Agenda Why event sourcing? Designing a domain model based on event sourcing Event sourcing and service design
  57. 57. @crichardson Event Store HTTP Request HTTP Adapter Event Handler Cmd Cmd Events Events Xyz Adapter Xyz Request microservice Aggregate Service Event Adapter
  58. 58. @crichardson Place Order example As a customer I want to place an order So that I get the needed products Given that my available credit is $1500 When I place a $250 order Then the order is created Then my available credit is $1250 Story Scenario Post conditions Pre conditions
  60. 60. … becomes eventually consistent (BASE) Updating multiple aggregates multi-step, event-driven flow each step updates one Aggregate Service creates saga to coordinate workflow A state machine Part of the domain, e.g. Order aggregate OR Synthetic aggregate Post-conditions eventually true Order Customer Created Credit reserved public Order createOrder() { … Creates Order … } Approved
  61. 61. @crichardson Need compensating transactions Pre-conditions might be false when attempting to update an aggregate Credit check might fail => cancel order Credit check succeeded but customer cancels order => undo credit reservation …
  62. 62. @crichardson Creating an order DSL concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
  63. 63. @crichardson Event handling in Account Durable subscription name Triggers BeanPostProcessor 1.Load Customer aggregate 2.Processes command 3.Applies events 4.Persists events
  64. 64. @crichardson Summary Event sourcing solves a variety of problems in modern application architectures ES-based architecture = choice of monolith or microservices Scala is a great language for implementing ES-based domain models: Case classes Pattern matching Recreating state = functional fold over events
  65. 65. @crichardson @crichardson