Download as PDF, PPTX
Uploaded byChris Richardson
Developing microservices with aggregates (devnexus2017)
The document discusses using domain-driven design aggregates and event sourcing to develop microservices architectures. It describes how aggregates partition domain models to allow microservices to own distinct capabilities. Event sourcing is presented as a way to implement event-driven architectures by persisting domain events rather than just the current state. Explicit sagas are proposed to orchestrate transactions across aggregates and microservices to maintain consistency in an eventually consistent way.
More Related Content
Similar to Developing microservices with aggregates (devnexus2017)
Developing microservices with aggregates (devnexus2017)
- 1. @crichardson Developing Microservices with Aggregates ChrisRichardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
- 2. @crichardson Presentation goal Show how DomainDriven Design Aggregates and Microservices are a perfect match
- 3.
- 4. @crichardson About Chris Consultant andtrainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder ofa startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
- 6.
- 7. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 8. @crichardson The Microservice architecture tacklescomplexity through modularization
- 9.
- 10.
- 11. @crichardson But there arechallenges…
- 12. @crichardson Product service Customer serviceOrderservice Domain model = tangled web of classes Order OrderLine Item quantity … Address street city … Customer Product name price ? ? creditLimit
- 13. @crichardson Reliance on ACIDtransactions to enforce invariants BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Simple and ACID
- 14. @crichardson But it violatesencapsulation… BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service
- 15. @crichardson .. and requires2PC BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION
- 16. @crichardson 2PC is notan option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
- 17. @crichardson Doesn’t fit withthe NoSQL DB “transaction” model
- 18. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 19. @crichardson Domain Driven Design- building blocks Entity Value object Services Repositories Aggregates Adopted Ignored (at least by me)
- 20. About Aggregates Cluster ofobjects that can be treated as a unit Graph consisting of a root entity and one or more other entities and value objects Typically business entities are Aggregates, e.g. customer, Account, Order, Product, …. Order OrderLine Item quantity productId productName productPrice customerId Address street city …
- 21. Aggregate: rule #1 Referenceother aggregate roots via identity (primary key) Order OrderLine Item quantity productId productName productPrice customerId Address street city …
- 22. @crichardson Foreign keys ina Domain Model?!?
- 23. @crichardson Domain model =collection of loosely connected aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
- 24. @crichardson Product service Customer serviceOrderservice Easily partition into microservices Order OrderLine Item quantity … Address street city … Customer Product name price
- 25. @crichardson Aggregate rule #2 Transaction = processingone command by one aggregate
- 26. @crichardson Product service Customer serviceOrderservice Transaction scope = service Order OrderLine Item quantity … Address street city … Customer Product name price
- 27.
- 28. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 29. @crichardson Customer management How tomaintain data consistency between aggregates? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
- 30. @crichardson Saga Using event-driven Sagasinstead of 2PC Distributed transaction Order Customer Local transaction Order Local transaction Customer Local transaction Order Event Event
- 31. @crichardson Order Service Saga-based, eventuallyconsistent order processing Customer Service Order created Credit Reserved Credit Check Failed Create Order OR Customer creditLimit creditReservations ... Order state total … approve()/reject() Event Handler Event Handler reserveCredit()
- 32. @crichardson Order Service Order id :4567 total: 343 state = CREATED Customer Service Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = APPROVED Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
- 33. @crichardson Complexity of compensating transactions ACIDtransactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions For example: CreditCheckFailed => cancel Order Money transfer destination account closed => credit source account Careful design required!
- 34. @crichardson How atomically update databaseand publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
- 35. @crichardson Failure = inconsistentsystem Order Service Order Database Message Broker insert Order publish OrderCreatedEvent X
- 36. @crichardson 2PC is notan option
- 37. @crichardson Reliably publish eventswhen state changes
- 38. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 39. @crichardson Event sourcing =event-centric persistence Application Database Event store update publish X
- 40. @crichardson Event sourcing For eachDDD aggregate: Identify (state changing) domain events e.g. use Event Storming Define Event classes For example, Order events: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
- 41. @crichardson Persists events NOT currentstate Order status …. 101 ACCEPTED Order table X
- 42.
- 43. @crichardson Persists events NOT currentstate Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
- 44. @crichardson Replay events torecreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
- 45. @crichardson The present isa fold over history currentState = foldl(applyEvent, initialState, events)
- 46. @crichardson Event Store Application architecture Order123 Customer 456 OrderCreated OrderApproved … CustomerCreated CustomerCreditReserved … CreateOrder UpdateOrder GetOrder Subscribe Order Service CreateCustomer UpdateCustomer GetCustomer Subscribe Customer Service
- 47. @crichardson Request handling inan event sourced application HTTP Handler Event Store pastEvents = findEvents(orderId) Order new() applyEvents(pastEvents) newEvents = processCmd(someCmd) saveEvents(newEvents) (optimistic locking) Order Service applyEvents(newEvents)
- 48. @crichardson Event Store publishesevents consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Customer update() Customer Service
- 49. @crichardson Event Store publishesevents consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) CQRS View update() Service Xyz send notifications …
- 50. Event store =database + message broker Home grown/DIY .NET geteventstore.com https:// www.lightbend.com/ lagom http://eventuate.io (mine) … Event Store Save aggregate events Get aggregate events Subscribe to events
- 51. @crichardson Benefits of eventsourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log temporal queries Preserved history More easily implement future requirements
- 52. @crichardson Drawbacks of event sourcing… Requiresapplication rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must detect and ignore duplicate events Idempotent event handlers Track most recent event and ignore older ones …
- 53. @crichardson … Drawbacks ofevent sourcing Querying the event store can be challenging Some queries might be complex/inefficient, e.g. accounts with a balance > X Event store might only support lookup of events by entity id Must use Command Query Responsibility Segregation (CQRS) to handle queries application must handle eventually consistent data
- 54. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 55. @crichardson Orders and Customers example Kafka RestAPI Event Store Event Store Adapter Customer Service Rest API Order Service Customer Aggregate Order Aggregate Rest API Order History Service MongoDB CQRS View Event Store Adapter Event Store Adapter Used to notify Order Service of invalid customer ids!
- 56. @crichardson Customer microservice Customer Service Customer Controller Spring Cloud Stream Kafka Customer Aggregate Rest API Customer EventHandlers Event Store Event Store Adapter Order*Event POST /customers
- 57. @crichardson The Customer aggregate MoneycreditLimit Map<OrderId, Money> creditReservations Customer List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditReservedEvent anEvent) { … } … State Behavior
- 58. @crichardson Familiar concepts restructured classCustomer { public void reserveCredit( orderId : String, amount : Money) { // verify // update state this.xyz = … } public List<Event> process( ReserveCreditCommand cmd) { // verify … return singletonList( new CreditReservedEvent(…); ) } public void apply( CreditReservedEvent event) { // update state this.xyz = event.xyz }
- 59.
- 60.
- 61.
- 62. @crichardson Creating a Customer save()concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
- 63. @crichardson Event handling inCustomers Durable subscription name 1. Load Customer aggregate 2. Processes command 3. Applies events 4. Persists events Triggers BeanPostProcessor Publish message to Spring Cloud Stream (Kafka) when Customer not found
- 64.
- 65. @crichardson Agenda The problem withDomain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
- 66. @crichardson Order Service Order Service⬄ Customer Service: tightly coupled, cyclic dependency 😢 Customer Service Order created Credit Reserved Credit Check Failed OR Must subscribe to all Order events that affect available credit! Order …
- 67. @crichardson Order Service Decouple bymaintaining available credit in OrderService Customer Service Available Credit Changed Customer Created Customer creditLimit availableCredit ... Order state total … AvailableCredit Tracker creditLimit availableCredit ... Simpler 😄 Still cyclic 😢
- 68. Use an explicitsaga Saga object that orchestrates the flow Created in response to an initial event, e.g. OrderCreated A state machine Receives events from aggregates Sends commands to aggregates Implemented as an event-sourced aggregate Saga Event Command
- 69. @crichardson Customer Service Order Service Using anOrderCreationSaga Order state total … AvailableCredit Tracker creditLimit availableCredit ... Order Creation Saga create() create() Reserve Credit Credit Reserved Approve Order Customer Update Available Credit OrderCreated Acyclic 😄
- 70. @crichardson Order saga statemachine NEW SUCCEEDED FAILED OrderCreatedEvent / reserveCredit() CustomerCreditReservedEvent / approveOrder() EntityNotFoundCommandExecutionError CustomerCreditLimitExceededEvent / rejectOrder()
- 71. @crichardson Saga as asingleton state machine A state machine state e.g. enum { NEW, …} A data object
- 72. @crichardson Event processing (State +Data, Event) (State’+ Data’, Commands)
- 73. @crichardson (OrderSagaState, OrderSagaData, Event) (Commands,OrderSagaState’, OrderSagaData’)
- 74. @crichardson OrderCreationSaga state machine definition eventsaga id starting + state transitions
- 75. @crichardson Initial action Triggerssaga creation 1. Send command 2. Transition to new state 3. Create data
- 76. @crichardson Event handler Triggering event 1. Sendcommand 2. Transition to new state
- 77. @crichardson Kafka-based command bus Order Saga Credit Tracking Aggregate CreditTrackingCommandTopic OrderSagaCommandFailure Topic Reserve Credit CustomerNotFound Per aggregate type Per saga type
- 78. @crichardson Summary Aggregates are thebuilding blocks of microservices Use event-driven sagas to maintain consistency between aggregates Event sourcing is a good way to implement a event-driven architecture Use explicit sagas to orchestrate transactions
- 79.