Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing microservices with aggregates (SpringOne platform, #s1p)

30,971 views

Published on

The Domain Model pattern is a great way to develop complex business logic. Unfortunately, a typical domain model is a tangled, birds nest of classes. It can’t be decomposed into microservices. Moreover, business logic often relies on ACID transactions to maintain consistency.

Fortunately, there is a solution to this problem: aggregates. An aggregate is an often overlooked modeling concept from the must read book Domain Driven Design. In this talk you will learn how aggregates enable you to develop business logic for the modern world of microservices and NoSQL. We will describe how to use aggregates to design modular business logic that can be partitioned into microservices. You will learn how aggregates enable you to use eventual consistency instead of ACID. We will describe the design of a microservice that is built using aggregates, and Spring Cloud.

Published in: Software

Developing microservices with aggregates (SpringOne platform, #s1p)

  1. 1. @crichardson Developing Microservices with Aggregates Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
  2. 2. @crichardson Presentation goal Show how Domain Driven Design Aggregates and Microservices are a perfect match
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
  5. 5. @crichardson About Chris Founder of a startup that is creating a platform that makes it easier for developers to write transactional microservices (http://eventuate.io)
  6. 6. @crichardson For more information http://learnmicroservices.io
  7. 7. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
  8. 8. @crichardson The Microservice architecture tackles complexity through modularization
  9. 9. @crichardson Microservice = Business capability
  10. 10. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Catalog Service Review Service Order Service … Service Catalog Database Review Database Order Database … Database HTML REST REST Apply X-axis and Z-axis scaling to each service independently
  11. 11. @crichardson Service boundaries enforce modularity
  12. 12. @crichardson But there are challenges…
  13. 13. @crichardson Product service Customer serviceOrder service Domain model = tangled web of classes Order OrderLine Item quantity … Address street city … Customer Product name price ? ? creditLimit
  14. 14. @crichardson Reliance on ACID transactions to enforce invariants BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Simple and ACID
  15. 15. @crichardson But it violates encapsulation… BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service
  16. 16. @crichardson .. and requires 2PC BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION
  17. 17. @crichardson 2PC is not a viable option Guarantees consistency BUT 2PC is best avoided Not supported by many NoSQL databases etc. CAP theorem 2PC impacts availability ….
  18. 18. @crichardson Doesn’t fit with the NoSQL DB “transaction” model
  19. 19. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
  20. 20. @crichardson Domain Driven Design: read it!
  21. 21. @crichardson Domain Driven Design - building blocks Entity Value object Services Repositories Aggregates Adopted Ignored (at least by me)
  22. 22. About Aggregates Cluster of objects that can be treated as a unit Graph consisting of a root entity and one or more other entities and value objects Typically business entities are Aggregates, e.g. customer, Account, Order, Product, …. Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  23. 23. Aggregate: rule #1 Reference other aggregate roots via identity (primary key) Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  24. 24. @crichardson Foreign keys in a Domain Model?!?
  25. 25. @crichardson Domain model = collection of loosely connected aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
  26. 26. @crichardson Product service Customer serviceOrder service Easily partition into microservices Order OrderLine Item quantity … Address street city … Customer Product name price
  27. 27. @crichardson Aggregate rule #2 Transaction = processing one command by one aggregate
  28. 28. @crichardson Product service Customer serviceOrder service Transaction scope = service Order OrderLine Item quantity … Address street city … Customer Product name price
  29. 29. @crichardson Transaction scope = NoSQL database “transaction”
  30. 30. @crichardson Aggregate granularity If an update must be atomic then it must be handled by a single aggregate Therefore Aggregate granularity is important
  31. 31. @crichardson Aggregate granularity Consistency Scalability/ User experience Customer Order Product Customer Order Product Customer Order Product
  32. 32. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
  33. 33. @crichardson Customer management How to maintain data consistency between aggregates? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
  34. 34. @crichardson Event-driven architecture
  35. 35. @crichardson Use event-driven, eventually consistent order processing Order Service Customer Service Order created Credit Reserved Credit Check Failed Create Order OR Customer creditLimit creditReservations ... Order state total … create() reserveCredit() approve()/reject()
  36. 36. @crichardson Order Management Order id : 4567 total: 343 state = CREATED Customer Management Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = APPROVED Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
  37. 37. @crichardson Complexity of compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions For example: CreditCheckFailed => cancel Order Money transfer destination account closed => credit source account Careful design required!
  38. 38. @crichardson How atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
  39. 39. @crichardson Failure = inconsistent system Order Service Order Database Message Broker insert Order publish OrderCreatedEvent X
  40. 40. @crichardson 2PC is not an option
  41. 41. @crichardson Reliably publish events when state changes
  42. 42. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
  43. 43. @crichardson Event sourcing = event-centric persistence Application Database Event store update publish X
  44. 44. @crichardson Event sourcing For each domain object (i.e. DDD aggregate): Identify (state changing) domain events, e.g. use Event Storming Define Event classes For example, Order events: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
  45. 45. @crichardson Persists events NOT current state Order status …. 101 ACCEPTED Order table X
  46. 46. @crichardson Persists events NOT current state Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
  47. 47. @crichardson Replay events to recreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
  48. 48. @crichardson The present is a fold over history currentState = foldl(applyEvent, initialState, events)
  49. 49. @crichardson Event Store Application architecture Order 123 Customer 456 OrderCreated OrderApproved … CustomerCreated CustomerCreditReserved … CreateOrder UpdateOrder GetOrder Subscribe Order Service CreateCustomer UpdateCustomer GetCustomer Subscribe Customer Service
  50. 50. @crichardson Request handling in an event sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Order new() applyEvents(pastEvents) newEvents = processCmd(someCmd) saveEvents(newEvents) (optimistic locking) Order Service applyEvents(newEvents)
  51. 51. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Customer update() Customer Service
  52. 52. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) CQRS View update() Service Xyz send notifications …
  53. 53. Event store = database + message broker Hybrid database and message broker Implementations: Home grown/DIY geteventstore.com by Greg Young http://eventuate.io (mine) Event Store Save aggregate events Get aggregate events Subscribe to events
  54. 54. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log temporal queries Preserved history More easily implement future requirements
  55. 55. @crichardson Drawbacks of event sourcing… Requires application rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must detect and ignore duplicate events Idempotent event handlers Track most recent event and ignore older ones …
  56. 56. @crichardson … Drawbacks of event sourcing Querying the event store can be challenging Some queries might be complex/inefficient, e.g. accounts with a balance > X Event store might only support lookup of events by entity id Must use Command Query Responsibility Segregation (CQRS) to handle queries application must handle eventually consistent data
  57. 57. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Example application
  58. 58. @crichardson Orders and Customers example Kafka Rest API Event Store Event Store Adapter Customer Service Rest API Order Service Customer Aggregate Order Aggregate Rest API Order History Service MongoDB CQRS View Event Store Adapter Event Store Adapter
  59. 59. @crichardson Customer microservice Customer Service Customer Controller Spring Cloud Stream Kafka Customer Aggregate Rest API Customer Event Handlers Event Store Event Store Adapter
  60. 60. @crichardson The Customer aggregate Money creditLimit Map<OrderId, Money> creditReservations Customer List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditReservedEvent anEvent) { … } … State Behavior
  61. 61. @crichardson Familiar concepts restructured class Customer { public void reserveCredit( orderId : String, amount : Money) { // verify // update state this.xyz = … } public List<Event> process( ReserveCreditCommand cmd) { // verify … return singletonList( new CreditReservedEvent(…); ) } public void apply( CreditReservedEvent event) { // update state this.xyz = event.xyz }
  62. 62. @crichardson Customer command processing
  63. 63. @crichardson Customer applying events
  64. 64. @crichardson Customer Controller
  65. 65. @crichardson Creating a Customer save() concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
  66. 66. @crichardson Event handling in Customers Durable subscription name 1. Load Customer aggregate 2. Processes command 3. Applies events 4. Persists events Triggers BeanPostProcessor Publish message to Spring Cloud Stream (Kafka) when Customer not found
  67. 67. @crichardson Summary Aggregates are the building blocks of microservices Use events to maintain consistency between aggregates Event sourcing is a good way to implement a event-driven architecture
  68. 68. @crichardson @crichardson chris@chrisrichardson.net http://learnmicroservices.io Questions?

×