Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ArchSummit Shenzhen - Using sagas to maintain data consistency in a microservice architecture

1,209 views

Published on

This is a talk I gave at QCON ArchSummit in Shenzhen.

The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.

In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.

Published in: Software
  • Be the first to comment

ArchSummit Shenzhen - Using sagas to maintain data consistency in a microservice architecture

  1. 1. @crichardson Using sagas to maintain data consistency in a microservice architecture Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
  2. 2. @crichardson Presentation goal Distributed data management challenges in a microservice architecture Sagas as the transaction model
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris
  5. 5. @crichardson About Chris
  6. 6. @crichardson About Chris
  7. 7. @crichardson About Chris
  8. 8. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
  9. 9. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
  10. 10. @crichardson For more information http://learnmicroservices.io
  11. 11. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
  12. 12. The microservice architecture structures an application as a set of loosely coupled services
  13. 13. @crichardson The microservice architecture tackles complexity through modularization
  14. 14. @crichardson The microservice architecture tackles complexity through modularization * might improve scalability too
  15. 15. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Customer Service Order Service … Service Customer Database Order Database … Database HTML REST REST
  16. 16. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Customer Service Order Service … Service Customer Database Order Database … Database HTML REST REST Database per service
  17. 17. @crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit
  18. 18. @crichardson How to maintain data consistency?!?!? Invariant: sum(open order.total) <= customer.creditLimit
  19. 19. @crichardson Cannot use ACID transactions BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION
  20. 20. @crichardson Cannot use ACID transactions BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service
  21. 21. @crichardson Cannot use ACID transactions BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service Distributed transactions
  22. 22. @crichardson 2PC is not an option
  23. 23. @crichardson 2PC is not an option Guarantees consistency
  24. 24. @crichardson 2PC is not an option Guarantees consistency BUT
  25. 25. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure
  26. 26. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2)
  27. 27. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks
  28. 28. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers)
  29. 29. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability
  30. 30. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
  31. 31. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
  32. 32. @crichardson From a 1987 paper
  33. 33. @crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service B Local transaction Service C Local transaction X Service C
  34. 34. @crichardson Order Service Create Order Saga
  35. 35. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() createOrder()
  36. 36. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() createOrder()
  37. 37. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder()
  38. 38. @crichardson If only it were this easy…
  39. 39. @crichardson Rollback using compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions Careful design required!
  40. 40. @crichardson Saga: Every Ti has a Ci T1 T2 …
  41. 41. @crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions
  42. 42. @crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions T1 T2 C1 FAILS
  43. 43. @crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order reject order() createOrder() FAIL Insufficient credit
  44. 44. @crichardson Sagas complicate API design Request initiates the saga. When to send back the response?
  45. 45. @crichardson Sagas complicate API design Request initiates the saga. When to send back the response? Option #1: Send response when saga completes: + Response specifies the outcome - Reduced availability
  46. 46. @crichardson Sagas complicate API design Request initiates the saga. When to send back the response? Option #1: Send response when saga completes: + Response specifies the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended): + Improved availability - Response does not specify the outcome. Client must poll or be notified
  47. 47. @crichardson Revised Create Order API createOrder() returns id of newly created order NOT fully validated getOrder(id) Called periodically by client to get outcome of validation
  48. 48. @crichardson Minimal impact on UI UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms) If it takes longer UI displays “processing” popup Server can push notification to UI
  49. 49. @crichardson Sagas complicate the business logic Changes are committed by each step of the saga
  50. 50. @crichardson Sagas complicate the business logic Changes are committed by each step of the saga Other transactions see “inconsistent” data, e.g. Order.state = PENDING more complex logic
  51. 51. @crichardson Sagas complicate the business logic Changes are committed by each step of the saga Other transactions see “inconsistent” data, e.g. Order.state = PENDING more complex logic Interaction between sagas and other operations e.g. what does it mean to cancel a PENDING Order? “Interrupt” the Create Order saga Wait for the Create Order saga to complete?
  52. 52. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas
  53. 53. @crichardson How to sequence the saga transactions? After the completion of transaction Ti “something” must decide what step to execute next Success: which T(i+1) - branching Failure: C(i - 1)
  54. 54. @crichardson Order Service Orchestration-based saga coordination Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() CreateOrderSaga
  55. 55. @crichardson Complex coordination logic is centralized 😀
  56. 56. @crichardson Services expose APIs that are invoked by saga 😄
  57. 57. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... OrderService
  58. 58. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... CreateOrder Saga OrderService create()
  59. 59. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… CreateOrder Saga OrderService create() create()
  60. 60. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create()
  61. 61. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create() creditReserved()
  62. 62. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create() approve() creditReserved()
  63. 63. @crichardson Saga orchestrators are state machines RESERVING CREDIT APPROVED REJECTED / customerService. reserveCredit() credit reserved / order.approve() credit limit exceed / order.reject() event [guard] / action Initial action
  64. 64. @crichardson Saga orchestrators are state machines RESERVING CREDIT APPROVED REJECTED / customerService. reserveCredit() credit reserved / order.approve() credit limit exceed / order.reject() event [guard] / action Initial action reply and action
  65. 65. @crichardson Create Order Saga code Eventuate Saga framework
  66. 66. @crichardson Create Order Saga code Stateless singleton: Behavior Eventuate Saga framework
  67. 67. @crichardson Create Order Saga code Enum Stateless singleton: Behavior Eventuate Saga framework State
  68. 68. @crichardson Create Order Saga code Enum Persistent data Stateless singleton: Behavior Eventuate Saga framework State
  69. 69. @crichardson Create Order Saga
  70. 70. @crichardson Create Order Saga State machine definition
  71. 71. @crichardson Initializing the saga
  72. 72. @crichardson Initializing the saga Create order
  73. 73. @crichardson Initializing the saga Invoke saga participant Create order
  74. 74. @crichardson Handling a reply
  75. 75. @crichardson Handling a reply Update Order
  76. 76. @crichardson Customer Service - command handling
  77. 77. @crichardson Customer Service - command handling Reserve credit
  78. 78. @crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant Saga must complete even if there are transient failures
  79. 79. @crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant command Saga must complete even if there are transient failures
  80. 80. @crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant command reply Saga must complete even if there are transient failures
  81. 81. @crichardson Use asynchronous messaging
  82. 82. @crichardson Create Order Saga messaging Order Service Message Broker Customer Service Begin TXN Read data Send reply Commit TXN Credit Reserved Credit Limit Exceeded Create Order Saga Orchestrator reserve credit Customer Service Request Channel Customer Service Reply Channel Order create() approve() reject()
  83. 83. @crichardson Create Order Saga messaging Order Service Message Broker Customer Service Begin TXN Read data Send reply Commit TXN Credit Reserved Credit Limit Exceeded Create Order Saga Orchestrator reserve credit Customer Service Request Channel Customer Service Reply Channel Order create() approve() reject() Commands
  84. 84. @crichardson Create Order Saga messaging Order Service Message Broker Customer Service Begin TXN Read data Send reply Commit TXN Credit Reserved Credit Limit Exceeded Create Order Saga Orchestrator reserve credit Customer Service Request Channel Customer Service Reply Channel Order create() approve() reject() Commands Replies
  85. 85. @crichardson Messaging must be transactional Service Database Message Broker update publish How to make atomic without 2PC?
  86. 86. @crichardson Option #1: Use database table as a message queue See BASE: An Acid Alternative, http://bit.ly/ebaybase Customer Service ORDER_ID CUSTOMER_ID TOTAL 99 CUSTOMER_CREDIT_RESERVATIONS table 101 1234 ID TYPE DATA DESTINATION MESSAGE table 84784 OrderCreated {…} … INSERT INSERT Message Publisher QUERY Message Broker Publish Local transaction
  87. 87. @crichardson Option #1: Use database table as a message queue ACID transaction See BASE: An Acid Alternative, http://bit.ly/ebaybase DELETE Customer Service ORDER_ID CUSTOMER_ID TOTAL 99 CUSTOMER_CREDIT_RESERVATIONS table 101 1234 ID TYPE DATA DESTINATION MESSAGE table 84784 OrderCreated {…} … INSERT INSERT Message Publisher QUERY Message Broker Publish Local transaction
  88. 88. @crichardson Option #1: Use database table as a message queue ACID transaction See BASE: An Acid Alternative, http://bit.ly/ebaybase DELETE Customer Service ORDER_ID CUSTOMER_ID TOTAL 99 CUSTOMER_CREDIT_RESERVATIONS table 101 1234 ID TYPE DATA DESTINATION MESSAGE table 84784 OrderCreated {…} … INSERT INSERT Message Publisher QUERY Message Broker Publish Local transaction ?
  89. 89. @crichardson Publishing messages Poll the MESSAGE table OR Tail the database transaction log
  90. 90. @crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table Entity type Event id Entity id Event data Event type Every state change event
  91. 91. @crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table Entity type Event id Entity id Event data Event type Order 901101 …OrderCreated Every state change event
  92. 92. @crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Event type Order 901101 …OrderCreated Every state change event
  93. 93. @crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated Every state change event
  94. 94. @crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loose coupling Use sagas to maintain data consistency across services Use transactional messaging to make sagas reliable
  95. 95. @crichardson @crichardson chris@chrisrichardson.net http://learnmicroservices.io Questions?

×