Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@crichardson
Managing data consistency in a
microservice architecture using
Sagas
Chris Richardson
Founder of eventuate.io...
@crichardson
Presentation goal
Distributed data management challenges
in a microservice architecture
Sagas as the transact...
@crichardson
About Chris
@crichardson
About Chris
Consultant and trainer
focusing on modern
application architectures
including microservices
(http...
@crichardson
About Chris
Founder of a startup that is creating
an open-source/SaaS platform
that simplifies the development...
@crichardson
For more information
http://learn.microservices.io
40%
discount
with code
ctwsaturn18
@crichardson
Agenda
ACID is not an option
Overview of sagas
Coordinating sagas
Countermeasures for data anomalies
Using re...
The microservice architecture
structures
an application as a
set of loosely coupled
services
@crichardson
Microservices enable
continuous delivery/deployment
Process:
Continuous delivery/deployment
Organization:
Sma...
@crichardson
Each team owns one or more
services
Catalog
Service
Review
Service
Order
Service
…
Service
Catalog
Team
Revie...
@crichardson
Microservice architecture
Browser
Mobile
Device
Store
Front UI
API
Gateway
Customer
Service
Order
Service
…
S...
@crichardson
Private database
!=
private database server
@crichardson
Loose coupling =
encapsulated data
Order Service Customer Service
Order Database Customer Database
Order tabl...
@crichardson
But…
How to maintain data
consistency?
How to implement queries?
@crichardson
How to maintain data consistency?
createOrder(customerId, orderTotal)
Pre-conditions:
• customerId is valid
P...
@crichardson
Cannot use ACID transactions
that span services
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE CUST...
@crichardson
2PC is not an option
Guarantees consistency
BUT
2PC coordinator is a single point of failure
Chatty: at least...
@crichardson
Basically
Available
Soft state
Eventually consistent
http://queue.acm.org/detail.cfm?id=1394128
ACID
@crichardson
Agenda
ACID is not an option
Overview of sagas
Coordinating sagas
Countermeasures for data anomalies
Using re...
@crichardson
From a 1987 paper
@crichardson
Saga
Use Sagas instead of 2PC
Distributed transaction
Service A Service B
Service A
Local
transaction
Service...
@crichardson
Order Service
Create Order Saga
Local transaction
Order
state=PENDING
createOrder()
Customer Service
Local tr...
@crichardson
But what about rollback?
BEGIN TRANSACTION
…
UPDATE …
…
INSERT ….
…
…. BUSINESS RULE VIOLATED!!!!
…
ROLLBACK ...
@crichardson
Rolling back sagas
Use compensating transactions
Developer must write application logic to “rollback” eventua...
@crichardson
Saga: Every Ti has a Ci
T1 T2 …
C1 C2
Compensating transactions
T1 T2 C1
FAILS
@crichardson
Order Service
Create Order Saga - rollback
Local transaction
Order
createOrder()
Customer Service
Local trans...
@crichardson
Writing compensating
transactions isn’t always easy
Write them so they will always succeed
If a compensating ...
@crichardson
Non-compensatable actions
For example, sending an email can’t be unsent.
Move actions that can’t be undone to...
@crichardson
Sagas complicate API design
Synchronous API vs Asynchronous Saga
Request initiates the saga. When to send bac...
@crichardson
Revised Create Order API
createOrder()
returns id of newly created order
NOT fully validated
getOrder(id)
Cal...
@crichardson
Minimal impact on UI
UI hides asynchronous API from the user
Saga will usually appear instantaneous (<= 100ms...
@crichardson
Agenda
ACID is not an option
Overview of sagas
Coordinating sagas
Countermeasures for data anomalies
Using re...
@crichardson
How to sequence the saga
transactions?
After the completion of transaction Ti “something” must
decide what st...
@crichardson
Use asynchronous, broker-
based messaging
Order Service
Customer
Service
….
Message broker
Guaranteed deliver...
@crichardson
Saga step = a transaction
local to a service
Service
Database Message Broker
update publish message/event
Tra...
@crichardson
Choreography: distributed decision making
vs.
Orchestration: centralized decision making
@crichardson
Option #1: Choreography-based
coordination using events
Order
Service
Customer
Service
Order created
Credit R...
@crichardson
Order Service: publishing
domain events
Publish event
Create order
@crichardson
Customer Service: consuming
domain events…
https://github.com/eventuate-tram/eventuate-tram-examples-customer...
@crichardson
Customer Service: consuming
domain events
Attempt to
Reserve credit
Publish event on
success
Publish event on...
@crichardson
More complex choreography example
Order
Service
Consumer
Service
Order created
Consumer Validated
Create Orde...
Benefits and drawbacks of
choreography
Benefits
Simple, especially when
using event sourcing
Participants are loosely
couple...
@crichardson
Order Service
Option #2: Orchestration-based saga
coordination
Local transaction
Order
state=PENDING
createOr...
@crichardson
A saga (orchestrator)
is a persistent object
that
tracks the state of the saga
and
invokes the participants
Saga orchestrator behavior
On create:
Invokes a saga participant
Persists state in database
Wait for a reply
On reply:
Loa...
@crichardson
Order Service
CreateOrderSaga orchestrator
Customer Service
Create Order
Customer
creditLimit
creditReservati...
@crichardson
CreateOrderSaga definition
Sequence of
steps
step = (Ti, Ci)
Build command
to send
Saga’s Data
@crichardson
Customer Service command
handler Route command
to handler
Reserve
credit
Make reply message
@crichardson
Eventuate Tram Sagas
Open-source Saga orchestration framework
Currently for Java
https://github.com/eventuate...
Benefits and drawbacks of
orchestration
Benefits
Centralized coordination
logic is easier to understand
Reduced coupling, e....
@crichardson
Agenda
ACID is not an option
Overview of sagas
Coordinating sagas
Countermeasures for data anomalies
Using re...
@crichardson
Lack of isolation complicates business
logic
Order Service
Local transaction
Order
state=PENDING
createOrder(...
@crichardson
How to cancel a PENDING
Order?
Don’t throw an OrderNotCancellableException
Questionable user experience
“Inte...
@crichardson
Countermeasure Transaction
Model
http://bit.ly/semantic-acid-ctm - paywall 😥
@crichardson
Sagas are ACD
Atomicity
Saga implementation ensures that all transactions are
executed OR all are compensated...
@crichardson
Lack of I anomalies
@crichardson
Outcome of
concurrent execution
!=
a sequential execution
@crichardson
Sounds scary
BUT
It’s common to relax isolation
to improve performance
@crichardson
Anomaly: Lost update
Ti: Create
Order
Tj: Approve
Order
Ti: Cancel
Order
Create
Order
Saga
Cancel
Order
Saga
...
@crichardson
Anomaly: Dirty reads …
Ti: Reserve
Credit
Ci: Unreserve
credit
Ti: Reserve
Credit
Reads
uncommitted
changes
S...
@crichardson
… Anomaly: Dirty reads
Ti: Unreserve
Credit
Ci: Reserve
credit
Ti: Reserve
Credit
Reads
uncommitted
changes
S...
@crichardson
Anomaly: non-repeatable/
fuzzy read
Ti: Begin
Revise Order
Tj: Finalize
Revise Order
Ti: Update
Order
Time
Or...
@crichardson
Countermeasures for
reducing impact
of isolation anomalies…
*
i.e. good enough, eventually consistent applica...
Saga structure
T1 C1
…
T2 C2
Tn+1
Tn+2
….
Compensatable transactions
Pivot transaction = GO/NO GO
Retriable transactions t...
Countermeasure: Semantic
lock
Compensatable transaction sets flag,
retriable transaction releases it
Indicates a possible d...
@crichardson
Countermeasure:
Commutative updates
Commutative: g(f(x)) = f(g(x))
For example:
Account.debit() compensates f...
@crichardson
Countermeasure: Pessimistic
view
Increase avail.
credit
Reduce avail.
credit
Cancel Order Delivery
Cancel Ord...
@crichardson
Countermeasure: Re-read
value
Ti: Read
Order
Tj: Re-read, then
update Order
Ti: Update
Order
Time
Saga 1
Saga...
@crichardson
Countermeasure: version file
Ti: Create
Order
Tj: Authorize
Credit Card
Ti: Cancel
Order
Time
Saga 1
Saga 2
Tj...
@crichardson
Countermeasure: By value
Business risk determines strategy
Low risk => semantic ACID
High risk => use 2PC/dis...
@crichardson
Agenda
ACID is not an option
Overview of sagas
Coordinating sagas
Countermeasures for data anomalies
Using re...
@crichardson
Use asynchronous, broker-
based messaging
Order Service
Customer
Service
….
Message broker
Guaranteed deliver...
@crichardson
Messaging must be
transactional
Service
Database Message Broker
update publish
How to
make atomic
without 2PC?
@crichardson
Option #1: Use database
table as a message queue
@crichardson
Option #1: Use database
table as a message queue
ACID
transaction
See BASE: An Acid Alternative, http://bit.l...
@crichardson
Publishing messages by
polling the MESSAGE table
Message table
Message
PublisherSELECT *
FROM MESSAGES
WHERE ...
@crichardson
Transaction log tailing
Order
Service
Datastore
ORDER table
Transaction log
Update
Transaction log
miner
Mess...
@crichardson
Transaction log tailing
Eventuate Local - reads MySQL binlog
Oracle Golden Gate
AWS DynamoDB streams
MongoDB ...
Transaction log tailing: benefits
and drawbacks
Benefits
No 2PC
No application changes
required
Guaranteed to be
accurate
Dr...
@crichardson
Option #2: Event sourcing:
event-centric persistence
@crichardson
Event sourcing: persists an
object as a sequence of events
Service
Event Store
save events
and
publish
Event ...
@crichardson
Guarantees:
state change
->
event is published
@crichardson
Implementing choreography-
based sagas is straightforward
@crichardson
Preserves history of domain objects
Supports temporal queries
Built-in auditing
@crichardson
Summary
Microservices tackle complexity and accelerate development
Database per service is essential for loos...
@crichardson
@crichardson chris@chrisrichardson.net
http://learn.microservices.io
Questions?
40%
discount
with code
ctwsat...
Upcoming SlideShare
Loading in …5
×

Saturn 2018: Managing data consistency in a microservice architecture using Sagas

22,089 views

Published on

A revised and extended version that I gave at Saturn 2018.

The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two phase commit (a.k.a. a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.

In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.

Published in: Software
  • Be the first to comment

Saturn 2018: Managing data consistency in a microservice architecture using Sagas

  1. 1. @crichardson Managing data consistency in a microservice architecture using Sagas Chris Richardson Founder of eventuate.io Author of Microservices Patterns Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io http://learn.microservices.io
  2. 2. @crichardson Presentation goal Distributed data management challenges in a microservice architecture Sagas as the transaction model
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
  5. 5. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
  6. 6. @crichardson For more information http://learn.microservices.io 40% discount with code ctwsaturn18
  7. 7. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies Using reliable and transactional messaging
  8. 8. The microservice architecture structures an application as a set of loosely coupled services
  9. 9. @crichardson Microservices enable continuous delivery/deployment Process: Continuous delivery/deployment Organization: Small, agile, autonomous, cross functional teams Architecture: Microservice architecture Enables Enables Enables Successful Software Development Services = testability and deployability Teams own services
  10. 10. @crichardson Each team owns one or more services Catalog Service Review Service Order Service … Service Catalog Team Review Team Order Team … Team Responsible for
  11. 11. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Customer Service Order Service … Service Customer Database Order Database … Database HTML REST REST Database per service
  12. 12. @crichardson Private database != private database server
  13. 13. @crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit availableCredit Change schema without coordinating with other teams
  14. 14. @crichardson But… How to maintain data consistency? How to implement queries?
  15. 15. @crichardson How to maintain data consistency? createOrder(customerId, orderTotal) Pre-conditions: • customerId is valid Post-conditions • Order was created • Customer.availableCredit -= orderTotal Customer class Invariant: availableCredit >= 0 availableCredit <= creditLimit Spans services
  16. 16. @crichardson Cannot use ACID transactions that span services BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service Distributed transactions
  17. 17. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
  18. 18. @crichardson Basically Available Soft state Eventually consistent http://queue.acm.org/detail.cfm?id=1394128 ACID
  19. 19. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies Using reliable and transactional messaging
  20. 20. @crichardson From a 1987 paper
  21. 21. @crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service B Local transaction Service C Local transaction X Service C
  22. 22. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() Initiates saga
  23. 23. @crichardson But what about rollback? BEGIN TRANSACTION … UPDATE … … INSERT …. … …. BUSINESS RULE VIOLATED!!!! … ROLLBACK TRANSACTION Really simple!
  24. 24. @crichardson Rolling back sagas Use compensating transactions Developer must write application logic to “rollback” eventually consistent transactions Careful design required!
  25. 25. @crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions T1 T2 C1 FAILS
  26. 26. @crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order reject order() createOrder() FAIL Insufficient credit
  27. 27. @crichardson Writing compensating transactions isn’t always easy Write them so they will always succeed If a compensating transaction fails => no clear way to recover Challenge: Undoing changes when data has already been changed by a different transaction/saga More on this later
  28. 28. @crichardson Non-compensatable actions For example, sending an email can’t be unsent. Move actions that can’t be undone to the end of the saga More on this later.
  29. 29. @crichardson Sagas complicate API design Synchronous API vs Asynchronous Saga Request initiates the saga. When to send back the response? Option #1: Send response when saga completes: + Response specifies the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended): + Improved availability - Response does not specify the outcome. Client must poll or be notified
  30. 30. @crichardson Revised Create Order API createOrder() returns id of newly created order NOT fully validated getOrder(id) Called periodically by client to get outcome of validation
  31. 31. @crichardson Minimal impact on UI UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms) If it takes longer UI displays “processing” popup Server can push notification to UI
  32. 32. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies Using reliable and transactional messaging
  33. 33. @crichardson How to sequence the saga transactions? After the completion of transaction Ti “something” must decide what step to execute next Success: which T(i+1) - branching Failure: C(i - 1)
  34. 34. @crichardson Use asynchronous, broker- based messaging Order Service Customer Service …. Message broker Guaranteed delivery ensures a saga complete when its participants are temporarily unavailable
  35. 35. @crichardson Saga step = a transaction local to a service Service Database Message Broker update publish message/event Transactional DB: BEGIN … COMMIT DDD: Aggregate NoSQL: single ‘record’
  36. 36. @crichardson Choreography: distributed decision making vs. Orchestration: centralized decision making
  37. 37. @crichardson Option #1: Choreography-based coordination using events Order Service Customer Service Order created Credit Reserved Credit Limit Exceeded Create Order OR Customer creditLimit creditReservations Order state total create() reserveCredit() approve()/ reject() Order events channel Customer events channel
  38. 38. @crichardson Order Service: publishing domain events Publish event Create order
  39. 39. @crichardson Customer Service: consuming domain events… https://github.com/eventuate-tram/eventuate-tram-examples-customers-and-orders Subscribe to event
  40. 40. @crichardson Customer Service: consuming domain events Attempt to Reserve credit Publish event on success Publish event on failure
  41. 41. @crichardson More complex choreography example Order Service Consumer Service Order created Consumer Validated Create Order Order events channel Consumer events channel Inventory Service Accounting Service Inventory events channel Inventory Reserved Accounting events channel Card Authorized
  42. 42. Benefits and drawbacks of choreography Benefits Simple, especially when using event sourcing Participants are loosely coupled Drawbacks Cyclic dependencies - services listen to each other’s events Overloads domain objects, e.g. Order and Customer know too much Events = indirect way to make something happen https://github.com/eventuate-examples/eventuate-examples-java-customers-and-orders
  43. 43. @crichardson Order Service Option #2: Orchestration-based saga coordination Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() CreateOrderSaga InvokesInvokesInvokes
  44. 44. @crichardson A saga (orchestrator) is a persistent object that tracks the state of the saga and invokes the participants
  45. 45. Saga orchestrator behavior On create: Invokes a saga participant Persists state in database Wait for a reply On reply: Load state from database Determine which saga participant to invoke next Invokes saga participant Updates its state Persists updated state Wait for a reply …
  46. 46. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create() approve() creditReserved() Customer command channel Saga reply channel
  47. 47. @crichardson CreateOrderSaga definition Sequence of steps step = (Ti, Ci) Build command to send Saga’s Data
  48. 48. @crichardson Customer Service command handler Route command to handler Reserve credit Make reply message
  49. 49. @crichardson Eventuate Tram Sagas Open-source Saga orchestration framework Currently for Java https://github.com/eventuate-tram/eventuate-tram-sagas https://github.com/eventuate-tram/eventuate-tram-sagas- examples-customers-and-orders
  50. 50. Benefits and drawbacks of orchestration Benefits Centralized coordination logic is easier to understand Reduced coupling, e.g. Customer knows less. Simply has API Reduces cyclic dependencies Drawbacks Risk of smart sagas directing dumb services
  51. 51. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies Using reliable and transactional messaging
  52. 52. @crichardson Lack of isolation complicates business logic Order Service Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction cancelOrder() ? Time
  53. 53. @crichardson How to cancel a PENDING Order? Don’t throw an OrderNotCancellableException Questionable user experience “Interrupt” the Create Order saga? Cancel Order Saga: set order.state = CANCELLED Causes Create Order Saga to rollback But is that enough to cancel the order? Cancel Order saga waits for the Create Order saga to complete? Suspiciously like a distributed lock But perhaps that is ok
  54. 54. @crichardson Countermeasure Transaction Model http://bit.ly/semantic-acid-ctm - paywall 😥
  55. 55. @crichardson Sagas are ACD Atomicity Saga implementation ensures that all transactions are executed OR all are compensated Consistency Referential integrity within a service handled by local databases Referential integrity across services handled by application Durability Durability handled by local databases
  56. 56. @crichardson Lack of I anomalies
  57. 57. @crichardson Outcome of concurrent execution != a sequential execution
  58. 58. @crichardson Sounds scary BUT It’s common to relax isolation to improve performance
  59. 59. @crichardson Anomaly: Lost update Ti: Create Order Tj: Approve Order Ti: Cancel Order Create Order Saga Cancel Order Saga Time Overwrites cancelled order
  60. 60. @crichardson Anomaly: Dirty reads … Ti: Reserve Credit Ci: Unreserve credit Ti: Reserve Credit Reads uncommitted changes Saga 1 Saga 2 Order rejected unnecessarily Time
  61. 61. @crichardson … Anomaly: Dirty reads Ti: Unreserve Credit Ci: Reserve credit Ti: Reserve Credit Reads uncommitted changes Saga 1 Saga 2 Credit limit exceeded! Time
  62. 62. @crichardson Anomaly: non-repeatable/ fuzzy read Ti: Begin Revise Order Tj: Finalize Revise Order Ti: Update Order Time Order has changed since Ti Saga 1 Saga 2
  63. 63. @crichardson Countermeasures for reducing impact of isolation anomalies… * i.e. good enough, eventually consistent application
  64. 64. Saga structure T1 C1 … T2 C2 Tn+1 Tn+2 …. Compensatable transactions Pivot transaction = GO/NO GO Retriable transactions that can’t fail
  65. 65. Countermeasure: Semantic lock Compensatable transaction sets flag, retriable transaction releases it Indicates a possible dirty read Flag = lock: prevents other transactions from accessing it Require deadlock detection, e.g. timeout Flag = warning - treat the data differently, e.g. Order.state = PENDING a pending deposit … … … Approve Order Create Pending Order Reject Order Order.state = PENDING Order.state = REJECTED Order.state = APPROVED
  66. 66. @crichardson Countermeasure: Commutative updates Commutative: g(f(x)) = f(g(x)) For example: Account.debit() compensates for Account.credit() Account.credit() compensates for Account.debit() Avoids lost updates
  67. 67. @crichardson Countermeasure: Pessimistic view Increase avail. credit Reduce avail. credit Cancel Order Delivery Cancel Order Reorder saga to reduce risk Cancel Order Delivery Cancel Order Increase avail. credit … …… … Won’t be compensated, so no dirty read
  68. 68. @crichardson Countermeasure: Re-read value Ti: Read Order Tj: Re-read, then update Order Ti: Update Order Time Saga 1 Saga 2 Verify unchanged, possibly restart Prevents lost update ~Offline optimistic lock pattern
  69. 69. @crichardson Countermeasure: version file Ti: Create Order Tj: Authorize Credit Card Ti: Cancel Order Time Saga 1 Saga 2 Tj: Reserve Credit Tj: Reverse Credit Card Payment 1. reverse() 2. authorize() “log” of changes: Enables operations to commute
  70. 70. @crichardson Countermeasure: By value Business risk determines strategy Low risk => semantic ACID High risk => use 2PC/distributed transaction
  71. 71. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Countermeasures for data anomalies Using reliable and transactional messaging
  72. 72. @crichardson Use asynchronous, broker- based messaging Order Service Customer Service …. Message broker Guaranteed delivery ensures a saga complete when its participants are temporarily unavailable
  73. 73. @crichardson Messaging must be transactional Service Database Message Broker update publish How to make atomic without 2PC?
  74. 74. @crichardson Option #1: Use database table as a message queue
  75. 75. @crichardson Option #1: Use database table as a message queue ACID transaction See BASE: An Acid Alternative, http://bit.ly/ebaybase DELETE ? Customer Service ORDER_ID CUSTOMER_ID TOTAL 99 CUSTOMER_CREDIT_RESERVATIONS table 101 1234 ID TYPE DATA DESTINATION MESSAGE table 84784 CreditReserved {…} … INSERT INSERT Message Publisher QUERY Message Broker Publish Local transaction reserveCredit()
  76. 76. @crichardson Publishing messages by polling the MESSAGE table Message table Message PublisherSELECT * FROM MESSAGES WHERE …. Message table Publish message DELETE …
  77. 77. @crichardson Transaction log tailing Order Service Datastore ORDER table Transaction log Update Transaction log miner Message Broker Publish Changes MESSAGE table
  78. 78. @crichardson Transaction log tailing Eventuate Local - reads MySQL binlog Oracle Golden Gate AWS DynamoDB streams MongoDB - Read the oplog
  79. 79. Transaction log tailing: benefits and drawbacks Benefits No 2PC No application changes required Guaranteed to be accurate Drawbacks Obscure Database specific solutions Tricky to avoid duplicate publishing
  80. 80. @crichardson Option #2: Event sourcing: event-centric persistence
  81. 81. @crichardson Event sourcing: persists an object as a sequence of events Service Event Store save events and publish Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated Every state change event
  82. 82. @crichardson Guarantees: state change -> event is published
  83. 83. @crichardson Implementing choreography- based sagas is straightforward
  84. 84. @crichardson Preserves history of domain objects Supports temporal queries Built-in auditing
  85. 85. @crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loose coupling Use ACID transactions within services Use orchestration-based or choreography-based sagas to maintain data consistency across services Use countermeasures to reduce impact of anomalies caused by lack of isolation
  86. 86. @crichardson @crichardson chris@chrisrichardson.net http://learn.microservices.io Questions? 40% discount with code ctwsaturn18

×