To deliver a large complex application rapidly, frequently and reliably, you often must use the microservice architecture. The microservice architecture is an architectural style that structures the application as a collection of loosely coupled services. One challenge with using microservices is that in order to be loosely coupled each service has its own private database. As a result, implementing transactions and queries that span services is no longer straightforward. In this presentation, you will learn how event-driven microservices address this challenge. I describe how to use sagas, which is an asynchronous messaging-based pattern, to implement transactions that span services. You will learn how to implement queries that span services using the CQRS pattern, which maintain easily queryable replicas using events.

1. @crichardson
Events to the rescue: solving
distributed data problems in a
microservice architecture
Chris Richardson
Microservice architecture consultant and trainer
Founder of Eventuate.io
Founder of the original CloudFoundry.com
Author of POJOs in Action and Microservices Patterns
@crichardson
chris@chrisrichardson.net
http://adopt.microservices.io
Copyright © 2021. Chris Richardson Consulting, Inc. All rights reserved

2. @crichardson
What you will learn
How to use events to implement
transactions and queries in a
microservice architecture

3. @crichardson
About Chris
http://adopt.microservices.io

4. @crichardson
Discounts
35% discount
ctweda21
$120 discount coupon
HIDKQIFC
http://adopt.microservices.io

5. @crichardson
Agenda
• Why loosely coupled microservices?
• Distributed data challenges in a microservice architecture
• Transactions in a microservice architecture
• Querying in a microservice architecture

6. @crichardson
Microservice architecture
= architectural style
Online store
createCustomer(creditLimit)
createOrder(customerId, orderTotal)
findOrdersForCustomer(customerId)
findRecentCustomers()
Order
Service
Customer
Service
REST API
API
Gateway
Small team
Loosely coupled
Independently deployable
<15 minute lead time
Order team
Customer team

7. @crichardson
Why microservices: success triangle
Process: Lean + DevOps/Continuous Delivery & Deployment
Organization:
Network of small,
loosely coupled, teams
Architecture:
Microservices
(sometimes)
IT must deliver software
rapidly, frequently, reliably and sustainably:
Enables:
Loose
coupling
Enables:
Testability
Deployability
Businesses must be
nimble, agile and
innovate faster
S/W
VUCA
• Deployment frequency: >= once per developer/day
• < 15 minute lead time
• …

8. @crichardson
Loose coupling is essential
Services collaborate, e.g. Order Service must
reserve customer credit
Some coupling - degree of connectedness - is
inevitable
BUT
You must design services to be loosely coupled
API
Order
Service
Customer
Service
reserveCredit()
createOrder()

9. @crichardson
Order
Service
Customer
Service
PUT /customer/id/credit
POST /order
Order Customer
HTTP 200 OK
…
Runtime coupling
=
Both services must be available
Reduced availability
Design-time coupling
=
lockstep changes
Reduced productivity
Infrastructure
Infrastructure coupling
=
Competing for same resources
Misconfiguration bugs
Reduced availability

10. @crichardson
Agenda
• Why loosely coupled microservices?
• Distributed data challenges in a microservice architecture
• Transactions in a microservice architecture
• Querying in a microservice architecture

11. Sharing database tables =
tight coupling
Order
Service
Customer
Service
Database Server
Customer
table
Order
table
Competing for
same resources
Risk of
misconfiguration
Infrastructure coupling
Design-time coupling

12. Customer DB Server
Order DB Server
Customer Database
Use a Database-per service
Order
Service
Customer
Service
Order Database
Customer
table
Order
table
API
API
findOrdersForCustomer()
createOrder()
Operations that
span services

13. Database Server
Traditional distributed transactions =
runtime coupling
Order
Service
Customer
Service
Database Server
Order
Database
Customer
Database
createOrder()
2PC/XA Distributed Transaction
reserveCredit()
TXN ID
All participants must be
available to commit

14. @crichardson
Queries that do cross-database
joins violate encapsulation
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND c.id = ?
Private to Customer
Service
Private to Order
Service
Find customer and their orders

15. Events to the rescue
An event is something
notable that has occurred in
a domain, e.g.
• Order Created
• Order Confirmed
• Order Cancelled
• ..
Order
emits
Order created
Aggregate
(a.k.a business object)
createOrder()
Command creates,
updates, deletes aggregate

16. @crichardson
messaging system
An event is a type of message
Sender Recipient
Message
Channel types:
• Point-to-point - deliver to one recipient
• Publish-subscribe - deliver to all recipients
Channel
Payload
Header
Message types:
• Message - generic
• Event - something that has happened
• Command - a request to perform an operation
• Reply - a reply to a command
• …

17. @crichardson
API
Services exchange events
Operations
Event
Publisher
Commands
Queries
Synchronous
REST/gRPC
Asynchronous
Messaging
Events
Event
Subscriber
API
Client
Invokes
Operations
Events
DB
adapter
Business
logic

18. @crichardson
Use asynchronous, broker-
based messaging
Order Service
Customer
Service
….
Message broker
Highly scalable and available
At-least once, ordered delivery, durable subscriptions
DB DB

19. @crichardson
Reliable messaging requires
atomicity
Service
Database
Message Broker
update publish
How
to make
atomic
without
2PC?
Aggregate
Request

20. @crichardson
Atomically updating state and
publishing events
https://microservices.io/patterns/data/transactional-outbox.html
DELETE
Database
Order Service
Transaction
OUTBOX table
…
ORDER table
…
INSERT
INSERT, UPDATE, DELETE
Message
Broker
Message
Relay
1
2 Read OUTBOX
table
3 Publish
ACID
transaction

21. @crichardson
Agenda
• Why loosely coupled microservices?
• Distributed data challenges in a microservice architecture
• Transactions in a microservice architecture
• Querying in a microservice architecture

22. @crichardson
From a 1987 paper

23. @crichardson
Sagas: event-based transactions
https://microservices.io/patterns/data/saga.html

24. @crichardson
Order Service
Create Order Saga
Local transaction
Order
state=PENDING
createOrder()
Customer Service
Local transaction
Customer
reserveCredit()
Order Service
Local transaction
Order
state=APPROVED
approve
order()
createOrder() Initiates saga
Order ID
message message

25. @crichardson
Order Service
Create Order Saga - “rollback”
using a compensating transaction
Local transaction
Order
state=PENDING
createOrder()
Customer Service
Local transaction
Customer
reserveCredit()
Order Service
Local transaction
Order
state=REJECTED
reject
Order()
createOrder()
FAIL
Insufficient credit
message message

26. How to sequence the steps of
a saga?
After the completion of
transaction Ti/Ci
“something” must decide
what step to execute next
Success: T(i+1)
Failure: C(i - 1)
T1 C1
T2 C2
T… C…
Compensating
transactions
= “undo”

27. @crichardson
Choreography: distributed decision making
vs.
Orchestration: centralized decision making

28. @crichardson
Choreography = event-driven
sagas
Coordination logic = code that publishes events + event
handlers
Whenever a saga participant updates a business object, it
publishes (domain) events announcing what it has done
Saga participants have event handlers that update business
objects => …

29. @crichardson
Message broker
Choreography-based Create Order
Saga
Order created
Credit Reserved
Credit Limit Exceeded
Create Order
OR
Customer
availableCredit
...
Order
state
total
…
create()
reserveCredit()
approve()/
reject()
Order events channel
Customer events channel
Order
Service
Customer
Service
https://github.com/eventuate-tram/eventuate-tram-examples-customers-and-orders

30. @crichardson
Agenda
• Why loosely coupled microservices?
• Distributed data challenges in a microservice architecture
• Transactions in a microservice architecture
• Querying in a microservice architecture

31. @crichardson
Querying using the API
Composition pattern
Simple
but involves
runtime
coupling
and is
sometimes too
inefficient
https://microservices.io/patterns/data/api-composition.html

32. @crichardson
CQRS: event-based queries
https://microservices.io/patterns/data/cqrs.html
Query that spans
multiple services
• A replica
• Efficiently queryable

33. @crichardson
Pick a database that efficiently
supports the queries
Command side
POST
PUT
DELETE
Aggregate
Message Broker
Events
Query side
GET /customers/id
Document
database
Query side
GET /orders?text=xyz
Text
database
Query side
GET …
Graph
database
Query side
RDBMS
BI/Reporting

34. @crichardson
Summary
Rapid, frequent, reliable and sustainable software delivery
requires services to be loosely coupled
Sharing databases = tight design-time coupling Database
per service
Traditional distributed transactions = tight runtime coupling
Use the Saga pattern to implement commands that span
services
Use the CQRS pattern to implement queries that span services

35. @crichardson
@crichardson chris@chrisrichardson.net
http://adopt.microservices.io
Questions?
ctweda21 HIDKQIFC