Dark energy and dark matter are useful metaphors for the repulsive forces, which encourage decomposition into services, and the attractive forces, which resist decomposition. You must balance these conflicting forces when defining a microservice architecture including when designing system operations (a.k.a. requests) that span services. In this talk, I describe the dark energy and dark matter forces. You will learn how to design system operations that span services using microservice architecture collaboration patterns: Saga, Command-side replica, API composition, and CQRS patterns. I describe how each of these patterns resolve the dark energy and dark matter forces differently.

1. @crichardson
Dark energy, dark matter and
microservice architecture
collaboration patterns
Chris Richardson
Microservice architecture consultant and trainer
Founder of the original CloudFoundry.com
Author of POJOs in Action and Microservices Patterns
Founder of Eventuate.io
@crichardson
chris@chrisrichardson.net
http://adopt.microservices.io
Copyright © 2022. Chris Richardson Consulting, Inc. All rights reserved

2. @crichardson
Presentation goal
How to design system
operations that span multiple
services

3. @crichardson
About Chris
http://adopt.microservices.io

4. @crichardson
Discounts
35% discount
au35ric
$100 discount coupon
GBXCCPJD until July 11th
http://adopt.microservices.io

5. @crichardson
Agenda
Dark energy, dark matter and de
fi
ning a microservice
architecture
Overview of service collaboration patterns
Designing system commands
Designing system queries

6. @crichardson
How to de
fi
ne an
Architecture…
Application
≪subdomain≫
Customer
management
≪aggregate≫
Customer
≪subdomain≫
Order
management
≪aggregate≫
Order
createCustomer()
createOrder()
fi
ndOrder()
fi
ndOrderHistory()
System operations
Distill
Requirements
Event storming
Functional requirements
As a consumer
I want to place an Order
So that I can ….
As a Restaurant
I want to accept an Order
So that I can ….
User stories
• SLA: Reliability/Latency
• Scalability
• …
System quality attributes
The “requests” that the
application implements
Customer Team
Order Team
About Subdomains
• Business capability/function/etc
• Logical view: packages and classes
• Team-sized
• Loosely coupled (Conways law)
1
2

7. @crichardson
Application
… how to de
fi
ne an
Architecture…
Application
≪subdomain≫
Customer
≪aggregate≫
Customer
≪subdomain≫
Order
≪aggregate≫
Order
createCustomer()
createOrder()
fi
ndOrder()
fi
ndOrderHistory()
System operations
Component
≪subdomain≫
≪subdomain≫
≪subdomain≫
≪subdomain≫
How to group
subdomains
into
components*?
Component Component
* Component = deployable, executable, e.g. WAR
fi
le, executable JAR, …
3
Deployable, Executable,
e.g. WAR
fi
le, executable
JAR, …

8. @crichardson
Application
… how to de
fi
ne an
Architecture
createCustomer()
createOrder()
fi
ndOrder()
fi
ndOrderHistory()
System operations
Component
≪subdomain≫
≪subdomain≫
Component
≪subdomain≫
≪subdomain≫
Component
Design
collaborations*
Distributed
4
*Collaboration =
interactions via REST,
messages etc.

9. @crichardson
Grouping subdomains into
components: together or separate?
≪subdomain≫
Customer
≪aggregate≫
Customer
≪subdomain≫
Order
≪aggregate≫
Order
Attraction
Repulsion
Generated by
system operations
Simple components
Team-sized services
Fast deployment pipeline
…
Dark energy: an anti-
gravity that’s accelerating
the expansion of the
universe
Dark matter: an invisible
matter that has a
gravitational effect on stars
and galaxies.
https://www.nasa.gov/feature/goddard/2020/new-hubble-data-explains-missing-dark-matter
Simple interactions
Prefer ACID or BASE
Minimize runtime coupling
…
https://chrisrichardson.net/post/microservices/2021/11/30/dark-matter-dark-energy.html

10. @crichardson
Dark energy: repulsive forces
subdomains in different services
https://chrisrichardson.net/post/microservices/2021/11/30/dark-matter-dark-energy.html
Service
Service
«Subdomain» A
«Aggregate»
X
«Subdomain» B
«Aggregate»
Y
Simple components
Team autonomy
Fast deployment pipeline
Support multiple technology stacks
Cost effective scaling
Segregate regulated software
Segregate highly available components

11. @crichardson
Dark matter: attractive forces
subdomains in same service
https://chrisrichardson.net/post/microservices/2021/11/30/dark-matter-dark-energy.html
Subdomain A
«Aggregate»
X
Subdomain B
«Aggregate»
Y
Service A Service B
Simple interactions
Prefer ACID over BASE
Minimize runtime coupling
Efficient communication
Minimize design time coupling
SystemOperation()
Generates

12. @crichardson
The role of the architect is to
balance con
fl
icting forces
Minimize attraction
Service
Subdomain
Subdomain
Minimize
repulsion
Service
Subdomain
Subdomain
Minimize
repulsion
Con
fl
icting goals

13. @crichardson
Balancing forces is an iterative process
Partition subdomains to
de
fi
ne services to resolve
dark energy forces
Design system operations
(service collaborations) to
resolve dark matter forces
Adjust service de
fi
nitions to enable
resolution of dark matter forces

14. @crichardson
Agenda
Dark energy, dark matter and de
fi
ning a microservice
architecture
Overview of service collaboration patterns
Designing system commands
Designing system queries

15. @crichardson
After partitioning subdomains into services:
• Simple interactions
• Prefer ACID over BASE
• Minimize runtime coupling
• Ef
fi
cient inter-service communication
• Minimize design time coupling
createCustomer()
Customer
Service
createOrder()
Order
Service
Restaurant
Service
Customer
Service
Collaboration
✅ ?
Local operation Distributed operation
≪entity≫
Restaurant
≪entity≫
Customer
≪entity≫
Order
≪entity≫
Customer
ACID transaction ??? transaction
Owned by
service

16. @crichardson
Distributed system operation =
ACID transaction
Customer
Service
Restaurant
Service
createOrder()
Order Service
Distributed ACID transaction
BEGIN transaction
…
COMMIT transaction

17. @crichardson
Pattern: Shared database schema
Order Service
Customer
Service
Order
table
Customer
table
createOrder()
ACID
Transaction
(Maybe 2PC)
https://microservices.io/patterns/data/shared-database.html
reserveCredit()
creditOrder()
… Service
… table
BEGIN transaction
…
COMMIT transaction

18. @crichardson
Bene
fi
ts and drawbacks: shared schema
Dark energy repulsive forces
Support multiple technology stacks ❌ DBs be able to participate in same transaction
Dark matter attractive forces
Simple interactions ✅ No service-service interaction!?
Prefer ACID over BASE ✅ ACID transaction
Minimize runtime coupling ❌ DB-level coupling/interference
Ef
fi
cient inter-service communication ✅ No service-service interaction!?
Minimize design time coupling
❌ Schema changes must be coordinated
❌ Duplicated business logic

19. @crichardson
Pattern: Private database schema
Order Service
Customer
Service
Order
table
Customer
table
API
createOrder()
2PC
Transaction
https://microservices.io/patterns/data/database-per-service.html
reserveCredit()
BEGIN transaction
…
COMMIT transaction

20. @crichardson
Private database + 2PC: Bene
fi
ts and drawbacks
Dark energy repulsive forces
Support multiple technology stacks
❌ Participants must use 2PC-
compatible technology
Dark matter attractive forces
Simple interactions ? Depends on operation
Prefer ACID over BASE
✅
Minimize runtime coupling ❌ all participants must be available
Ef
fi
cient inter-service communication ? Depends on operation
Minimize design time coupling ? Depends on operation

21. @crichardson
Distributed system operation =
Basically Available, Soft state, Eventually
consistent transaction
Service A
Transaction scope
=
service
distributedOperation()
BEGIN transaction
…
COMMIT transaction
Service B
BEGIN transaction
…
COMMIT transaction
Service C
BEGIN transaction
…
COMMIT transaction

22. @crichardson
ACID vs. BASE
2PC BASE
Dark energy repulsive forces
Support multiple technology stacks ❌ ✅
Dark matter attractive forces
Prefer ACID over BASE ✅ ❌
Minimize runtime coupling ❌ ✅ (Mostly)

23. @crichardson
Eventually consistent service
collaboration patterns
API Composition
Provider
Service
Service
query()
Composer
Provider
Service
Provider
Service
CQRS
Provider
Service
Service
query()
View
Provider
Service
Provider
Service
Event
Event
Event
Service
Service Service
Transaction
Compensating
transaction
Transaction
Compensating
transaction
Transaction
Compensating
transaction
command()
Saga
Service
Service
command()
Replica Source
Event
Command-side replica
Commands
Queries

24. @crichardson
Agenda
Dark energy, dark matter and de
fi
ning a microservice
architecture
Overview of service collaboration patterns
Designing system commands
Designing system queries

25. @crichardson
Let’s imagine that you are implementing
the createOrder() command…
Service Operation Potential failure
Order Service Create Order Invalid quantities
Restaurant Service
Validate and price menu
items (Order subtotal)
Invalid menu items
Customer Service Reserve credit Insuf
fi
cient credit
Kitchen Service Create Ticket Out of stock ingredient

26. @crichardson
Command patterns
Service
Service Service
Transaction
Compensating
transaction
Transaction
Compensating
transaction
Transaction
Compensating
transaction
command()
Saga
Service
Service
command()
Replica Source
Event
Command-side replica

27. @crichardson
From a 1987 paper Service
Service Service
Transaction
Compensating
transaction
Transaction
Compensating
transaction
Transaction
Compensating
transaction
command()
Saga

28. @crichardson
Saga = sequence of local transactions
Service A
Local ACID
transaction
Service B
Local ACID
transaction
Service C
Local ACID
transaction
https://microservices.io/patterns/data/saga.html

29. @crichardson
The CreateOrderSaga
Step Participant Transaction
Compensating
transaction
1 Order Service createOrder() rejectOrder()
2 Restaurant Service validateAndPrice() -
3 Order Service updateSubtotal() -
4 Customer Service reserveCredit() releaseCredit()
5 Kitchen Service createTicket()
6 Order Service approveOrder() Can fail!
Undo
Lock
Unlock
Unlock
Semantic lock countermeasure = Semantic ACID
Can fail!
Need to rollback

30. @crichardson
Saga coordination =
sequencing of steps
Choreography: distributed decision making
vs.
Orchestration: centralized decision making

31. @crichardson
Choreography-based Create Order
Saga
Order Created
Credit Reserved
Create Order
Order
state
total
…
create()
…
Order events channel
Customer events channel
Order
Service
Customer
Service
Restaurant
Service
Restaurants events channel
Order total calculated
MenuItems Priced
Ticket Created
Kitchen
Service
Kitchen events channel
Order credit reserved
approve()

32. @crichardson
Order Service
CreateOrderSaga orchestrator
Restaurant
Service
Order
state
total…
reserveCredit()
CreateOrder
Saga
OrderService
create()
create()
…
Items Priced
Restaurant command channel
Saga reply channel
Create Order Order ID
Customer Service
Customer command channel
validateAndPrice()
Credit Reserved
updateSubtotal()
Kitchen Service
Kitchen command channel
approve()

33. @crichardson
Bene
fi
ts and drawbacks of Sagas
Dark energy repulsive forces
Support multiple technology stacks
✅
Dark matter attractive forces
Simple interactions ? Depends on command
Prefer ACID over BASE ❌
Minimize runtime coupling ? Depends on command
Ef
fi
cient inter-service communication ? Depends on command
Minimize design time coupling ? Depends on command

34. Force: Prefer ACID over BASE ❌
No automatic undo =>
extra work to implement
compensating
transactions
Potential complexity of
undoing changes to data
that was changed by
another transaction
Create Order Saga Create Order Saga
Order Service:
createOrder()
Customer Service:
reserveCredit()
Order Service:
createOrder()
Customer Service:
reserveCredit()
Kitchen Service:
createTicket()
Customer Service:
releaseCredit()
FAILS!
Compensating transaction:
• Undoes change to available credit
• Fortunately: add/subtract commute
Updates
available credit

35. Force: Prefer ACID over BASE ❌
Sagas are ACD - lack I =
isolation => concurrent
execution => potential data
anomalies:
Dirty reads
Lost updates
Fuzzy reads
Must implement
countermeasures = application-
level ACID, e.g. Semantic lock
Cancel Order Saga Create Order Saga
Order Service:
cancelOrder()
Customer Service:
releaseCredit()
Order Service:
createOrder()
Customer Service:
reserveCredit()
Kitchen Service:
createTicket()
Customer Service:
reserveCredit()
Dirty read:
Exceeds customer
credit limit
FAILS!
Compensating transaction

36. @crichardson
Force: Minimize runtime coupling -
depends on when response is sent
Order
Service
createOrder()
Order Created
1
Response
2
3
✅
Order
Service
createOrder()
Ticket Created
1
Response
7
8
Self-Contained
Restaurant
Service
Customer
Service
Kitchen Service
Tightly coupled ❌

37. @crichardson
Force: minimize design-time
coupling
Order
Service
Restaurant
Service
Customer
Service
Order
Service
Restaurant
Service
Customer
Service
Choreography Orchestration
Participants depend on each other Orchestrator depends on participants
Inappropriate
knowledge of
Orders? Simple API

38. @crichardson
Sagas are potentially complex
BUT
They can be simple
It depends
Order Service
createOrder()
Ful
fi
llment Service
Order Created

39. @crichardson
Service
Service
command()
Replica Source
Event
Command-side replica

40. @crichardson
Using a command-side replica
Order Service
createOrder()
Restaurant
Service
Orders
Restaurants
Restaurant events
Replica
Validate and price line
items without invoking
Restaurant Service
Restaurant
MenuItem
MenuItem
createRestaurant()

41. @crichardson
Bene
fi
ts and drawbacks of Command-side replica
Dark energy repulsive forces
Support multiple technology stacks ✅ e.g. even different DBs
Dark matter attractive forces
Simple interactions
✅ Eliminates interaction with participant
❌ Other cmds must publish events
Prefer ACID over BASE ❌ Risk of stale/inconsistent data
Minimize runtime coupling
✅ Command does not interact with
participant
Ef
fi
cient inter-service communication
✅ Command is less distributed
❌ Cost of replication - storage + events
Minimize design time coupling
? Risk of tightly coupling view to
providers

42. @crichardson
Force: Minimize runtime coupling -
non-local but self-contained operation
Order Service
createOrder()
Orders
Restaurants
Response
Outcome of
validating and
pricing line
items
https://microservices.io/patterns/decomposition/self-contained-service.html

43. @crichardson
Agenda
Dark energy, dark matter and de
fi
ning a microservice
architecture
Overview of service collaboration patterns
Designing system commands
Designing system queries

44. @crichardson
Let’s imagine that you are
implementing the
fi
ndOrder() query
Service Operation
Order Service Status of order: line items
Kitchen Service Status of ticket: ready for pickup
Delivery Service Status of delivery: ETA
Accounting Service Status of payment: total including tax

45. @crichardson
Query patterns
API Composition
Provider
Service
Service
query()
Composer
Provider
Service
Provider
Service
CQRS
Provider
Service
Service
query()
View
Provider
Service
Provider
Service
Event
Event
Event

46. @crichardson
API Composition pattern
Order Service
Order
status
Delivery
Service
Delivery
Accounting
Service
Order
status
API Gateway
GET /orders/id
GET /orders?orderId=id
GET /orders/id
GET /deliveries?orderId=id
status
eta
Kitchen
Service
Ticket
status
prepareBy
GET /tickets?orderId=id
{… response …}
API
Composer
Providers
Composer

47. @crichardson
Bene
fi
ts and drawbacks of API Composition
Dark energy repulsive forces
Support multiple technology stacks
✅
Dark matter attractive forces
Simple interactions ? Possibility of many roundtrips
Prefer ACID over BASE
❌ Multiple queries - potential for inconsistencies
Minimize runtime coupling
❌ Use resiliency patterns (eg. Circuit breaker) to
reduce impact
Ef
fi
cient inter-service communication
? Parallelization can reduce latency
Possibility of many roundtrips or large data transfers
Minimize design time coupling ❌ Composer coupled to providers

48. @crichardson
Inef
fi
cient API composition:
fi
ndOrderHistory()
Order Service
Order
status
Delivery Service
Delivery
Accounting Service
Order
status
API Gateway
GET /orders?
consumerId=abc&
text=noodles&offset=ABC
GET /orders?
consumerId=abc&
text=noodles&offset=ABC
GET /deliveries?
consumerId=abc&
text=noodles&offset=ABC
GET /orders?
consumerId=abc&
text=noodles&offset=ABC
X X
No line items for text search
Different sort order
status
eta
Order
Line
Item
Composer

49. @crichardson
CQRS
Provider
Service
Service
query()
View
Provider
Service
Provider
Service
Event
Event
Event

50. @crichardson
Implementing a query using CQRS
Order
Service
Delivery
Service
Order events
Delivery events
fi
ndOrderHistory()
Order events channel
Delivery events channel
Order
History
Service
Replica
View
Database
https://microservices.io/patterns/data/cqrs.html
… Service
… events channel
… events

51. @crichardson
Bene
fi
ts and drawbacks of CQRS
Dark matter repulsive forces
Support multiple technology stacks
✅ View DB different than Provider DBs
Dark matter attractive forces
Simple interactions
✅ Query is local
❌ Other cmds must publish events
Prefer ACID over BASE ❌ Out of order events => Inconsistencies
Minimize runtime coupling ✅
Ef
fi
cient inter-service communication
✅ Query is self-contained
❌ Cost of replication - storage + events
Minimize design time coupling
? Risk of view being tightly coupled to
providers

52. @crichardson
Summary: a microservice architecture
must balance con
fl
icting forces
Service
Subdomain
Subdomain
Service
Subdomain
Subdomain
Minimize dark
energy repulsive
forces
Con
fl
icting
Minimize dark matter
attractive forces
systemOperation()
Minimize dark
energy repulsive
forces
generates

53. @crichardson
Summary: design system operations
using the collaboration patterns
API Composition
Provider
Service
Service
query()
Composer
Provider
Service
Provider
Service
CQRS
Provider
Service
Service
query()
View
Provider
Service
Provider
Service
Event
Event
Event
Service
Service Service
Transaction
Compensating
transaction
Transaction
Compensating
transaction
Transaction
Compensating
transaction
command()
Saga
Service
Service
command()
Replica Source
Event
Command-side replica
Distributed ACID transactions
X

54. @crichardson
Summary:
Adjust service boundaries if dark
matter forces cannot be suf
fi
ciently
resolved

55. @crichardson
@crichardson chris@chrisrichardson.net
http://adopt.microservices.io
Questions?
au35ric GBXCCPJD