Event-Driven Microservices architecture has gained a lot of attention recently. The trend in the industry is to move away from Monolithic applications to Microservices to innovate faster. While Microservices have their benefits, implementing them is hard. This talk focuses on the challenges faced and how to solve them. It covers topics like using Domain Driven Design to break functionality into small parts. Various communication patterns among Microservices are also discussed. One major drawback is the problem of distributed data management, as each Microservice has its own database. Event-Driven Architecture enables a way to make microservices work together and the talks show how to use architectural patterns like Event Sourcing & CQRS to implement them. Another implementation challenge is to manage transactions that update entities owned by multiple services in an eventually consistent fashion. This challenge is solved using sagas, which can be thought of as Long running transactions that use compensating actions to handle failures. The objective of the talk is to show how to implement highly distributed Event Driven Microservices architecture that are scalable and easy to maintain.

1. Patterns & Practices for Implementing
Event-driven Microservices
Nikhil Barthwal
@nikhilbarthwal
nikhilbarthwal@hotmail.com
www.nikhilbarthwal.com

2. Agenda (1/2)
• Motivations to move from Monolithic to Microservices
• Understanding the Distributed Data in Microservices
• Using Domain Driven Design for partitioning data
• Problems with Distributed Data
• Event Driven Microservices
2

3. Agenda (2/2)
• Communication Patterns for Microservices
• Event Sourcing
• Command Query Responsibility Segregation
• Introducing Sagas
• Resources for further reading
3

4. Monolithic: The old school approach …
• Large code base → Slower development
• More time to test & deploy
• Too much team interdependency
Database
Web Service
Easy to Develop, Test and Deploy …
But doesn’t scale!
4

5. Microservices: The Brave new world ...
• Smaller code base to understand & change
• Autonomous teams
• Faster testing & deployment
Assembly of Loosely
coupled services!
Image Source: https://www.slideshare.net/JontheBeach/distributed-sagas-a-protocol-for-coordinating-microservices
5

6. Loose Coupling → Distributed Data
Database per Service!
Faster Innovation!
Loosely coupled services
Reduces team interdependencies
Image Source: http://microservices.io/patterns/microservices.html
6

7. How to create service boundaries?
An approach to software development for complex needs
by connecting the implementation to an evolving model.
• Entities
• Value Objects
• Bounded Context
• Aggregates
Infrastructure Layer
Domain Layer
Application Layer
User Interface
Use Domain Driven Design (DDD)!
7

8. DDD: Aggregates
• Cluster of domain objects that
can be treated as a single unit
• One of its component is root
• All outside reference would
only go to the aggregate root
Domain Model
=
Collection of Aggregates
8

9. Data Partition Strategies: Use DDD
Aggregates
provide
boundaries
for
partition
9

10. Problems with Distributed Data
• How do we keep data consistent?
• How do we query scattered data?
10

11. Cannot use ACID Transactions
BEGIN TRANSACTION
SELECT ADDRESS FROM CUSTOMERS WHERE CUSTOMER_ID = XXX
SELECT PRICE FROM PRODUCTS WHERE CUSTOMER_ID = YYY
INSERT INTO ORDERS ...
COMMIT TRANSACTION
Private to Customer service
Private to Product service
11

12. Cannot use 2-Phase Commit
• Coordinator is a single point of failure
• Very chatty, with O(4n) messages, O(n2) with retries
• Locks would decrease throughputs
• Impacts availability (CAP theorem limitations)
Guarantees consistency, but …
12

13. Consistency Eventual Consistency
Availability
Partition (Network)
Eventual Consistency
Use Event Driven Microservice Architecture!
13

14. Event Driven Architecture
• Event occurs when a change happens in system
• All listeners get notified of the event, may take action
• Highly distributed/loosely coupled architecture
• Often used for asynchronous flows of information
14

15. Communication: Direct Messaging
Benefits:
• Easier implementation
• Better Error control
Drawbacks:
• Increased coupling
• Difficult to scale
Order Service
Product
Service
Customer
Service
Fulfillment
Service
Payment
Service
Shipping
Service
15

16. Communication: Via Message broker
Benefits:
• More scalable
• Increased Decoupling
Drawbacks:
• Requires Broker as Highly
Available component
Order Service
Product
Service
Customer
Service
Fulfillment
Service
Payment
Service
Shipping
Service
Message Broker
16

17. Communication Patterns
Request/Response
(synchronous)
Event-Driven
(asynchronous)
Tier-3: Little to No impact for Customer
Tier-2: Impacting Customers indirectly
Tier-1: Directly Customer facing
17

18. Event Sourcing: Introduction
• Modeling state changes as sequence of events
• Storing the event that could trigger the state change
• Enables rolling back to particular time in history
Examples:
18

19. Event Sourcing: Snapshots
Alternative to playing events: Snapshots
19

20. Event Sourcing: Multiple views
Image Source: https://www.confluent.io/wp-content/uploads/2016/09/Event-sourced-based-architecture.jpeg
Adding applications that process event …
but create a different view!
20

21. Event Sourcing: Benefits & Drawbacks
Benefits:
• 100% accurate audit logging
• Easy temporal queries
• Applications can process
same events but create views
Drawbacks:
• Adds Complexity
• No Strict Consistency
• Longer bootup times
(Snapshots can help)
21

22. Command Query Responsibility
Segregation
CQRS Pattern
Must for Event Sourcing!
Read
Write
Image Source: https://martinfowler.com/bliki/CQRS.html
22

23. CQRS: Benefits & Drawbacks
Benefits:
• Needed for Event Sourcing
• Improved separation of
concerns
• Supports scalable multiple
denormalized views
Drawbacks:
• Increased complexity
• Potential code duplication
• Replication lag as No Strict
Consistency
23

24. Sagas: Introduction
• Based on a 1987 paper
• Initially for a single database running
on one node
• Now adapted for distributed systems
with asynchrony and partial failure
24

25. Transaction & Rollback Transaction
• Every Transaction has a Rollback transaction
• This logic must be included in the service
T1 T2 T3
R1 R2 R3
Transactions →
Rollback Transactions →
25

26. Sagas: Example
This action initiates the saga
Check Credit Service Withdrawal Service Deposit Service
Deposit Check
Long running transactions …
use compensating actions to handle failures!
Does Sender have
enough money in his
account?
Withdraw the money
(unless it is on hold)?
Deposit the money
(unless account is
deactivated)?
26

27. Resources
• Using DDD to develop Microservices: https://www.infoq.com/news/2016/02/ddd-
microservices
• Overview of Saga Pattern: http://microservices.io/patterns/data/saga.html
• Distributed Sagas: https://www.slideshare.net/JontheBeach/distributed-sagas-a-
protocol-for-coordinating-microservices
27

28. Summary
• Microservices enable faster innovation
• DDD can used to define service boundaries
• Data is distributed which create challenges
• Use Event Driven Architecture for eventual consistency
• Understand Event Sourcing & CQRS
• Use Sagas to maintain data consistency across services
28

29. Thank you!
29
Nikhil Barthwal
Questions?
@nikhilbarthwal
nikhilbarthwal@hotmail.com
www.nikhilbarthwal.com