This document discusses workflow automation in microservices architectures. It notes the challenges of coordinating microservices and losing visibility of end-to-end flows. It proposes using workflow engines to orchestrate microservice interactions through commands and events while ensuring end-to-end flow manageability through monitoring. Key points are balancing orchestration with loose coupling, using BPMN to support integration use cases, and having ownership of workflows within service boundaries.

1. Workflow Automation in
Microservices Architectures
@NiallDeehan

2. Who I am?
Niall Deehan Neil De Han

3. What? Why?
Microservices & Workflow Automation
How?
t

4. What? Why?
Microservices & Workflow Automation
How?
t

5. Microservices…
Service
B
Some
Service
Some
Service Some
Service
Service
C
Service
A
Service
D
Monolith
Functionality
A
Functionality
B
Functionality
C
Functionality
D

6. Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Microservices…

7. Challenges

8. What we wanted
Photo by Lijian Zhang, available under Creative Commons SA 2.0 License / CC BY-SA 4.0

9. How do we know whats going on?
Photo by pxhere, available under Creative Commons CC0 1.0 License.

10. Example: http://github.com/berndruecker/flowing-retail
Checkout
Payment
Inventory
Shipment
?

11. Order
Checkout
Payment
Inventory
Shipment
An „orchestration process“?

12. Every workflow must be
owned by one microservice.
No exceptions!

13. Order
Checkout
Payment
Inventory
Shipment

14. Otherwise you end up with SOA again!
Checkout
Payment
Inventory
Shipment
Order

15. No BPM(N) monoliths
https://blog.bernd-ruecker.com/avoiding-the-bpm-monolith-when-using-bounded-contexts-d86be6308d8

16. Workflows live inside service boundaries

17. Order
Placed
Payment
Received
Goods
Fetched
Notification
Order
Checkout
Payment
Inventory
Shipment
Event-driven architecture

18. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched

19. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched

20. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

21. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

22. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

23. Finding the right balance
33
Observe Microservice Paradigms
● Loose Coupling
● Autonomy
● Event Collaboration
Ensure Flow Manageability
• End-To-End Visibility
• Meet Business Process SLA
• Support Continuous Process
Improvement

24. Tracking
Checkout Inventory
Payment Shipment
Event Bus
Workflow
Engine
https://www.confluent.io/kafka-summit-sf18/the_big_picture

25. Tracking + BI
Checkout Inventory
Payment Shipment
Event Bus
Workflow
Engine
Camunda
Optimize
Other reporting
/ BIO solutions

26. But you can start acting upon certain events

27. Order
Checkout
Payment
Inventory
Shipment
Orchestration

28. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
Fetch the goods
before the
payment

29. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Fetch the goods
before the
payment
Goods
fetched
Order
placed
Payment
received
Goods
shipped

30. Orchestration makes it easy to change the flow

31. Order
Commands vs. Events
Checkout
Payment
Inventory
Shipment
Order
placed
Retrieve
payment

32. Order
It is about where to decide about the coupling!
Checkout
Payment
Inventory
Shipment
Order
placed
Retrieve
payment
Order decides
. to listen to the event
. to issue the command

33. Microservices =
Distributed systems

34. Order
Checkout
Payment
Inventory
Shipment
Stateful orchestration
This orchestration
requires state

35. BPMN supports integration use cases
Aggregator
Re-Sequencer
Combined with timeouts:

36. BPMN supports integration use cases
Compensation

37. Order
Checkout
Payment
Inventory
Shipment
Orchestration

38. Order
Checkout
Payment
Inventory
Shipment
Orchestration
Orchestration is independant of
communication channel!
This might be REST, Messaging,
Event-Bus, …

39. Workflow Engine
Typical ways of communication
Checkout
Payment
Order
Shipment Checkout
Payment
Order
Shipment
Checkout
PaymentOrder
Shipment
Message/Event Bus REST-ish

40. Workflow Engine
Typical ways of communication
Checkout
Payment
Order
Shipment Checkout
Payment
Order
Shipment
Checkout
PaymentOrder
Shipment
Message/Event Bus REST-ish
Pub/SubOwnership

41. Checkout
Payment
Order
Shipment
Workflow
Engine
Workflow
Engine
Monitoring
Checkout
Payment
Order
Shipment
Workflow
Engine
Monitoring
Checkout
Payment
Order
Shipment
Workflow
Engine
Workflow
Engine
Monitoring
How many workflow engines?

42. Monitoring
Checkout Inventory
Payment Shipment
Workflow
Engine
Order
Workflow
Engine
Monitor
e.g.
Elastic
Links back
Optimize

43. Example
https://github.com/berndruecker/flowing-retail/tree/master/kafka/java/monitor

44. What we currently build with some customers…
Camunda
Optimize
Elastic
17
Registration
requested
Credit
checked
Address
checked
Customer
registered
@berndruecker

45. Recap
• Microservices have its merits
• You need to balance orchestration and chorography
• Track, monitor or manage the flow – depending on your scenario
• Distributed systems need stateful orchestration
• Visibility is essential to survive
• Workflow automation is an essential building block,
make sure to use BPMN

46. Thank you!