Slides from my talk at Camunda Con Live on 24th of April 2020 about orchestrating and integrating microservices and the connection of choreography, observability and workflow automation

1. 1
Microservices Orchestration
and Integration
using Workflow Automation
Bernd Ruecker
Co-Founder and Chief Technologist

2. Service
A
Microservices
@berndruecker
Monolith
Service
B
Service
C
C
B
A
D

3. Change is the only constant
„ We don‘t know what we will need tomorrow.
But we do know that we will need something.
We have to be able to move quickly!
CIO of an insurance company in 2019

4. Service
A
Autonomous Microservices
@berndruecker
Service
B
Service
C

5. The era of APIs
API
API
API
API
API
API
API
API
@berndruecker

6. The era of APIs
API
API
API
API
API
API
API
External Services
Standard Software
API
Functions
@berndruecker

7. Connecting the dots…
API
API
API
API
API
API
API
Microservices
External Services
Standard Software
API
Functions
@berndruecker

8. Example
@berndruecker

9. Example
Payment
Seat
ReservationBooking
Ticket
Generation

10. Example
@berndruecker
sync

11. Weaknesses
Payment
Seat
ReservationBooking
Ticket
Generation
REST

12. Weaknesses: Latency creep
Payment
Seat
ReservationBooking
Ticket
Generation
REST
300 ms
1150 + x ms
600 ms
250 ms

13. Weaknesses: Availabiliy erosion
Payment
Seat
ReservationBooking
Ticket
Generation
REST
99 % uptime
99 % uptime
99 % uptime
96 % uptime

14. Example
@berndruecker

15. And it is even hard to implement
Payment
Seat
ReservationBooking
Ticket
Generation
REST

16. And it is even hard to implement
Payment
Seat
ReservationBooking
Ticket
Generation
REST

17. Workflow automation can help
Payment
Seat
ReservationBooking
Ticket
Generation
REST
Workflow Automation:
Is stateful
Can wait
Can retry
Can escalate
Can compensate

18. Typical pattern
Payment
Seat
ReservationBooking
Ticket
Generation
REST
Simulate synchronicty by waiting
(callback or polling)

19. Can your company
leverage your
hipster architecture?
Shutterstock
You need to
change business
processes and
customer
experience!

20. @berndruecker
happy
case
failure
case
Redesign your business process accordingly!
Or some interface
to poll for status
Sync in happy case
Async response

21. @berndruecker
Redesign your business process accordingly!

22. @berndruecker
https://www.reactivemanifesto.org/

23. Yeah!
Let‘s go reactive.

24. Phil Calcado at QCon NYC 2019

25. API
API
API
API
API
API
API
Microservices
External Services
Standard Software
„What the hell just happened?“
@berndruecker

26. Example:
order fulfillment via
dash button
Photo by 0xF2, available under Creative Commons BY-ND 2.0
license. https://www.flickr.com/photos/0xf2/29873149904/
@berndruecker

27. Three steps…
@berndruecker

28. (Micro-)services
Checkout
Payment
Inventory
Shipment
@berndruecker

29. Order
Placed
Payment
Received
Goods
Fetched
Notification
Checkout
Payment
Inventory
Shipment
Event-driven architecture
@berndruecker

30. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
@berndruecker

31. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
@berndruecker

32. The danger is that it's very easy to
make nicely decoupled systems with
event notification, without realizing
that you're losing sight of that larger-
scale flow, and thus set yourself up for
trouble in future years.
https://martinfowler.com/articles/201701-event-driven.html
@berndruecker

33. The danger is that it's very easy to
make nicely decoupled systems with
event notification, without realizing
that you're losing sight of that larger-
scale flow, and thus set yourself up for
trouble in future years.
https://martinfowler.com/articles/201701-event-driven.html
@berndruecker

34. The danger is that it's very easy to
make nicely decoupled systems with
event notification, without realizing
that you're losing sight of that larger-
scale flow, and thus set yourself up for
trouble in future years.
https://martinfowler.com/articles/201701-event-driven.html
@berndruecker

35. Process Events Monitoring
Camunda
Optimize
Order
placed
Payment
received
Goods
fetched
Goods
shipped
@berndruecker
Order
placed
Payment
received
Goods
fetched
Goods
shipped

36. Demo

37. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
@berndruecker

39. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
Fetch the goods
before the
payment
@berndruecker

40. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Goods
fetched
Order
placed
Payment
received
Goods
shipped
@berndruecker

41. What we wanted
Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and P..19 / CC BY-SA 4.0
@berndruecker

42. Order
Extract the end-to-end responsibility
Checkout
Payment
Inventory
ShipmentPayment
received
Order
placed
Retrieve
payment
@berndruecker

43. Order
Events & Commands
Checkout
Payment
Inventory
ShipmentPayment
received
Order
placed
Retrieve
payment
@berndruecker
Event
Command
Fact,
happened in the past,
immutable
Intend,
Want s.th. to happen

44. Order
It is not about the protocol!
Checkout
Payment
Inventory
Shipment
Order
placed
Retrieve
payment
It can still be messaging!
@berndruecker

45. Order
It is about where to decide about the coupling!
Checkout
Payment
Inventory
Shipment
Order
placed
Retrieve
payment
Order decides
. to listen to the event
. to issue the command
@berndruecker

46. Order
Checkout
Payment
Inventory
Shipment

47. Demo

48. One of the largest online fashion retailers
in the world (> 14.000 employees) does
every order fulfillment worldwide
with Camunda & Kafka
@berndruecker

49. Workflows live inside service boundaries
@berndruecker

50. No BPM(N) monoliths
https://blog.bernd-ruecker.com/avoiding-the-bpm-monolith-when-using-bounded-contexts-d86be6308d8
@berndruecker

51. Your ITarchitecture
Choreography
Orchestration
@berndruecker
Your services
or applications

52. Monolith Chaos
Choreography
Orchestration
@berndruecker
Process Monitoring
Your services
or applications

53. Your ITarchitecture
Process Monitoring
Monolith Chaos
Choreography
Orchestration
Your services
or applications
Balance choreography and orchestration
@berndruecker

54. Summary
• Business agility is essential to survive, this is the case
for microservices
• Collaboration of services is the new challenge
• Balance orchestration and choreography
• From choreography to orchestration can be a journey
• Workflow automation is a essential building block

55. Thank you!
@berndruecker