SlideShare a Scribd company logo
1 of 91
Download to read offline
@berndruecker
Are you done yet?
Mastering
long-running processes
in modern architectures
Let‘s talk about food
How does ordering Pizza work?
Pizza
Place
You
Phone Call
Synchronous blocking communication
Feedback loop (ack, confirmation or rejection)
Temporal coupling (e.g. busy, not answering)
Pizza
Place
You
Email
Asynchronous non-blocking communication
No temporal coupling
Pizza
Place
You
A feedback loop might make sense
(ack, confirmation or rejection)
Email
Confirmation Email
@berndruecker
Feedback loop != result
Pizza
Place
You
Email
Confirmation Email
Pizza Delivery
Feedback (ACK, confirmation, rejection)
Result
@berndruecker
Only the first communication step is synchronous / blocking
Pizza
Place
You
PUT/order
HTTP 200
Pizza Delivery
The task of
Pizza making is
long running
@berndruecker
Synchronous blocking behavior for the result?
Bad user experience
Does not scale well
@berndruecker
Scalable Coffee Making
https://www.enterpriseintegrationpatterns.com/ramblings/18_starbucks.html
Photo by John Ingle
@berndruecker
Long running
Long running
Long running basically means waiting
When do services want to wait? Some business reasons…
Human work Waiting for response
Let some time pass
Why is waiting a pain?
Persistent state How to solve the technical challenges without adding accidential
complexity?
Monitoring &
Operations
Visibility
Versioning
Scheduling &
Timeouts
Domain Logic
Scalability &
Resilience
Warning:
Contains Opinion
bernd.ruecker@camunda.com
@berndruecker
http://berndruecker.io/
Bernd Ruecker
Co-founder and
Chief Technologist of
Camunda
Workflow Engine aka Process Engine aka Orchestration Engine
Live hacking
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot
Customer Unboarding
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot
@berndruecker
There are also technical reasons
why services need to wait…
When do services need to wait? Some technical reasons…
Asynchronous communication
Wait for
responses
Wait for
availability
Especially failure scenarios
Unavailability of peers
Live hacking
https://github.com/berndruecker/customer-onboarding-camunda-8-springboot
Photo by Tookapic, available under Creative Commons CC0 1.0 license.
„There was an error
while sending your
boarding pass“
Check-in
Web-UI
Me
Current situation
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Failure will happen.
Accept it!
But keep it local!
Be resilient.
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part
Stateful
Retry
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!
Stateful
Retry
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Stateful
Retry
Possible situation – much better!
The failure
never leaves
this scope!
„But the customer wants a synchronous response!“
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
„Eh – no!“
Let‘s add some money…
@berndruecker
Booking Payment
Credit
Card
Payment
Stateful retry by using an orchestration engine
Credit
Card
REST
Distributed systems introduce more complexity
Credit
Card
Payment
REST
It is impossible to
differentiate certain
failure scenarios.
Independant of
communication style!
Service
Provider
Client
Distributed systems introduce more complexity
Credit
Card
Payment
REST
Distributed systems introduce more complexity
Credit
Card
Payment
REST
Cancel
charge
Using an orchestration engine allows to do long
running processes reliably, and end-to-end
retrievePayment
HTTP 200 OK
HTTP 202 ACCEPTED
Payment
Happy case: Synchronous response
Otherwise: asynchronous
Embrace asynchronicity
This allows to extend payment Options
@berndruecker
Further requirements when extending payment Options
begin commit
{local TX}
Customer Credit Service
begin commit
{local TX}
Credit Card Service
@berndruecker
Distributed transactions using compensation *
Compensation
@berndruecker
If you want to
dive into code…
https://github.com/berndruecker/flowing-retail
Long running capabilities
are essential to design
good service boundaries
(= a good architecture)
@berndruecker
Example
Booking Payment
Retrieve
Payment
@berndruecker
Example
Booking Payment
Credit
Card
Retrieve
Payment
@berndruecker
Example
Booking Payment
Credit
Card
Retrieve
Payment
Rejected
@berndruecker
Example
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Rejected
@berndruecker
???
Payment
failed
Who is responsible to deal with problems?
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Payment
received
@berndruecker
Payment
failed
(Potentially) long running services
Booking Payment
Credit
Card
Retrieve
Payment
Rejected
Payment
received
@berndruecker
Long running services
Booking Payment
Credit
Card
@berndruecker
Orchestration != monolithic processes
Booking Payment
Credit
Card
@berndruecker
Long running avoids „god services“
Booking Payment
If the credit
card was
rejected, the
customer can
provide new
details
Credit
Card
Retrieve
Payment
Rejected
Rejected
@berndruecker
A few
smart god services
tell
anemic CRUD services
what to do
Sam Newmann
Long running capabilities
are essential to design
good service boundaries
(= a good architecture)
@berndruecker
Being able to implement
long running services
makes it easy to distribute
responsibilities correctly
@berndruecker
Being able to implement
long running services
makes it easy to embrace
async/non-blocking
@berndruecker
You need a
process orchestration capability
that is available as-a-service
within your organization
to create competitive architectures
What we see most successful: Center of Excellence
https://camunda.com/process-orchestration/automation-center-of-excellence/
Focus: Enablement and platform
Business Unit
or Domain
Business Unit
or Domain
Business Unit
or Domain
Process Automation
Center of Excellence
Solution
Delivery
Solution
Delivery
Solution
Delivery
Solution
Delivery
Solution
Delivery
enable
But isn’t a central CoE
harming
team autonomy?
(This is a rhetorical question - the answer is of course NO)
@berndruecker
Center of
Excellence
Domain
Autonomy of teams?
Golden Paths
https://engineering.atspotify.com/2020/08/how-we-use-golden-paths-to-solve-fragmentation-in-our-software-ecosystem/
“We found that
rumour-driven development
simply wasn’t scalable”
@berndruecker
https://backstage.io/ (OSS, Made with at Spotify) @berndruecker
At Twilio, we do it by offering what we call the paved path.
These are mature services that you can just pull off the shelf,
adopt, and get up and running super quickly.
But if a team wants to diverge from that path they can make
that decision. You just still have to pass the same rigorous bar
of quality, reliability, and security.
So what you do is you create the incentive structure for teams
to take the paved path. It’s a lot easier. But if they really have
to go a different route, you make it possible for them to do
that.
Graphical models?
@berndruecker
BPMN
Business Process
Model and Notation
ISO Standard
@berndruecker
Living documentation for long-running behaviour
@berndruecker
Living documentation for long-running behaviour
@berndruecker
BizDevOps
@berndruecker
Decisions about
long running behavior
need to be elevated
to the business level
You need to redesign
your customer journeys
to leverage your
hipster architecture
Shutterstock
Yeah - everything
asynchronously :-)
Recap
• You need capabilities for long running behavior for technical and
business reasons
• Process orchestration platforms are a great fit
• This allows you to design better service boundaries, implement quicker
with less accidental complexity, embrace asynchronoucity and provide
a better customer experience (plus increase operational efficiency and
reduces risk)
• Organize central enablement (center of excellence, platform as-a-
service) for easy adoption at scale
@berndruecker
Want to learn more?
http://camunda.com/
https://www.berndruecker.io/ | bernd.ruecker@camunda.com

More Related Content

Similar to QCon London: Mastering long-running processes in modern architectures

opening keynote on the state of eCommerce
opening keynote on the state of eCommerceopening keynote on the state of eCommerce
opening keynote on the state of eCommerce
webhostingguy
 
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
Wednesday Webinars
 
Covenant Kickoff
Covenant KickoffCovenant Kickoff
Covenant Kickoff
covweb
 

Similar to QCon London: Mastering long-running processes in modern architectures (20)

MuCon London 2017: Break your event chains
MuCon London 2017: Break your event chainsMuCon London 2017: Break your event chains
MuCon London 2017: Break your event chains
 
JavaOne 2015 Devops and the Darkside CON6447
JavaOne 2015 Devops and the Darkside CON6447JavaOne 2015 Devops and the Darkside CON6447
JavaOne 2015 Devops and the Darkside CON6447
 
Reactive Summit 2020 - How state helps you to stay reactive
Reactive Summit 2020 - How state helps you to stay reactiveReactive Summit 2020 - How state helps you to stay reactive
Reactive Summit 2020 - How state helps you to stay reactive
 
Long running processes in DDD
Long running processes in DDDLong running processes in DDD
Long running processes in DDD
 
MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...
 
opening keynote on the state of eCommerce
opening keynote on the state of eCommerceopening keynote on the state of eCommerce
opening keynote on the state of eCommerce
 
20 Minutes on Desktop Analytics: Top Uses in the Back Office
20 Minutes on Desktop Analytics:  Top Uses in the Back Office 20 Minutes on Desktop Analytics:  Top Uses in the Back Office
20 Minutes on Desktop Analytics: Top Uses in the Back Office
 
Expertise Hour: The Dos and Don'ts of Web Chat with Johan Jacobs
 Expertise Hour: The Dos and Don'ts of Web Chat with Johan Jacobs Expertise Hour: The Dos and Don'ts of Web Chat with Johan Jacobs
Expertise Hour: The Dos and Don'ts of Web Chat with Johan Jacobs
 
One trunk one pipeline one truth
One trunk one pipeline one truthOne trunk one pipeline one truth
One trunk one pipeline one truth
 
JCon 2021 - Loosely or lousily coupled
JCon 2021 - Loosely or lousily coupledJCon 2021 - Loosely or lousily coupled
JCon 2021 - Loosely or lousily coupled
 
CamundaCon NYC 2023 Keynote - Shifting into overdrive with process orchestration
CamundaCon NYC 2023 Keynote - Shifting into overdrive with process orchestrationCamundaCon NYC 2023 Keynote - Shifting into overdrive with process orchestration
CamundaCon NYC 2023 Keynote - Shifting into overdrive with process orchestration
 
Approval Forms: Automate Your Forms With An Intranet
Approval Forms: Automate Your Forms With An IntranetApproval Forms: Automate Your Forms With An Intranet
Approval Forms: Automate Your Forms With An Intranet
 
JAX 2017 talk: Orchestration of microservices
JAX 2017 talk: Orchestration of microservicesJAX 2017 talk: Orchestration of microservices
JAX 2017 talk: Orchestration of microservices
 
The Business Value of PaaS Automation - Kieron Sambrook-Smith - Presentation ...
The Business Value of PaaS Automation - Kieron Sambrook-Smith - Presentation ...The Business Value of PaaS Automation - Kieron Sambrook-Smith - Presentation ...
The Business Value of PaaS Automation - Kieron Sambrook-Smith - Presentation ...
 
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
Wednesday Webinar Pt. II: Using online services and mobile apps to save time ...
 
Covenant Kickoff
Covenant KickoffCovenant Kickoff
Covenant Kickoff
 
2019 - Lost in transaction
2019 - Lost in transaction2019 - Lost in transaction
2019 - Lost in transaction
 
Introducing safex pay june2017
Introducing safex pay june2017Introducing safex pay june2017
Introducing safex pay june2017
 
Introducing safex pay june2017
Introducing safex pay june2017Introducing safex pay june2017
Introducing safex pay june2017
 
DevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh CityDevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh City
 

More from Bernd Ruecker

More from Bernd Ruecker (20)

iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
 
JCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problemsJCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problems
 
JFall - Process Oriented Integration
JFall - Process Oriented IntegrationJFall - Process Oriented Integration
JFall - Process Oriented Integration
 
JavaLand 2023 - Process Oriented Integration
JavaLand 2023 - Process Oriented IntegrationJavaLand 2023 - Process Oriented Integration
JavaLand 2023 - Process Oriented Integration
 
CraftConf: Surviving the hyperautomation low code bubbl
CraftConf: Surviving the hyperautomation low code bubblCraftConf: Surviving the hyperautomation low code bubbl
CraftConf: Surviving the hyperautomation low code bubbl
 
Mastering Data for Higher Business Impact - at Commerzbank Innovation Summit
Mastering Data for Higher Business Impact - at Commerzbank Innovation SummitMastering Data for Higher Business Impact - at Commerzbank Innovation Summit
Mastering Data for Higher Business Impact - at Commerzbank Innovation Summit
 
Camunda Chapter Hamburg - Surviving the hyperautomation low code bubble
Camunda Chapter Hamburg - Surviving the hyperautomation low code bubbleCamunda Chapter Hamburg - Surviving the hyperautomation low code bubble
Camunda Chapter Hamburg - Surviving the hyperautomation low code bubble
 
CamundaCon 2022 Keynote: The Process Orchestration Journey
CamundaCon 2022 Keynote: The Process Orchestration JourneyCamundaCon 2022 Keynote: The Process Orchestration Journey
CamundaCon 2022 Keynote: The Process Orchestration Journey
 
JFS 2021 - The Process Automation Map
JFS 2021 - The Process Automation MapJFS 2021 - The Process Automation Map
JFS 2021 - The Process Automation Map
 
CamundaCon 2021 Keynote :From Human Workflow to High-Throughput Process Autom...
CamundaCon 2021 Keynote :From Human Workflow to High-Throughput Process Autom...CamundaCon 2021 Keynote :From Human Workflow to High-Throughput Process Autom...
CamundaCon 2021 Keynote :From Human Workflow to High-Throughput Process Autom...
 
Kafka Summit 2021 - Apache Kafka meets workflow engines
Kafka Summit 2021 - Apache Kafka meets workflow enginesKafka Summit 2021 - Apache Kafka meets workflow engines
Kafka Summit 2021 - Apache Kafka meets workflow engines
 
Process Automation Forum April 2021 - Practical Process Automation
Process Automation Forum April 2021 - Practical Process AutomationProcess Automation Forum April 2021 - Practical Process Automation
Process Automation Forum April 2021 - Practical Process Automation
 
Micronaut Webinar 2021 - Process Automation Introduction
Micronaut Webinar 2021 - Process Automation IntroductionMicronaut Webinar 2021 - Process Automation Introduction
Micronaut Webinar 2021 - Process Automation Introduction
 
Webinar "Communication Between Loosely Coupled Microservices"
Webinar "Communication Between Loosely Coupled Microservices"Webinar "Communication Between Loosely Coupled Microservices"
Webinar "Communication Between Loosely Coupled Microservices"
 
Automating Processes in Modern Architectures
Automating Processes in Modern ArchitecturesAutomating Processes in Modern Architectures
Automating Processes in Modern Architectures
 
OOP 2021 - Leverage the full potential of your hipster architecture
OOP 2021 - Leverage the full potential of your hipster architectureOOP 2021 - Leverage the full potential of your hipster architecture
OOP 2021 - Leverage the full potential of your hipster architecture
 
GOTOpia 2020 - Balancing Choreography and Orchestration
GOTOpia 2020 - Balancing Choreography and OrchestrationGOTOpia 2020 - Balancing Choreography and Orchestration
GOTOpia 2020 - Balancing Choreography and Orchestration
 
CamundaCon 2020 Keynote - The Return of Process Automation
CamundaCon 2020 Keynote - The Return of Process AutomationCamundaCon 2020 Keynote - The Return of Process Automation
CamundaCon 2020 Keynote - The Return of Process Automation
 
Destination Automation: Automating Processes in Modern Hipster Architectures
Destination Automation: Automating Processes in Modern Hipster ArchitecturesDestination Automation: Automating Processes in Modern Hipster Architectures
Destination Automation: Automating Processes in Modern Hipster Architectures
 
Kafka Summit 2020: If an event is published to a topic and no one is around t...
Kafka Summit 2020: If an event is published to a topic and no one is around t...Kafka Summit 2020: If an event is published to a topic and no one is around t...
Kafka Summit 2020: If an event is published to a topic and no one is around t...
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 

QCon London: Mastering long-running processes in modern architectures