Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QCon NYC 2019 - Workflow automation reinvented

286 views

Published on

Slides from my talk "Workflow Automation Reinvented" at QCon 2019 in New York City

Published in: Technology
  • Be the first to comment

QCon NYC 2019 - Workflow automation reinvented

  1. 1. Workflow Automation Reinvented @berndruecker
  2. 2. What people think, when I say „Workflow Automation“…
  3. 3. What I think, when I say „Workflow Automation“ …
  4. 4. What people think, when I say „BPM“… Death by properties panel Low-code is great! (You can get rid of your developers!) @berndruecker
  5. 5. What I think, when I say „BPM“ +
  6. 6. Why workflow automation?
  7. 7. https://thenewstack.io/5-workflow-automation-use-cases-you-might-not-have-considered/
  8. 8. <= milliseconds seconds minutes, weeks, … Use cases for workflow automation
  9. 9. <= milliseconds seconds minutes, weeks, … Business processes automation Use cases for workflow automation
  10. 10. Real-life examples
  11. 11. <= milliseconds seconds minutes, weeks, … Business processes automation Use cases for workflow automation
  12. 12. <= milliseconds seconds minutes, weeks, … Business processes automation Use cases for workflow automation
  13. 13. <= milliseconds seconds minutes, weeks, … Business processes automation Use cases for workflow automation Communication in distributed systems
  14. 14. Payment Ever called a RESTAPI? Credit Card REST
  15. 15. Distributed systems
  16. 16. Payment Stateful retry Credit Card REST
  17. 17. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST
  18. 18. It is impossible to differentiate certain failure scenarios. Independant of communication style! Service Provider Client
  19. 19. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST
  20. 20. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST Cancel charge
  21. 21. Live hacking
  22. 22. Warning: Contains Opinion
  23. 23. Berlin, Germany mail@berndruecker.io @berndruecker Bernd Ruecker Co-founder and Chief Technologist of Camunda
  24. 24. Live hacking https://github.com/berndruecker/flowing-retail/tree/master/rest/java/payment-camunda
  25. 25. + Code, UI, … BPMN XML Workflow engine BPMN - Business Process Model and Notation ISO Standard
  26. 26. Long running services* provide a better API! @berndruecker * Services that have the capability to keep state and thus can be potentially long running
  27. 27. Example Order Payment Retrieve Payment @berndruecker
  28. 28. Example Order Payment Credit Card Retrieve Payment @berndruecker
  29. 29. Example Order Payment Credit Card Retrieve Payment Rejected @berndruecker
  30. 30. Example Order Payment If the credit card was rejected, the customer can provide new details Credit Card Retrieve Payment Rejected Rejected @berndruecker
  31. 31. Example Order Payment Client of dumb endpoints easily become a god services. If the credit card was rejected, the customer can provide new details Credit Card Retrieve Payment Rejected Rejected @berndruecker
  32. 32. Payment failed Who is responsible to deal with problems? Order Payment If the credit card was rejected, the customer can provide new details Credit Card Retrieve Payment Rejected Payment received @berndruecker
  33. 33. Payment failed Long running services Order Payment Credit Card Retrieve Payment Rejected Payment received Smart endpoints are potentially long-running @berndruecker
  34. 34. Payment Credit Card Rejected Payment failed Long running services Order Retrieve Payment Payment received @berndruecker
  35. 35. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation long runningalways short running short running, but potentially long running Use cases for workflow automation Communication in distributed systems
  36. 36. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation long runningalways short running short running, but potentially long running Use cases for workflow automation Communication in distributed systems
  37. 37. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation Communication in distributed systems long runningalways short running short running, but potentially long running Use cases for workflow automation Orchestration
  38. 38. Microservices… Service B Some Service Some Service Some Service Service C Service A Service D Monolith Functionality A Functionality B Functionality C Functionality D
  39. 39. Order fulfillment example: dash button Photo by 0xF2, available under Creative Commons BY-ND 2.0 license. https://www.flickr.com/photos/0xf2/29873149904/ @berndruecker
  40. 40. (Micro-)services Checkout Payment Inventory Shipment @berndruecker
  41. 41. Order Placed Payment Received Goods Fetched Notification Event-driven architecture @berndruecker Checkout Payment Inventory Shipment
  42. 42. Peer-to-peer event chains Checkout Payment Inventory Shipment Order placed Payment received Goods shipped Goods fetched @berndruecker
  43. 43. Peer-to-peer event chains Checkout Payment Inventory Shipment Order placed Payment received Goods shipped Goods fetched @berndruecker
  44. 44. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  45. 45. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  46. 46. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  47. 47. Peer-to-peer event chains Checkout Payment Inventory Shipment Order placed Payment received Goods shipped Goods fetched Fetch the goods before the payment @berndruecker
  48. 48. Peer-to-peer event chains Checkout Payment Inventory Shipment Fetch the goods before the payment Goods fetched Order placed Payment received Goods shipped @berndruecker
  49. 49. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0 @berndruecker
  50. 50. Order Extract the end-to-end responsibility Checkout Payment Inventory Shipment *Commands have an intent about what needs to happen in the future Payment received Order placed Retrieve payment @berndruecker
  51. 51. Order It still can be messaging! Checkout Payment Inventory Shipment Order placed Retrieve payment @berndruecker
  52. 52. Workflows implement stateful orchestration logic @berndruecker
  53. 53. No BPM(N) monoliths https://blog.bernd-ruecker.com/avoiding-the-bpm-monolith-when-using-bounded-contexts-d86be6308d8 @berndruecker
  54. 54. Workflows implement stateful orchestration logic @berndruecker
  55. 55. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation Communication in distributed systems long runningalways short running short running, but potentially long running Use cases for workflow automation Orchestration
  56. 56. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation Communication in distributed systems long runningalways short running short running, but potentially long running Use cases for workflow automation Orchestration Consistency
  57. 57. The classical example book hotel book car book flight 1. 2. 3. Failure! book trip
  58. 58. Pat Helland “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
  59. 59. Pat Helland Grown-Ups Don’t Use Distributed Transactions “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
  60. 60. The classical example Saga book hotel book car book flight cancel hotel cancel car 1. 2. 3. 5.6. Failure book trip Trigger compensations
  61. 61. BPMN Saga Pattern (implemented by BPMN compensation) https://github.com/berndruecker/trip-booking-saga-java
  62. 62. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation Distributed Transactions Orchestration Communication in distributed systems long runningalways short running short running, but potentially long running Use cases for workflow automation
  63. 63. <= milliseconds seconds minutes, weeks, … Business IT Business processes automation Distributed Transactions Decision Automation Orchestration Communication in distributed systems long runningalways short running short running, but potentially long running Use cases for workflow automation
  64. 64. Decisions with DMN
  65. 65. See more on http://berndruecker.io/
  66. 66. Workflow Automation is important in modern architectures!
  67. 67. Thoughts on the state machine | workflow engine market
  68. 68. Thoughts on the state machine | workflow engine market OSS Workflow or Orchestration Engines Stack Vendors, Pure Play BPMS Low Code Platforms Homegrown frameworks to scratch an itch Integration Frameworks Cloud Offerings Uber, Netflix, AirBnb, ING, … AWS Step Functions, Azure Durable Functions, … Camunda, Zeebe, jBPM, Activiti, Flowable, Mistral, … PEGA, IBM, SAG, … Apache Airflow, Spring Data Flow, … Apache Camel, Balerina, … Data Pipelines
  69. 69. Does it support stateful operations? Does it support the necessary flow logic? Does it support BizDevOps? Does it scale?
  70. 70. https://netflix.github.io/conductor/metadata/kitchensink/
  71. 71. AWS Step Functions https://read.acloud.guru/how-the-saga-pattern-manages-failures-with-aws-lambda-and- step-functions-bc8f7129f900
  72. 72. Flow language is important! Think of more complicated scenarios…
  73. 73. Proper Operations Visibility + Context @berndruecker
  74. 74. Biz Dev Leverage state machine & workflow engine Living documentation Visibility in testing Operate with visibility and context Understand and discuss business processes Evaluate optimizations in-sync with implementation improve communication improve communication Ops @berndruecker
  75. 75. Example: Storage Persistent State RDMS
  76. 76. Source: Spring StateMachine docs
  77. 77. Example: Storage Persistent State Persistent change RDMS Do it yourself
  78. 78. Workflow automation at scale! low latency, high-throughput low frequency, latency doesn‘t matter What people think workflow automation can do What we currently teach workflow automation to be able to do What workflow automation can already do today
  79. 79. Why Zeebe? Horizontally scalable and resilient
  80. 80. https://zeebe.io/blog/2019/03/zeebe-comparison-evaluation-criteria/
  81. 81. Aspects to consider Camunda Spring State Machine Workflow Definition BPMN 2.0 (graphical, XML, Model API) Java API, UML-Generator Visual? YES No Tooling Modeler, Cockpit, Optimize - Storage Runtime RDMS up to you Storage History RDMS - Scalability Stateless Engine, RDMS is limit, Sharding possible up to you Fault tolerance If RDMS is HA up to you Supported programming languages Java, REST, Language Clients (JS, C#) Java …
  82. 82. My personal pro-tip for a shortlist ;-) OSS Workflow or Orchestration Engines Stack Vendors, Pure Play BPMS Low Code Platforms Homegrown frameworks to scratch an itch Integration Frameworks Cloud Offerings Data Pipelines Camunda & Zeebe
  83. 83. Thank you! @berndruecker
  84. 84. mail@berndruecker.io @berndruecker https://berndruecker.io https://medium.com/berndruecker https://github.com/berndruecker https://www.infoq.com/articles/events- workflow-automation Contact: Slides: Blog: Code: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/

×