Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe

478 views

Published on

Slides from my talk at the Kafka Summit San Francisco 2018 about orchestration vs. choreography as well as track vs. manage event flows. Source code for live demos available on https://github.com/berndruecker/flowing-retail

Published in: Technology
  • Be the first to comment

Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe

  1. 1. @berndruecker The Big Picture Monitoring and Orchestration of Your Microservices Landscape with Kafka and Zeebe
  2. 2. Some Service Some Service Some Service Some Service Some Service Some Service Some Service Microservices…
  3. 3. Data, data, data, … Source: Microservices orchestration survey, July 2018, 354 responses
  4. 4. 92 %
  5. 5. The microservice community favours an alternative approach: smart endpoints and dumb pipes. [… Microservices …] are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool. https://www.martinfowler.com/articles/microservices.html
  6. 6. De-coupled services with event backbone Checkout Inventory Payment Shipment Kafka
  7. 7. Event-driven Checkout Payment Inventory Shipment Order Placed Payment Received Goods Fetched
  8. 8. Peer-to-peer event chain Checkout Payment Inventory Shipment
  9. 9. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  10. 10. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  11. 11. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  12. 12. Challenges?
  13. 13. Microservice pioneers have become aware
  14. 14. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0
  15. 15. Finding the right balance 16 Observe Microservice Paradigms ● Loose Coupling ● Autonomy ● Event Collaboration Ensure Flow Manageability • End-To-End Visibility • Meet Business Process SLA • Support Continuous Process Improvement
  16. 16. Workflow automation | orchestration is an essential building block in microservice architectures Of course I am opinioated – but I truly believe this!
  17. 17. Workflow automation | orchestration is an essential building block in microservice architectures Of course I am opinioated – but I truly believe this! Orchestration is not a central thing! „It is not BPEL“
  18. 18. Warning: Contains Opinion
  19. 19. HQ Germany (Berlin) + Offices in US bernd.ruecker@camunda.com @berndruecker Bernd Ruecker Co-founder and Developer Advocate of Camunda
  20. 20. Different levels of leveraging workflow automation Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  21. 21. A possible journey from pogo to beauty… Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  22. 22. Tracking Checkout Inventory Payment Shipment Kafka Zeebe
  23. 23. Tracking + BI Checkout Inventory Payment Shipment Kafka Zeebe Camunda Optimize Other reporting / BIO solutions
  24. 24. Tracking + BI Checkout Inventory Payment Shipment Kafka Workflow Engine Camunda Optimize Other reporting / BIO solutions
  25. 25. Tracking + BI Checkout Inventory Payment Shipment Kafka Zeebe Camunda Optimize Other reporting / BIO solutions This is totally non-invasive But also limited in power
  26. 26. But you can start acting upon certain events
  27. 27. Live hacking https://github.com/berndruecker/flowing-retail/
  28. 28. Demo architecture Checkout Inventory Payment Shipment Kafka Zeebe Spring Boot Services Spring Cloud Streams Kafka Connect Zeebe Simple Monitor
  29. 29. Why so shy? Photo by LuidmilaKot, available under Creative Commons CC0 1.0 License.
  30. 30. A possible journey away from pogo… Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  31. 31. Changing the flow Checkout Payment Inventory Shipment Fetch the goods before the payment
  32. 32. Changing the flow Checkout Payment Inventory Shipment Fetch the goods before the payment
  33. 33. Photo by born1945, available under Creative Commons BY 2.0 license.
  34. 34. Order Checkout Payment Inventory Shipment Important responsibilities need a home
  35. 35. Distributed systems
  36. 36. Distributed systems
  37. 37. Distributed systems
  38. 38. Order Checkout Payment Inventory Shipment Stateful orchestration This orchestration requires state
  39. 39. Order Checkout Payment Inventory Shipment
  40. 40. Now it is easy to change the flow
  41. 41. Manage the flow Checkout Inventory Payment Shipment Kafka Zeebe Order Owned heredeployed here
  42. 42. Live hacking https://github.com/berndruecker/flowing-retail/
  43. 43. Demo architecture Checkout Inventory Payment Shipment Kafka Zeebe Spring Boot Services Spring Cloud Streams Order Alternative: Kafka Connect
  44. 44. BPMN Business Process Model and Notation ISO Standard
  45. 45. BPMN supports integration use cases Aggregator Re-Sequencer Combined with timeouts: Zeebe supports Message TTL
  46. 46. BPMN supports integration use cases Compensation
  47. 47. Biz Dev Ops
  48. 48. Hybrid architectures Order Credit Card Customer Credit Payment Pub/Sub e.g. Kafka Pub/Sub Work Distribution e.g. External Task Request/Response e.g. REST
  49. 49. Order Checkout Payment Inventory Shipment Every service owns its workflow!
  50. 50. Central Zeebe or … Checkout Inventory Payment Shipment Kafka Zeebe Order Owned heredeployed here Owned here
  51. 51. …decentral Zeebe Checkout Inventory Payment Shipment Kafka ZeebeOrderZeebe
  52. 52. Workflow automation at scale! low latency, high-throughput low frequency, latency doesn‘t matter What people think workflow automation can do What we currently teach workflow automation to be able to do What workflow automation can already do today
  53. 53. Vodafone, Liongate & WDW Presented at CamundaCOn Berlin 2018
  54. 54. Recap • In event-driven-systems it is easy to loose sight of the flow • Visibility is essential to survive • You need to balance orchestration and chorography • Distributed systems need stateful orchestration • Track, monitor or manage the flow – it might be a journey • Workflow automation is an essential building block • BPMN is very powerful and provides BizDevOps
  55. 55. Thank you!
  56. 56. mail@berndruecker.io @berndruecker https://berndruecker.io https://medium.com/berndruecker https://github.com/berndruecker https://www.infoq.com/articles/events- workflow-automation Contact: Slides: Blog: Code: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/

×