Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

3 Common Pitfalls Microservices Integration QCONSP2018_Camunda

12 views

Published on

Integrating microservices and taming distributed systems is hard. In this talk I will present three challenges I’ve observed in real-life projects and discuss how to avoid them:

Communication is complex. With everything being distributed failures are normal so you need sophisticated failure handling strategies (e.g. stateful retry);
Asynchronicity requires you to handle timeouts. This is not only about milliseconds, systems get much more resilient when you can wait for minutes, hours or even longer;
Distributed transactions cannot simply be delegated to protocols like XA. So you need to solve the requirement to retain consistency in case of failures.
I will not only use slides but demonstrate concrete source code examples available on GitHub.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

3 Common Pitfalls Microservices Integration QCONSP2018_Camunda

  1. 1. 3 Common Pitfalls in Microservice Integration (Bonus : And how to avoid them ) credit to Bernd Ruecker Patricio Zambrano Technical Consultant, Camunda Inc.
  2. 2. 2 Microservices Agenda • Introduction • 3 Common Challenges and How to Avoid Them • Conclusion
  3. 3. Raise your hand • REST • Microservices • Java
  4. 4. Distributed systems
  5. 5. Distributed systems Challenges of asynchronicity Distributed Transactions Communication is complex
  6. 6. 7 Some Microservices…. Some Service Some Service Some Service Some Service Some Service Some Service Some Service Failure will happen. Accept it! But keep it local! Be resilient.
  7. 7. Let‘s start with a simple example Credit Card Payment REST Payment Requestor Application What happens if the Credit Card Service is Super Slow?
  8. 8. Live hacking https://github.com/flowing/flowing-retail/blob/master/payment- rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV2.java
  9. 9. Circuit Breaker Photo by CITYEDV, available under Creative Commons CC0 1.0 license.
  10. 10. Failing Fast is important…. ..but not enough
  11. 11. Photo by https://www.archdaily.com/560641/liverpool-insurgentes-department-store-rojkind-arquitectos
  12. 12. Check-in Web-UI You Current situation
  13. 13. Check-in Barcode Generator Web-UI You Output Mgmt Current situation
  14. 14. Current situation Check-in Barcode Generator Web-UI You Output Mgmt Circuit breaker
  15. 15. Another screenshot
  16. 16. Check-in Barcode Generator Web-UI Me Output Mgmt Current situation – the bad part
  17. 17. Current situation – the bad part Check-in Barcode Generator Web-UI Me Output Mgmt
  18. 18. Current situation – the bad part Check-in Barcode Generator Web-UI Me Output Mgmt Stateful Retry
  19. 19. Another Example We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it on time.
  20. 20. Check-in Barcode Generator Web-UI Me Output Mgmt Possible Solution – Much better? Stateful Retry The failure never leaves this scope!
  21. 21. Persist thing (Entity, Document, Actor, …) State machine or workflow engine Typical concerns DIY = effort, accidental complexity Complex, proprietary, heavyweight, slow, don‘t scale, developer adverse Scheduling, Versioning, operating, visibility, scalability, … Handling State Typical concerns
  22. 22. Current Players in the State Machine Market • AWS Step Function • UBER Cadence • Netflix Conductor • Camunda  • Zeebe  • jBPM • Activiti
  23. 23. Performance: Zeebe vs. Kafka Vs. Apache Kafka
  24. 24. Current Players in the State Machine Market • AWS Step Function • UBER Cadence • Netflix Conductor • Camunda  (Raise of hand?) • Zeebe  • jBPM • Activiti
  25. 25. In the previous demo…. Credit Card Payment REST Payment Requestor Application What if I want my Payment to be Asynchronous and Retry itself when my Credit Card Service Slow?
  26. 26. Live hacking https://github.com/flowing/flowing-retail/blob/master/payment- rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV3.java
  27. 27. Demo Credit Card Payment REST Payment Requestor Application What if I want a Synchronous response when everything is fast?
  28. 28. Live hacking https://github.com/flowing/flowing-retail/blob/master/payment- rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV3.java
  29. 29. Payment Now you have a state machine! Credit Card REST
  30. 30. has to implement Retry has to implement Idempotency Client Service Provider Most important factors to consider in distributed systems (so far.. )
  31. 31. We are processing your payment. Do not leave this page. And for god sake – do not reload! It is a business problem anyway! Bad Example..
  32. 32. We are processing your payment. Do not leave this page. And for god sake – do not reload! It is a business problem anyway! We are currently processing your request. Don‘t worry, it will happen safely – even if you loose connection. Feel free to reload this page any time! Better…
  33. 33. It is impossible to differentiate certain failure scenarios(and Code Exceptions). Independant of communication style! Service Provider Client
  34. 34. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST
  35. 35. Distributed systems introduce complexity you have to tackle! Credit Card Payment REST Do it reliably
  36. 36. Check-in Barcode Generator Web-UI Me Output Mgmt Workflows live within service boundaries
  37. 37. Different Architecture Options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  38. 38. Different architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  39. 39. Different architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  40. 40. Different architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  41. 41. generateBoardingPass HTTP 200 OK HTTP 202 ACCEPTED Check-in A synchronous response is possible in the happy case, otherwise it is switched to asynchronous processing. First Sync then Async
  42. 42. The customer wants a synchronous response… Check-in Barcode Generator Web-UI Me Output Mgmt !Eh – no!
  43. 43. Synchronous communication is the crystal meth of distributed programming Todd Montgomery and Martin Thompson in “How did we end up here” at GOTO Chicago 2015
  44. 44. Challenges of asynchronicity
  45. 45. Check-in Barcode Generator Web-UI Me Output Mgmt Asynchronous communication You need to monitor timeouts
  46. 46. Check-in Barcode Generator Web-UI Me Output Mgmt Remember… The failure never leaves this scope!
  47. 47. Workflow…
  48. 48. Workflow…
  49. 49. has to implement Timeout, Retry has to implement Idempotency Client Service Provider
  50. 50. Who uses a message bus?
  51. 51. Who has no problems operating a message bus? Dead messages | No context | Inaccesible payload | Hard to redeliver | Home-grown message hospitals | …
  52. 52. Other Architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  53. 53. Other Architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  54. 54. Other Architecture Options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  55. 55. Distributed Transactions Distributed Transactions
  56. 56. Distributed systems • ACID Transactions • Scalability • Troubleshooting TM • Still Required!!
  57. 57. Distributed transactions using compensation * Compensation
  58. 58. Eventual consistency Temporarily inconsistent state But only temporarily!
  59. 59. Demo Time Payment Requestor Application Node.js App
  60. 60. Live hacking https://github.com/flowing/flowing-retail/blob/master/payment- rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV6.java
  61. 61. has to implement Timeout, Retry, Compensation has to offer Compensation has to implement Idempotency Client Service Provider
  62. 62. has to implement Timeout, Retry, Compensation has to offer Compensation has to implement Idempotency Client Service Provider Don‘t forget about state
  63. 63. Before mapping processes explicitly with BPMN and DMN, the truth was buried in the code and nobody knew what was going on. Jimmy Floyd, 24 Hour Fitnesse
  64. 64. Event-driven example also available InventoryPaymentOrder ShippingCheckout Monitor https://github.com/flowing/flowing-retail/ Kafka Human Tasks H2 H2 https://github.com/flowing/flowing- retail/tree/master/zeebe
  65. 65. Workflows live inside service boundaries KafkaorZeebe
  66. 66. Sales-Order & Order-Fulfillment via Camunda for every order worldwide (Q2 2017: 22,2 Mio)
  67. 67. Camunda Value Technical Use Cases Business Process Examples Improving development, operations and visibility of automated workflows and decisions. Straight-Through Processing Microservice Orchestration Human Workflow Management Business Rule Automation E-Commerce: Order Execution Finance: Stock Trading Insurance: Claim Settlement Telco: OSS/BSS …….. Some of the Workflow Engine Use Cases and… what we talked about 72
  68. 68. # Be aware of complexity of distributed systems # Know strategies and tools to handle it e.g. Circuit breaker (Hystrix) Workflow engine for stateful retry, waiting, timeout and compensation (Camunda)
  69. 69. https://www.infoworld.com/article/3254777/application-development/ 3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html
  70. 70. Where to learn more #6
  71. 71. Contact Us • Andreas Stange | International Sales • +49-172-862-2730 | Berlin • Mauricio Bitencourt | Customer Delivery & Success • +55 51 984.087.798 | Sao Paulo
  72. 72. Q&A ?
  73. 73. Camunda Ecosystem Model > Execute > Improve
  74. 74. What is ZEEBE? • Zeebe scales orchestration of workers and microservices using visual workflows. Zeebe is horizontally scalable and fault tolerant so that you can reliably process all your transactions as they happen.

×