Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JFuture - Battle of the circuit breakers

466 views

Published on

There are two ways to implement the circuit breaker pattern: white-box à la Hystrix/Resilience4J or black-box à la Istio. Both have pros and cons. Come to this talk to hear more about that!

Kubernetes in general, and Istio in particular, have changed a lot the way we look at Ops-related constraints: monitoring, load-balancing, health checks, etc. Before those products became available, there were already available solutions to handle those constraints.

Among them are different libraries e.g. Hystrix and Resilience4J in the Java ecosystem. In particular, they both provide an implementation of the Circuit Breaker pattern, which prevents a network or service failure from cascading to other services. But now Istio also provides the same capability.

In this talk, we will have a look at how Istio and one among Hystrix/Resilience4J implement the Circuit Breaker pattern, and what pros/cons each of them has. Bonus, you'll be able to choose what library we will focus for the demo.

After this talk, you'll be able to decide which one is the best fit in your context.

Published in: Software
  • Be the first to comment

  • Be the first to like this

JFuture - Battle of the circuit breakers

  1. 1. @nicolas_frankel Istio vs. Hystrix/Resilience4J Battle of the Circuit Breakers
  2. 2. @nicolas_frankel • Developer Advocate • Developer until last September • DevOps and Cloud curious Me, myself and I
  3. 3. @nicolas_frankel Hazelcast HAZELCAST IMDG is an operational, in-memory, distributed computing platform that manages data using in-memory storage, and performs parallel execution for breakthrough application speed and scale. HAZELCAST JET is the ultra fast, application embeddable, 3rd generation stream processing engine for low latency batch and stream processing.
  4. 4. @nicolas_frankel • Some introduction • The problem • The circuit-breaker pattern • Istio implementation • Hystrix implementation • Demo Agenda
  5. 5. @nicolas_frankel • Componentization via Services • Smart endpoints and dumb pipes • Decentralized Governance • Decentralized Data Management • Infrastructure Automation • Design for failure • Evolutionary Design • Organized around Business Capabilities • Products not Projects µservice: a tentative definition https://martinfowler.com/articles/microservices.html
  6. 6. @nicolas_frankel • Componentization via Services • Smart endpoints and dumb pipes • Decentralized Governance • Decentralized Data Management • Infrastructure Automation • Design for failure • Evolutionary Design • Organized around Business Capabilities • Products not Projects µservice: a tentative definition https://martinfowler.com/articles/microservices.html
  7. 7. @nicolas_frankel • Microservices are an organizational solution to an organizational problem • They are ill-adapted to most orgs Word of warning https://martinfowler.com/bliki/MicroservicePrerequisites.html
  8. 8. @nicolas_frankel "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations." Conway’s Law
  9. 9. @nicolas_frankel https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/
  10. 10. @nicolas_frankel https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/
  11. 11. @nicolas_frankel “I see you have a poorly structured monolith. Would you like me to convert it into a poorly structured set of microservices?” Rant of the day https://twitter.com/architectclippy/status/570025079825764352
  12. 12. @nicolas_frankel Webservice, not microservice Semantics!
  13. 13. @nicolas_frankel • “Anything that can go wrong will go wrong” • Apply that to webservices architecture Reminder: Murphys’s law
  14. 14. @nicolas_frankel
  15. 15. @nicolas_frankel • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology doesn't change • There is one administrator • Transport cost is zero • The network is homogeneous Reminder: Fallacies of distributed computing https://yourlogicalfallacyis.com/
  16. 16. @nicolas_frankel • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology doesn't change • There is one administrator • Transport cost is zero • The network is homogeneous Reminder: Fallacies of distributed computing https://yourlogicalfallacyis.com/
  17. 17. @nicolas_frankel A sample webservice architecture F B C1 C2
  18. 18. @nicolas_frankel A sample webservice architecture F B C1 C2
  19. 19. @nicolas_frankel A sample webservice architecture F B C1 C2
  20. 20. @nicolas_frankel A sample webservice architecture F B C1 C2
  21. 21. @nicolas_frankel A sample webservice architecture F B C1 C2
  22. 22. @nicolas_frankel “A service client should invoke a remote service via a proxy that functions in a similar fashion to an electrical circuit breaker.” https://microservices.io/patterns/reliability/circuit-breaker.html Enter the Circuit Breaker pattern
  23. 23. @nicolas_frankel Circuit Breaker state machine
  24. 24. @nicolas_frankel • Number of failed calls • Elapsed time strategy: • Fixed • Doubling • Something else • Number of successful calls Configuration options
  25. 25. @nicolas_frankel What to do in the case of timeout? The most important configuration option
  26. 26. @nicolas_frankel 1. Recommendation webservice • “People also bought xyz” 2. Pricing webservice 3. Payment webservice 4. Logging webservice Use-case: e-commerce webshop
  27. 27. @nicolas_frankel • Fire-and-forget • Asynchronous calls Logging
  28. 28. @nicolas_frankel • Synchronous req/response • Optional • Fallback options • Display no recommendations • Static recommendations set Recommendation
  29. 29. @nicolas_frankel • Synchronous req/response • Required • But better sell at a slightly outdated price! • Fallback options • Accept outdated data from another source • In-memory cache Pricing
  30. 30. @nicolas_frankel • Synchronous req/response • Required • Fallback options • Accept potentially bad payments 🤔 Payment
  31. 31. @nicolas_frankel Available strategies Strategy Implementations Fits Black Box ● Proxies ● Service meshes Fail fast White Box Libraries ● Hystrix ● Resilience4J Fallbacks relying on business logic
  32. 32. @nicolas_frankel “A service mesh is a configurable infrastructure layer for a microservices application. It makes communication between service instances flexible, reliable, and fast. The mesh provides service discovery, load balancing, encryption, authentication and authorization, support for the circuit breaker pattern, and other capabilities.” https://www.nginx.com/blog/what-is-a-service-mesh/ Service mesh
  33. 33. @nicolas_frankel • Open Source service mesh • Leverages Kubernetes • Implements the sidecar pattern • Uses the Envoy proxy under the hood Istio
  34. 34. @nicolas_frankel Sidecar pattern
  35. 35. @nicolas_frankel Istio from a birds-eye view https://istio.io/docs/concepts/what-is-istio/
  36. 36. @nicolas_frankel apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: foo spec: host: foo trafficPolicy: outlierDetection: consecutiveErrors: 3 interval: 10s baseEjectionTime: 1m maxEjectionPercent: 80 Circuit-breaker configuration in Istio Number of consecutive errors that open the circuit breaker Interval between two checks Duration of opening Percentage of evicted instances
  37. 37. @nicolas_frankel • No fallback Cons of Istio
  38. 38. @nicolas_frankel A talk in which you’re the hero! Go to slide 39 Go to slide 44
  39. 39. @nicolas_frankel “Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.” Hystrix
  40. 40. @nicolas_frankel • Provided by Netflix • Currently in maintenance mode ⚠️ • Superseded by Resilience4J • But not equivalent Hystrix
  41. 41. @nicolas_frankel • Wraps calls into “commands” • Run commands asynchronously from a thread pool • Measure success/failures • Circuit-breaker implementation • Fallback logic Hystrix features
  42. 42. @nicolas_frankel • A lot of configuration options • Hard to fine-tune • No big picture Cons of Hystrix
  43. 43. @nicolas_frankel • Easy Hystrix integration • Also: • Service discovery: Eureka • Declarative REST client: Feign • Client-side LB: Ribbon • etc. Spring Cloud Netflix
  44. 44. @nicolas_frankel “Resilience4j is a lightweight fault tolerance library inspired by Netflix Hystrix, but designed for Java 8 and functional programming.” Resilience4J
  45. 45. @nicolas_frankel • Circuit Breaker • Rate Limiter • Retry • Cache • etc. Resilience4J’s features
  46. 46. @nicolas_frankel • Each feature is designed as a function • Uses Java 8 functional interfaces • e.g. Supplier • Based on function composition • Based on Vavr • Functional Programming in Java Resilience4J’s design principles
  47. 47. @nicolas_frankel • Need to be very familiar with Functional Programming • No big picture Cons of Resilience4J
  48. 48. @nicolas_frankel
  49. 49. @nicolas_frankel • https://blog.frankel.ch/ • @nicolas_frankel • https://git.io/JenH9 Thanks

×