Kubernetes in general, and Istio in particular, have changed a lot the way we look at Ops-related constraints: monitoring, load-balancing, health checks, etc. Before those products became available, there were already available solutions to handle those constraints. Among them is Resilience4J, a Java library. From the site: "Resilience4j is a fault tolerance library designed for Java8 and functional programming." In particular, Resilience4J provides an implementation of the Circuit Breaker pattern, which prevents a network or service failure from cascading to other services. But now Istio also provides the same capability. In this talk, we will have a look at how Istio and Resilience4J implement the Circuit Breaker pattern, and what pros/cons each of them has. After this talk, you’ll be able to decide which one is the best fit in your context.

1. @nicolas_frankel
Istio vs. Hystrix/Resilience4J
Battle of the Circuit Breakers

2. @nicolas_frankel

3. @nicolas_frankel
• Developer Advocate
• Developer until last September
• DevOps and Cloud curious
Me, myself and I

4. @nicolas_frankel
Hazelcast
HAZELCAST IMDG is an operational,
in-memory, distributed computing
platform that manages data using
in-memory storage, and performs
parallel execution for breakthrough
application speed and scale.
HAZELCAST JET is the ultra fast,
application embeddable, 3rd
generation stream processing
engine for low latency batch
and stream processing.

5. @nicolas_frankel
• Some introduction
• The problem
• The circuit-breaker pattern
• Istio implementation
• Hystrix implementation
• Demo
Agenda

6. @nicolas_frankel
• Componentization via Services
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
• Organized around Business Capabilities
• Products not Projects
µservice: a tentative definition
https://martinfowler.com/articles/microservices.html

7. @nicolas_frankel
• Componentization via Services
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
• Organized around Business Capabilities
• Products not Projects
µservice: a tentative definition
https://martinfowler.com/articles/microservices.html

8. @nicolas_frankel
• Microservices are an
organizational solution to an
organizational problem
• They are ill-adapted to most orgs
Word of warning
https://martinfowler.com/bliki/MicroservicePrerequisites.html

9. @nicolas_frankel
"organizations which design systems
... are constrained to produce
designs which are copies of the
communication structures of these
organizations."
Conway’s Law

10. @nicolas_frankel
https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/

11. @nicolas_frankel
https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/

12. @nicolas_frankel
“I see you have a poorly structured
monolith. Would you like me to
convert it into a poorly structured set
of microservices?”
Rant of the day
https://twitter.com/architectclippy/status/570025079825764352

13. @nicolas_frankel
Webservice, not microservice
Semantics!

14. @nicolas_frankel
• “Anything that can go wrong will go
wrong”
• Apply that to webservices
architecture
Reminder: Murphys’s law

15. @nicolas_frankel

16. @nicolas_frankel
• The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology doesn't change
• There is one administrator
• Transport cost is zero
• The network is homogeneous
Reminder: Fallacies of distributed
computing
https://yourlogicalfallacyis.com/

17. @nicolas_frankel
• The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology doesn't change
• There is one administrator
• Transport cost is zero
• The network is homogeneous
Reminder: Fallacies of distributed
computing
https://yourlogicalfallacyis.com/

18. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

19. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

20. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

21. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

22. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

23. @nicolas_frankel
“A service client should invoke a remote service via a
proxy that functions in a similar fashion to an electrical
circuit breaker.”
https://microservices.io/patterns/reliability/circuit-breaker.html
Enter the Circuit Breaker pattern

24. @nicolas_frankel
Circuit Breaker state machine

25. @nicolas_frankel
• Number of failed calls
• Elapsed time strategy:
• Fixed
• Doubling
• Something else
• Number of successful calls
Configuration options

26. @nicolas_frankel
What to do in the case of timeout?
The most important configuration
option

27. @nicolas_frankel
1. Recommendation webservice
• “People also bought xyz”
2. Pricing webservice
3. Payment webservice
4. Logging webservice
Use-case: e-commerce webshop

28. @nicolas_frankel
• Fire-and-forget
• Asynchronous calls
Logging

29. @nicolas_frankel
• Synchronous req/response
• Optional
• Fallback options
• Display no recommendations
• Static recommendations set
Recommendation

30. @nicolas_frankel
• Synchronous req/response
• Required
• But better sell at a slightly
outdated price!
• Fallback options
• Accept outdated data from
another source
• In-memory cache
Pricing

31. @nicolas_frankel
• Synchronous req/response
• Required
• Fallback options
• Accept potentially bad payments 🤔
Payment

32. @nicolas_frankel
Available strategies
Strategy Implementations Fits
Black Box ● Proxies
● Service meshes
Fail fast
White Box Libraries
● Hystrix
● Resilience4J
Fallbacks relying
on business logic

33. @nicolas_frankel
“A service mesh is a configurable infrastructure
layer for a microservices application. It makes
communication between service instances flexible,
reliable, and fast. The mesh provides service
discovery, load balancing, encryption,
authentication and authorization, support for the
circuit breaker pattern, and other capabilities.”
https://www.nginx.com/blog/what-is-a-service-mesh/
Service mesh

34. @nicolas_frankel
• Open Source service mesh
• Leverages Kubernetes
• Implements the sidecar pattern
• Uses the Envoy proxy under the
hood
Istio

35. @nicolas_frankel
Sidecar pattern

36. @nicolas_frankel
Istio from a birds-eye view
https://istio.io/docs/concepts/what-is-istio/

37. @nicolas_frankel
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: foo
spec:
host: foo
trafficPolicy:
outlierDetection:
consecutiveErrors: 3
interval: 10s
baseEjectionTime: 1m
maxEjectionPercent: 80
Circuit-breaker configuration in Istio
Number of consecutive errors that open the circuit breaker
Interval between two checks
Duration of opening
Percentage of evicted instances

38. @nicolas_frankel
• No fallback
Cons of Istio

39. @nicolas_frankel
A talk in which you’re the hero!
Go to slide 39 Go to slide 44

40. @nicolas_frankel
“Hystrix is a latency and fault tolerance
library designed to isolate points of access
to remote systems, services and 3rd party
libraries, stop cascading failure and enable
resilience in complex distributed systems
where failure is inevitable.”
Hystrix

41. @nicolas_frankel
• Provided by Netflix
• Currently in maintenance mode ⚠️
• Superseded by Resilience4J
• But not equivalent
Hystrix

42. @nicolas_frankel
• Wraps calls into “commands”
• Run commands asynchronously
from a thread pool
• Measure success/failures
• Circuit-breaker implementation
• Fallback logic
Hystrix features

43. @nicolas_frankel
• A lot of configuration options
• Hard to fine-tune
• No big picture
Cons of Hystrix

44. @nicolas_frankel
• Easy Hystrix integration
• Also:
• Service discovery: Eureka
• Declarative REST client: Feign
• Client-side LB: Ribbon
• etc.
Spring Cloud Netflix

45. @nicolas_frankel
“Resilience4j is a lightweight fault
tolerance library inspired by Netflix
Hystrix, but designed for Java 8 and
functional programming.”
Resilience4J

46. @nicolas_frankel
• Circuit Breaker
• Rate Limiter
• Retry
• Cache
• etc.
Resilience4J’s features

47. @nicolas_frankel
• Each feature is designed as a
function
• Uses Java 8 functional interfaces
• e.g. Supplier
• Based on function composition
• Based on Vavr
• Functional Programming in Java
Resilience4J’s design principles

48. @nicolas_frankel
• Need to be very familiar with
Functional Programming
• No big picture
Cons of Resilience4J

49. @nicolas_frankel

50. @nicolas_frankel

51. @nicolas_frankel
• https://blog.frankel.ch/
• @nicolas_frankel
• https://git.io/JenH9
Thanks