This document discusses various application resiliency patterns including timeout, isolation, circuit breaker, adaptive throttle, and fallback. It describes how these patterns can help applications recover quickly from difficulties, manage sustained loads and traffic spikes, and prevent cascading failures. Specific patterns like circuit breaker and adaptive throttle are explained in more detail with examples of how they work. The document also covers configuration of these patterns and lists some other resiliency patterns like deferred work and asynchronous execution.

1. 1
Application Resiliency Patterns
Kiran Sama
ksama@visa.com

2. Application Resiliency Patterns | Aug 19, 2016
What is Application Resiliency?
• To recover quickly from difficulties
• To manage the sustained loads and spike in traffic
• Stop cascading failures
• Service degradation
2

3. Application Resiliency Patterns | Aug 19, 2016
Everything Is Healthy
3

4. Application Resiliency Patterns | Aug 19, 2016
Resources get saturated under load
4

5. Application Resiliency Patterns | Aug 19, 2016
Resiliency Patterns
Timeout
o Isolation
Fail Fast
Circuit Breaker
Adaptive Throttle
o Fallback
5

6. Application Resiliency Patterns | Aug 19, 2016
Timeout
• Always wait with timeout
• Preserve responsiveness independent of dependency latency
• Measure response times
• HTTP, JDBC and LDAP
• Connection Timeout, Read Timeout and Connection Pool
Timeout
• Configurable Timeouts
• Self Adapting Timeouts
• 99.5 percentile response time at full load without failures
6

7. Application Resiliency Patterns | Aug 19, 2016
Isolation
7
• One latent dependency does not effect rest of the application
• When dependency recovers, thread pool will clear up

8. Application Resiliency Patterns | Aug 19, 2016
Isolation on Server Side
8
• Dependency between clients
• Load on one client effects other

9. Application Resiliency Patterns | Aug 19, 2016
Isolation on Server Side
9
• Partitioning the service increases the stability
• Tradeoff: Resource cost increases
• Hybrid approach of shared and individual pools is efficient
• Separation Granularity
 Servers in a cluster
 Thread pools in an application

10. Application Resiliency Patterns | Aug 19, 2016
Circuit Breaker
10

11. Application Resiliency Patterns | Aug 19, 2016
Hystrix Circuit Breaker
11

12. Application Resiliency Patterns | Aug 19, 2016 12
Circuit Breaker in Resource Manager

13. Application Resiliency Patterns | Aug 19, 2016
Http Calls Using Resource Manager
13
• All the rest and soap calls are made through
ResourceManager will be converted to Hystrix commands.
• All the calls to one external system will be grouped as one
command by default
• API granularity for circuits to isolate end point specific
failures
• Failures: TimeoutException,ConnectException,UnKnownHo
stException and Status Code of 500 or more
• Fallback: Fail fast

14. Application Resiliency Patterns | Aug 19, 2016
Configuration Mechanism
14
• Configs can be controlled at ENV level via rm_configuration.xml
• Runtime refresh of configuration
• Configs can be controlled at multiple levels
 Value for all (Ex A,B,C,D,E…->A,B,C,D,E..)
 Value for one client application (Ex A -> B,C,D,E…)
 Value for one client to one server (Ex A->B)
• Important circuit breaker configs
 hystrix.command.default.circuitBreaker.enabled=true
 hystrix.command.default.circuitBreaker.requestVolumeThreshold=20
 hystrix.command.default.circuitBreaker.errorThresholdPercentage=50
 hystrix.command.default.circuitBreaker.forceOpen=false
 hystrix.command.default.circuitBreaker.forceClose=false

15. Application Resiliency Patterns | Aug 19, 2016
Adaptive Throttle
15

16. Application Resiliency Patterns | Aug 19, 2016
Adaptive Throttle
• Calculate success rate over last n seconds
• Client only allows successful throughput and little extra
to adapt to check server stability for next period
• This will result in more throughput as it always allows
some requests
16

17. Application Resiliency Patterns | Aug 19, 2016
Fallback
17
• It’s not optional
• Fallback should be part of the initial design
• Fallback Strategies
 Caching responses
 Stale Data
 Queue up the requests
 Remote cluster

18. Application Resiliency Patterns | Aug 19, 2016
More Patterns
• Defer Work, Queue it up
– Eventually consistency helps stability
• Asynchronous execution
– Stops cascading failures
– Blocks chain reactions
• Handshaking Pattern
– Server and client to communicate about capacity and timeouts
• Steady State
– Data purging
– Log archival
– Limit in memory caching
18