How to avoid cascading failures in microservices

NaradaCode
PAGE
1
DEVOPS INDONESIA
Thomas Ivantoro Prasetyo
NaradaCode
Jakarta, 5 Mei 2020
How to Avoid Cascading Failures in Microservices

NaradaCode
How to Avoid
Cascading Failures in
Microservices
DevOps Indonesia Meetup
May 05th, 2020

NaradaCode
Thomas Ivantoro Prasetyo
Solution Architect, Head of SME
IT Solution Architect Channel & Payment
IT Application Architect,
IT Development Supervisor
Squad Leader Sales Management System
Technology Innovation Manager, NaradaCode
thomas.ivantoro@naradacode.com
https://www.linkedin.com/in/thomasivantoro/
Previous Experience

NaradaCode
We provide end-to-end service to improve your digital capability
Lean Digital Transformation Company
Enterprise DevOps Lean Digital Governance Competency Development
Enterprise DevOps focuses on how
enterprise can leverage modern IT
and technology to drive business
faster and achieve the maximum
outcome
Lean Digital Governance ensure
enterprise has enough controls and
still ensure the speed of digital to
stay ahead of competition
Complete competency development
and mindset changing program to
upskill your people in order to
accelerate digital transformation

NaradaCode
DevOps Indonesia Meetup, May 05th 2020
 Microservices Benefits and Complexities
 Cascading Failure in Microservices
 Strategy to Avoid Cascading Failures
 Demo – Handling Cascading Failures
ThingstoDiscuss

NaradaCode
Let’s get to know each other alittlebit

NaradaCode
How to avoid cascading failure in Microservices
Microservices
Benefit &
Complexities

NaradaCode
Microservices–BenefitsandComplexities
Microservices provide benefits …but come with costs
 Strong Module Boundaries
Microservices reinforce modular structure, which is
particularly important for larger teams.
⚠ Distribution
Distributed systems are harder to program, since
remote calls are slow and are always at risk of failure.
 Independent Deployment
Simple services are easier to deploy, and since they are
autonomous, are less likely to cause system failures
when they go wrong. Especially if you have automation
pipeline.
⚠ Eventual Consistency
Maintaining strong consistency is extremely difficult for
a distributed system, which means everyone has to
manage eventual consistency.
 Technology Diversity
With microservices you can mix multiple languages,
development frameworks and data-storage
technologies.
⚠ Operational Complexity
You need a mature operations team to manage lots of
services, which are being redeployed regularly.
Source: https://martinfowler.com/articles/microservice-trade-offs.html#boundaries

NaradaCode
Microservices–DesignForFailuresPrinciple
• Fault tolerance of each services
• Isolate Failures
• Fail Fast
• Graceful service degradation with good fallback as components fail separately

NaradaCode
Cascading Failures
in Microservices

NaradaCode
CascadingFailuresinMicroservices(1/2)
Service A Service B
Service C
(struggle)
• Responses started slow down,
hanging
• Running out of resources
• Network problem
• Coding problem
• etc
• Many of the request are failed
because timeout

NaradaCode
CascadingFailuresinMicroservices(2/2)
Service A
Service B
(struggle)
Service C
(struggle)
Service F
Service E
Service B run out of resources to call Service F
Huge request hanging

NaradaCode
CascadingFailuresinMicroservices
It’s horrible because:
• We don’t exactly know when it's going to happen
• When it's already gone down, hard to find root cause
• Small problem in one microservice can cause massive problem and chaos

NaradaCode
Strategy to avoid
Cascading Failures

NaradaCode
HowtoAvoidCascadingFailuresinMicroservices
 Circuit Breaker
A circuit breaker is an automatically
operated electrical switch designed to protect
an electrical circuit from damage caused by
excess current from an overload or short
circuit. Its basic function is to interrupt
current flow after a fault is detected. Unlike
a fuse, which operates once and then must
be replaced, a circuit breaker can be reset
(either manually or automatically) to resume
normal operation. (Wikipedia)

NaradaCode
 Circuit Breaker
Microservice B
(struggle)
Microservice A
Circuit
Breaker
Open Circuit:

NaradaCode
 Circuit Breaker
As always Martin fowler write very well about this subject.

NaradaCode
 BulkHead
Bulkhead is used in the industry to partition a ship into sections
Watertight bulkheads
Cargo holds and boiler rooms flooded after
hull is pierced by iceberg

NaradaCode
 BulkHead
Protect limited resources from being exhausted
Microservice A
Connection Pool for X and Y
Microservice X Microservice Y
Microservice A
Pool for X
Microservice X Microservice Y
Pool for Y

NaradaCode
As mentioned in Design for Failure Principle, the Purpose is to:
• Isolate failure & isolate resources, to avoid failure to cascade
• Fail fast, degrade system gracefully with good fallback without sacrificing user
experiences
• Fail Fast, to save resources and network (save cost)

NaradaCode

NaradaCode
Problem with Embedded Components:
– Polyglot language microservices
– Need to build components in every single microservices
– Memory leak might impact performance of components
Microservice A
Connection Pool
Circuit Breaker

NaradaCode
 Sidecar Design Pattern
Microservice
A
Sidecar
Microservice
B
Sidecar
Microservice
C
Sidecar
Control Plane

NaradaCode
Circuit Breaker and Connection Pooling in Sidecar
Sidecar
Pool for B
Microservice
B
Sidecar
Pool for A
Microservice
A
PodPod
Circuit
Breaker A
Circuit
Breaker B

NaradaCode
Benefits of using Sidecar:
– Extends functionalities without changing primary application
– Reduce complexity code on primary application
– Use of Polyglot language in microservices without code duplication
– Isolate resources from primary application
Microservice
A
Sidecar
Microservice
B
Sidecar
Microservice
C
Sidecar
Control Plane
- Circuit Breaker
- TCP & HTTP Pool
- Timeout
- Retries
- etc

NaradaCode
ManyOtherWays
There are many other ways to avoid cascading failures and to achieve
application resiliency or might be self healing:
 Request Timeout
 Retries
 Fallback
 Rate Limiter
 Event-driven (CQRS, Event Sourcing)
 …

NaradaCode
Demo on
Avoiding Cascading
Failures

NaradaCode
Sample: Book Info page, Online Book Store
Product
Page
Reviews
Details
Front-end Back-end
Demo

NaradaCode
• Enhancement Book Reviews to add Review Ratings
• In this sample they do Canary Release with A/B Testing
V.1
V.2 V.3
V.1, Reviews without ratings.
User: All Users (except A, B, chaos)
V.2 , Ratings color is BLACK
User: A
V.3 , Ratings color is RED
User: B and chaos
Demo

NaradaCode
Microservices Design & Technology Stack
Product
Page
Reviews
V.3
Ratings
Details
Front-end Back-end
Reviews
V.2
Reviews
V.1
Demo

NaradaCode
Scenario:
1. User (with name chaos) will trigger Fault Injection to simulate timeout (hanging) on
Reviews Service V.3
2. User chaos will make consecutive failures at least 3 times. Review Service V.3 have Circuit
Breaker with this configuration:
If there are three consecutive errors
(502, 503, 504) within 20 seconds,
Circuit Breaker will be open and
Review Service V.3 will be ejected
for 3 minutes.
Demo

NaradaCode
Expected Result
1. Circuit Breaker on Reviews Service V.3 will be opened automatically, stopping access to this
service.
2. Book Info page of user B, Book Reviews section will not be shown for 3 minutes.
3. Book Info page of user A, Book Reviews section will be shown same as before.
4. Book Info page of other users (or without login), Book Reviews section will be shown same
as before.
5. After 3 minutes, Circuit Breaker on Reviews Service V.3 will be closed automatically, Reviews
Service V.3 ready to serve.
6. Book Info page of user B, Book Reviews section will be shown same as before.
Demo

NaradaCode
PatternofResilience
Do not try to use all of them, use only if “Value > Cost + Risk“
Pattern of resilience by Uwe
Friedrichsen (CTO Codecentric)

NaradaCode
Thankyou!Let’skeepconnected
thomas.ivantoro@naradacode.com
https://www.linkedin.com/in/thomasivantoro/

How to avoid cascading failures in microservices

Recommended

Recommended

More Related Content

Similar to How to avoid cascading failures in microservices

Similar to How to avoid cascading failures in microservices (20)

More from DevOps Indonesia

More from DevOps Indonesia (20)

Recently uploaded

Recently uploaded (20)

How to avoid cascading failures in microservices