The document discusses the circuit breaker pattern in the context of NGINX and microservices, highlighting the importance of resilience in application architecture. It covers NGINX's capabilities as a circuit breaker, the significance of understanding failure profiles, and various resiliency options like rate limiting and caching. The presentation emphasizes the inevitability of failure in applications and the need for proactive measures to mitigate its impact.

1. NGINX MRA AMA #7:
The Circuit Breaker Pattern
Wednesday, January 10th, 2018
1

2. MORE INFORMATION AT NGINX.COM
Who Are We?
Floyd Smith
Director, Content
Marketing
Formerly with Apple,
Alta Vista, Google,
startups, and an
author
Chris Stetson
Chief Architect & Sr.
Director, Microservices
Engineering
Formerly with Razorfish
and Huge
Charles Pretzer
Technical Architect
Former software
architecture
consultant and
engineering lead

3. MORE INFORMATION AT
NGINX.COM
Agenda
1. Introducing NGINX and NGINX Plus
2. The Circuit Breaker pattern
3. Embracing Failure
4. Understand Your Failure Profile
5. NGINX Plus as the Circuit Breaker
6. Other Resiliency Options
7. Questions
3 3

4. MORE INFORMATION AT
NGINX.COM
1. Introducing NGINX and
NGINX Plus

5. MORE INFORMATION AT NGINX.COM
About NGINX, Inc.
• NGINX OSS released 2003
• NGINX Plus first released in 2013
• NGINX, Inc. is VC-backed by leading investors in enterprise
software
• Offices in SF, Sunnyvale, Singapore, Cork, Cambridge, & Moscow
• 1,200+ commercial customers
• 180+ employees

6. >50%of the top 100,000
busiest websites
6Source: W3Techs Web Technology Survey

7. >40%of sites on AWS
7Source: SumoLogic

8. 8

9. Internet
Web Server
Serve content from disk
Application Gateway
FastCGI, uWSGI, Passenger…
Reverse Proxy
Caching, load balancing…
HTTP traffic
Where NGINX Plus Fits

10. MORE INFORMATION AT NGINX.COM
Public/Private/Hybrid CloudBare Metal Containers
10
NGINX Plus Works in All Environments

11. MORE INFORMATION AT NGINX.COM
NGINX Products and Roadmap
• Start with Gus Robertson keynote at nginx.conf 2017
• See Owen Garrett’s Roadmap presentation….
• …and Chris and Rachael’s Controller demo
• Also, Chris Richardson series, Intro to Microservices…
• …and Chris Stetson series, NGINX MRA
• …and much more; contact Sales for free evaluation

12. MORE INFORMATION AT
NGINX.COM
2. The Circuit Breaker Pattern

13. MORE INFORMATION AT
NGINX.COM
Architectural
Changes:
Monolith to
Microservices 1

14. MORE INFORMATION AT
NGINX.COM
Architectural
Changes:
Monolith to
Microservices 2

15. MORE INFORMATION AT
NGINX.COM
Microservices
Reference
Architecture
• Docker containers
• Polyglot services
• Design incorporates
Twelve-Factor App
principles

16. MORE INFORMATION AT
NGINX.COM
Circuit Breaker
• Active health checks
• Avoid catastrophic
failure
• Greater resiliency

17. MORE INFORMATION AT
NGINX.COM
3. Embracing Failure

18. Your App
Will Fail
18

19. MORE INFORMATION AT
NGINX.COM
Layer Cake
• Cloud VMs
• Docker containers
• Application libraries
• Dynamic networking

20. You Can’t
Prevent Failure
20

21. MORE INFORMATION AT
NGINX.COM
Resizer
• Compute & memory
intensive
• Interfaced through
uploader

22. MORE INFORMATION AT
NGINX.COM
Resizer
• Compute & memory
intensive
• Interfaced through
uploader
File uploaded
Image expanded in memory 4k x 3k
Rotated and/or flipped
Resized to 3 sizes

23. MORE INFORMATION AT
NGINX.COM
At high volume
• Docker image crashed
• Taking out the underlying
VM
• Causing cluster to
become unstable

24. MORE INFORMATION AT
NGINX.COM
Health Check
• Uploader interfaces with
resizer
• Routes through N+
• Health check prevents
failure

25. MORE INFORMATION AT
NGINX.COM
4. Understand Your Failure Profile

26. Understand
Your Service
26

27. MORE INFORMATION AT
NGINX.COM
Dimensions of
failure
• Memory
• CPU
• Disk
Memory
CPU
Disk

28. MORE INFORMATION AT
NGINX.COM
Develop the
failure profile
• Establish thresholds
• Know the triggers
• Understand the
metrics

29. MORE INFORMATION AT
NGINX.COM
5. NGINX Plus as the Circuit Breaker

30. MORE INFORMATION AT
NGINX.COM
Circuit
Breakers
• Active health checks
• Retry
• Caching

31. MORE INFORMATION AT
NGINX.COM
Active Health
Check
• Check regularly
• Don’t over do it
• Internal
location /health-check-resizer {
internal;
health_check uri=/health match=conditions fails=1 interval=3s;
proxy_pass https://resizer;
proxy_ssl_protocols TLSv1.2; proxy_http_version
1.1;
}

32. MORE INFORMATION AT
NGINX.COM
Match
• Get specific data match conditions {
status 200-399;
header Content-Type ~ "application/json";
body ~ '{
"deadlocks":{"healthy":true},
"Disk":{"healthy":true},
"Memory":{"healthy":true}
}';
}

33. MORE INFORMATION AT
NGINX.COM
Don’t kill a
recovering
service
• Use the slow start
feature to allow a
service to recover
upstream resizer {
server resizer slow_start=30s;
zone backend 64k;
least_time last_byte;
keepalive 300;
}

34. MORE INFORMATION AT
NGINX.COM
6. Other Resiliency Options

35. MORE INFORMATION AT
NGINX.COM
Moderated
Delivery
• Use rate limiting to
keep your services
from failing in the first
place
http {
# Moderated delivery limit_req_zone $server_addr
zone=moderateReqs:1m rate=100r/s;
# ...
server {
# ...
limit_req zone=moderateReqs burst=150; limit_req_status 503;
# ...
}
}

36. MORE INFORMATION AT
NGINX.COM
Caching to
mitigate failure
• Providing service
continuity
• Read-oriented
services
• proxy_cache_use_
stale

37. MORE INFORMATION AT
NGINX.COM
Backup server
• Another service
continuity option
• Generic over custom
upstream personal-ad-server {
server personal-ad-server;
server generic-ad-server backup;
zone backend 64k;
least_time last_byte;
keepalive 300;
}

38. MORE INFORMATION AT
NGINX.COM
7. Q&A

39. Thank You
39