Presentation on scaling patterns for microservices and APIs.

1. GLUE CON 2015
Architectural Patterns for Scaling Microservices
and APIs

2. Scalability is not a Singleton
You have
less than 5
seconds to
deliver
content.
Maintain availability
Maintain performance
Average
cost of
critical app
failure per
hour is
$500,000 to
$1 million.

3. Scale is achieved by optimally distributing
load
 TCP Connections
 Response time
 HTTP Requests
 Least connections
 Fastest response
 Fastest app
 Round Robin
Thresholds Methods

4. The Scale Cube
 X-axis
 Y-axis
 Z-axis
http://microservices.io/articles/scalecube.html

5. Scalability: Out, Up, and Across
 Creating scalability
domains
 Balance
performance and
availability

6. X-Axis
 Like clustering
 Leverages cloning
 Typically operates at layer
4 (TCP)

7. Y-Axis
 Like routing
 Uses identifiable variables
(usually from HTTP headers)
 Operates at layer 7 (HTTP)

8. Z-Axis
 Like sharding
 Uses one or more identifiable variables (usually from HTTP headers)
 Operates at layer 7 (HTTP)

9. APIs and Microservices
API as Façade API as Endpoint
SERVICE
API
API
SERVICE
API
SERVIC
E
A
P
I
SERVIC
E
A
P
I

10. API as Endpoint
API
Standard X-axis Scaling Pattern
APIAPI

11. API Façade: Interpolated in Network
Standard Y-axis Scaling Pattern
Microservice A
Microservice B
Microservice C
API

12. One pattern is rarely enough

13. Most often combined in 2-3 tiers
 Y-axis routing pattern used to
distribute API calls to X-axis
scaled service clusters

14. 2-Tier Interpolated API
Y-Axis Scaling X-Axis Scaling
API

15. Top Mistakes Regardless of
Pattern
• Choosing the wrong thresholds
• Scaling too late leads to time outs &
performance degradation
• Not monitoring
• Services fail, users get angrified by time outs
• Not scaling the scaler
• Overwhelming the upstream scaler is a Very
Bad Thing™

16. @lmacvittie
F5 Networks
THANK YOU!