Diagram different network load balancing strategies

1. Load Balancing
Enabling High Availability
for Cloud Native Design

2. 2
(Inter)network
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
(Network)LoadBalancing
...
Well Known
Endpoint
scale
Client
Load Balancing

3. 3
Network
μService Architecture — Cloud Native
(HTTP)Route/Proxy
Client
Load Balancing Reality
LB
μS
…
μS
μS
LB
μS
…
μS
μS
LB
μS
…
μS
μS
LB
μS
…
μS
μS
LB
μS
…
μS
μS
LB
μS
…
μS
μS
Optional

4. Elastic Scaling
Cloud Native Design aka: μService Architecture
It’s all about:
4
Scaling → Redundancy
Compute → Instance Redundancy
Storage → State Redundancy
Network → Path Redundancy

5. 5
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
(Network)LoadBalancing
...
Well Known
Endpoint
scale
Single Path = Single Point of Failure
Network
Client
Single
Point of
Failure

6. Solutions — Load Balancer End Point is SPOF
6
Suitability Description Comments
LB in front of LB Moves the problem, does not fix anything.
DNS “LB”
• DNS is not reactive enough.
• Clients implement multi-A rec inconsistently.
VIP (Virtual IP)
• Master-Slave: eventual overload.
• Failure  100% of active connections lost.
NAT (Network Addr Translation)
• No existing OSS Solutions that are HA
• Commercial solutions expensive (Citrix)
SDN – per Flow management Recently became a viable solution.
Client Managed
 Ideal: client knows about multiple endpoints.
− Client must be modified.

7. 7
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LoadBalancer
...
Well Known
Endpoint
scale
Load Balancer in front of Load Balancer
Network
Client
LBLBLB
SPOF
Removed
New
SPOF

8. 8
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LB–1
...
scale
Well Known
Endpoint
DNS Load Balancing
Client
LB–2
No SPOF
Network
Client
Client
DNS
DNS
Optional
MirrorA Record
ep.com 1.1.1.1
ep.com 1.1.1.2
Update
Latency
Inconsistent
Balancing
Behavior
1.1.1.1
1.1.1.2

9. 9
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LB–1
...
scale
Well Known
Endpoint
VIP Load Balancing
Client
LB–2
Master/Slave
Network
Client
Client
1.1.1.10
1.1.1.2
VIP
1.1.1.10
failover
Problem: during failover
100% of the existing (TCP)
connections will be
DROPPED because of
TCP state loss.
Latency

10. 10
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LB–1
...
scale
Well Known
Endpoint
NAT Layer 4 + Protocol Proxy (L7) Load Balancing
LB–2
No SPOF
10.2.2.1
10.2.2.2
Border
Gateway
Border
Gateway
Network
Client
Client
L4
Rt+NAT
L4 Statefull
Router + NAT
(firewall)
L4
Rt+NAT
BGP
“spray”
State
Sync
1.1.1.1
Translated
IP(s)
1.1.1.1

11. 11
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LB–1
...
scale
Well Known
Endpoint
SDN-LB (L4) + Protocol Proxy (L7) Load Balancing
LB–2
No SPOF
Border
Gateway
Border
Gateway
Network
Client
Client
BGP
“spray”
1.1.1.1
1.1.1.1
HA Switch
Fabric
(OpenFlow)
2 x 2
Altoline
OpenSwitch
was
(OpenHalon)
[HA Pair]

12. 12
(μ)Service
(μ)Service
(μ)Service
...
Cloud / Engineered System
LB–1
...
scale
Well Known
Endpoint
Client Load Balancing
Client
LB–2
No SPOF
Network
Client
Client
DNS
DNS
Optional
MirrorA Record
ep.com 1.1.1.1
ep.com 1.1.1.2
1.1.1.1
1.1.1.2
LB-lib
LB-lib
LB-lib
Intelligent
Client Library

13. Citations
13
- “Making applications scalable with Load Balancing”, Willy Tarreau, http://1wt.eu/articles/2006_lb/
TBD

14. Thank you
Eric Gustafson gustafson@hpe.com
14