The document discusses the architecture and functionality of a load-balancing system involving container clusters and hosts. It emphasizes high availability, efficient load balancing techniques, and the use of IPVS with Maglev hashing scheduler for handling traffic. It also outlines connection management during load balancer disruptions and suggests dynamic updates to the IPVS configuration for reliability.

4. Container Cluster(s)
Container Cluster(s)
Container Cluster(s)
Container Host Container Host Container Host

5. Internet
LB
LB
LB
Container
Host
Container
Host
Container
Host
LB
LB
LB
Router
Router
Container Cluster (s)

6. L4-LB L4-LB L4-LB L4-LB
VIP: 10.0.0.1:80
Destination IP 1
Destination IP 2
Destination IP N
VIP: 10.0.0.1:80
Destination IP 1
Destination IP 2
Destination IP N
VIP: 10.0.0.1:80
Destination IP 1
Destination IP 2
Destination IP N
VIP: 10.0.0.1:80
Destination IP 1
Destination IP 2
Destination IP N

7. Container Cluster(s)
Container Host Container Host Container Host
restart
restart
restart
stop stop
stop
start
start
start

8. L4-LB L4-LB L4-LB L4-LB
VIP: 10.0.0.1:80
DIP 1 ... N
VIP: 10.0.0.1:80
DIP 1 ... N
VIP: 10.0.0.1:80
DIP 1 ... N
VIP: 10.0.0.1:80
DIP 1 ... N
Container Host Container Host Container Host
restart
restart
restart
stop stop
stop
start
start
start

9. Internet
LB
LB
LB
Container
Host
Container
Host
Container
Host
LB
LB
LB
Router
Router
start
stop
restart
Container Cluster (s)

10. Container Host
Container Host
Container Host
Container Host
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB

11. Internet
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB

12. Container Host
Container Host
Container Host
Container Host
L4-LB

13. Large Scale
Load Balancing
High
Availability

15. L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
Host: 1.1.1.1
VIP: 10.0.0.1
Host: 1.1.1.2
VIP: 10.0.0.1
Host: 1.1.1.3
VIP: 10.0.0.1
Host: 1.1.1.4
VIP: 10.0.0.1
Host: 1.1.1.5
VIP: 10.0.0.1
Host: 1.1.1.6
VIP: 10.0.0.1

16. Host: 1.1.1.1
IP Advertise
L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
VIP: 10.0.0.1
Host: 1.1.1.2
VIP: 10.0.0.1
Host: 1.1.1.3
VIP: 10.0.0.1
Host: 1.1.1.4
VIP: 10.0.0.1
Host: 1.1.1.5
VIP: 10.0.0.1
Host: 1.1.1.6
VIP: 10.0.0.1

17. PATH= Hashing(
IP.Src,
IP.Dst,
IP.Protocol,
Port.Src,
Port.Dst )
Mod N
L4-LB L4-LB
Host: 1.1.1.3
VIP: 10.0.0.1
Host: 1.1.1.5
VIP: 10.0.0.1

18. Internet
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB

19. PATH= Hashing()
Mod ( N - 1 )
L4-LBL4-LBL4-LB L4-LB
Host: 1.1.1.1
VIP: 10.0.0.1
Host: 1.1.1.3
VIP: 10.0.0.1
Host: 1.1.1.5
VIP: 10.0.0.1
Host: 1.1.1.6
VIP: 10.0.0.1

20. 192.68.0.2
Container[0]
Container[1]
Container[2]
Container[3]
L4-LB
1. Connection
Established With
Container[2]
2. A Load
Balancer Down
3. ECMP Disruption.
Forward packet
another l4
4. Container[0] has no
idea about connection
with container[2]
5. Container[0] Send RST.
Connection Closed

21. 192.68.0.3
Container[0]
Container[1]
Container[2]
Container[3]
L4-LB
L4-LB
1. Connection
Established With
Container[3]
2. A Load
Balancer Down
3. ECMP Disruption.
Forward packet
another l4 although
the serving LB is
alive. 4. Container[1] has no
idea about connection
with container[3]
5. Container[1] Send RST.
Connection Closed
L4-LB

22. Internet
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB
L4-LB

23. Container Host
Container Host
Container Host
Container Host
L4-LB

24. Not
Reliable
Not
scalable
High
Availability

25. BGP/
ECMP
High
Availability

27. 192.68.0.1 Hashing(IP) % 2
Container[0]
Container[1]
Hashing(IP) % 2
Hashing(IP) % 2
Hashing(IP) % 2
192.68.0.2
192.68.0.3
192.68.0.4

28. 192.68.0.1 Hashing(IP) % 4
Hashing(IP) % 4
Hashing(IP) % 4
Hashing(IP) % 4
192.68.0.2
192.68.0.3
192.68.0.4
Container[0]
Container[1]
Container[2]
Container[3]

29. 192.68
.0.2
Container[0]
192.68
.0.3
Container[2]
192.68
.0.4
Container[0]
Container[3]
Container[2]
192.68
.0.1
Container[1]
Container[3]
Container[1]
Guarantee
to remap
K/n only

30. 192.68
.0.2
Container[0]
192.68
.0.3
Container[2]
192.68
.0.4
Container[0]
Container[3]
Container[2]
192.68
.0.1
Container[1]
Container[3]
Container[1]
Efficient
Load
Balancing

32. Consistent Hashing
Backend Selection
Packet Processing
Packet Forwarding

33. B0 B1 B2
3 0 3
0 2 4
4 4 5
1 6 6
5 1 0
2 3 1
6 5 2
Backend = 3
Table Size = 7
Permutate
Hashing
Permutation table ( 3x 7 )
Preference list
of Backends
Lookup_table ( size =7,empty )

34. Population
Permutation table ( 3x 7 )
B1 B0 B1 B0 B2 B2 B0
Lookup_table ( size =7 )
B0 B1 B2
3 0 3
0 2 4
4 4 5
1 6 6
5 1 0
2 3 1
6 5 2
[0] [1] [2] [3] [4] [5] [6]
Assign backends by
preference list

35. B1 B0 B1 B0 B2 B2 B0
each backend will receive an
almost equal numberof
connections.
[0] [1] [2] [3] [4] [5] [6]
L4-LB
L4-LB
B0 =3 * connection
B1 =2 * connection
B2 =2 * connection

36. Population
Permutation table ( 2x 7 )
B0 B2
3 3
0 4
4 5
1 6
5 0
2 1
6 2
Assign backends by
preference list
Before
B1
B0
B1
B0
B2
B2
B2
B1 is
Removed
Re-Permutate
Hashing
Lookup_table ( size =7 )
After
B1 -> B0
B0
B1 -> B0
B0
B2
B2
B2
Same Value By
hashing
Lookup_table ( size =7 )

38. Backend SelectionPacket ProcessingPacket Forwarding

40. Internet
L4-LB
L4-LB
L4-LB
L4-LB

42. Consistent Hashing
Backend Selection:
FUNCTION
Packet Processing:
NETFILTER
Packet Forwarding:
NAT, DR, IP Tunneling

43. Weighted
Round Robin
Round Robin
Source Hashing
least
connection

44. Makefile
Maglev Hashing
Scheduling
Module

45. Maglev Hashing
Scheduling
Round Robin
Source Hashing
least
connection
Weighted
Round Robin

47. ...
Lookup_table Size=251
...
Lookup_table Size=131071
Disruption % VS Memory usage

49. New Service
Old Service
5%
95%

50. Container Host
Container Host
Container Host
Container Host
W: 10
W: 40
W: 80

51. Container Host
Container Host
Container Host
Container Host
W: 10
W: 40 -> 0
W: 80

52. IPVS 1 VIP: 10.0.0.1:80 MH
DIP: 172.16.0.1:80
DIP: 172.16.0.2:80
DIP: 172.16.0.3:80
DIP: 172.16.0.10:80
IPVS 2 VIP: 10.0.0.1:80 MH
DIP: 172.16.0.3:80
DIP: 172.16.0.1:80
DIP: 172.16.0.5:80
DIP: 172.16.0.7:80
ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1
ip_vs_mh_permutate() &
ip_vs_mh_populate()
ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2
ip_vs_mh_permutate() &
ip_vs_mh_populate()
IPVS MH IPVS MH

53. 192.68.0.2
IPVS
IPVS MH
IPVS
IPVS MH
IPVS
IPVS MH
Container[0]
Container[1]
Container[2]
Container[3]
[2]
[2]
[2]

55. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH
Add IP of containers
dynamically To IPVS
Add IP of containers
dynamically To IPVS

56. VIP: 10.0.0.1:80 MH
DIP: 172.16.0.1:80
DIP: 172.16.0.2:80
DIP: 172.16.0.3:80
DIP: 172.16.0.10:80
IPVS 2 VIP: 10.0.0.1:80 MH
DIP: 172.16.0.3:80
DIP: 172.16.0.1:80
DIP: 172.16.0.5:80
DIP: 172.16.0.7:80
K/n
IPVS MH

57. Container Host
Container Host
Container Host
Container Host
L4-LB
Only K/n Disruption

58. IPVS MH
IPVS 1 VIP: 10.0.0.1:80 MH
DIP: 172.16.0.1:80
DIP: 172.16.0.2:80
DIP: 172.16.0.3:80
DIP: 172.16.0.10:80
IPVS 2 VIP: 10.0.0.1:80 MH
DIP: 172.16.0.1:80
DIP: 172.16.0.2:80
DIP: 172.16.0.3:80
DIP: 172.16.0.10:80
IPVS MH
ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1 ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2

59. Container Host
Container Host
Container Host
Container Host
IPVS
No Disruption

60. 192.68.0.2
Container[0]
Container[1]
Container[2]
Container[3]
IPVS
1. Connection
Established With
Container[2]
2. A Load
Balancer Down
3. ECMP Disruption.
Forward packet
another l4
4. IPVS MH can forward
the packet to same
destination by hashing.
IPVS MH
5. Continue the
Established connection.

61. 192.68.0.3
Container[0]
Container[1]
Container[2]
Container[3]
IPVS
1. Connection
Established With
Container[3]
2. A Load
Balancer Down
3. ECMP Disruption.
Forward packet
another l4 although
the serving LB is
alive.
IPVS
IPVS MH
4. IPVS MH can forward
the packet to same
destination by hashing
With no connection info.
5. Continue the
Established connection.

62. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH
Add IP of containers
dynamically To IPVS
Add IP of containers
dynamically To IPVS

63. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH
Add IP of containers
dynamically To IPVS
Add IP of containers
dynamically To IPVS

64. BGP/
ECMP
IPVS Maglev
Hashing
Scheduler

65. Linux Kernel >= 4.18
Choose M in Kernel menuconfig
to use IP_VS_MH

66. echo 1 > /proc/sys/net/ipv4/vs/sloppy_tcp
echo 2 > /proc/sys/net/ipv4/vs/conn_reuse_mode