Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing

1,353 views

Published on

그림이 정상 출력되는 다음 링크의 자료를 확인해 주세요.
https://www.slideshare.net/deview/233-network-load-balancing-maglev-hashing-scheduler-in-ipvs-linux-kernel

Published in: Technology
  • 그림이 정상 출력되는 다음 링크의 자료를 확인해 주세요. https://www.slideshare.net/deview/233-network-load-balancing-maglev-hashing-scheduler-in-ipvs-linux-kernel
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing

  1. 1. Container Cluster(s) Container Cluster(s) Container Cluster(s) Container Host Container Host Container Host
  2. 2. Internet LB LB LB Container Host Container Host Container Host LB LB LB Router Router Container Cluster (s)
  3. 3. L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N
  4. 4. Container Cluster(s) Container Host Container Host Container Host restart restart restart stop stop stop start start start
  5. 5. L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N Container Host Container Host Container Host restart restart restart stop stop stop start start start
  6. 6. Internet LB LB LB Container Host Container Host Container Host LB LB LB Router Router start stop restart Container Cluster (s)
  7. 7. Container Host Container Host Container Host Container Host L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
  8. 8. Internet L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
  9. 9. Container Host Container Host Container Host Container Host L4-LB
  10. 10. Large Scale Load Balancing High Availability
  11. 11. L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB Host: 1.1.1.1 VIP: 10.0.0.1 Host: 1.1.1.2 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.4 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
  12. 12. Host: 1.1.1.1 IP Advertise L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1 Host: 1.1.1.2 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.4 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
  13. 13. PATH= Hashing( IP.Src, IP.Dst, IP.Protocol, Port.Src, Port.Dst ) Mod N L4-LB L4-LB Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1
  14. 14. Internet L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
  15. 15. PATH= Hashing() Mod ( N - 1 ) L4-LBL4-LBL4-LB L4-LB Host: 1.1.1.1 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
  16. 16. 192.68.0.2 Container[0] Container[1] Container[2] Container[3] L4-LB 1. Connection Established With Container[2] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 4. Container[0] has no idea about connection with container[2] 5. Container[0] Send RST. Connection Closed
  17. 17. 192.68.0.3 Container[0] Container[1] Container[2] Container[3] L4-LB L4-LB 1. Connection Established With Container[3] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 although the serving LB is alive. 4. Container[1] has no idea about connection with container[3] 5. Container[1] Send RST. Connection Closed L4-LB
  18. 18. Internet L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
  19. 19. Container Host Container Host Container Host Container Host L4-LB
  20. 20. Not Reliable Not scalable High Availability
  21. 21. BGP/ ECMP High Availability
  22. 22. 192.68.0.1 Hashing(IP) % 2 Container[0] Container[1] Hashing(IP) % 2 Hashing(IP) % 2 Hashing(IP) % 2 192.68.0.2 192.68.0.3 192.68.0.4
  23. 23. 192.68.0.1 Hashing(IP) % 4 Hashing(IP) % 4 Hashing(IP) % 4 Hashing(IP) % 4 192.68.0.2 192.68.0.3 192.68.0.4 Container[0] Container[1] Container[2] Container[3]
  24. 24. 192.68 .0.2 Container[0] 192.68 .0.3 Container[2] 192.68 .0.4 Container[0] Container[3] Container[2] 192.68 .0.1 Container[1] Container[3] Container[1] Guarantee to remap K/n only
  25. 25. 192.68 .0.2 Container[0] 192.68 .0.3 Container[2] 192.68 .0.4 Container[0] Container[3] Container[2] 192.68 .0.1 Container[1] Container[3] Container[1] Efficient Load Balancing
  26. 26. Consistent Hashing Backend Selection Packet Processing Packet Forwarding
  27. 27. B0 B1 B2 3 0 3 0 2 4 4 4 5 1 6 6 5 1 0 2 3 1 6 5 2 Backend = 3 Table Size = 7 Permutate Hashing Permutation table ( 3x 7 ) Preference list of Backends Lookup_table ( size =7,empty )
  28. 28. Population Permutation table ( 3x 7 ) B1 B0 B1 B0 B2 B2 B0 Lookup_table ( size =7 ) B0 B1 B2 3 0 3 0 2 4 4 4 5 1 6 6 5 1 0 2 3 1 6 5 2 [0] [1] [2] [3] [4] [5] [6] Assign backends by preference list
  29. 29. B1 B0 B1 B0 B2 B2 B0 each backend will receive an almost equal numberof connections. [0] [1] [2] [3] [4] [5] [6] L4-LB L4-LB B0 =3 * connection B1 =2 * connection B2 =2 * connection
  30. 30. Population Permutation table ( 2x 7 ) B0 B2 3 3 0 4 4 5 1 6 5 0 2 1 6 2 Assign backends by preference list Before B1 B0 B1 B0 B2 B2 B2 B1 is Removed Re-Permutate Hashing Lookup_table ( size =7 ) After B1 -> B0 B0 B1 -> B0 B0 B2 B2 B2 Same Value By hashing Lookup_table ( size =7 )
  31. 31. Backend SelectionPacket ProcessingPacket Forwarding
  32. 32. Internet L4-LB L4-LB L4-LB L4-LB
  33. 33. Consistent Hashing Backend Selection: FUNCTION Packet Processing: NETFILTER Packet Forwarding: NAT, DR, IP Tunneling
  34. 34. Weighted Round Robin Round Robin Source Hashing least connection
  35. 35. Makefile Maglev Hashing Scheduling Module
  36. 36. Maglev Hashing Scheduling Round Robin Source Hashing least connection Weighted Round Robin
  37. 37. ... Lookup_table Size=251 ... Lookup_table Size=131071 Disruption % VS Memory usage
  38. 38. New Service Old Service 5% 95%
  39. 39. Container Host Container Host Container Host Container Host W: 10 W: 40 W: 80
  40. 40. Container Host Container Host Container Host Container Host W: 10 W: 40 -> 0 W: 80
  41. 41. IPVS 1 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.3:80 DIP: 172.16.0.1:80 DIP: 172.16.0.5:80 DIP: 172.16.0.7:80 ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1 ip_vs_mh_permutate() & ip_vs_mh_populate() ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2 ip_vs_mh_permutate() & ip_vs_mh_populate() IPVS MH IPVS MH
  42. 42. 192.68.0.2 IPVS IPVS MH IPVS IPVS MH IPVS IPVS MH Container[0] Container[1] Container[2] Container[3] [2] [2] [2]
  43. 43. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
  44. 44. VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.3:80 DIP: 172.16.0.1:80 DIP: 172.16.0.5:80 DIP: 172.16.0.7:80 K/n IPVS MH
  45. 45. Container Host Container Host Container Host Container Host L4-LB Only K/n Disruption
  46. 46. IPVS MH IPVS 1 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS MH ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1 ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2
  47. 47. Container Host Container Host Container Host Container Host IPVS No Disruption
  48. 48. 192.68.0.2 Container[0] Container[1] Container[2] Container[3] IPVS 1. Connection Established With Container[2] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 4. IPVS MH can forward the packet to same destination by hashing. IPVS MH 5. Continue the Established connection.
  49. 49. 192.68.0.3 Container[0] Container[1] Container[2] Container[3] IPVS 1. Connection Established With Container[3] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 although the serving LB is alive. IPVS IPVS MH 4. IPVS MH can forward the packet to same destination by hashing With no connection info. 5. Continue the Established connection.
  50. 50. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
  51. 51. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
  52. 52. BGP/ ECMP IPVS Maglev Hashing Scheduler
  53. 53. Linux Kernel >= 4.18 Choose M in Kernel menuconfig to use IP_VS_MH
  54. 54. echo 1 > /proc/sys/net/ipv4/vs/sloppy_tcp echo 2 > /proc/sys/net/ipv4/vs/conn_reuse_mode

×