Download as PDF, PPTX
Uploaded byHungWei Chiu
Load Balancing 101
This document provides an overview of load balancing, explaining its functionality and different algorithms, particularly in the context of Kubernetes. It covers the distinction between layer 4 and layer 7 load balancing, various implementations, and the role of middleware. Additionally, it discusses real-world applications, including gRPC protocol handling in a Kubernetes environment.
More Related Content
![[20200720]cloud native develoment - Nelson Lin](https://cdn.slidesharecdn.com/ss_thumbnails/20200720cloudnativedeveloment-200824031228-thumbnail.jpg?width=640&height=640&fit=bounds)
![Connection load balancing with mikrotik [workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/20161013loadbalancing-161013083020-thumbnail.jpg?width=640&height=640&fit=bounds)
Load Balancing 101
- 1.
- 2. Who Am I •Hung-Wei Chiu (hwchiu) • Member of Technical Staff at Open Networking Foundation • https://www.hwchiu.com • Co-Organizer of SDNDS-TW/CNTUG
- 3. Outline • What’s Load-Balancing •How it works • Kubernetes
- 4. Load Balancing • Efficientlydistribute incoming network traffic across a server pool • Distribute Algorithm
- 5. Functionality • Ensure highavailability and reliability • Only send requests to servers that are online • Provides the flexibility to add or subtract servers as demand dictates • Distributes client requests or network load efficiently across multiple servers. https://www.nginx.com/resources/glossary/load-balancing/
- 6. Distribute Algorithm • RoundRobin • Header Hash • Layer 3 • Layer 4 • Layer 7 • Server Status • Connection number • …etc
- 7. OSI / TCP-IP OSI– TCP/IP https://techdifferences.com/difference-between-tcp-ip-and-osi-model.html Layer1 Layer2 Layer3 Layer4 Layer5 Layer6 Layer7
- 8.
- 9.
- 10.
- 11. True/False ? • KubernetesService (Layer 4) • ClusterIP • NodePort • Kubernetes Ingress (Layer 7) • Different Implementation • TCP/UDP/HTTP
- 12.
- 13. Client Client Client Client Server Server Server Load Balancer Typical Implementation 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.41.2.3.4 192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 src dest src dest
- 14. Client Client Client Client Server Server Server Load Balancer Typical Implementation 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.41.2.3.4 192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 src dest src dest HTTPS HTTPCertificate
- 15. How it works •Type • Client-Side • Middleware • Connection Statue • Proxy • Transparency
- 16. Client-Side • Client directlysends request to backend server. • Query all possible addresses of backend servers • Apply distributed algorithm • Send request to backend server • Someone should provide address list of all backend server
- 17.
- 18.
- 19.
- 20. Client Client Client Client Server Server Server Address list Provider Client-Side LB 192.168.1.2 192.168.1.3 192.168.1.4 Sendto 192.168.1.3 client_ip 192.168.1.3 192.168.1.3 client_ip
- 21. Real Case • gRPCis a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • gRPC in Kubernetes environment • Server type: Headless • Return a set of PodIPs • https://medium.com/google-cloud/loadbalancing- grpc-for-kubernetes-cluster-services-3ba9a8d8fc03
- 22. Middleware • Client sendspackets to a middleware • Middleware handles the traffic load-balancing • Client doesn’t need to know the IP address of backend servers, only the middleware. • Different implementation • Proxy • Transparency
- 23. Proxy • Usually, ademand daemon to handle the incoming traffic. • Establish a new network connection. • One to one • One to many • SSL-terminator
- 24. Client Client Client Client Server Server Server Load Balancer Proxy 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 1.2.3.4192.168.1.2 NGINX 1.2.3.4 client_ip 192.168.1.2 1.2.3.4 Different TCP connections
- 25. Real Case • gRPCis a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • Layer 4 proxy can’t identify gRPC connection. • All gRPC request will be forward to same backend server. • Layer 7 proxy if supports gRPC • One to Many • Identify gRPC format • Envoy/LinkerD https://www.bugsnag.com/blog/envoy
- 26.
- 27.
- 28.
- 29.
- 30. Transparency • Modify packetsto make it be routed to backend server. • Keep the same connection, no additional one. • Client still doesn’t know the IP address of backend servers.
- 31. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection
- 32. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection
- 33. Real Case • gRPCis a layer 7 protocol, multiple gRPC request rely on the same TCP connection. • gRPC load-balancing can’t work correctly in this mode. • Only single TCP connection. • TCP is a connection based protocol (3 way handshake)
- 34. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection Clone packets TCP needs connection status.
- 35. Client Client Client Client Server Server Server Transparency 1.2.3.4 192.168.1.2 192.168.1.3 192.168.1.4 client_ip 1.2.3.4 client_ip192.168.1.2 1.2.3.4 client_ip 192.168.1.2 client_ip Same tcp connection TCP needs connection status.
- 36. Example • Kubernetes Service(by default) • ClusterIP/NodeIP • Packet are modified by iptables • Connection persistence are maintained by kernel conntrack.
- 37. Node1 Node2 Node3 BackendBackendBackend clusterIP
- 38. Node1 Node2 Node3 BackendBackendBackend clusterIP Kube-proxy Kube-proxy Kube-proxy Insert iptables rules Insert iptables rules Insert iptables rules
- 39. Node1 Node2 Node3 BackendBackendBackend clusterIP Client 1. Choose the backend server 2. Change the packet destination IP address 3. Forward packet Iptables..
- 40. Node1 Node2 Node3 BackendBackendBackend clusterIP 10.104.8.48 10.244.0.4 10.244.0.5 10.244.0.6
- 41. Node3 Backend Client Iptables.. PacketsPacketsPackets Match ClusterIP Choosebackend server Change Packet IP ip/protocol Distributed algo Action
- 42. Node3 Backend Client Iptables.. PacketsPacketsPackets Match ClusterIP Choosebackend server Change Packet IP ip/protocol Distributed algo Action
- 44. EP1 EP2 EP3 P< 0.2 P < 0.25 EP1 EP2 EP3 EP4 EP5 P < 0.33 P < 0.5 EP4 EP5 P = 4/5 * 1/4 = 1/5 P = 4/5 * 3/4 * 1/3 = 1/5
- 45. Question • How tomake the connection consistent? https://medium.com/@chirag.singla/tcp-3-way-handshake-and-latency-aae5a2a6a68b
- 46.
- 47. Answer • Only thefirst packet meets NAT rules • Kernel modifies the packets for you • You need the conntrack tool to control it. • You should choose the destination before you see the real data packets.
- 48. Kubernetes service • Peoplealways said kubernetes only supports layer 4 load-balancing • Is it possible to provide layer 7 load-balancing in ? • Without modifying kubernetes source code
- 49. I think • Fornormal TCP implementation, No. • Should be Yes for customized TCP protocol • Real zero-RTT TCP. • How about other protocols ? • UDP ?
- 50. DEMO • Source code •https://github.com/cloud-native-taiwan/k8s- course/pull/5/files • Modify the iptables linux kernel module • Statistic module • Parse the UDP payload and try to match