This document discusses load balancing for gRPC services on Kubernetes. It explains that gRPC uses HTTP/2, which relies on long-lived TCP connections, making traditional load balancing at the TCP level ineffective. Instead, load balancing must be done at the HTTP layer by understanding the protocol. The document recommends using a lightweight proxy like Linkerd, which can provide request-level load balancing for gRPC services in Kubernetes without requiring changes to gRPC clients.

1. gRPC Load Balancing on
Kubernetes without Tears
Orig Author: William Morgan (Buoyant)
Slide by : Soohan Ahn

2. Orignial blog post link
• https://kubernetes.io/blog/2018/11/07/grpc-load-
balancing-on-kubernetes-without-tears/

3. Why does gRPC need special load
balancing?
• gRPC is built on HTTP/2
• HTTP/2 is designed to have a single long-lived TCP
connection

4. Why does gRPC need special load
balancing?
• gRPC is built on HTTP/2
• HTTP/2 is designed to have a single long-lived TCP
connection
• Reduces the overhead of connection management
• Connection balancing doesn't work well

5. Why does gRPC need special load
balancing?
• Connection balancing doesn't work well
• Once the connection is established, there’s no more balancing to be
done.
• All requests will get pinned to a single destination pod.

6. On http/1.1?
• In contrast to HTTP/2, HTTP/1.1 cannot multiplex requests.
• Only one HTTP request can be active at a time per TCP
connection.
Each blue box : Connection

7. Each blue box : Connection
On http/1.1?
• Additionally, long-lived HTTP/1.1 connections typically expire
after some time, and are torn down by the client (or server).
Connection may expired.

8. How do we do on gRPC?
• Connection balancing -> Request balancing

9. How do we do on gRPC?
• We need to make decisions at L5/L7 rather than L3/L4,
• Need to understand the protocol sent over the TCP connections.

10. On Kubernetes?
• Use https://godoc.org/google.golang.org/grpc/balancer.
• Too complex on the dynamic env.
• Headless services
• On outdated gRPC clients, is not possible to maintain load balancing
pool automatically.
• Use a lightweight proxy.
• Such as Linkerd.
• https://linkerd.io/

11. Linkerd
• A CNCF-hosted service mesh for Kubernetes.
• Also functions as a service sidecar.