Slides from Michael Pleshavkov - Platform Integration Engineer, NGINX about HTTP load balancing on Kubernetes with NGINX. You will learn how to configure load balancing for a web application using a Kubernetes Ingress resource and how to deploy and use NGINX Ingress controller.

1. Load Balancing Applications on
Kubernetes with NGINX
Michael Pleshakov – Platform Integration Engineer, NGINX Inc.

2. MORE INFORMATION AT NGINX.COM
Agenda
• Kubernetes and its key features
• Application delivery on Kubernetes: Ingress and
Ingress controllers (ICs)
• Introduce NGINX IC
• Demo: Delivering a simple web application using
Ingress with NGINX IC.
• Advanced configuration of NGINX IC
• Summary and Q&A
2
Michael Pleshakov
Platform Integration
Engineer, NGINX
michael@nginx.com

3. MORE INFORMATION AT NGINX.COM
Kubernetes
• Kubernetes has become the leading technology for
container orchestration
• 29% of our users said that Kubernetes was part of
their planned technology stack (from 2017 NGINX
user survey)
• One of the largest open source communities with
contributions from thousands of organizations
• Vibrant ecosystem of service providers and vendors
• All major cloud providers (AWS, Azure and GCP) offer
a managed Kubernetes solution

4. MORE INFORMATION AT NGINX.COM
Kubernetes Keys Features
Kubernetes:
• Runs containerized applications across a cluster of machines
• Manages applications – scaling, rolling upgrades
• Adds resilience to applications by restarting failed workloads
• Connects applications – internal service discovery and load balancing
• Supports running stateful applications
• And more …
How do you successfully deliver applications?

5. MORE INFORMATION AT NGINX.COM
Meet NGINX
• NGINX -- open source layer 4/layer 7 load
balancing solution, as well as a content-
cache and a web server:
• 63% of top 10K The busiest sites choose
NGINX (W3Techs Web server ranking 23-
Jan-2018)
• #1 downloaded application image on
DockerHub
• Releases:
• v0.1 - 04 Oct 2004
• v1.13.9 – 20 Feb 2018

6. MORE INFORMATION AT NGINX.COM
Application Delivery on Kubernetes
app
app
app
Kubernetes Cluster
Application pods
Users
Internet
How do we
deliver the app?

7. MORE INFORMATION AT NGINX.COM
Application Delivery Requirements
Requirement Meaning
1. Stable Public Endpoint DNS name/static public IP address and port
2. Performance Ensuring clients can access an application quickly with no delays
3. Scalability Scaling the number of application backends according with the demand
4. Reliability Mitigating failures of application backends
5. Ease of configuration Creating, deploying and maintaining the AD configuration is easy
6. Visibility Understanding how the application is being delivered in real-time and over
period of time
7. Security Using TLS to secure the client connections to the application
8. Routing Routing client requests at L7 (host header, URI, cookies)

8. MORE INFORMATION AT NGINX.COM
Ingress Resource
1. apiVersion: extensions/v1beta1
2. kind: Ingress
3. metadata:
4. name: hello-ingress
5. spec:
6. tls:
7. - hosts:
8. - hello.example.com
9. secretName: hello-secret
10. rules:
11. - host: hello.example.com
12. http:
13. paths:
14. - path: /
15. backend:
16. serviceName: hello-svc
17. servicePort: 80
Ingress:
• Built-in Kubernetes resource
• Configuration for an edge load
balancer (or ADC)
Ingress features:
• L7 routing based on the host
header and URI
• TLS termination

9. MORE INFORMATION AT NGINX.COM
Ingress Controller
• Kubernetes only provides Ingress
resource, not a load balancer
• Ingress Controller (IC) – software that
applies Ingress rules to a particular load
balancer
• Several IC implementations for
software/hardware/cloud load balancers
• It is common to refer to both a load
balancer and the IC software as the IC
Kubernetes
API
Ingress
Controller
Load
Balancer
Watches Ingress resources
Configures

10. MORE INFORMATION AT NGINX.COM
NGINX IC
Kubernetes
API
Ingress
Controller
Watches Ingress resources
Configures
• NGINX and the IC are in the same
pod in the same container.
• The IC generates configuration files
according to the created Ingress
resources and reloads NGINX
NGINX Ingress Controller Pod

11. MORE INFORMATION AT NGINX.COM
Application Delivery on Kubernetes
app
app
app
Kubernetes Cluster
Application pods
Users
Internet
How do we
deliver an app?

12. MORE INFORMATION AT NGINX.COM
Application Delivery with NGINX IC
app
app
app
Kubernetes Cluster
Application pods
Users
Internet
How do we expose
NGINX IC?
NGINX
IC pod

13. MORE INFORMATION AT NGINX.COM
How To Expose NGINX IC?
• On-premises
– Exposing the NGINX IC on a subset of
cluster nodes
– A load balancer in front of NGINX IC
• Cloud
– Cloud load balancer in front of NGINX IC

14. MORE INFORMATION AT NGINX.COM
On-premises: IC Nodes
• NGINX IC is deployed as a
DaemonSet on a subset of nodes (IC
nodes)
• Port mapping is used to expose
NGINX IC pods ports 80 and 443 on
those nodes
• Clients use the public IP of those
nodes to access the IC.
• It is recommended configure HA
between the IC nodes
IC Node 1 IC Node 2 Regular Node(s)
Public IP 1 Public IP 2

15. MORE INFORMATION AT NGINX.COM
Basic TCP
Load
Balancer
Public IP
On-premises:
Load Balancer + NodePorts
• NGINX IC is deployed as a
Deployment
• NGINX IC is exposed through a
Service with Type=NodePort
• The load balancer distributes client
connections among all nodes of the
cluster at the noderports.
• Clients connects to the NGINX IC
through the public IP of the load
balancer
• The load balancer must be HA

16. MORE INFORMATION AT NGINX.COM
Cloud
Load
Balancer
Public IP
Cloud Load Balancer
• NGINX IC is deployed as a
Deployment
• NGINX IC is exposed through a
Service with Type=LoadBalancer
• The cloud load balancer distributes
client connections among all nodes
of the cluster at the noderports.
• Clients connects to the NGINX IC
through the public IP of cloud the
load balancer
• The cloud load balancer is HA

17. MORE INFORMATION AT NGINX.COM
Application Delivery with NGINX IC
app
app
app
Kubernetes Cluster
Application pods
Users
Internet
Basic TCP
Load
Balancer
Public IP

18. MORE INFORMATION AT NGINX.COM
Demo
Delivering a simple web application using Ingress
with NGINX IC.

19. MORE INFORMATION AT NGINX.COM
Application Delivery Requirements
Requirement Kubernetes Kubernetes Ingress with NGINX Plus
Ingress Controller
1. Stable Public Endpoint V
2. Performance V V
3. Scalability V V
4. Reliability V V
5. Ease of configuration V
6. Visibility V
7. Security V
8. Routing V

20. MORE INFORMATION AT NGINX.COM
Limitations Of Ingress Resource
1. kind: Ingress
2. metadata:
3. name: hello-ingress
4. spec:
5. tls:
6. - hosts:
7. - hello.example.com
8. secretName: hello-secret
9. rules:
10. - host: hello.example.com
11. http:
12. paths:
13. - path: /
14. backend:
15. serviceName: hello-svc
16. servicePort: 80
Ingress features:
• L7 routing based on the host
header and URL
• TLS termination

21. MORE INFORMATION AT NGINX.COM
Supporting Advanced Use Cases
Using advanced NGINX features:
• Rewriting the URI of a request
Customizing NGINX behavior:
• Enabling HTTP/2
• Choosing a load balancing method
• Changing the SSL protocols and ciphers

22. MORE INFORMATION AT NGINX.COM
Customizing NGINX Behavior
1. kind: ConfigMap
2. apiVersion: v1
3. metadata:
4. name: nginx-config
5. data:
6. http2: "True"
7. ssl-protocols: "TLSv1.2"
8. lb-method: "least_conn"
1. apiVersion: extensions/v1beta1
2. kind: Ingress
3. metadata:
4. name: hello-ingress
5. annotations:
6. nginx.org/lb-method: "ip_hash"
7. spec:
8. tls:
9. - hosts:
10. - hello.example.com
11. secretName: hello-secret
12. rules:
13. - host: hello.example.com
14. http:
15. paths:
16. - path: /
17. backend:
18. serviceName: hello-svc
19. servicePort: 80

23. MORE INFORMATION AT NGINX.COM
Using Advanced Features
1. apiVersion: extensions/v1beta1
2. kind: Ingress
3. metadata:
4. name: hello-ingress
5. annotations:
6. nginx.org/rewrite: "serviceName=hello rewrite=/hi"
7. spec:
8. tls:
9. - hosts:
10. - hello.example.com
11. secretName: hello-secret
12. rules:
13. - host: hello.example.com
14. http:
15. paths:
16. - path: /
17. backend:
18. serviceName: hello-svc
19. servicePort: 80
hello.example.com/foo
->
hello.example.com/hi/foo

24. MORE INFORMATION AT NGINX.COM
Snippets
1. apiVersion: extensions/v1beta1
2. kind: Ingress
3. metadata:
4. name: hello-ingress
5. annotations:
6. nginx.org/location-snippets: |
7. proxy_set_header X-Custom-Header-1 foo;
8. proxy_set_header X-Custom-Header-2 bar;
9. spec:
10. tls:
11. - hosts:
12. - hello.example.com
13. secretName: hello-secret
14. rules:
15. - host: hello.example.com
16. http:
17. paths:
18. - path: /
19. backend:
20. serviceName: hello-svc
21. servicePort: 80
• Snippets allow to use native
NGINX configuration
• Available as ConfigMap keys
(for global, http, server and
location contexts) and
Annotations (for server and
location contexts)

25. MORE INFORMATION AT NGINX.COM
Changing the Template
1. {{range $upstream := .Upstreams}}
2. upstream {{$upstream.Name}} {
3. zone {{$upstream.Name}} 256k;
4. {{if $upstream.LBMethod }}{{$upstream.LBMethod}};{{end}}
5. {{range $server := $upstream.UpstreamServers}}
6. server {{$server.Address}}:{{$server.Port}};{{end}}
7. {{if $upstream.StickyCookie}}
8. sticky cookie {{$upstream.StickyCookie}};
9. {{end}}
10. {{if $.Keepalive}}keepalive {{$.Keepalive}};{{end}}
11.}{{end}}
12.
13.{{range $server := .Servers}}
14.server {
15. {{range $port := $server.Ports}}
16. listen {{$port}}{{if $server.ProxyProtocol}} proxy_protocol{{end}};
17. {{- end}}
18. {{if $server.SSL}}
19. {{- range $port := $server.SSLPorts}}
20. . . .

26. MORE INFORMATION AT NGINX.COM
Supporting Advanced Use Cases
Method Complexity (1-3)
ConfigMap 1
Annotations 1
Snippets 2
Changing the Template 3

27. MORE INFORMATION AT NGINX.COM
NGINX and NGINX Plus
NGINX Open Source
• Webserver
• Web accelerator
• Application Gateway
• Basic media streaming
• Basic Reverse proxy
Community supported, build-your-
own for third-party modules
NGINX Plus
• Application Delivery features:
• Advanced load balancing and health checks
• Web Application Firewall
• Adaptive streaming for video
• Enterprise Management features:
• Dynamic configuration
• Monitoring and status
• JWT Authentication
• Cache purge management
Fully tested and supported by NGINX Inc.

28. MORE INFORMATION AT NGINX.COM
NGINX Ingress Controllers
• NGINX/NGINX Plus Ingress Controllers --
https://github.com/nginxinc/kubernetes-ingress
• NGINX Ingress Controller --
https://github.com/kubernetes/ingress-nginx

29. MORE INFORMATION AT NGINX.COM
NGINX Ingress Controllers
Aspect of Feature kubernetes/ingress-nginx nginxinc/kubernetes-ingress
with NGINX
nginxinc/kubernetes-
ingress with NGINX Plus
Authors Kubernetes community NGINX Inc and community NGINX Inc and community
NGINX version Custom NGINX build with
third-party modules
NGINX official mainline build NGINX Plus
Commercial support No No Included
Standard Ingress Yes Yes Yes
Annotations Yes Yes Yes
ConfigMaps Yes Yes Yes
TCP/UDP Extension Yes Coming soon Coming soon
JWT Validation No No Yes
Extended Status Yes, Via a third-party module No Yes
Prometheus Yes No Yes
Dynamic Reconfiguration No No Yes

30. MORE INFORMATION AT NGINX.COM
NGINX IC -- Summary
a
p
p
a
p
p
a
p
p
Kubernetes Cluster
Application pods
Users
Intern
et
Load
Balancer
• HA and scalable solution for
application delivery on Kubernetes
• High performance and stability
• Flexible deployment – NGINX is
deployed as a k8s application – and
configuration
• Advanced features of NGINX via the
ConfigMap and Annotations