The document discusses challenges in implementing Kubernetes, particularly with multitenancy and complex applications. It introduces the NGINX ingress controller as a solution for managing ingress traffic and highlights the use of custom resources for advanced load balancing and routing capabilities. Additionally, it emphasizes the role of NGINX service mesh in securing communication and managing traffic within Kubernetes clusters.

1. Relevez les défis
Kubernetes avec
NGINX
BENJAMIN SIMON
Specialist Solution Engineer
@ benjamin.simon@f5.com

2. | ©2021 F5
2
Introduction

3. | ©2021 F5
3
Kubernetes stitches together a numberof servers (‘nodes’) to create a flat platform for running Containers:
What is Kubernetes?
Kubernetes
Master
API Server
Scheduler
Controller-
Manager
etcd
Kubernetes
Node
Kubelet
Kubernetes
Node
Kubelet
Kubernetes
Node
Kubelet
Ingress
Controller
Internal
Network
KubeProxy
KubeProxy
KubeProxy
External Load
Balancer
• BIG CIS
• NGINX
• Cloud LB
Users

4. | ©2021 F5
4
Challenges in implementing containers
SOURCE: CNCF 2020 SURVEY

5. | ©2021 F5
5
Two Challenges at Scale
Multitenancy (Teams)
How can multiple teams and
applications share a Container
environment safely and securely?
Complex Applications
How can you provide the
advanced capabilities that
complex applications require?

6. | ©2021 F5
7
Ingress Controller

7. | ©2021 F5
8
• Accepts traffic from outside the Kubernetes
platform, and load-balances it to pods
(containers) running inside the platform
• Configured using the Kubernetes API, with
objects called ‘Ingress Resources’
• Monitors the pods running in Kubernetes, and
automatically updates the load balancing rules if,
for example, pods are added or removed from a
service
A SPECIALIZED LOAD BALANCER FOR KUBERNETES ENVIRONMENTS:
The Ingress Controller
Internal
Network
Users
Ingress
Controller

8. | ©2021 F5
9
L7 Logic
(Ingress)
An Overly Simplified Picture

9. | ©2021 F5
10
NGINX Ingress Controller resources
Kubernetes
Ingress Resource
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
tls:
- hosts:
- hello.example.com
secretName: hello-secret
rules:
- host: hello.example.com
http:
paths:
- path: /
backend:
serviceName: service-b
servicePort: 80
NGINX
Ingress Resource
• Expose more NGINX/NGINX Plus
features e.g. TCP, GRPC load
balancing, traffic splitting, rate limits…
• Easy to use, familiar concepts, familiar
processes
• Cloud native -- support Prometheus,
Helm, OpenTracing, mTLS
• Special focus on security, stability and
self-service

10. | ©2021 F5
11
Ingress Resource are limited to Basic SSL/TLS and
HTTP Load Balancing
Customized using Annotations and Config Maps
Annotations Problems:
− Can quickly grow biggerthan the spec
− Lack validation
− Not suited for granular configuration
− Not portable
Problems with Ingress Controllers
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp
annotations:
nginx.org/lb-method: "ip_hash"
nginx.org/ssl-services: "webapp"
nginx.org/proxy-connect-timeout: "10s"
nginx.org/proxy-read-timeout: "10s"
nginx.org/proxy-send-timeout: "10s"
nginx.org/rewrites: "serviceName=webapp rewrite=/v1"
nginx.com/jwt-key: "webapp-jwk"
nginx.com/jwt-realm: "Webb App"
nginx.com/jwt-token: "$cookie_auth_token"
nginx.com/jwt-login-url: "https://login.example.com""
spec:
rules:
- host: webapp.example.com
. . .

11. | ©2021 F5
12
Reduce complexity with CRDs

12. | ©2021 F5
13
Introduction of Custom Ressources
The VirtualServer and VirtualServerRoute resources
are new load balancingconfiguration,an alternative to
native Ingress Ressource
The resources enable use cases not supported with
the Ingress resource, such as traffic splitting,
advanced content-basedrouting,and more …
NGINX Ingress Resources – Rich Capabilities

13. | ©2021 F5
15
Full control over how NGINX IC
handles the request:
• Simply forward to an upstream
• Return a redirect
• Return a direct response
Rewrite, hide or ignore headers:
• Set custom headers from request
parameters
• Add debugging response headers
• Rewrite URLs
Control how each Request is Handled

14. | ©2021 F5
16
Apply sophisticated Routing
Implement smart routing:
• A|B testing and canary releases using splits
• Identify debug traffic and route to new, test instance
• Separate requestsby API method or version
• Perform Blue-Green Deployments

15. | ©2021 F5
17
Enable Cross Namespace use cases
apiVersion: k8s.nginx.org/v1alpha1
kind: VirtualServer
metadata:
name: api-fe
namespace: frontend-ns
spec:
host: api.example.com
tls:
secret: api-ssl-secret
routes:
- path: /games/api
route: games-ns/games-route
- path: /stats/api
route: stats-ns/stats-route
Admin team
frontend-ns
Games team
games-ns
Stats team
stats-ns
/games/api
/stats/api

16. | ©2021 F5
18
Enable Cross Namespace use cases
apiVersion:k8s.nginx.org/v1alpha1
kind: VirtualServerRoute
metadata:
name: games-route
namespace: games-ns
spec:
host: api.example.com
upstreams:
- name: games
service: games-svc
port: 80
subroutes:
- path: /games/api
upstream: games
Admin team
frontend-ns
Games team
games-ns
Stats team
stats-ns
/games/api
/stats/api

17. | ©2021 F5
19
Demo

18. | ©2021 F5
20
Secure Ingress Traffic

19. | ©2021 F5
21
NGINX App Protect
Manage
CI/CD
Friendly
Secure
Highperforming
Security protection beyond signatures
Trusted Signatures from F5
Simple CI/CD integration
Designed formodern infrastructures
Rapid feedback loop forsecurity remediations
Unified F5 declarative interface
Security statistics via syslog
Backed by F5 Support

20. | ©2021 F5
22
Host
TLS
Upstreams
Routes
- Path
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
optional
Host
TLS
Upstreams
Routes
- Path
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
optional
NGINX Ingress Resources – Rich Capabilities
Host
TLS
Policies
Upstreams
Routes
- Path
Policies
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
VirtualServer
pass
redirect
return
proxy
pass
redirect
return
proxy
Host
Upstreams
Subroutes
- Path
Policies
Action
Split
Match
ErrorPage
pass
redirect
return
proxy
VirtualServerRoute
Policies
Access Control
Rate Limiting
Auth (JWT, OIDC)
MTLS (Ingress/Egress)
App Protect WAF

21. | ©2021 F5
23
Example of policy consumption

22. | ©2021 F5
24
NGINX Ingress Resources – Distributed Configuration

23. | ©2021 F5
25
NGINX Ingress Resources – Distributed Configuration
NetOps DevOps-FE
DevOps-NG
Identity
DevSecOps

24. | ©2021 F5
26
Demo

25. | ©2021 F5
27
NGINX Service Mesh

26. | ©2021 F5
28
L7 Logic (Ingress)
L3-L4 Networking
L3 – L7 Network
Management ==
Service Mesh
An Overly Simplified Picture

27. | ©2021 F5
29
Service Mesh controlscommunicationsbetween pods and external apps
What DoesA Service Mesh Do?
Secure Traffic
End-to-end encryption (Mutual TLS / mTLS), ACLs
Manage All Service Traffic
Load Balance, Circuit breaker, B|G, Rate Limiting…
Orchestration
Injection and sidecar management, K8s API integration
Measure Traffic
Generate transaction traces and real-time monitoring

28. | ©2021 F5
30
NGINX Service Mesh Components
• NGINX Service Mesh runs within a K8s cluster
• Securely manages ingress/egress traffic to
external services
• Can be deployed in any K8s cluster platform

29. | ©2021 F5
31
Mutual TLS
Instrumentation
Tracing
Rate Limiting
Traffic Splitting
Access Control
Egress Control
Secure traffic in a zero-trust network
Monitor performance, latency and availability
Debug transactions and locate faults
Protect vulnerable apps, limitgreedy clients
Implement A|B, Canary and Blue-Green upgrades
Implement allow-lists to accurately govern traffic
Manage and broker traffic to external services
Spotlight on Use Cases

30. | ©2021 F5
32
Demo

31. | ©2021 F5
33
Questions ?