NGINX KIC in Cloud Native Kubernetes
From Network to Identity Layer
Koen Vanderpoorten
Solution Engineer BELUX
| ©2020 F5
2
PUBLIC CLOUD KUBERNETES
| ©2020 F5
3
So everything is perfect…
…or is it not?
| ©2020 F5
4
Challenge of k8s loadbalancer services in public clouds?
Users
Operational, Financial and Security Challenges
Cloud Native Kubernetes
| ©2020 F5
5
• Accepts traffic from outside the Kubernetes
platform, and load-balances it to pods
(containers) running inside the platform
• One single IP address and load balancer for
multiple applications, routing is based on URI’s
(L7 info)
• Monitors the pods running in Kubernetes, and
automatically updates the load balancing rules if,
for example, pods are added or removed from a
service
The Ingress Controller
Internal
Network
Users
Ingress
Controller
A specialized load balancer for Kubernetes environments:
Load
Balancer
| ©2020 F5
6
Ingress and Annotations – the pros and cons
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp
annotations:
nginx.org/lb-method: "ip_hash"
nginx.org/ssl-services: "webapp"
nginx.org/proxy-connect-timeout: "10s"
nginx.org/proxy-read-timeout: "10s"
nginx.org/proxy-send-timeout: "10s"
nginx.org/rewrites: "serviceName=webapp rewrite=/v1"
nginx.com/jwt-key: "webapp-jwk"
nginx.com/jwt-realm: "Webb App"
nginx.com/jwt-token: "$cookie_auth_token"
nginx.com/jwt-login-url: "https://login.example.com""
spec:
rules:
- host: webapp.example.com
. . .
Implement features missing in Ingress spec
Lack validation
Not suited for granular configuration
Good Things:
Can quickly grow bigger than the spec
Bad Things:
Not portable
| ©2020 F5
7
Enter NGINX KIC CRD’s
| ©2021 F5
8
Host
TLS
Upstreams
Routes
- Path
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
optional
Host
TLS
Upstreams
Routes
- Path
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
optional
NGINX Ingress Resources – Rich Capabilities
Host
TLS
Policies
Upstreams
Routes
- Path
Policies
Action
Split
Match
Route
ErrorPage
pass
redirect
return
proxy
delegation
VirtualServer
pass
redirect
return
proxy
pass
redirect
return
proxy
Host
Upstreams
Subroutes
- Path
Policies
Action
Split
Match
ErrorPage
pass
redirect
return
proxy
VirtualServerRoute
NGINX server configuration
NGINX http configuration
Server and HTTP snippets
NGINX location configuration
Location snippets
Policies
Access Control
Rate Limiting
Auth (JWT, OIDC)
MTLS (Ingress/Egress)
App Protect WAF
| ©2020 F5
9
Conditional Routing
Split Routing
Error Pages
Rate Limiting
Authentication
Web Application Firewall
NGINX Snippets
Multiple versions, multiple clients
A|B Testing for safe production deployment
Implement ‘Circuit Breakers’ to contain failures
Protect vulnerable apps, limit greedy clients
Offload identity checking from apps and centralize
Protect from known and unknown vulnerabilities
“to-the-metal” with NGINX configuration
Use Cases
| ©2021 F5
10
NGINX Ingress Resources – Distributed Configuration
| ©2021 F5
11
NGINX Ingress Resources – Distributed Configuration
NetOps DevOps-FE
DevOps-NG
Identity
DevSecOps
| ©2021 F5
12
NGINX
Ingress Controller
WITH KUBERNETES & NGINX INGRESS CONTROLLER
Manage Complexity in Production
Apps
Teams
Basic
Ingress
Solutions
1 Many
Many
1
| ©2020 F5
13
Why not use the Community version of NGINX Ingress Controller?
WAIT, THERE’S MORE THAN ONE?
Footprint
Latency
Timeouts
Security
Community: 500MB
NGINX Plus: 120MB
Community: Slowed by timeouts
NGINX Plus: Dynamically reconfigures
Community: 8809
NGINX Plus: 0
Community: OpenResty = CVE problems
NGINX Plus: Proactive CVE patching,
Integrated WAF &
service mesh
Driven by innovation at the
expense of feature stability
Driven by enterprise-ready stability
without compromising innovation
| ©2020 F5
14
2020 PERFORMANCE TEST RESULTS
Latency in a Dynamic Deployment
| ©2020 F5
15
UNNECESSARY RISK IN YOUR K8S ENVIRONMENTS
Delays in CVE patching
Competitor
| ©2021 F5
16
THE COST OF ADDING SECURITY IS LOW COMPARED WITH ITS IMPACT
NGINX App Protect Performance
0
0,5
1
1,5
2
2,5
Throughput (MB/sec)
No Protection NGINX App Protect ModSec
0
2000
4000
6000
8000
10000
12000
14000
Requests/sec
No Protection NGINX App Protect ModSec
0
100
200
300
400
500
600
700
800
Latency (ms)
No Protection NGINX App Protect ModSec
| ©2021 F5
17
AppProtect + KIC + NSM = Mesh Ingress WAF
• AppProtect runs in NGINX+ KIC at the edge
• Provides L7 WAF security for all mesh-based
ingress
• mTLS between KIC and NSM
NGINX
AppProtect
| ©2021 F5
18
Mutual TLS
Instrumentation
Tracing
Rate Limiting
Traffic Splitting
Access Control
Egress Control
Secure traffic in a zero-trust network
Monitor performance, latency and availability
Debug transactions and locate faults
Protect vulnerable apps, limit greedy clients
Implement A|B, Canary and Blue-Green upgrades
Implement allow-lists to accurately govern traffic
Manage and broker traffic to external services
Spotlight on Service Mesh - Use Cases
| ©2021 F5
19
Demo Time
Deploying NGINX in Cloud Native Kubernetes

Deploying NGINX in Cloud Native Kubernetes

  • 1.
    NGINX KIC inCloud Native Kubernetes From Network to Identity Layer Koen Vanderpoorten Solution Engineer BELUX
  • 2.
    | ©2020 F5 2 PUBLICCLOUD KUBERNETES
  • 3.
    | ©2020 F5 3 Soeverything is perfect… …or is it not?
  • 4.
    | ©2020 F5 4 Challengeof k8s loadbalancer services in public clouds? Users Operational, Financial and Security Challenges Cloud Native Kubernetes
  • 5.
    | ©2020 F5 5 •Accepts traffic from outside the Kubernetes platform, and load-balances it to pods (containers) running inside the platform • One single IP address and load balancer for multiple applications, routing is based on URI’s (L7 info) • Monitors the pods running in Kubernetes, and automatically updates the load balancing rules if, for example, pods are added or removed from a service The Ingress Controller Internal Network Users Ingress Controller A specialized load balancer for Kubernetes environments: Load Balancer
  • 6.
    | ©2020 F5 6 Ingressand Annotations – the pros and cons apiVersion: extensions/v1beta1 kind: Ingress metadata: name: webapp annotations: nginx.org/lb-method: "ip_hash" nginx.org/ssl-services: "webapp" nginx.org/proxy-connect-timeout: "10s" nginx.org/proxy-read-timeout: "10s" nginx.org/proxy-send-timeout: "10s" nginx.org/rewrites: "serviceName=webapp rewrite=/v1" nginx.com/jwt-key: "webapp-jwk" nginx.com/jwt-realm: "Webb App" nginx.com/jwt-token: "$cookie_auth_token" nginx.com/jwt-login-url: "https://login.example.com"" spec: rules: - host: webapp.example.com . . . Implement features missing in Ingress spec Lack validation Not suited for granular configuration Good Things: Can quickly grow bigger than the spec Bad Things: Not portable
  • 7.
    | ©2020 F5 7 EnterNGINX KIC CRD’s
  • 8.
    | ©2021 F5 8 Host TLS Upstreams Routes -Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional Host TLS Upstreams Routes - Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional NGINX Ingress Resources – Rich Capabilities Host TLS Policies Upstreams Routes - Path Policies Action Split Match Route ErrorPage pass redirect return proxy delegation VirtualServer pass redirect return proxy pass redirect return proxy Host Upstreams Subroutes - Path Policies Action Split Match ErrorPage pass redirect return proxy VirtualServerRoute NGINX server configuration NGINX http configuration Server and HTTP snippets NGINX location configuration Location snippets Policies Access Control Rate Limiting Auth (JWT, OIDC) MTLS (Ingress/Egress) App Protect WAF
  • 9.
    | ©2020 F5 9 ConditionalRouting Split Routing Error Pages Rate Limiting Authentication Web Application Firewall NGINX Snippets Multiple versions, multiple clients A|B Testing for safe production deployment Implement ‘Circuit Breakers’ to contain failures Protect vulnerable apps, limit greedy clients Offload identity checking from apps and centralize Protect from known and unknown vulnerabilities “to-the-metal” with NGINX configuration Use Cases
  • 10.
    | ©2021 F5 10 NGINXIngress Resources – Distributed Configuration
  • 11.
    | ©2021 F5 11 NGINXIngress Resources – Distributed Configuration NetOps DevOps-FE DevOps-NG Identity DevSecOps
  • 12.
    | ©2021 F5 12 NGINX IngressController WITH KUBERNETES & NGINX INGRESS CONTROLLER Manage Complexity in Production Apps Teams Basic Ingress Solutions 1 Many Many 1
  • 13.
    | ©2020 F5 13 Whynot use the Community version of NGINX Ingress Controller? WAIT, THERE’S MORE THAN ONE? Footprint Latency Timeouts Security Community: 500MB NGINX Plus: 120MB Community: Slowed by timeouts NGINX Plus: Dynamically reconfigures Community: 8809 NGINX Plus: 0 Community: OpenResty = CVE problems NGINX Plus: Proactive CVE patching, Integrated WAF & service mesh Driven by innovation at the expense of feature stability Driven by enterprise-ready stability without compromising innovation
  • 14.
    | ©2020 F5 14 2020PERFORMANCE TEST RESULTS Latency in a Dynamic Deployment
  • 15.
    | ©2020 F5 15 UNNECESSARYRISK IN YOUR K8S ENVIRONMENTS Delays in CVE patching Competitor
  • 16.
    | ©2021 F5 16 THECOST OF ADDING SECURITY IS LOW COMPARED WITH ITS IMPACT NGINX App Protect Performance 0 0,5 1 1,5 2 2,5 Throughput (MB/sec) No Protection NGINX App Protect ModSec 0 2000 4000 6000 8000 10000 12000 14000 Requests/sec No Protection NGINX App Protect ModSec 0 100 200 300 400 500 600 700 800 Latency (ms) No Protection NGINX App Protect ModSec
  • 17.
    | ©2021 F5 17 AppProtect+ KIC + NSM = Mesh Ingress WAF • AppProtect runs in NGINX+ KIC at the edge • Provides L7 WAF security for all mesh-based ingress • mTLS between KIC and NSM NGINX AppProtect
  • 18.
    | ©2021 F5 18 MutualTLS Instrumentation Tracing Rate Limiting Traffic Splitting Access Control Egress Control Secure traffic in a zero-trust network Monitor performance, latency and availability Debug transactions and locate faults Protect vulnerable apps, limit greedy clients Implement A|B, Canary and Blue-Green upgrades Implement allow-lists to accurately govern traffic Manage and broker traffic to external services Spotlight on Service Mesh - Use Cases
  • 19.