Gateway APIs, Envoy Gateway, and API Gateways

Gateway APIs and API Gateways @mt165
Modern Ingress Demystiﬁed
Gateway APIs and API
Gateways
Matt Turner
KubeHuddle, Edinburgh UK | October 2022 @mt165 | mt165.co.uk

THE ENTERPRISE SERVICE MESH COMPANY

Outline
● Recap: Ingress
● Gateway API
● Envoy Gateway
● [Envoy] [API] Gateway
● Where Next?

Ingress Networking

Service
A
Backend
*.example.com
Service
A
Service
A

Service
A
Backend
Load
Balancer
*.example.com
Service
A
Service
A

Service
A
Backend
Load
Balancer
Node
port
*.example.com
Service
A
Service
A

Backend
Load
Balancer
Node
port
Cluster
IP
*.example.com
Service
A
Service
A
Service
A

Backend
Proxy
Proxy
Proxy
Proxy
Ingress
Load
Balancer
Node
port
Cluster
IP
*.example.com
Service
A
Service
A
Service
A

Service
A
Backend
Proxy
Proxy
Proxy
Proxy
Ingress
Load
Balancer
Node
port
Cluster
IP
*.example.com
Cluster
IP
Service
A
Service
A

Service
A
Backend
Proxy
Proxy
Proxy
Proxy
Ingress
Load
Balancer
Node
port
*.example.com
Service
A
Service
A
Ingress
Controller
Service
A

Service
A
Backend
Proxy
Proxy
Proxy
Proxy
Ingress
Load
Balancer
Node
port
*.example.com
Service
A
Service
A
Ingress
Controller
Service
A
K8s API
Server
etcd

Service
A
Backend
Proxy
Proxy
Proxy
Proxy
Ingress
Load
Balancer
Node
port
*.example.com
Service
A
Service
A
Ingress
Controller
Service
A
K8s API
Server
etcd
Ingress

😅

The Ingress API

Ingress API
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
spec:
rules:
- host: example.com
http:
paths:
- path: /test
backend:
service:
name: test
port:
number: 80

Ingress API
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: example.com
http:
paths:
- path: /test
backend:
service:
name: test
port:
number: 80

Ingress API
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: example.com
http:
paths:
- path: /test
backend:
service:
name: test
port:
number: 80

Ingress API
kind: Ingress
metadata:
annotations:
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /test
pathType: Prefix
backend:
service:
name: test
port:
number: 80

Ingress API
kind: IngressClass
metadata:
name: nginx
spec:
controller: example.com/nginx-ingress-controller

Ingress API
kind: IngressClass
metadata:
name: nginx
spec:
controller: example.com/nginx-ingress-controller
---
kind: Deployment
spec:
template:
spec:
containers:
- name: nginx
args:
- /nginx-ingress-controller
- '--ingress-class=k8s.io/nginx'
- '--controller-class=example.com/nginx-ingress-controller'

Ingress API
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
…

Ingress API: Implementations
● Nginx
● Haproxy
● Apache
● Traefik
● Contour
● Ambassador
● Kong
● Tyk
● Avi
● Istio
● etc

😅😅

Gateway API

xkcd, Creative Commons Attribution 2.5

Recap: The Storage API
StorageClass
PersistentVolume
Pod
PersistentVolume
Claim
Pod
Pod
(AWS, gp2)
(10Gi, ReadOnce)
PersistentVolume
Claim
Infra provider / cluster builder:
Infra admin:
App dev:
PersistentVolume

The Gateway API

The Gateway API
● Not built-in yet; packaged as CRDs
● >1 resource
● gateway.networking.k8s.io
○ GatewayClass/v1beta1
○ Gateway/v1beta1
○ HTTPRoute/v1beta1
○ TLSRoute/v1alpha1 - SNI routing
○ GRPCRoute/v1alpha1
○ TCPRoute/v1alpha1
○ UDPRoute/v1alpha1

GatewayClass
apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
name: my-class
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller

kind: Gateway
metadata:
name: my-gateway
spec:
gatewayClassName: my-class
listeners:
- name: http
protocol: HTTP
port: 80
- name: https
protocol: HTTPS
port: 443
Gateway

HTTPRoute
kind: HTTPRoute
metadata:
name: http-log
spec:
parentRefs: [{name: my-gateway}]
hostnames: ["www.example.com"]
rules:
- matches:
- path: {value: /http-log, type: PathPrefix}
backendRefs:
- {group: "", kind: Service, name: http-log, port: 80, weight: 1}

What else does it look like?
● Heavily based on the Istio API
● In turn, Istio implements the Gateway API
○ Currently beta
○ Will be default when gw-api hits v1
● Also implemented by the SMI Meshes (Linkerd2, Consul, Open Service Mesh,
etc)

A Mesh API?
● Gain resources to describe East-West (service mesh)
● GAMMA group trying to get meshes to adopt the GW API, and conversely to
get GW API to model mesh concerns
(https://gateway-api.sigs.k8s.io/contributing/gamma/)

Reference Implementation: Envoy Gateway

What’s a standard?
Nginx-ingress currently the de facto standard
● Surely the most common, certainly when you discount cloud providers’
ingress
● Only one mentioned in the main upstream docs

An Envoy-Based Gateway
● But nginx isn’t very modern
○ Reads its config from a file, not an API
○ The operator hides this, and that’s fine; that’s its job
○ But those reload events cause the drop of in-flight requests, which isn’t ok
○ Plus other operational issues
○ Hard to extend
● Envoy is more modern, and designed for this kinda stuff
○ xDS API
○ It’s proven itself as Ingress, Sidecar, even GFE
● A new gateway in town!

Another One?
● Contour, Emissary (formerly Ambassador) agreed to rebase onto the EG
code, but will keep their brands, add value

Demo!

🍺
An offering to the demo gods...

👀

An Adventure in Metrics
● Pod: <EG Operator>
○ Container <main>: no metrics port
○ Container kube-rbac-proxy: https metrics port, just controller_runtime’s default stats
● Pod: <Envoy Instance>
○ Container Envoy: prom-format metrics on admin at localhost:19000 (unreachable)

Other Features
● cert-manager has experimental support (hard to demo locally)

A Work-in-Progress
● v0.2 targeting KubeCon
○ Full compliance to the Gateway API
○ Doesn’t seem to mean other basics, like metrics
● To follow the project
○ https://github.com/envoyproxy/gateway
○ Envoy Slack #gateway-dev

Gateway API API Gateway

What Even is an API Gateway?
You might think
● TLS termination
● Load Balancing
● L7 Routing
● WAF
● Rate-limiting and quotas
● Bot-blocking
● OIDC auth
● Caching
● Body validation and transformation
● Version and staging support
● etc

What Even is an API Gateway?
“Basic” features
● TLS termination
● Load Balancing
● L7 Routing
“API Gateway” features
● WAF
● Rate-limiting and quotas
● Bot-blocking
● OIDC auth
● Caching
● Body validation and transformation
● Version and staging support
● etc

Envoy (API) Gateway
● Extensible code
● Extensible API
● New, clean codebase

Gateway API models API Gateways
● Gain resources to describe API Gateway features
○ Auth one in progress
● On-going discussion about making the API extensible to model the different
features in all the implementations, but in a consistent, first-class way
● “GEP” - Gateway Enhancement Proposal
(https://gateway-api.sigs.k8s.io/contributing/gep/)
● Graduation path
○ Vendor extension
○ GW-API extension
○ GW-API core

A Work-in-the-Future
● Needs the API
● Needs the Extensions
● None being worked on yet (that I know of)
● Except Coraza: a Golang implementation of mod_security

Recap

Where Next?
● Release of Envoy Gateway 0.2 for KubeCon Detroit
● Emissary, Contour rebasing eventually
● Get Gateway API into upstream k8s
● Extend the GW-API to model API-GW concerns
○ Solve problems like modelling deploy of redis for global ratelimits
● Build API-GW feature plugins
● Gateway API v1?
● Envoy Gateway 1.0?

Recap
● Ingress API sucks
● Gateway API doesn’t
○ Ingress
○ East-West
○ API Gateway
● Envoy Gateway exists. It hasn’t got far but you can try it at home.
● Envoy Gateway will become an API Gateway
○ That needs lots of work

Thanks!
Slides
Videos
Demo code
mt165.co.uk
Questions @mt165

Gateway APIs, Envoy Gateway, and API Gateways

More Related Content

Similar to Gateway APIs, Envoy Gateway, and API Gateways

More from Matt Turner

Recently uploaded

Gateway APIs, Envoy Gateway, and API Gateways