Ports, pods and proxies

© 2021 Kasten, Inc.
Proxies, Pods and Ports
Aaron Alpar
Member of Technical Staff
Kasten by Veeam

1. Introduction
2. Setup - the .yaml files
3. Kubernetes API Proxies
4. Using kubectl's --raw opton
5. Port forwarding with kubectl port-forward-ing
6. kubectl logs and kubectl attach
Agenda

Pods have various access points exposed via kubectl
● As proxy through the API server
● Through port-forwarding
● Through stdin, stdout and stderr

The Deployment
$ cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: demo-1
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80

The Service
$ cat service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: demo-1
labels:
run: nginx
spec:
selector:
app: nginx
ports:
- name: http
protocol: TCP
port: 80

API Server Proxy
● Direct access to API server
● Forwards HTTP connections from local port to API server
● Forwarded connections will be amended with authentication state
● Similar to using --raw

API Server Proxy
$ kubectl proxy
Starting to serve on 127.0.0.1:8001
$ curl http://localhost:8001/
{
"paths": [
"/.well-known/openid-configuration",
"/api",
"/api/v1",
"/apis",
"/apis/",
"/apis/admissionregistration.k8s.io",
"/apis/admissionregistration.k8s.io/v1",
...

API Server Proxy
$ kubectl get namespaces
NAME STATUS AGE
default Active 22h
demo-1 Active 3m46s
kube-node-lease Active 22h
$ curl http://localhost:8001/api/v1/namespaces
{
"kind": "NamespaceList",
"apiVersion": "v1",
"metadata": {
"resourceVersion": "2812175"
},
"items": [
{
"metadata": {
"name": "default",
...

API Server Proxy
$ kubectl get pods -n demo-1 nginx-deployment-66b6c48dd5-wst5h
NAME READY STATUS RESTARTS AGE
nginx-deployment-66b6c48dd5-wst5h 1/1 Running 0 32m
$ curl
http://localhost:8001/api/v1/namespaces/demo-1/pods/nginx-deployment-66b6c48dd5-wst5h
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "nginx-deployment-66b6c48dd5-wst5h",
...

API Server Proxy
Access to HTTP and HTTPS services in
● Services
● Pods

API Server Proxy: Service Ports
$ curl http://localhost:8001/api/v1/namespaces/demo-1/services/nginx-service:http/proxy/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
...

$ curl http://localhost:8001/api/v1/namespaces/demo-1/services/nginx-service:http/proxy/
...
$ cat service.yaml
...
selector:
app: nginx
ports:
- name: http
protocol: TCP
port: 80
...

Proxy URL form for a Service port
/api/v1/namespaces/namespace_name/services/[https:]service_name[:port_name]/proxy

API Server Proxy: Pod Ports
$ curl
http://localhost:8001/api/v1/namespaces/demo-1/pods/nginx-deployment-66b6c48dd5-t8446:80/proxy/
<!DOCTYPE html>
<html>
<head>
<style>
body {
width: 35em;
margin: 0 auto;
}
...

$ curl
http://localhost:8001/api/v1/namespaces/demo-1/pods/nginx-deployment-66b6c48dd5-t8446:80/p
roxy/
<!DOCTYPE html>
<html>
<head>
<style>
body {
width: 35em;
margin: 0 auto;
}
...

$ curl http://localhost:8001/api/v1/namespaces/demo-1/pods/nginx-deployment-66b6c48dd5-t8446:80/proxy/
...
$ kubectl get pod -n demo-1 nginx-deployment-66b6c48dd5-t8446 -o yaml
...
containers:
- image: nginx:1.14.2
name: nginx
ports:
- containerPort: 80
protocol: TCP
...

Proxy URL form for a Pod port
/api/v1/namespaces/namespace_name/pods/[https:]pod_name[:port_number]/proxy

API Server Proxy
● kubectl proxy provides access to API server HTTPS endpoint
● Can be used to access service and pod endpoints
● Useful for debugging

kubectl -v9
$ kubectl -v9 get pods -A
I1213 08:00:16.355063 98152 loader.go:372] Config loaded from file:
/Users/aalpar/.kube/config
I1213 08:00:16.360303 98152 round_trippers.go:435] curl -k -v -XGET -H "Accept:
application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8
s.io,application/json" -H "User-Agent: kubectl/v1.21.5 (darwin/amd64) kubernetes/aea7bba"
'https://127.0.0.1:64926/api/v1/pods?limit=500'
I1213 08:00:16.371606 98152 round_trippers.go:454] GET
https://127.0.0.1:64926/api/v1/pods?limit=500 200 OK in 11 milliseconds
...

kubectl -v9
● Non truncated kubectl debugging output
● Useful for getting curl versions of kubectl commands
● curl commands can be used in conjunction with kubectl proxy
● Values less than 9 provide less output

Raw Option
● kubectl --raw used to submit URL to the API server
● Can be used with get, create, delete

Raw Option
$ kubectl get --raw /api/v1/namespaces | jq
{
"kind": "NamespaceList",
"apiVersion": "v1",
"metadata": {
"resourceVersion": "2812175"
},
"items": [
{
"metadata": {
"name": "default",
...

Raw Option
$ kubectl get --raw /api/v1/namespaces/demo-1/services/nginx-service | jq
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "nginx-service",
"namespace": "demo-1",
"uid": "8fa61327-0158-4d1c-a48a-2364150545d8",
"resourceVersion": "9485",
"creationTimestamp": "2021-12-12T17:42:25Z",
"labels": {
...

The Service as JSON
$ cat service.json
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"labels": {
"run": "nginx"
}
},
"spec": {
"selector": {
"app": "nginx"
},
"ports": [
{
"name": "http",
"protocol": "TCP",
"port": 80
}
]
}
}

Raw Option
$ kubectl delete --raw /api/v1/namespaces/demo-1/services/nginx-service
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"nginx-servic
e","kind":"services","uid":"8fa61327-0158-4d1c-a48a-2364150545d8"}}
$ kubectl create --raw /api/v1/namespaces/demo-1/services -f service.json | jq
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"uid": "5f3677d9-0f59-4197-91ec-f4f989f9a8a6",
"creationTimestamp": "2021-12-12T18:37:58Z",
"labels": {
...

Raw Option
$ kubectl get --raw /api/v1/services?limit=2 | jq
{
"kind": "ServiceList",
"apiVersion": "v1",
"metadata": {
"continue":
"eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6NjM4MzgsInN0YXJ0IjoiZGVmYXVsdC9wb3N0Z3Jlc3FsXHUwMDAwIn0",
"remainingItemCount": 3
},
"items": [
{
...

Raw Option
● Access to GET, POST and DELETE method for API service calls
● Can add URL query arguments

Port Forwarding
● Forwards TCP ports to Pods and Services
● Uses HTTP/2 streams
● Useful for using local web-browser to view cluster services
● Will pass-through TCP protocols other than HTTP and HTTPS

Port Forwarding
$ kubectl port-forward -n demo-1 pod/nginx-deployment-66b6c48dd5-wst5h :80
Forwarding from 127.0.0.1:54688 -> 80
Forwarding from [::1]:54688 -> 80
$ curl http://localhost:54688/
Handling connection for 54688
<!DOCTYPE html>
<html>
<head>
<style>

Logs
● Forwards stdout and stderr from Pods to logging subsystem
● Logs can be retrieved for the previous container instance
● stderr and stdout merged
● Logs are rotated automatically
● kubectl logs only accesses last log rotation

Logs
$ kubectl logs -n demo-1 deployment/nginx-deployment --all-containers
127.0.0.1 - - [30/Aug/2021:20:07:00 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0
(Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1
Safari/605.1.15" "-"
...
$ kubectl logs --prefix -n demo-1 -lapp=nginx --all-containers
[pod/nginx-deployment-66b6c48dd5-wst5h/nginx] 127.0.0.1 - - [30/Aug/2021:20:07:00 +0000]
"GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15" "-"
...

Attach
● Forwards stdin, stdout and stderr
● Can allocate tty - effects signal handling
● Distinguish between stdout and stderr
● Attaches stdout, stderr and stdin of PID 1

Attach
$ kubectl attach -n demo-1 nginx-deployment-66b6c48dd5-wst5h
If you don't see a command prompt, try pressing enter

Attach
$ kubectl run -it --rm busybox --image=busybox
If you don't see a command prompt, try pressing enter.
/ # tty
/dev/pts/0
/ # ls -l /proc/1/fd
total 0
lrwx------ 1 root root 64 Dec 13 21:26 0 -> /dev/pts/0
lrwx------ 1 root root 64 Dec 13 21:26 10 -> /dev/tty
/ #
Session ended, resume using 'kubectl attach busybox -c busybox -i -t' command when the pod
is running
pod "busybox" deleted

Attach
$ kubectl run -i --rm busybox --image=busybox
tty
not a tty
ls -l /proc/1/fd
total 0
lr-x------ 1 root root 64 Dec 13 21:27 0 -> pipe:[1343480]
l-wx------ 1 root root 64 Dec 13 21:27 1 -> pipe:[1343481]
l-wx------ 1 root root 64 Dec 13 21:27 2 -> pipe:[1343482]
pod "busybox" deleted

Attach
$ kubectl run -it busybox --image=busybox --attach=false
pod/busybox created
$ kubectl attach -it busybox
/ # pwd
/
/ #

Attach
$ kubectl run -it busybox --image=busybox --attach=false
pod/busybox created
$ kubectl attach -i busybox 1>stdout.out 2>stderr.out
echo "stdout" >&1
echo "stderr" >&2
^D
$ cat stdout.out
/ # echo "stdout" >&1
stdout
/ # echo "stderr" >&2
stderr
/ #
$ cat stderr.out

Attach
$ kubectl run -i busybox --image=busybox --attach=false
pod/busybox created
$ kubectl attach -i busybox 1>stdout.out 2>stderr.out
echo "stdout" >&1
echo "stderr" >&2
^D
$ cat stdout.out
stdout
$ cat stderr.out
stderr

● kubectl attach routes stdout, stderr and stdin to client
● stdout and stderr will be "flattened" if a tty is allocated
Attach

● kubectl proxy forwards HTTP traffic to API server
● kubectl -v9 can be used to get curl commands for use with kubectl
proxy
● kubectl with --raw option will meet most needs where HTTP GET,
DELETE or POST is needed
● kubectl attach copies stdout, stderr and stdin to client
● kubectl attach -t folds stderr into stdout
● Port-forwarding forwards TCP traffic for a port via HTTP/2 to a pod or service

References
https://kubernetes.io/docs/concepts/cluster-administration/proxies/
https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/

THANK YOU
Aaron Alpar
Member of Technical Staff
Kasten by Veeam

Ports, pods and proxies

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Ports, pods and proxies

Similar to Ports, pods and proxies (20)

More from LibbySchulze

More from LibbySchulze (20)

Recently uploaded

Recently uploaded (11)

Ports, pods and proxies