Kubernetes Basis: Pods, Deployments, and Services

Copyright © JianKai Wang 2020-2021
Kubernetes Basis
JianKai Wang 王建凱
gljankai@gmail.com
https://jiankaiwang.no-ip.biz/
https://github.com/jiankaiwang/k8s-basis
Pods, Deployments, and Services

Using Kubernetes
● From public cloud service providers
○ Google Kubernetes Engine (GKE) on Google Cloud Platform
○ Amazon Elastic Kubernetes Service (EKS) on AWS
○ Azure Kubernetes Service (AKS) on Microsoft Azure
● From bare metal servers
○ Raspberry Pi device arrays
○ From on-premise machines
● Using Kubernetes-IN-Docker tool (KIND)
● ...
2

Using Kubernetes by GKE on GCP
3
[Reference] Google Cloud API Document. JianKai Wang. 2021/04 fetched from
https://github.com/jiankaiwang/GCPNotes/blob/master/api_doc/gcloud_api.md .
[Reference] Google Cloud Platform Reference. JianKai Wang. 2021/04 fetched from https://github.com/jiankaiwang/GCPNotes.
Conﬁgure deployment environment on GCP Create a k8s cluster on GCP
# gcloud command hierarchy
gcloud [-h]
auth # get authorized info
list
config # get project configure
list [--all]
project
set # set the resource configure
compute/zone <zone_name>
get-value # get the value of the parameter to the config
project
# list the active account information
gcloud auth list
# show the information about the project
gcloud config list project
# set the default compute zone
gcloud config set compute/zone us-central1-a
# gcloud commands to create a k8s cluster
container
clusters
create <CLUSTER-NAME> # create a k8s cluster
[--num-nodes <num>] [--scopes <cloud-platform>]
get-credentials <CLUSTER-NAME>
delete <CLUSTER-NAME> # delete a k8s cluster
# e.g. creating a cluster
gcloud container clusters create <CLUSTER-NAME>
# e.g. get authentication credentials for the cluster
gcloud container clusters get-credentials <CLUSTER-NAME>
# e.g. after get authentication credentials
# you now can use kubectl
kubectl ...

Kubernetes-IN-Docker (KIND)
4
github.com/kubernetes-signs/kind
Create a k8s cluster using KIND.
# for linux & Mac
curl -Lo ./kind "https://kind.sigs.k8s.io/dl/v0.10.0/kind-$(uname)-amd64"
chmod +x ./kind
# change or create a soft link the path of executor
mv ./kind /some-dir-in-your-PATH/kind
ln -s $PWD/kind /usr/local/bin
# create a k8s cluster using kind
# kind-expose.yaml for type NodePort
kind create cluster --wait 5m --name local <--config kind-expose.yaml>
# check the k8s cluster
kubectl cluster-info
# delete the k8s cluster
kind delete cluster

Bare-metal examples from scratch on Raspberry Pi array
[Reference] Create a kubernetes clsuter on the bare metal machines. JianKai Wang. 2021/04 fetched from
https://gist.github.com/jiankaiwang/7120c3c57c508b61b2ae4e9dc2a100e1.
5
● Install the toolkits
● Setup the Node Network
○ Main node
○ Worker node
● Setup the Pod Network
○ using kube-ﬂannel as the example
○ calico
● Setup the Dashboard
○ required to set the authentication

Basic K8s commands
6
[Reference] Basic K8s commands. JianKai Wang. 2021/04 fetched from
https://github.com/jiankaiwang/ITHandbook/blob/master/kubernetes/reference.md.
Useful kubectl commands
kubectl
help <command-name>
version
cluster-info # get the cluster information
create # create the objects
apply # apply the changes
get # get the k8s objects
describe # show the metadata
run # run a pod
label # update the objects' labels
attach # attach the pod
logs # show the logs
expose # create a service using `expose`
scale # scale the deployment
port-forward # port-forwarding from local to the object
set # make changes on objects
rollout # roll out the object
exec # execute the commands in the pod
cp # copy from the container to the local or vice versa
explain # explain the object and its meta data
edit # edit the running objects
delete # delete the objects

Pod: the minimum unit for scaling in K8s
● Pod
○ running a collection of containers inside
● Why decoupling applications is required?
○ resource management
○ diﬀerent users: web vs. git
○ security issues
○ separating applications to separate containers
○ symbiotic applications/containers
○ containers in a pod run on the same machine
○ containers in a pod have the same execution environment
■ each container has it own cgroup in the pod
■ same IP address, port space, hostname, IPC
● What should I put in a Pod?
7
serving pod
web git
FS

Pod Manifest
● Pod Manifest
○ a text file in YAML (for reading) or JSON
○ represention of k8s API objects
○ declarative configuration
■ desired state fo the world
■ vs. imperative one (like apt-get)
○ metadata and spec sections
○ multiple containers in spec
● Send it to K8s API server.
● Store it in persistent storage (etcd).
● Scheduler schedules pods onto nodes with it.
○ relibility issues to put pods on different nodes
● Kubelet creates and monitors the pods.
● ReplicaSets are better for multiple pod instances.
8
a example running on docker
# run a container using the docker command
docker run -d --rm --name nginx -p 8080:80
nginx:latest
the Pod manifest example using YAML
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
name: http
protocol: TCP

Operating Pods
● Creating a pod via
○ kubectl run
○ kubectl apply -f <ya,l>
● Getting a pod’s information via
○ kubectl get pods
● Update a pod via
○ kubectl edit pod on the editor
○ kubectl apply a configuration file
● Show pod’s details using `kubectl describe`.
● Deleting a pod via
○ kubectl delete pods/<name>
○ kubectl delete -f <yaml>
○ Grace peroid (30 secs)
9
Basic commands to operate pods.
# creating a pod, using image from dockerhub by default
kubectl run nginx --image=nginx:latest
kubectl apply -f nginx.yaml
# getting the pod's information
kubectl get pods/nginx [-o wide|json|yaml]
# update the pod's by èdit` or àpply`
kubectl edit pods/nginx
kubectl apply -f nginx.yaml
# delete the pod by `delete`
kubectl delete pods/nginx
# get more information about the pod by `describe`
kubectl describe pods/nginx

Kubernetes Core Components
10
ETCD
Scheduler
Controller
Kube API Server Kube Proxy
Kubelet
Container
Runtime
Interface
Control Plane Node

Accessing Pods
● Port forwarding
○ Pod is required in running.
○ a tunnel from local to the pod
● Get logs
○ using `logs`
○ add `-f` to follow the log streaming
○ add `--previous` to get logs from
previous instances (if it exists)
● Running commands on container (not pod)
○ use `exec`
○ add `-it` to interact with
● Copying ﬁles to or from containers
○ using `cp` between containers and local
11
Basic commands to operate pods.
# port forwarding from local:8080 to pod:80
kubectl port-forward --address 0.0.0.0
pods/nginx 8080:80
# get log information from a pod
kubectl logs [-f] [--previous] pods/nginx
# run a single command
kubectl exec pods/nginx -- date
# interaction mode
kubectl exec -it pods/nginx -- /bin/bash
# like the general cp command
kubectl cp nginx:/etc/nginx/conf.d/default.conf
./default.conf

Health Checks
● Process health check
○ insufficient: e.g. deadlock
○ restart the pod if necessary
● liveness: functional check
○ application-specific logic, e.g. loading pages
○ defined in pod manifest
○ defined per container/application
● readiness: ready check
○ decided when to start receiving the requests
○ removed from load balancers
○ similar to liveness check
● types of health checks
○ tcpSocket: non-HTTP apps, like db
○ exec: scripts-/ programs-based, zero (success)
12
liveness checks defined in pod manifest
...
spec:
containers:
name: nginx
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 5
timeoutSeconds: 1
periodSeconds: 10
failureThreshold: 3
ports:
...
readiness checks defined in pod manifest
...
spec:
containers:
name: nginx
readinessProbe:
... (similar to the liveness)

Resource Management
● Maximally active (Cost): utilization metric
● utilization: resource used / resource purchased
● resource requests
○ specify the minimum amount of resources
● resource limits
○ specify the maximum amount of resources
● restriction on the containers
● the sum of all containers’ resources stands for the
pod’s ones
● hardware
○ CPU, RAM
○ CPU requests: cpu-shares functionality
○ RAM requests: termiate pods if out of RAM
13
resources deﬁned in pod manifest
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
name: nginx
livenessProbe:
...
resources:
requests:
cpu: "500m" # half a CPU free
memory: "128Mi" # 128 MB
limits:
cpu: "1000m" # 1.0 CPU free
memory: "256Mi" # 256 MB
ports:
...

Persisting Data with Volumes
● Stateless Data State
● defined in the Pod manifest
○ volumes in Pod
○ volumeMounts in container at different paths
● Using volumes
○ emptyDir
■ lifespan with node and pod
■ communication, synchronization, cache
○ persistent data
■ cloud services: gcpPersistentDisk, etc.
■ on-premise: nfs, iscsi, etc.
○ host filesystem
14
defined volumes in pod manifest
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
volumes:
- name: dataset
hostPath:
path: "/tmp/dataset"
- name: caching
emptyDir: {}
containers:
name: nginx
ports:
...
livenessProbe:
...
resources:
...
volumeMounts:
- mountPath: "/data"
name: dataset
- mountPath: "/cache"
name: caching

Labels and Annotations
● k8s scales applications in both size and complexity
● work in sets of things
○ e.g. manipulate multiple HTTP portals at once
● labels
○ key/value pairs
○ attached to k8s objects such as Pods, ReplicaSets
○ attaching identifying information to k8s objects
○ foundation for grouping objects
● annotations
○ key/value pairs
○ storage mechanism to hold non-identifying information
○ for tools and libraries
15

Labels
● providing identifying metadata for objecs
● for grouping, viewing, and operating
● k8s uses labels to handle sets of objects
instead of singletons, not objects’ names
● both key and value are strings
● recommended key format
16
the preﬁx the separator the name
DNS domain /
253 chs limit
dash(-), underscore(_), dot(.)
63 chs /
alphanumeric
kubernetes.io / appversion
A quick example
# applying labels, replicas will be removed
kubectl run nginx
--image=nginx:latest
--replicas=2
--labels="ver=1,app=web,env=prod"
# show pods with labeling information
kubectl get pods -o wide --show-labels

env=prod
app=apple
Manipulate objects with labels
17
Pods with labels
kubectl run apple-prod
--labels="ver=1,app=apple,env=prod"
kubectl run apple-test
--labels="ver=2,app=apple,env=test"
kubectl run banana-prod
--labels="ver=2,app=banana,env=prod"
kubectl run banana-staging
--labels="ver=2,app=banana,env=staging"
# show all pods with labels
kubectl get pods --show-labels
kubectl get pods -L app -L env -L ver
alpha-prod
alpha-test
beta-prod
beta-staging
app=banana
app=2

Labels and Selectors
● modify or update labels to objects using
`kubectl label <object_types>`
○ objects like pods, deployments
● selectors
○ filter objects based on a set of labels
○ use simple boolean language like the
“key=value” format
○ use `,` for the AND operation
○ use ìn` for asking if the set of values
exists at the label
○ show objects with a specific “label” only
18
A quick example
# modify the eisting label
kubectl label pods banana-staging
--overwrite "ver=2"
# add an new label
kubectl label pods banana-staging
"newkey=newlabel"
# show all pods with labels
kubectl get pods --show-labels
kubectl get pods -L app -L env -L ver
# the way of using selector
kubectl get pods --selector="ver=2"
kubectl get pods --selector="ver=2,env=prod"
kubectl get pods --selector="app in (banana, candy)"
kubectl get pods --selector="newkey"
kubectl get pods --selector="ver=2,!newkey"

Label selectors in API objects
● supported operators
● deﬁned in manifest
19
selector in the command line selector in the API objects or manifest
--selector="app=apple,ver in (1, 2)" selector:
matchLabels:
app: apple
matchExpressions:
- {key: ver, operator: In, values: [1, 2]}
the operator description
key=value
key!=value
key in (value1, value2)
key notin (value1, value2)
key
!key
whose key is the value
whose key is not the value
whose key’s value in the set
whose key’s value not in the set
whose objects own the key
whose objects didn’t own the key

Annotations
● k8s is a purposefully decoupled system
○ no hierarchy and all objects operate independently
● provide a place to store additional metadata for k8s objects
○ store some opaque data
○ assisting tools or libraries
■ where an object came from
■ how to use it
■ policy around that object
● examples
○ specialized scheduling policy
○ CI/CD: git hash, timestamp, PR number, etc.
○ enable deployment object to keep track of ReplicaSets for rollouts
○ prototype alpha functionionality in k8s
20

Defining Annotations
● defined in the object scope
● same format as the label keys
○ the “namespace” part of the key is more important for tools
○ the value is allowed of any format
■ e.g. JSON format in a string
● defined in common metadata section
● both labels and annotations provide the flexibility to k8s
21
example in manifest
...
metadata:
annotations:
demo.com/icon-url: "https://demo.com/icon.png"
...

Service discovery
● Kubernetes is a dynamic system.
○ horizontal Pod autoscaling (ReplicaSet)
● Service discovery ﬁnds which processes are listening at which addresses for which services.
○ resolves the information quickly and reliably
○ is a low latency system
○ stores a richer deﬁnition of what that service is
● Domain Name System (DNS) is a traditional service discovery system.
○ but it falls short in dynamic attributes of k8s
22

Service Object
● Service discovery in k8s starts with a Service object.
○ step.1-a ﬁrst create a pod via `kubectl run`
○ step.1-b use a deployment to create/monitor the pods
○ step.2 ﬁnal use `kubectl expose` to create a service
■ [not recommended] you can expose a pod as well
23
[not recommended] Expose a pod. [recommended] Expose a deployment.
# you can create a pod using `run`
kubectl run nginx
--image=nginx:latest --port 80
# [not recommended] expose the pod
kubectl expose pods/nginx
--type=NodePort --port 8080
--target-port 80
# port-forward due to the inner IP
kubectl port-forward
svc/nginx-deployment 8081:8080
# create a deployment
kubectl apply
-f ./nginx-deployment.yaml
# expose a service
kubectl expose deployment/nginx-deployment
--type=NodePort --port 8080
--target-port 80
# port-forward due to the inner IP
kubectl port-forward
svc/nginx-deployment 8081:8080
nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx-app
replicas: 3
template:
metadata:
labels:
app: nginx-app
spec:
containers:
name: nginx
ports:
- containerPort: 80
name: http
protocol: TCP

Node
Control-Plane
Service Types
24
types descriptions
ClusterIP
The service will be assigned a virtual
IP address and system balances the
loading using the selector. This mode
is the default value.
NodePort
The service will be allowed to access
by a node’s port. You have to make
sure the cluster IP of the service
located in the same domain for
enabling access.
LoadBalancer
It’s provided by the cloud service
provider. Generally the cloud service
provider would also assign a external
IP address to the service and create a
load balancer in front of it.
NodePort:8080
ServicePort:<auto>
DeploymentPort:<80>
PodPort:<80>
Port-forward:8081
k8s cluster (10.x.x.x) network interface
HTTP requests
X

A NodePort Example
● provided with a exposure of all of
the nodes’ ports to the service
25
Service and Deployment Example
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
# target the pod, not deployment
app: nginx-app
ports:
- protocol: TCP
port: 8080
targetPort: 80
nodePort: 30000
apiVersion: apps/v1
kind: Deployment
metadata:
spec:
selector:
matchLabels:
app: nginx-app
replicas: 3
template:
metadata:
labels:
app: nginx-app
spec:
containers:
name: nginx
ports:
- containerPort: 80
name: http
protocol: TCP
Flow
# create a deployment
kubectl apply -f ./deployments/nginx-deployment.yaml
# create a service
kubectl apply -f ./services/nginx-service.yaml
# now you can access the service
# via each node IP with the same port
http://<node-1-IP>:30000/
http://<node-2-IP>:30000/
...

Service DNS
● Cluster IP is virtual and also stable.
○ appropriate to give it a DNS address
○ DNS caching is not required.
● Namespace
○ a space for highly corelated services
○ using service name to connect one of
Pods identiﬁed by a service
● DNS service takes responsible for service
discovery in k8s.
○ no more IP-based logic
○ instead of using domain name service
● DNS service is a built-in service.
○ managed by k8s itself
○ k8s auto assigns to Pods and Services
26
DNS for pods (kubelet setup)
# kubelet set for each pod at /etc/resolv.conf
kubectl exec -it pods/nginx-deployment-568fc89568-4s926
-- cat /etc/resolv.conf
# return one of Pods on the service
kubectl exec -it svc/nginx-deployment
-- cat /etc/resolv.conf
an example
# within a pod
apt-get update && apt-get install -y dnsutils
# start a DNS lookup tool
nslookup
# lookup a service's DNS name (auto generated)
# nginx-deployment: service name
# default: which namespace the service locates
# svc: is a service object
# cluster.local: the unix-like default name
> server nginx-deployment.default.svc.cluster.local
Default server: nginx-deployment.default.svc.cluster.local
Address: 10.96.54.72#53

Readiness Check
● initialization may cost a sec to several mins
● check whether pods are ready or not
● readiness check targets the pod scope
● declaration is similar to the liveness check
● only ready pods are sent traﬃc
27
readiness probe
apiVersion: apps/v1
kind: Deployment
metadata:
spec:
selector:
...
replicas: 3
template:
metadata:
...
spec:
containers:
name: nginx
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
httpGet:
path: /
port: 80
periodSeconds: 2
initialDelaySeconds: 0
failureThreshold: 3
successThreshold: 1

Cloud Integration
● provided by the cloud service provider
○ GKE on GCP
○ AKS on Azure
○ EKS on AWS
● type deﬁnition
○ LoadBalancer
● mechanism
○ the same with NodePort
○ cloud services created a new load balancer
○ balanced the traﬃc to the nodes
● Types
○ GKE: a EXTERNEL-IP (public) would be exposed
○ EKS: DNS-based loadbalancer,
a hostname exposed
28
LoadBalancer Example in YAML
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: LoadBalancer
selector:
# target the pod, not deployment
app: nginx-app
ports:
- protocol: TCP
port: 8080
targetPort: 80
nodePort: 30000
in CommandLine
kubectl expose deployment/nginx-deployment
--type=LoadBalancer --port 8080
--target-port 80
using edit
# service from cli
kubectl edit svc/nginx-deployment
# service from yaml
kubectl edit svc/nginx-service

Endpoints
● use services without using cluster IP
● k8s creates a buddy Endpoint for each
service object
○ contains an IP address
○ to each pod exposed as the
service
● scenario
○ ask the k8s API for looking up
endpoints of services and calling
them
● k8s uses selectors to select the pods
for the service
29
get Endpoints
# get the services’ endpoints information
kubectl get endpoints
# delete the deployment and recreate it again to watch the endpoints
# from terminal.1
kubectl get ednpoints nginx-service --watch
NAME ENDPOINTS AGE
nginx-service 10.244.2.12:80,10.244.2.13:80,10.244.2.14:80 42h
nginx-service 10.244.2.12:80,10.244.2.13:80 42h
nginx-service 10.244.2.13:80 42h
nginx-service <none> 42h
nginx-service 10.244.2.15:80 42h
nginx-service 10.244.2.15:80,10.244.2.16:80 42h
nginx-service 10.244.2.15:80,10.244.2.16:80,10.244.2.17:80 42
# from terminal.2
kubectl delete deployment/nginx-deployment
kubectl apply -f ./deployments/nginx-deployment.yaml
# select the pods for the service by selectors
kubectl get pods -o wide --selector=app=nginx-app

Node
Kube-Proxy and Cluster IPs
● ClusterIPs are virtual IPs
○ is assigned by the API server
○ load-balance traﬃc across all endpoints by kube-proxy on every node
○ most users use DNS services to ﬁnd cluster IPs
● Kube-proxy
○ watches for new services via API server
○ programs a set of iptables to rewrite the destinations of packets
● K8s service address range
○ using --service-cluster-ip-range
○ on kube-apiserver
● in-cluster and out-of-cluster connection
○ open source projects: Hashicorp’s Consul
30
apiserver
kube-proxy
clients
Cluster IP
(iptables)
Backend 1
Backend 2
Backend N

Conclusion
● Pods are the atomic units where running multiple containers for
scaling, and are the destinations of service requests.
● Deployments keep the track of the Pods to run upon the
desired or requested states.
● Services expose the Pods to the internal or external network
environment for providing services.
31
JianKai Wang 王建凱
gljankai@gmail.com
https://jiankaiwang.no-ip.biz/
https://www.linkedin.com/in/wangjiankai/

Kubernetes Basis: Pods, Deployments, and Services

More Related Content

What's hot

Similar to Kubernetes Basis: Pods, Deployments, and Services

More from Jian-Kai Wang

Recently uploaded

Kubernetes Basis: Pods, Deployments, and Services