This talk provides a 101 introdution to Kubernetes from a user point of view.
Aimed at service providers, it was presented at the GPN Annual Meeting 2019. https://conferences.k-state.edu/gpn/
1. An overview of the
Kubernetes architecture
Presented by Igor Sfiligoi, UCSD
Workshop at the Great Plains Network Annual Meeting 2019
GPN Annual Meeting 2019 - Kubernetes Architecture 1
2. Outline
• Kubernetes history
• Basic building blocks
• Provided bells and whistles
• Scheduling
• User interface
GPN Annual Meeting 2019 - Kubernetes Architecture 2
3. Kubernetes
• Now maintained by
Cloud Native Computing Foundation
https://kubernetes.io
Originally created by Google
• With very large and active
development community
Open source
• But also available out-of-the-box on
all major Clouds (GCP, AWS and Azure)
Can be deployed on-prem
GPN Annual Meeting 2019 - Kubernetes Architecture 3
4. Container based
• Typically Docker based
Containers are the
basic building block
• Creating custom ones almost trivial
Standard images for
many applications exist
• If state needed, must be held outside
Just remember
containers are stateless
GPN Annual Meeting 2019 - Kubernetes Architecture 4
5. Container Orchestration
• Once you have many containers on many nodes, you need something to manage the whole
• This is usually referred to as Orchestration
Attribution: https://kubernetes.io
GPN Annual Meeting 2019 - Kubernetes Architecture 5
6. Packing containers into pods
The smallest concept is actually the Pod
A Pod is a set of containers
• Having a single Container in a Pod OK
Containers within a Pod are
guaranteed to run alongside
• And can share (ephemeral) state
Pod
Container
Container
https://kubernetes.io/docs/concepts/workloads/pods/pod/
GPN Annual Meeting 2019 - Kubernetes Architecture 6
7. Packing Pods into Deployments
• If it terminates for whatever reason, it is gone
A Pod is ephemeral
• Initially launches a single Pod (no obvious benefit)
• If a Pod is removed, a new Pod is automatically re-submitted
A Deployment is persistent
• E.g. for load balancing and horizontal scaling
A Deployment can also manage multiple replicas
Great
for service
applications
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
GPN Annual Meeting 2019 - Kubernetes Architecture 7
8. Configuration
management
• Kubernetes provides an easy mechanism to inject
information into the Container images at runtime
Most applications need to be configured
Three types of information
Environment variables Whole files Secrets
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
https://kubernetes.io/docs/concepts/configuration/secret/
GPN Annual Meeting 2019 - Kubernetes Architecture 8
9. Linking to external storage
• Most applications will need it!
External storage essential for persistency
• Local storage
• Distributed storage, e.g. CEPH, NFS, etc.
• Custom filesystems via CSI – e.g. CVMFS
Kubernetes provides the necessary hooks at Pod launch time
https://kubernetes.io/docs/concepts/storage/volumes/
https://kubernetes-csi.github.io/docs/
GPN Annual Meeting 2019 - Kubernetes Architecture 9
10. Networking
Each container get its own private IP address
A Deployment can be registered as a Service
• Gets its own IP address and DNS entry
• Traffic routes to the Pods in Deployment based on selected policy (e.g. RR)
Service can also serve as a NAT
• Routing traffic from WAN using the Kubernetes public IPs
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
GPN Annual Meeting 2019 - Kubernetes Architecture 10
11. Networking
Each container get its own private IP address
A Deployment can be registered as a Service
• Gets its own IP address and DNS entry
• Traffic routes to the Pods in Deployment based on selected policy (e.g. RR)
Service can also serve as a NAT
• Routing traffic from WAN using the Kubernetes public IPs
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
• Unprivileged Pods better for regular users to minimize risk
Privileged Pods can get access to the host/public IP
• E.g. due to the use of X.509
Useful for Network Servers tied to a specific node
GPN Annual Meeting 2019 - Kubernetes Architecture 11
12. Pod scheduling
Kubernetes comes with a pretty decent scheduler
Will match Pods to available resources (CPU, Memory, GPU, etc.)
• Nodes advertise what is available
• Pods specify what they require, may also limit itself to a subset of Nodes
• A Pod will start on a Node only if a match can be made
There is also a notion of Priorities
• If a match for a higher priority Pod cannot be made,
the scheduler will kill one or more lower priority Pods to make space for it (if at all possible)
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
GPN Annual Meeting 2019 - Kubernetes Architecture 12
13. The DaemonSet
• E.g. a Monitoring probe
Sometimes an application must run on all the nodes
• Like a Deployment, but with fixed all-nodes scheduling
The DaemonSet automates this
https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
GPN Annual Meeting 2019 - Kubernetes Architecture 13
14. Users and Permissions
Kubernetes does not really have a concept of a “User”
Permissions are set as part of the Namespace concept
• Anyone having access to a Namespace can operate on the objects inside that Namespace
• Including creating, monitoring and modifying them
Namespace conceptually provides virtual-private Kubernetes clusters
• But very little additional restrictions within
• And relatively hard coordinating Pods in separate Namespaces
https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
GPN Annual Meeting 2019 - Kubernetes Architecture 14
15. Users and Permissions
Kubernetes does not really have a concept of a “User”
Permissions are set as part of the Namespace concept
• Anyone having access to a Namespace can operate on the objects inside that Namespace
• Including creating, monitoring and modifying them
Namespace conceptually provides virtual-private Kubernetes clusters
• But very little additional restrictions within
• And relatively hard coordinating Pods in separate Namespaces
PRP Nautilus provides
user management as a
side concept.
https://nautilus.optiputer.net
GPN Annual Meeting 2019 - Kubernetes Architecture 15
17. YAML
Everywhere
• Both for creating/configuring
Pods/Deployments/Services
• And for querying their (detailed) status
Most interactions with Kubernetes
will involve YAML documents
• Describes itself as
“a human friendly markup language”
• Uses Python-indentation
to indicate nesting
YAML is actually quite easy to use
https://en.wikipedia.org/wiki/YAML
GPN Annual Meeting 2019 - Kubernetes Architecture 17
20. Installing kubectl
• Just a static binary
• Available for all major platforms
(Linux, MacOS, Windows)
• Detailed download instructions at
https://kubernetes.io/docs/tasks/tools/install-kubectl/
• Can be used over WAN
• Just put the config file in
~/.kube/config
Get yours from
PRP’s Nautilus
GPN Annual Meeting 2019 - Kubernetes Architecture 20
22. Acknowledgents
This work was partially funded by
US National Science Foundation (NSF) awards
CNS-1456638, CNS-1730158,
ACI-1540112, ACI-1541349,
OAC-1826967, OAC 1450871,
OAC-1659169 and OAC-1841530.
GPN Annual Meeting 2019 - Kubernetes Architecture 22