Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
kubernetes 1001
hung-wei chiu
Microsoft MVP
Devops @ Thundertoken
Co-organizer of SDNDS-TW
Co-organizer of CNTUUG
Network/Kubernetes/SDN
h...
Why Kubernetes 1001 ?
All-In-One Solution ? Unrealistic Expectation?
Do you know what you want ?
Before Kubernetes
We need to know container first.
What is Container ?
Container / VM
https://blog.docker.com/2018/08/containers-replacing-
virtual-machines/
Container / VM
https://blog.docker.com/2018/08/containers-replacing-
virtual-machines/
Relationship between VMs/Containers
✖Containers Are More Agile then VMs
✖Containers Enable Hybrid and Multi-
Cloud Adoptio...
Resource isolation
https://blog.docker.com/2018/08/containers-replacing-
virtual-machines/
How Docker Works
✖Mount namespaces
✖IPC namespaces
✖PID namespaces
✖Network namespace
✖User namespaces
✖UTS namespaces
Persistent Storage ?
Where is my data ?
VM -> Container
OS
Docker
BusyBox
b1
Docker run --name b1 busybox
OS
Docker
BusyBox
b2
touch …
apk add …
????
Docker run --name b2 busybox
We Need To Know How Container
Works First
Image, series of read-only layers
DockerFile Image
RUN APK add ….
COPY
RUN Yarn …
a1b2c3d3xxxxx
a1b2c3d3xxxxx
a1b2c3d3xxxxx
Image Container
902b87aaaec9
4dcef5c50d60
c34ce3c1fcc0c
9a61b6b1315e
Read Only
902b87aaaec9
4dcef5c50d60
c34ce3c1fcc0c
9a6...
Container
https://docs.docker.com/glossary/?term=Union%20file
%20system
Container Layer
Container
Container Layer
Containe...
When the container is deleted, the
writable layer is also deleted.
The underlying image remains
unchanged
So, Persistent Data ?
https://docs.docker.com/storage/volumes/
Docker volume create vol
Docker run –d –v vol:/app nginx
Docker run –d -v /home/nginx:/app
nginx
How about advance storage functions ?
Networking
Container -> WAN
WAN -> Container
Container -> Container
OS
Docker
Nginx
OS
Docker
Nginx BusyBox
WAN
OS
Docker
Nginx
WAN
Docker Use Bridge Network To Provide
Network Connectivity by default.
Linux bridge/Kernel
Routing/Gateway/Iptables …
Linux Bridge Network
✖Create a linux bridge
✖Create a linux network namespace
✖Create a veth pair
✖Attach the veth pair in...
br0 br0
br0br0br0
Container
vth1
vth1vth1
Linux Host Linux Host Linux Host
Linux HostLinux HostLinux Host
ContainerContain...
Container -> Container
Container <-> WAN
Docker run –p 12345:80 nginx
How About Advanced Networking
Features?
Docker provides the basic functionality
of storage/network
Containers Cluster ?
OS
Docker
Nginx
OS
Docker
Redis
OS
Docker
Backend
1
OS
Docker
Backend
2
Network Connectivity
OS
Docker
Nginx
OS
Docker
Redis
OS
Docker
Backend
1
OS
Docker
Backend
2
Shared Storage
Data Sync
OS
Docker
Nginx
OS
Docker
Redis
OS
Docker
Backend
1
OS
Docker
Backend
2
Disaster Recovery
OS
Docker
Nginx
OS
Docker
Redis
...
OS
Docker
Nginx
OS
Docker
Redis
OS
Docker
Backend
1
OS
Docker
Backend
2
Load Balancing/HA
Backend
1
Access Control
Service Discovery
Computing Resources (CPU/GPU)
Service Mesh
Container Deployment
……
Container Orchestrator ?
Take a Break
https://kubernetes.io/docs/home/
Kubernetes is becoming the Linux of the
cloud
Jim Zemlin, Linux Foundation
Before kubernetes
✖Google has been running
containerized workloads in
production.
○ Virtually everything runs as a contain...
Kubernetes architecture
Users Control Plane Nodes
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Contr...
Kubernetes architecture
Users Control Plane Nodes
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Contr...
Kubernetes architecture
Users Control Plane Nodes
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Contr...
Kubernetes architecture
Users Control Plane Nodes
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Contr...
Kubernetes architecture
Users Control Plane Nodes
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Contr...
Control Plane
Like Docker Daemon
api-server
✖Validates and configures data for the
api objects.
✖Services REST operations
controller
✖Control loop that watches the shared
state of the cluster
✖Make changes attempting to move
the current state
h...
scheduler
✖Watches newly created pods and
selected a node for them to run on.
✖Decisions
○ Resource requirements
○ Hardwar...
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Predicate
PodSelectorMat...
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 2
Host 3
Host 4
Hos...
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6
Host 7
Host 2
Host 3
Host 4
Hos...
https://docs.google.com/presentation/d/1Gp-
2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej-
huo/edit#slide=id.g1e639c415b_0_56
Core Primitives
DaemonSet
Node
ConfigMap
StatefulSet
Job
Labels
Replica Set
Secret
Deployment
Ingress
Service
Network Policy
CRD
POD
Workloads
✖Pod
✖Deployment
✖Daemon Set
✖Job
✖Cron Job
✖Stateful Set
✖Replica Set
pod
✖A single instances of application in
Kubernetes
✖Group of containers
✖Those containers shares
○ IP address
○ File Sys...
pod
https://kubernetes.io/docs/concepts/workloads/pods/po
d/
replica Set
✖Maintain a stable set of replica Pods
running at any given time.
✖Guarantee the availability of a
specified n...
Replica Set
replica=3
Node Node Node Node
Pod Pod Pod
deployment
✖Rollouts as a Service
✖Update
○ Rolling update
○ Recreate
✖Manage Replica Set and Pod
Deployment
- replicas: 3
- version: v1
Replica Set
replica=3
Pod Pod Pod
Deployment
Deployment
- replicas: 3
- version: v1
Replica Set
replica=3
Pod Pod Pod
Deployment
Deployment
- replicas: 0
- version: v2...
Deployment
- replicas: 3
- version: v1
Replica Set
replica=3
Pod Pod Pod
Deployment
Deployment
- replicas: 1
- version: v2...
Deployment
- replicas: 2
- version: v1
Replica Set
replica=2
Pod Pod
Deployment
Deployment
- replicas: 1
- version: v2
Rep...
Deployment
- replicas: 2
- version: v1
Replica Set
replica=3
Pod Pod
Deployment
Deployment
- replicas: 2
- version: v2
Rep...
Deployment
- replicas: 1
- version: v1
Replica Set
replica=1
Pod
Deployment
Deployment
- replicas: 2
- version: v2
Replica...
Deployment
- replicas: 1
- version: v1
Replica Set
replica=1
Pod
Deployment
Deployment
- replicas: 3
- version: v2
Replica...
Deployment
- replicas: 0
- version: v1
Replica Set
replica=0
Deployment
Deployment
- replicas: 3
- version: v2
Replica Set...
Daemon Set
✖Ensure that all Nodes run a copy of a
Pod.
✖Pods are added to node once it is
added to the cluster.
✖Usage
○ S...
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Controller
Node
CLI
Deploy DaemonSet
Fluentd
Node
Fluen...
https://www.flaticon.com/free-icon/boy_145867
API Server
Scheduler
Controller
Node
CLI
Deploy DaemonSet
Fluentd
Node
Fluen...
Job
✖Creates one or more Pods
✖Ensures a specified number of them
successfully terminate.
✖You container process should no...
Daemon Job ?
Storage
Before We Talk About Kubernetes
How Would You Design The System
Infrastructure For Storage
We know
✖Container is in a read-write layer.
✖Each container has its own file
system
✖How about Pod (Group of containers)?...
Goals
✖Support different storage system.
○ Block Device
○ File System
○ Public Cloud Storage Service
✖Provide storage func...
Workloads
✖Volume
✖Persistent Volume
✖Persistent Volume Claim
✖Storage Class
Why Persistent Volume (PV) and
Persistent Volume Claim (PVC)
We Need One Abstraction Layer To
Handle All Storage Connection
Persistent volume
✖Abstraction layer for storage
providers
○ Parameters
○ Configurations
✖Take a NFS as example.
○ NFS Ser...
Persistent volume
✖Common Options
○ Capacity
○ Volume Mode
○ Access Mode
○ Reclaim Policy
○ Mount Options
Now, We Need Another Abstraction
Layer For Pod, Which is Used To Choose
Persistent Volume (PV)
Persistent volume claim
✖Don’t need to consider storage
backend.
✖Choose a PV to bind it
○ Resources requirements
✖Pod dec...
https://kubernetes.io/docs/concepts/storage/volumes/
How Pod Use PV/PVC ?
volume
✖Pod is a group of containers
✖Data will be lost if we restart Pod
✖We want to share files between those
containers...
https://kubernetes.io/docs/concepts/storage/volumes/
https://kubernetes.io/docs/concepts/storage/volumes/
Summary
✖Administrator prepare a storage
backend first
✖Create a PV first
✖Create a PVC which binds to above PV
✖Create Po...
Network
network
✖Network Connectivity
○ Container to Container (Same Node)
○ Container to Container (Cross Node)
✖Service
○ Wan to...
Network connectivity
✖Container Network Plugin (CNI)
✖Container to Container (Same Node)
○ Simplest approach is bridge mod...
Pod network
✖Group of Containers share same
network environment
✖Communicate by localhost
○ Use same IP address
○ Port con...
Pod network
Container
Nginx
Container
Redis
Pod
eth0
172.17.17.2
:80 :1234
Pod infrastructure
Pod
172.17.17.2
PID/Mount/Network/UTC
Namespace
Container
Pause
eth0
Pod infrastructure
Pod
172.17.17.2
PID/Mount/Network/UTC
Namespace
Container
Pause
eth0
Container
Nginx
Container
Redis
Al...
Kubernetes Service
Kubernetes Service
Before We Talk About Service, We Must
Know Why Service Exist.
Deployment
Node1
Nginx
Node2
Nginx
Node3
Nginx
Kubernetes Cluster
✖Deployment:
○ Ngnix
○ Replica: 3
10.123.234.56 10.123.2...
Access
✖How does application access those
Nginx servers?
✖IP address
○ 10.123.234.56:80
○ 10.123.234.57:80
○ 10.123.234.58...
Deployment
Node1
Nginx
Node2
Nginx
Node3
Nginx
Kubernetes Cluster
✖Deployment:
○ Ngnix
○ Replica: 3
10.123.234.56 10.123.2...
Deployment
Node1
Nginx
Node2
Nginx
Node3
Nginx
Kubernetes Cluster
✖Deployment:
○ Ngnix
○ Replica: 3
10.123.234.56 10.123.2...
Access
✖How does application access those
Nginx servers?
✖IP address
○ 10.123.234.56:80
○ 10.123.234.57:80
○ 10.123.234.58...
✖That’s Why We Need Service
Service
Node1
Nginx
Node2
Nginx
Node3
Nginx
Kubernetes Cluster
10.123.234.56 10.123.234.57 10.123.234.58
App
Service Nginx
Service
✖Application to Service
○ We use the DNS to access the service.
○ $(service).$(namespace).cluster.local
✖Service t...
INGRESS
✖Manage external access to the
services in a cluster, typically HTTP
✖Provide load balancing, SSL
termination and ...
https://www.hwchiu.com/ingress-1.html
summary
✖Kubernetes use CNI to provide the
basic network function for Pods
✖Service provide a DNS entry for all
backend se...
Kubernetes Limitation
Ask Yourself Before Using it
Do I Really Need Kubernetes ?
How Powerful Kubernetes Is ?
Flexible Infrastructure
✖Plugin Based
○ Container Runtime Interface
○ Device Plugin
○ Container Storage Interface
○ Contai...
https://docs.google.com/presentation/d/1Gp-
2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej-
huo/edit#slide=id.g1e639c415b_0_56
Container Runtime Interface
CRI
✖Is container omniscient ?
✖Containerlized applications
○ Dockerfile ?
○ Refactor?
✖Treat container as Virtual Machine...
Device Plugin
✖Third-party plugin
○ Nvidia GPU
○ RDMA
○ SRIOV
○ AMD GPU
○ Intel GPU/FPGA/Quick-Assist
✖Are those plugin pr...
GPU
✖GPU Device Plugin
✖GPU virtualization
✖GPU Dispatches
○ Node1: 1
○ Node2: 1
○ Node3: 0
✖Pod require 2 GPU
○ ?
✖Two Po...
GPU
✖https://github.com/AliyunContainerS
ervice/gpushare-scheduler-extender
✖https://github.com/NVIDIA/k8s-
device-plugin
Storage
✖Container Storage Interface
✖Connect to storage provider
✖Can kubernetes handle all storage
issues ?
Storage
✖FileSystem
○ Zfs/ext4/btrfs/…etc
✖Block Device
✖Distributed FS
○ Ceph/GlusterFS/BeeGFS
✖RAID/LVM
✖Read/Write Cache
Summary
✖Kubernetes doesn’t provide any
storage function.
✖It rely on backend storage provider.
✖Choose a proper storage t...
Network
✖Container Container Interface
✖A binary to setup the networking
function
✖Can kubernetes handle all networking
is...
Network
✖Network Topology
○ Fat-Tree, Leaf-Spine,
○ LAG, MC-LAG, Bonding
✖Routing related
○ BGP, OSPF, DSR, RIP
○ ECMP
✖Ne...
Network
✖SDN concept
○ Switch
○ Controller
✖Logical Network
○ VLAN/VXLAN/GRE/NVGRE
✖High Performance Network
○ DPDK/RDMA/S...
What you want?
✖IPv4 Address
○ Multiple addresses?
✖Connect to Host
○ Veth
○ Host-local
○ SRIOV ?
✖Routing
○ Static/Dynami...
summary
✖CNI provide the network connectivity
✖Service/Ingress may conflict with CNI
✖Need experience to debug networking
...
summary
✖Know what you want first
✖Evaluation
✖Check third-party solution
○ Production Ready?
○ Testing?
✖Check your resou...
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Kubernetes 1001
Upcoming SlideShare
Loading in …5
×

Kubernetes 1001

205 views

Published on

In this slide, I briefly introduce the container and how docker implement it, including the image and container itself. also show how docker setup the networking connectivity by default bridge network.

Published in: Software
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Kubernetes 1001

  1. 1. kubernetes 1001
  2. 2. hung-wei chiu Microsoft MVP Devops @ Thundertoken Co-organizer of SDNDS-TW Co-organizer of CNTUUG Network/Kubernetes/SDN https://blog.hwchiu.com
  3. 3. Why Kubernetes 1001 ?
  4. 4. All-In-One Solution ? Unrealistic Expectation? Do you know what you want ?
  5. 5. Before Kubernetes We need to know container first.
  6. 6. What is Container ?
  7. 7. Container / VM https://blog.docker.com/2018/08/containers-replacing- virtual-machines/
  8. 8. Container / VM https://blog.docker.com/2018/08/containers-replacing- virtual-machines/
  9. 9. Relationship between VMs/Containers ✖Containers Are More Agile then VMs ✖Containers Enable Hybrid and Multi- Cloud Adoption ✖Integrate Containers with Your Existing IT Process ✖Containers Save on VM Licensing ✖What About Bare Metal ✖What About Security https://blog.docker.com/2018/08/containers-replacing- virtual-machines/
  10. 10. Resource isolation https://blog.docker.com/2018/08/containers-replacing- virtual-machines/
  11. 11. How Docker Works ✖Mount namespaces ✖IPC namespaces ✖PID namespaces ✖Network namespace ✖User namespaces ✖UTS namespaces
  12. 12. Persistent Storage ? Where is my data ? VM -> Container
  13. 13. OS Docker BusyBox b1 Docker run --name b1 busybox OS Docker BusyBox b2 touch … apk add … ???? Docker run --name b2 busybox
  14. 14. We Need To Know How Container Works First
  15. 15. Image, series of read-only layers
  16. 16. DockerFile Image RUN APK add …. COPY RUN Yarn … a1b2c3d3xxxxx a1b2c3d3xxxxx a1b2c3d3xxxxx
  17. 17. Image Container 902b87aaaec9 4dcef5c50d60 c34ce3c1fcc0c 9a61b6b1315e Read Only 902b87aaaec9 4dcef5c50d60 c34ce3c1fcc0c 9a61b6b1315e https://docs.docker.com/glossary/?term=Union%20file %20system Container Layer Read Write Storage Driver Storage Driver Storage Driver Storage Driver
  18. 18. Container https://docs.docker.com/glossary/?term=Union%20file %20system Container Layer Container Container Layer Container Container Layer Read Write Read Write Read Write Read Only 902b87aaaec9 4dcef5c50d60 c34ce3c1fcc0c 9a61b6b1315e
  19. 19. When the container is deleted, the writable layer is also deleted.
  20. 20. The underlying image remains unchanged
  21. 21. So, Persistent Data ?
  22. 22. https://docs.docker.com/storage/volumes/
  23. 23. Docker volume create vol Docker run –d –v vol:/app nginx Docker run –d -v /home/nginx:/app nginx
  24. 24. How about advance storage functions ?
  25. 25. Networking
  26. 26. Container -> WAN WAN -> Container Container -> Container
  27. 27. OS Docker Nginx OS Docker Nginx BusyBox WAN OS Docker Nginx WAN
  28. 28. Docker Use Bridge Network To Provide Network Connectivity by default.
  29. 29. Linux bridge/Kernel Routing/Gateway/Iptables …
  30. 30. Linux Bridge Network ✖Create a linux bridge ✖Create a linux network namespace ✖Create a veth pair ✖Attach the veth pair into the namespace and linux bridge ✖Set the ip address ✖Set the route rules ✖Set the iptables
  31. 31. br0 br0 br0br0br0 Container vth1 vth1vth1 Linux Host Linux Host Linux Host Linux HostLinux HostLinux Host ContainerContainerContainer vth0vth0vth0
  32. 32. Container -> Container
  33. 33. Container <-> WAN
  34. 34. Docker run –p 12345:80 nginx
  35. 35. How About Advanced Networking Features?
  36. 36. Docker provides the basic functionality of storage/network
  37. 37. Containers Cluster ?
  38. 38. OS Docker Nginx OS Docker Redis OS Docker Backend 1 OS Docker Backend 2 Network Connectivity
  39. 39. OS Docker Nginx OS Docker Redis OS Docker Backend 1 OS Docker Backend 2 Shared Storage Data Sync
  40. 40. OS Docker Nginx OS Docker Redis OS Docker Backend 1 OS Docker Backend 2 Disaster Recovery OS Docker Nginx OS Docker Redis OS Docker Backend 2 Backend 1
  41. 41. OS Docker Nginx OS Docker Redis OS Docker Backend 1 OS Docker Backend 2 Load Balancing/HA Backend 1
  42. 42. Access Control Service Discovery Computing Resources (CPU/GPU) Service Mesh Container Deployment ……
  43. 43. Container Orchestrator ?
  44. 44. Take a Break
  45. 45. https://kubernetes.io/docs/home/
  46. 46. Kubernetes is becoming the Linux of the cloud Jim Zemlin, Linux Foundation
  47. 47. Before kubernetes ✖Google has been running containerized workloads in production. ○ Virtually everything runs as a container. ✖Borg: The predecessor to Kubernetes ○ Long-rumored internal container- oriented cluster-management system. ○ Pod ○ Services ○ Label https://kubernetes.io/blog/2015/04/borg-predecessor-to- kubernetes/
  48. 48. Kubernetes architecture Users Control Plane Nodes https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node (VM) Node (Bare Metal) Node (Container) CLI DISPATCH
  49. 49. Kubernetes architecture Users Control Plane Nodes https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node (VM) Node (Bare Metal) Node (Container) CLI DISPATCH I want to deploy a container
  50. 50. Kubernetes architecture Users Control Plane Nodes https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node (VM) Node (Bare Metal) Node (Container) CLI DISPATCH Find a target node
  51. 51. Kubernetes architecture Users Control Plane Nodes https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node (VM) Node (Bare Metal) Node (Container) CLI DISPATCH Dispatch Container
  52. 52. Kubernetes architecture Users Control Plane Nodes https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node (VM) Node (Bare Metal) Node (Container) CLI DISPATCH Running Container
  53. 53. Control Plane Like Docker Daemon
  54. 54. api-server ✖Validates and configures data for the api objects. ✖Services REST operations
  55. 55. controller ✖Control loop that watches the shared state of the cluster ✖Make changes attempting to move the current state https://drive.google.com/file/d/1iOsAa4HwXrNMfkkTJF A1mHt6glgpOYbL/view
  56. 56. scheduler ✖Watches newly created pods and selected a node for them to run on. ✖Decisions ○ Resource requirements ○ Hardware/Software/Policy constraints ○ Affinity ○ Anti-Affinity
  57. 57. Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7
  58. 58. Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Predicate PodSelectorMatches MatchNodeSelector NoDiskConflict …
  59. 59. Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host 2 Host 3 Host 4 Host 5 Host 6 Predicate Priority Node Affinity Image Locality Selector Spread
  60. 60. Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host 1 Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host 2 Host 3 Host 4 Host 5 Host 6 Host 6 Predicate Priority Select
  61. 61. https://docs.google.com/presentation/d/1Gp- 2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej- huo/edit#slide=id.g1e639c415b_0_56
  62. 62. Core Primitives
  63. 63. DaemonSet Node ConfigMap StatefulSet Job Labels Replica Set Secret Deployment Ingress Service Network Policy CRD POD
  64. 64. Workloads ✖Pod ✖Deployment ✖Daemon Set ✖Job ✖Cron Job ✖Stateful Set ✖Replica Set
  65. 65. pod ✖A single instances of application in Kubernetes ✖Group of containers ✖Those containers shares ○ IP address ○ File System ○ Network namespace
  66. 66. pod https://kubernetes.io/docs/concepts/workloads/pods/po d/
  67. 67. replica Set ✖Maintain a stable set of replica Pods running at any given time. ✖Guarantee the availability of a specified number of identical Pods.
  68. 68. Replica Set replica=3 Node Node Node Node Pod Pod Pod
  69. 69. deployment ✖Rollouts as a Service ✖Update ○ Rolling update ○ Recreate ✖Manage Replica Set and Pod
  70. 70. Deployment - replicas: 3 - version: v1 Replica Set replica=3 Pod Pod Pod Deployment
  71. 71. Deployment - replicas: 3 - version: v1 Replica Set replica=3 Pod Pod Pod Deployment Deployment - replicas: 0 - version: v2 Replica Set replica=0 Deployment
  72. 72. Deployment - replicas: 3 - version: v1 Replica Set replica=3 Pod Pod Pod Deployment Deployment - replicas: 1 - version: v2 Replica Set replica=1 Pod Deployment
  73. 73. Deployment - replicas: 2 - version: v1 Replica Set replica=2 Pod Pod Deployment Deployment - replicas: 1 - version: v2 Replica Set replica=1 Pod Deployment
  74. 74. Deployment - replicas: 2 - version: v1 Replica Set replica=3 Pod Pod Deployment Deployment - replicas: 2 - version: v2 Replica Set replica=2 Pod Pod Deployment
  75. 75. Deployment - replicas: 1 - version: v1 Replica Set replica=1 Pod Deployment Deployment - replicas: 2 - version: v2 Replica Set replica=2 Pod Pod Deployment
  76. 76. Deployment - replicas: 1 - version: v1 Replica Set replica=1 Pod Deployment Deployment - replicas: 3 - version: v2 Replica Set replica=3 Pod Pod Pod Deployment
  77. 77. Deployment - replicas: 0 - version: v1 Replica Set replica=0 Deployment Deployment - replicas: 3 - version: v2 Replica Set replica=3 Pod Pod Pod Deployment
  78. 78. Daemon Set ✖Ensure that all Nodes run a copy of a Pod. ✖Pods are added to node once it is added to the cluster. ✖Usage ○ Storage Daemon ○ Log collection Daemon ○ Monitoring Daemon
  79. 79. https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node CLI Deploy DaemonSet Fluentd Node Fluentd Node Fluentd
  80. 80. https://www.flaticon.com/free-icon/boy_145867 API Server Scheduler Controller Node CLI Deploy DaemonSet Fluentd Node Fluentd Node Fluentd Node Fluentd
  81. 81. Job ✖Creates one or more Pods ✖Ensures a specified number of them successfully terminate. ✖You container process should not be a daemon.
  82. 82. Daemon Job ?
  83. 83. Storage
  84. 84. Before We Talk About Kubernetes
  85. 85. How Would You Design The System Infrastructure For Storage
  86. 86. We know ✖Container is in a read-write layer. ✖Each container has its own file system ✖How about Pod (Group of containers)? ○ Restart Pod in different Node ○ Keep same data?
  87. 87. Goals ✖Support different storage system. ○ Block Device ○ File System ○ Public Cloud Storage Service ✖Provide storage function for Pods ○ Access identical storages by Pod ○ Different Pods ✖Easy to maintain/manager storage requirements ○ Capacity ○ Access Mode ○ Performance
  88. 88. Workloads ✖Volume ✖Persistent Volume ✖Persistent Volume Claim ✖Storage Class
  89. 89. Why Persistent Volume (PV) and Persistent Volume Claim (PVC)
  90. 90. We Need One Abstraction Layer To Handle All Storage Connection
  91. 91. Persistent volume ✖Abstraction layer for storage providers ○ Parameters ○ Configurations ✖Take a NFS as example. ○ NFS Server IP ○ NFS Server Export Path
  92. 92. Persistent volume ✖Common Options ○ Capacity ○ Volume Mode ○ Access Mode ○ Reclaim Policy ○ Mount Options
  93. 93. Now, We Need Another Abstraction Layer For Pod, Which is Used To Choose Persistent Volume (PV)
  94. 94. Persistent volume claim ✖Don’t need to consider storage backend. ✖Choose a PV to bind it ○ Resources requirements ✖Pod declare volume by it
  95. 95. https://kubernetes.io/docs/concepts/storage/volumes/
  96. 96. How Pod Use PV/PVC ?
  97. 97. volume ✖Pod is a group of containers ✖Data will be lost if we restart Pod ✖We want to share files between those containers. ✖Volume abstraction solves those problems
  98. 98. https://kubernetes.io/docs/concepts/storage/volumes/
  99. 99. https://kubernetes.io/docs/concepts/storage/volumes/
  100. 100. Summary ✖Administrator prepare a storage backend first ✖Create a PV first ✖Create a PVC which binds to above PV ✖Create Pods which use volume to source above PVC
  101. 101. Network
  102. 102. network ✖Network Connectivity ○ Container to Container (Same Node) ○ Container to Container (Cross Node) ✖Service ○ Wan to Container ✖Ingress ○ Wan to Container ✖Network Policy
  103. 103. Network connectivity ✖Container Network Plugin (CNI) ✖Container to Container (Same Node) ○ Simplest approach is bridge mode ○ Same as Docker default network ✖Container to Container (Cross Node) ○ Overlay Network (VXLAN/GRE) ○ L3 Routing ○ … etc
  104. 104. Pod network ✖Group of Containers share same network environment ✖Communicate by localhost ○ Use same IP address ○ Port conflict ✖How does it works ?
  105. 105. Pod network Container Nginx Container Redis Pod eth0 172.17.17.2 :80 :1234
  106. 106. Pod infrastructure Pod 172.17.17.2 PID/Mount/Network/UTC Namespace Container Pause eth0
  107. 107. Pod infrastructure Pod 172.17.17.2 PID/Mount/Network/UTC Namespace Container Pause eth0 Container Nginx Container Redis All user-defined containers are attached to Pause container.
  108. 108. Kubernetes Service
  109. 109. Kubernetes Service
  110. 110. Before We Talk About Service, We Must Know Why Service Exist.
  111. 111. Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster ✖Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.58
  112. 112. Access ✖How does application access those Nginx servers? ✖IP address ○ 10.123.234.56:80 ○ 10.123.234.57:80 ○ 10.123.234.58:80
  113. 113. Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster ✖Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.58
  114. 114. Deployment Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster ✖Deployment: ○ Ngnix ○ Replica: 3 10.123.234.56 10.123.234.57 10.123.234.75
  115. 115. Access ✖How does application access those Nginx servers? ✖IP address ○ 10.123.234.56:80 ○ 10.123.234.57:80 ○ 10.123.234.58:80 ○ 10.123.234.75:80 ✖Connect directly to server by IP address ?
  116. 116. ✖That’s Why We Need Service
  117. 117. Service Node1 Nginx Node2 Nginx Node3 Nginx Kubernetes Cluster 10.123.234.56 10.123.234.57 10.123.234.58 App Service Nginx
  118. 118. Service ✖Application to Service ○ We use the DNS to access the service. ○ $(service).$(namespace).cluster.local ✖Service to Pods ○ Service maintains all IP addresses of all Pods. ○ We call it endpoints
  119. 119. INGRESS ✖Manage external access to the services in a cluster, typically HTTP ✖Provide load balancing, SSL termination and name-based virtual hosting.
  120. 120. https://www.hwchiu.com/ingress-1.html
  121. 121. summary ✖Kubernetes use CNI to provide the basic network function for Pods ✖Service provide a DNS entry for all backend servers ✖Ingress manage a interface to handle HTTP issues
  122. 122. Kubernetes Limitation
  123. 123. Ask Yourself Before Using it
  124. 124. Do I Really Need Kubernetes ?
  125. 125. How Powerful Kubernetes Is ?
  126. 126. Flexible Infrastructure ✖Plugin Based ○ Container Runtime Interface ○ Device Plugin ○ Container Storage Interface ○ Container Network Interface ✖Developing life cycle ✖Support by third-party
  127. 127. https://docs.google.com/presentation/d/1Gp- 2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej- huo/edit#slide=id.g1e639c415b_0_56
  128. 128. Container Runtime Interface
  129. 129. CRI ✖Is container omniscient ? ✖Containerlized applications ○ Dockerfile ? ○ Refactor? ✖Treat container as Virtual Machine ✖Micro Service ?
  130. 130. Device Plugin ✖Third-party plugin ○ Nvidia GPU ○ RDMA ○ SRIOV ○ AMD GPU ○ Intel GPU/FPGA/Quick-Assist ✖Are those plugin production-ready ? ○ Stable?
  131. 131. GPU ✖GPU Device Plugin ✖GPU virtualization ✖GPU Dispatches ○ Node1: 1 ○ Node2: 1 ○ Node3: 0 ✖Pod require 2 GPU ○ ? ✖Two Pods use 1 GPU in Node 1 ○ ?
  132. 132. GPU ✖https://github.com/AliyunContainerS ervice/gpushare-scheduler-extender ✖https://github.com/NVIDIA/k8s- device-plugin
  133. 133. Storage ✖Container Storage Interface ✖Connect to storage provider ✖Can kubernetes handle all storage issues ?
  134. 134. Storage ✖FileSystem ○ Zfs/ext4/btrfs/…etc ✖Block Device ✖Distributed FS ○ Ceph/GlusterFS/BeeGFS ✖RAID/LVM ✖Read/Write Cache
  135. 135. Summary ✖Kubernetes doesn’t provide any storage function. ✖It rely on backend storage provider. ✖Choose a proper storage to meet your requirement ✖Learn the concept/knowledge about storage
  136. 136. Network ✖Container Container Interface ✖A binary to setup the networking function ✖Can kubernetes handle all networking issues ?
  137. 137. Network ✖Network Topology ○ Fat-Tree, Leaf-Spine, ○ LAG, MC-LAG, Bonding ✖Routing related ○ BGP, OSPF, DSR, RIP ○ ECMP ✖Network protocol ○ IPv4/IPv6/Multicast/Broadcast/TCP/UDP /MPTCP/STCP/QUIC ✖Network tools ○ Iptables/tun/tap
  138. 138. Network ✖SDN concept ○ Switch ○ Controller ✖Logical Network ○ VLAN/VXLAN/GRE/NVGRE ✖High Performance Network ○ DPDK/RDMA/Smart NIC
  139. 139. What you want? ✖IPv4 Address ○ Multiple addresses? ✖Connect to Host ○ Veth ○ Host-local ○ SRIOV ? ✖Routing ○ Static/Dynamic ✖Overlay network
  140. 140. summary ✖CNI provide the network connectivity ✖Service/Ingress may conflict with CNI ✖Need experience to debug networking issues
  141. 141. summary ✖Know what you want first ✖Evaluation ✖Check third-party solution ○ Production Ready? ○ Testing? ✖Check your resources

×