In this meetup, Oleg, CTO at Kublr, walks you through the basics of K8s persistence management functionality and how it can be used to simplify managing persistent applications across different environments - in the cloud or on premise. Oleg will use a demo environment with clusters in different clouds to show K8s persistence in practice. We will cover: • Persistent data abstractions in K8s: persistent volumes (PV) and their attributes • PV specifics in different clouds • Using PV in K8s: persistent volume claims (PVC) and storage classes (SC) • Automatic volume provisioning • Persistence and scheduling interrelationships • Practical examples Kubernetes (K8s) is a powerful and flexible open source container orchestration system. The power of K8s comes from its modularity and simplicity of basic concepts. Each of these basic concepts build on the other and, from the most basic elements to more advanced ones, each is responsible for its own well-defined logic and behavior.

1. Kubernetes Persistence 101
Oleg Chunikhin | CTO, Kublr

2. Introductions
Oleg Chunikhin
CTO, Kublr
• 20 years in software architecture &
development
• Working w/ Kubernetes since its release in 2015
• Software architect behind Kublr—an enterprise
ready container management platform
• Twitter @olgch

3. Enterprise Kubernetes Needs
Developers SRE/Ops/DevOps/SecOps
• Self-service
• Compatible
• Conformant
• Configurable
• Open & Flexible
• Governance
• Org multi-tenancy
• Single pane of glass
• Operations
• Monitoring
• Log collection
• Image management
• Identity management
• Security
• Reliability
• Performance
• Portability
@olgch; @kublr

4. @olgch; @kublr
Automation
Ingress
Custom
Clusters
Infrastructure
Logging Monitoring
Observability
API
Usage
Reporting
RBAC IAM
Air Gap TLS
Certificate
Rotation
Audit
Storage Networking Container
Registry
CI / CD App Mgmt
Infrastructure
Container Runtime Kubernetes
OPERATIONS SECURITY &
GOVERNANCE

5. Kubernetes Persistence
@olgch; @kublr
• Pods and Volumes
• PV and PVC
• Provisioning and Storage Class
• Volume Types, in-tree, out-of-tree, CSI and Flex
• Examples

6. Pods and Volumes
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Spec

7. PV and PVC
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Persistent Volume
Volume
Claim
Spec
Spec
Storage
Class

8. Volumes Types
@olgch; @kublr
Category Types
(Pod)
Volume
Persistent
Volume
External storage
(in-tree)
AWS EBS, Azure Disk and File, GCE Disk, CephFS, RBD, Cinder, FC, Flocker,
Gluster, Photon, Portworx, Quobyte, ScaleIO, Storageos, Vsphere, iSCSI, NFS ✔ ✔
External storage
(out-of-tree)
CSI, Flex
~100 CSI drivers ✔ ✔
Local Host Path, Local
✔
(HP)
✔(L)
Ephemeral Local empty, git (deprecated)
✔ ❌
Ephemeral Any Ephemeral
✔ ❌
Metadata Config Map, Secret, Downward API, Projected
✔ ❌
Indirection Persistent Volume Claim (resolved to a Persistent Volume)
✔ ❌

9. Storage Lifecycle: Provisioning and Use
@olgch; @kublr
Lifecycle Process
Pod Volume 1. Storage is provisioned admin
2. Pod is started with Volume referring to the Storage
Static
Provisioning
1. Storage is provisioned admin
2. PV is created referring to the Storage admin
3. PVC is created referring to the PV user
4. PVC is “bound” to the PV K8S controller
5. Pod is started with Volume referring to the PVC
Dynamic
Provisioning
1. Storage Class is created with parameters for provisioner admin
2. PVC is created referring to a Storage Class object user or K8S
3. Storage is provisioned and PV is created provisioner
4. PVC is “bound” to the PV K8S controller
5. Pod is started with Volume referring to the PVC

10. Storage Lifecycle: Pod Volume
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Spec
1 Provision storage
2
Pod reference storage

11. Storage Lifecycle: Static Provisioning
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Persistent Volume
Volume
Claim
Spec
Spec
1 Provision storage
3 PVC references PV
2 PV references storage
4 PVC bound to PV
5 Pod reference PVC

12. Storage Lifecycle: Dynamic Provisioning
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Persistent Volume
Volume
Claim
Spec
Spec
2 PVC with SC
5 Pod reference PVC
1 Storage Class
Storage
Class
Provisioner
3
Provision storage
Create PV
4 PVC bound to PV

13. Storage Lifecycle: Expansion
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Persistent Volume
Volume
Claim
Spec
Spec
2 PVC Change size
5 Pod use storage
1 Storage Class
Storage
Class
Controller
3
Resize storage
Update PV
Kubelet /
Controller
4
Resize FS

14. Storage Lifecycle: De-provisioning
@olgch; @kublr
Kubernetes Cluster
Namespace
Pod
Container 1 Container 2
Volume
Volume
Mount
Volume
Device
“actual”
storage
Persistent Volume
Volume
Claim
Spec
Spec
Reclaim Policy (PV, SC)
• Delete (default)
• Retain
• Recycle (deprecated)
Object in use protection: PV / PVC used by Pod stay “Terminating” if removed

15. All Parameters in One List
@olgch; @kublr
Attribute Object Description
mountPath / devicePath VM, VD path within container
subPath / subPathExpr VM path within volume
readOnly VM read-only mount
mountPropagation VM None, HostToContainer, Bidirectional
mountOptions PV, SC mount options for this volume, e.g. ["ro", "soft"]
accessModes PVC, PV RWO, ROX, RWX
resources / capacity PVC, PV volume resources (size)
volumeMode PVC, PV Filesystem (default) or Block
storage parameters PV, V parameters
selector / nodeAffinity / allowedTopologies PVC, PV, SC node constraints
volumeName / claimRef PVC, PV binding between PVC and PV
storageClassName PVC, PV link from PVC/PV to SC
dataSource PVC snapshotting / cloning data source: snapshot, PVC, CRD
provisioner SC provisioner ID
parameters SC provisioner parameters
volumeBindingMode SC Immediate (default), WaitForFirstConsumer (after scheduling)
allowVolumeExpansion SC Expansion enabled
persistentVolumeReclaimPolicy / reclaimPolicy PV, SC Delete (default), Retain, Recycle (deprecated)

16. Azure AWS
Examples
@olgch; @kublr
persistence-101-azure persistence-101-aws
Azure
Disk EFS
EBS
EBS
EBS
subdir
SC: ebs-immediate-retain
kubernetes.io/aws-ebs
in-tree Azure SC: kublr-system
kubernetes.io/azure-disk
in-tree AWS
NFS subdir
provisioner
SC: nfs-client
example.io/nfs
SC: ebs-wait-delete
kubernetes.io/aws-ebs

17. Beyond this demo
• Topological / zonal constraints
• PVC Templates: Stateful Sets, Ephemeral volumes
• Snapshots
• Cloning
• Backup: Velero
• Managed storage: Rook / Ceph, Gluster, Portworx, …
• ...
@olgch; @kublr

18. References
https://github.com/kublr/persistence-101-demo
https://kubernetes.io/docs/concepts/storage/
https://kubernetes-csi.github.io/docs/drivers.html
https://kubernetes-csi.github.io/docs/introduction.html
https://rook.io/
https://velero.io/
@olgch; @kublr

19. Q&A
@olgch; @kublr

20. Oleg Chunikhin
CTO
oleg@kublr.com
@olgch
Kublr | kublr.com
@kublr
Signup for our newsletter
at kublr.com
@olgch; @kublr