Presented at the Openstack summit vancouver https://www.openstack.org/videos/vancouver-2018/running-stateful-applications-in-containers-with-k8s-persistent-volumes-and-statefulsets

1. Running Stateful Applications in
Containers with K8S Persistent
Volumes and StatefulSets
OpenStack Summit, Vancouver
Kapil Arora
Solution Architect, NetApp EMEA
May 23, 2018

2. Agenda
1) Stateful Apps and Persistent Data
2) K8S Persistent Data Concepts
3) Static vs Dynamic Provisioning
4) NetApp Trident
5) PostgreSQL Demo
2 © 2018 NetApp, Inc. All rights reserved.

3. Stateful Apps and Persistent Data
3 © 2018 NetApp, Inc. All rights reserved.

4. Stateful Apps
§ SQL Databases
§ MySQL, PostgreSQL, SQL Server
§ NoSQL Databases
§ MongoDB, ElasticSearch, Redis, Cassandra
§ Pub Sub Systems
§ Kafka
§ Time series Databases
§ InfluxDB, Graphite
§ Big Data
§ Splunk
For Example
4 © 2018 NetApp, Inc. All rights reserved.

5. Persistent Data
§ Object
§ S3, Swift
§ File
§ NFS, SMB, Manila, GlusterFS
§ Block
§ iSCSI, FC, EBS, Cinder
For Example
5 © 2018 NetApp, Inc. All rights reserved.

6. Kubernetes Volume Plugins
6 © 2018 NetApp, Inc. All rights reserved.

7. Kubernetes Volume Plugins
§ Reference block
device and
mounted file
system
§ Accesible by all
containers in a pod
§ Lifetime of a
volume is same as
the pod (or longer)
7 © 2018 NetApp, Inc. All rights reserved.
Kubernetes Node
Volume
POD
kubelet Docker
NFSVol1
NFSiSCSI
Cinder
Azure
Disk
GCE
PD
EBS
RDB
VOLUME PLUGINS
and more…

8. § Remote
§ GCEPersistentDisk
§ AWSElasticBlockStore
§ AzureFile, AzureDisk
§ iSCSI , FC
§ Flocker
§ NFS
§ RBD, CephFS, GlusterFS
§ Cinder
§ VsphereVolume, VMware Photon
§ Quobyte Volumes
§ Portworx Volumes
§ ScaleIO Volumes
§ StorageOS
Kubernetes Volume Plugins
§ Ephemeral
§ Empty Dir
§ Secret
§ ConfigMap
§ Local Persistent Volume (beta)
§ Out-of-tree
§ CSI (beta)
§ Other
§ Host path
8 © 2018 NetApp, Inc. All rights reserved.

9. Example
§ Cinder Volume in a Pod
9 © 2018 NetApp, Inc. All rights reserved.

10. Example
§ EBS Volume in a Pod
10 © 2018 NetApp, Inc. All rights reserved.

11. Example
§ iSCSI volume in a Pod
11 © 2018 NetApp, Inc. All rights reserved.

12. Decouple storage implementation from consumption
PV and PVC Abstraction
12 © 2018 NetApp, Inc. All rights reserved.

13. Administrator creates a PV
§ PV: PersistentVolume API Object
• Configured for backing storage device
• NFS, iSCSI, Cinder, AWS EBS, GCE, Azure
• Includes connection information for the
storage volume
§ Example: NFS PV
13 © 2018 NetApp, Inc. All rights reserved.

14. User Creates PVC
§ PVC
§ PersistentVolumeClaim API Object
• Created by a user to request storage
§ User provides:
§ Size
§ Access mode
§ Storage class (optional)
• Kubernetes assigns a PV to meet
the requirements requested in the
PVC
• Does not require an exact match for
capacity
14 © 2018 NetApp, Inc. All rights reserved.

15. User provides PVC name in the Pod/Deployment
§ PVC claim name
§ mount path
15 © 2018 NetApp, Inc. All rights reserved.

16. Kubernetes Static Provisioning
16 © 2018 NetApp, Inc. All rights reserved.
K8SUser / Developer
Cluster / IT Admin
PV
5 GB
RWO
iSCSI
POD
PVC
5 GB
RWO
K8S PV
Controller
PersistentVolumeClaim PersistentVolume
Storage
POD
PVC
8 GB
RWO PV
10 GB
RWO
NFS

17. Storage Classes and Dynamic
Provisioning
17 © 2018 NetApp, Inc. All rights reserved.

18. StorageClass
§ Administrators can classify different storage types in the K8s environment using Storage
classes
§ Example
§ SSD/HDD
§ QoS
§ Dedupe/compression
§ Protocol: iSCSI/NFS
§ Etc.
§ Abstract underlying storage
§ E.g. Gold::Silver::Bronze
18 © 2018 NetApp, Inc. All rights reserved.

19. SC Example
§ Cinder
19 © 2018 NetApp, Inc. All rights reserved.

20. SC Example
§ NetApp ONTAP
§ NFS
§ SSD
20 © 2018 NetApp, Inc. All rights reserved.

21. Kubernetes Dynamic Provisioining
21 © 2018 NetApp, Inc. All rights reserved.
OpenShiftUser / Developer
Cluster / IT Admin
PV
8 GB
RWO
NFS
POD
PVC
8 GB
RWO
GOLD
K8S PV
Controller
PersistentVolumeClaim StorageClass
Gold
SSD
block
iSCSI
Bronze
HDD
NetApp
NetApp
Provisioner
(Trident)

22. NetApp Trident
22 © 2018 NetApp, Inc. All rights reserved.

23. Trident for Kubernetes
§ NetApp's Open source dynamic storage provisioner supporting:
§ ONTAP
§ SolidFire
§ E-Series
§ Automates Volume Creation and Mapping
§ Compatible with:
§ Kubernetes
§ OpenShift Origin & RedHat OCP
§ Snapshot and Cloning enabled
§ Available on GitHub: https://github.com/NetApp/trident
23 © 2018 NetApp, Inc. All rights reserved.

24. Resource Quotas
§ Kubernetes Resource Quotas
§ https://kubernetes.io/docs/concepts/policy/resource-quotas/
§ Blog
§ https://netapp.io/2017/06/09/self-provisioning-storage-kubernetes-without-worry/
§ Set Limits on
§ requests.storage
§ value.storageclass.storage.k8s.io/requests.storage
§ persistentvolumeclaims
§ value.storageclass.storage.k8s.io/persistentvolumeclaims
24 © 2018 NetApp, Inc. All rights reserved.

25. Future
§ Container Storage Interface (CSI)
§ Moving towards an industry standard, orchestrator-agnostic basic
storage abstraction
§ Standard API
§ Now Beta!
§ Spec
§ https://github.com/container-storage-interface/spec
§ Blog
§ https://kubernetes.io/blog/2018/04/10/container-storage-interface-
beta/
25 © 2018 NetApp, Inc. All rights reserved.

26. Demo : PostgreSQL
26 © 2018 NetApp, Inc. All rights reserved.

27. 27 © 2018 NetApp, Inc. All rights reserved.
Kubernetes-cluster
psql StatefulSet
(master)
psql
pod-0
(replica)
psql
pod-1
(replica)
psql
pod-2
pv-1 pv-3pv-2
psql
service
master
psql
service
psql
service
replica
pvc-1 pvc-3pvc-2
Dynamic Storage
Provisioner
Helm / Tiller

28. Why use Helm?
§ Standardized deployments
§ Build a catalog for users
§ Repeatable and predictable deployments
§ Follow best practices
28 © 2018 NetApp, Inc. All rights reserved.

29. StatefulSets
When and Why?
29 © 2018 NetApp, Inc. All rights reserved.
State
+
Scale
Unique
Network IDs
Ordered
Deployment
&
Termination

30. 30 © 2018 NetApp, Inc. All rights reserved.
Trident
postgresql-
pv-0
postgresql-statefulset
(replica=3)
postgresql-pod-0
postgresql-
container
postgresql-
pvc-0
postgresql-
service
port: 5432
postgresql-pod-1
postgresql-
container
postgresql-pod-2
postgresql-
container
postgresql-
pv-1
postgresql-
pvc-1
postgresql-
pv-2
postgresql-
pvc-2
Deploy postgresqlDB 3 replica StatefulSet

31. 31 © 2018 NetApp, Inc. All rights reserved.

32. § Stateful Apps run on Kubernetes
§ PV, PVC and StorageClass
§ Static vs Dynamic provisioning
§ NetApp Trident
§ CSI
32
Summary
© 2018 NetApp, Inc. All rights reserved.

33. § Blog and Video
§ https://netapp.io/2018/03/21/deploy-postgresql-using-kubernetes-helm-trident/
§ PostgreSQL sample Helm Chart
§ https://github.com/NetAppEMEA/kubernetes-netapp
§ NetApp Trident
§ https://github.com/NetApp/trident
33 © 2018 NetApp, Inc. All rights reserved.
Resources

34. thePub: NetApp’s Developer and Open Source Community
Engage with NetApp Engineers, Partners, and Customers
© 2018 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL ---34
GitHub
github.com/NetApp/trident
thePub
netapp.io
Twitter
@NetAppPub
Slack
netapp.io/slack
NetApp website
nt-ap.com/2CneVeR

35. 35
Thank You
© 2018 NetApp, Inc. All rights reserved.