This document discusses persistent storage for containers and Kubernetes. It outlines some of the challenges with container storage including lack of storage pets and data needing to follow containers. It then describes eight principles of cloud native storage including being application centric, platform agnostic, and consistently available. The document explains how Docker and Kubernetes manage persistent container storage using named volumes, volume plugins, persistent volumes and claims. It introduces StorageOS as a platform that manages container storage and integrates with Kubernetes, providing high availability, replication and other features.

1. Cloud Native stateful apps require persistent Cloud
Native storage
Docker Amsterdam Meetup
Chris Brandon, Co-Founder
© StorageOS Ltd.

2. About me
• Co-Founder of StorageOS helping to
deliver persistent storage for containers
• 24 years design and implementation of
storage solutions for mission critical
application
• Formerly VP Solutions Engineering for
Oracle (previous 2 startups acquired by
Oracle)
2
@cebrando
© StorageOS Ltd.

3. Why is persistence for
containers sotricky?
3© StorageOS, Inc

4. 4© StorageOS, Inc
Why do I need persistence? @cebrando

5. First Challenge: No Storage Pets
5© StorageOS, Inc
@chira001

6. Second Challenge:
Data needs to Follow
© StorageOS, Inc 6
@cebrando

7. Third
Challenge:
Humans
are
Fallible
© StorageOS, Inc 7
@cebrando

8. 8
Eight Principles of
Cloud Native Storage

9. Eight Principles of Cloud Native Storage
Storage should be presented to and consumed by
applications, not by operating systems or
hypervisors
1 Application
centric
© StorageOS, Inc
@cebrando

10. Eight Principles of Cloud Native Storage
Storage should be presented to and consumed by
applications, not by operating systems or
hypervisors
The storage platform should be able to run
anywhere. Upgrades and scaling is non-disruptive.
1 Application
centric
2 Platform
agnostic
© StorageOS, Inc
@cebrando

11. Eight Principles of Cloud Native Storage
Storage should be presented to and consumed by
applications, not by operating systems or
hypervisors
The storage platform should be able to run
anywhere. Upgrades and scaling is non-disruptive.
Storage resources should be declared and
composed just like all other resources required by
applications and services.
1 Application
centric
2 Platform
agnostic
3 Declarative
& composable
© StorageOS, Inc
@cebrando

12. Eight Principles of Cloud Native Storage
Storage should be presented to and consumed by
applications, not by operating systems or
hypervisors
The storage platform should be able to run
anywhere. Upgrades and scaling is non-disruptive.
Storage resources should be declared and
composed just like all other resources required by
applications and services.
Storage resources and services should be easy
to be provisioned, consumed, moved and
managed via an API.
1 Application
centric
2 Platform
agnostic
3 Declarative
& composable
4 API driven
© StorageOS, Inc
@cebrando

13. Eight Principles of Cloud Native Storage
5 Natively
secure
Storage services should integrate and inline
security features such as encryption and RBAC.
© StorageOS, Inc
@cebrando

14. Eight Principles of Cloud Native Storage
5 Natively
secure
6 Agile
Storage services should integrate and inline
security features such as encryption and RBAC.
The platform should be able to move application
data between locations, dynamically resize and
snapshot volumes.
© StorageOS, Inc
@cebrando

15. Eight Principles of Cloud Native Storage
5 Natively
secure
6 Agile
7 Performant
Storage services should integrate and inline
security features such as encryption and RBAC.
The platform should be able to move application
data between locations, dynamically resize and
snapshot volumes.
The storage platform should offer deterministic
performance in complex distributed environments.
© StorageOS, Inc
@cebrando

16. Eight Principles of Cloud Native Storage
5 Natively
secure
6 Agile
7 Performant
8 Consistently
available
Storage services should integrate and inline
security features such as encryption and RBAC.
The platform should be able to move application
data between locations, dynamically resize and
snapshot volumes.
The storage platform should offer deterministic
performance in complex distributed environments.
The storage platform should ensure high
availability, durability, consistency with a
predictable, proven data model.
© StorageOS, Inc
@cebrando

17. 17
How does Docker manage persistent container
storage?

18. Key Technologies Behind Docker Image & Container Management
• Docker containers comprise a layered image and a writable ‘Container Layer’
• New, or modified data is stored in the writable container layer
• When a container is deleted its writable layer is removed leaving just the
underlying image layers behind
18
Stackable Image
Layers
Container Layer
Copy-on-write
&
© StorageOS Ltd.
@cebrando

19. Local Named Volumes
19
• Share data locally between host and
containers
• Data persists after a container has been
removed
PERSISTENCE
$ docker volume create --name mydata
$ docker run --rm -v mydata:/data:rw alpine ash -c
"echo hello world > /data/myfile"
$ sudo cat /var/lib/docker/volumes/mydata/_data/myfile
hello world
© StorageOS Ltd.
@cebrando

20. Volume Plugins
20
• Integrates external storage providers
• Data persists across nodes
STORAGE PROVIDER
$ docker run --rm -v myvol:/data
--volume-driver=storageos
alpine ash -c "echo hello world > /data/myfile"
$ docker volume create --driver storageos --opt size=1 myvol
© StorageOS Ltd.
@cebrando

21. 21
How does Kubernetes manage persistent
container storage?

22. Kubernetes Storage Model: Persistent Volumes and Claims
© StorageOS, Inc
Registers PVs in the pool
Pool of Persistent Volumes
PV PV PV PV
Administrator
@cebrando

23. Kubernetes Storage Model: Persistent Volumes and Claims
© StorageOS, Inc
Registers PVs in the pool
Claims a PV from the pool
claim
Pool of Persistent Volumes
PV PV PV PV
Administrator
Developer
@cebrando

24. Kubernetes Storage Model: Persistent Volumes and Claims
© StorageOS, Inc
claim
Registers PVs in the pool
Claims a PV from the pool
References claim in pod
claim
Pool of Persistent Volumes
PV PV PV
Pod
PV
Administrator
Developer
@cebrando

25. Dynamic Provisioning with Storage Classes
© StorageOS, Inc
Registers Storage Classes
Storage Classes
NS
PVstandard
fast
Administrator
@cebrando

26. Dynamic Provisioning with Storage Classes
© StorageOS, Inc
Registers Storage Classes
claim
Storage Classes
NS
PVslowfast
Administrator
Developer
Claims a PV from the pool
@cebrando

27. Dynamic Provisioning with Storage Classes
© StorageOS, Inc
claim
Registers Storage Classes
References claim in pod
claim
Storage Classes
NS
PV
Pod
slowfast
Administrator
Developer
Claims a PV from the pool
fast
PV
@cebrando

28. Developer View
• Volumes Provisioned and expanded via
– Docker Run
– Kubernetes Persistent Volume Claim
– StorageOS CLI
• CSI Open API available for
programmatic control
• Namespaces segregate scope of control
aligned to K8s namespaces
• Kubernetes RBAC manages permissions
• StorageOS supports rules based management using labels for
– Features (replication, caching)
– Placement (affinity / anti-affinity)
– Security behaviors
© StorageOS, Inc. 28
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: fast0001
annotations:
volume.beta.kubernetes.io/storage-class: fast
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

29. What is StorageOS?
© StorageOS, Inc

30. What is StorageOS?
© StorageOS, Inc
Platform Agnostic
(Cloud and
On-Premises
Stateful workloads
(including Databases
and Messaging)
Horizontally
scalable
K8S, OpenShift
and Docker
integration
High
availability &
failover
Replication, Inline
Compression and
Encryption

31. • Manages config,
health, scheduling,
policy, provisioning and
recovery
• API is accessed by
plugins, CLI, GUI
• Runs as a container
How it Works
31
• Manages data access
requests
• Pools aggregated
storage for
presentation
• Runs as a container
DATA PLANE
CONTROL PLANE
© StorageOS, Inc.

32. Deploy Any Stateful App in Production with Docker & Kubernetes
32© StorageOS, Inc
Get in touch
chris.brandon@storageos.com
@Storage_OS
https://slack.storageos.com/
Try out in your browser, with zero
downloads or configuration:
play.storageos.com
Full documentation at docs.storageos.com
@cebrando