This document discusses persistent storage options for Linux containers. It notes that while some containerized applications are stateless, most require persistence for storing application and configuration data. It evaluates options like NFS, GlusterFS, Ceph RBD, and block storage, noting that persistent storage needs to be scalable, resilient, flexible, software-defined, and open. It provides examples of using Gluster and Ceph storage with containers. The document concludes that most containerized apps will need persistent storage and that software-defined storage allows for hyperconverged applications and storage on premises or in hybrid clouds.

1. Persistent Storage for Linux Containers
Grant Della Vecchia
Senior Storage Solutions Architect, Red Hat

2. Linux Containers in brief
A Software packaging concept that typically includes an application and all of its runtime dependencies
• Higher Quality Software
Releases
• Shorter Test Cycles
• Easier Application
Management
HOST OS
SERVER
CONTAINER
LIBS
APP
CONTAINER
LIBS
APP
Benefits

3. Greater Portability, Automation, and Integration
Code Registry
Push image
Code & Build Test Deploy
Pull image

4. The Red Hat Stack – From PaaS to Storage
DevOps Tools and User Experience
Language Runtimes and Middleware
Databases and Other Services
Container Orchestration and Management
Container API
Storage
Container Host

5. Typical workloads for Containers
Base: 194 IT operations and development decision-makers at enterprises in APAC, EMEA, and North America
Source: A commissioned study conducted by Forrester Consulting on behalf of Red Hat, January 2015
“For which workloads or application use cases have you used/do you anticipate to use containers?”
Scalable, Cost Effective, Distributed Storage for Containers

6. Why Containers require persistent Storage?
• Web & presentation layers should be stateless ... but ...
• That’s probably < 5% of all application instances in a DC
• Most other infrastructure applications / services require persistence
for storing application and configuration data
• Building a different infrastructure for stateful vs. stateless apps ?
• Modern IT requires Standardization (e.g. Cloud Computing)
• Software-defined DC is a hybrid cloud enabler

7. The requirements for Container Storage?
• Scalable – Scale out capacity & performance
• Resilient – Likely there will be important data
• Flexible – Allow different application access profiles
• Software-defined – To allow flexible deployment models
• Open – For customer choice and lowest TCO
RED HAT
STORAGE➔

8. Persistent Storage requires Redundancy
• Red Hat Storage includes important data redundancy features
• 2-way and 3-way replication
• Erasure Coding
• Geo-Replication
• Snapshots

9. Current Options for Persistent Storage
Shared Filesystems:
• NFS
• GlusterFS
Block Storage:
• Ceph RBD
• iSCSI or FC
• GCE Persistent Disk
• AWS Elastic Block Store

10. Storage Options Comparison
NFS-Filer GlusterFS Ceph
RBD
iSCSI / FC GCE PD AWS EBS
Scalability - + ++ - ++ ++
Availability + ++ ++ + + +
Cost + + + ++ - -
Deploymen
t
Flexibility
- ++ ++ - - -
Data
Privacy
++ ++ ++ ++ -- --
Sharing
Data
++ ++ +/- - - -
Use Cases Generic Data
Sharing
Generic Data
Sharing
Very Large
Deployments,
Object Store
High
Performance
Block
Cheap Block
Storage
Cheap Block
Storage

11. Configuration Examples

12. Node 1
NGINX
Container
Node 2
NGINX
Container
Persistent Storage for OSE/RHEL Atomic
Node 1
NGINX
Container
Node 3
MySQL
Container
File share
File share
Block Device /
Object Store
Atomic Host Atomic HostOPENSHIFTOPENSHIFT
Gluster
Ceph
Gluster

13. Host OS
Container Container
Host OS
Container Container
/share
(Gluster/NFS)
Container Volume mapping options
Gluster Volume
Brick 1 Brick 2 Brick 3 Brick n
/share
(Gluster/NFS)
/share/sub1
➔ /data
/share/sub2
➔ /data

14. Host OS
Container Container
Host OS
Container Container
Mapping Ceph RBDs to containers
/shared ➔ /data
Ceph
Pool
OSD
1
OSD
2
OSD
3
OSD
n
Ceph RBD +
FS
Ceph RBD + FS
/host-share ➔
/data /local-mnt
/shared ➔ /data
/host-share
Ceph RBD + FS

15. Hyperconverged Containers and Storage
• Lower TCO
• Unified Orchestration
• Ease of Use
• Greater control
Container
JBoss
NGINX
NGINX
Container
MASTER
Kubernetes
Node 2
Container
Elastic
Spark
MongoDB
Container
Kubernetes
Node 3
Redis
PostgreSQL
Container
Kubernetes
Node 1
Kubernetes
Node 4

16. App Container App Container App Container App Container
Containerized Gluster Storage
/shared-volume /shared-volume
Gluster Container
(privileged)
/bricks/brick2
Gluster Container
(privileged)
/bricks/brick1
/shared-volume
(Gluster-Fuse or NFS)
Host OS Host OS
Gluster Volume
/shared-volume /shared-volume

17. Container Container Container Container
Containerized Ceph Storage
/local-mnt1
Container
RHCS
Container
RHCS
Ceph RBDs (and Object Storage)
Host OS Host OS
Ceph Pool
/local-mnt2 /local-mnt3 /local-mnt4

18. Flexible Deployment Options
x86 x86 x86
Physical
VM VM VM
Virtualized
VM VM VM
Public / Private Cloud
OpenStack,
AWS, Azure, GCE
VMware, RHEV, KVM
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4

19. Customer
Case Studies

20. Customer Case Study: CapitalOne
Business Challenge:
• A leading diversified bank with 65 million customers
• Fast growing business and customer base
• Need to be disruptive and different
• Analytics plays a big role in growth strategy
Solution Description:
• Predefined docker images with a wide variety of analytics
tools
• Self-service Portal for developers to pick and instantiate
• Integrated monitoring and metrics
• Automated lifecycle management of containers
• High availability through MESOS
• Shared and consolidated Storage Platform with Gluster
Solution Benefits:
• More agile application development
• Larger choice of technologies
• Optimal resource usage and
performance

21. Customer Case Study: CapitalOne

22. Customer Case Study: Verizon
D
o
c
k
e
r
c
o
n
t
a
i
n
e
r
s
r
u
n
n
i
Business Challenge:
• Verizon is the largest provider of cellular services in the United States with
more than 100 million subscribers. Pretty much every subscriber takes ‘selfies’
and snaps of the kittens and these need to be backed up. Whilst many users
use Apple’s iCloud, or Yahoo’s Flickr, Verizon also has their own branded
Cloud offering – Verizon Cloud.
• Verizon wished to use a file-based format rather than converting all these
images to Objects and to have an extremely efficient architecture in terms of
the use of their server infrastructure.
Solution Description:
• Simplified Deployment via containers
• Seamless upgrade and rollback
• Dedicated and Containerized Gluster for FSaaS
• Dynamic scaling (up) of capacity as needed
• Performance and health metrics collection via container monitoring agents
and sending alerts.
Solution Benefits:
• Quicker time to market for new services
• Increased subscriber ‘stickiness’ and improving customer
satisfaction.
• Reduced infrastructure Costs
• Infrastructure can grow as demand grows

23. ETH/IB
app app app
app app app
Converged Computing Architecture
Mixed App and Storage Workloads
App-only servers
Converged servers
app
app +
Storage-only servers
app app app app
app app app app
app app app app
app app
app
Storage stack imposes only
3% - 10% load on compute
processing
• Applications and storage
stacks can co-exist on same
compute substrate.
• This achieves higher server
utilization and lower
operational costs across the
cloud
Customer Case Study: Verizon

24. Summary
• Most containerized applications will require Persistent Storage
• Software-defined Storage allows hyper-convergence for applications and
storage
• Red Hat Storage and OpenShift Enterprise provide a complete PaaS
solution with full deployment flexibility from on-premise to hybrid clouds

25. Thank You