This presentation discusses the need for containerizing the storage and how Docker is used to deliver storage as a micro serivce

1. Storage as microservice (a.k.a OpenEBS)
Built using Docker
Docker Chennai Meetup
@openebs
#containerizedstorageforcontainers
3 June 2017

2. Let us revisit containers
Image source : https://newsroom.netapp.com/blogs/containers-vs-vms/
VM
Container

3. Docker -> Portability

4. Docker registry → DevOps bridging

5. Storage controllers software version control
Monolithic Storage Kernel
iSCSI
Software
A
Replication
Snapshot
Encryption
B
QoS
&
Mgmt
C
volume
1
volume
2
Storage software is
same for all
volumes

6. Should we containerize storage ?
Version control of application software is made easy
through Docker images
How about the storage software ?

7. When storage is containerized
Docker Hub

8. Storage can also be a micro service
Containerized Storage
Kernel
(OpenEBS)
volume
1
Storage Container
VSM 1
Software
A, B, C
volume
2
Storage Container
VSM 2
Software
A’, B’, C

9. Dockerized storage
https://hub.docker.com/r/openebs/jiva/

10. What is OpenEBS ?
➔ OpenEBS is
containerized
storage for
containers
➔ OpenEBS solves the persistent storage issue for
containers
Legacy
storage
X
Application
Pod
★ Application pod management
automatically takes care of storage
volume management

11. Dedicated mode of deployment
Separate hosts

12. Network
Data
OpenEBS Maya
Hyperconverged mode of deployment
K8s master
K8s minions
VSM or Storage PodApplication Pod
TCMU/iSCSI
OpenEBS Maya Storage
Orchestrator
Application ( yaml )

13. What is Maya ?
Maya

14. Manifests
express
intent
K8S used with special care for apps requiring
persistence because they require brittle tight coupling.
Container 1 Container 2 Container 2
Hard wired connections via plug-in
stateless stateful
NAS SAN S3 NAS SAN S3
Without OpenEBS DevOps workflow is broken

15. Manifests
express
intent
stateless stateful
No changes to DevOps workflow even for containers requiring
persistence. Users manifest their intent and the storage and
storage controllers adjust automatically as needed.
OpenEBS
Policy engine
Remote Storage
Local Storage
OpenEBS Storage
Hosts
OpenEBS VSMs
OpenEBS Storage Cluster
APIs
Containers and underlying storage,
local on host or dedicated storage
pods OR remote S3 or EBS storage
all grouped into a storage cloud that
just works. Billing by AWS.
Manifests
express
intent
With OpenEBS the workflow works

16. DevOps operations with OpenEBS
OpenEBS got the storage ops off my back!
Because of OpenEBS we are rolling into
production.

17. How is OpenEBS built ?
Storage containerization
Storage orchestration
Deep storage operations
Core Storage features
is built using
is integrated with
is built into (OpenEBS Maya)
is built into (OpenEBS)
Smart Capacity Management
Volume LatencyMonitoring and Auto-healing
Adapt to varying workloads - ApplicationAware Storage (Zones)
AWS EBS Like APIs
Storage Analytics!
Enterprise HA
Intelligent QoS
DistributedScaleout
Smart Tieringfor higher performance

18. OpenEBS - Word of mouth
Containerized storage for containers
Calico / Flannel owns networking for K8S. OpenEBS owns
storage.
The only open source solution built for this. CEPH is not built
to be containerized.
I run (use) OpenEBS just like I run (use) K8S (EBS).
OpenEBS got the storage ops off my back! Because of OpenEBS
we are rolling into production.

19. Persistent storage is still an issue today
https://clusterhq.com/2016/07/17/dockercon-ecosystem-clusterhq/
"Containers have moved
beyond ephemeral storage
to persistent storage to
hold data and protect your
applications."- 451
Research
Mesosphere Declares
‘Container 2.0,’ the Stateful
Era
Kubernetes Version 1.3
include additional
resources for supporting
stateful containers

20. Persistence Storage Types
https://portworx.com/stateful-enterprise-applications-docker/
Flocker
Convoy
Rex-ray
Torus
Infinit
PortWorx
OpenEBS*
Docker
Plugin
K8s Plugin

21. Typical Use
cases

22. Where does OpenEBS fit?
DevOps and containers are the future
Which is already here, but unevenly distributed
Real computing needs persistence
But - persistence for DevOps & containers is broken

23. Usecase :
Reuse existing SAN in
your cloud to build
container friendly storage
platform

24. Build container friendly storage platform
AWS
VMs
EBSDocker
Application Pod (Application has
container-native storage)
● The existing SAN storage (EBS or GCP PD or
any other block storage) in your cloud can be
integrated into K8s application pods by
OpenEBS
● Persistent storage need not be managed
separately in your cloud
● Your persistent storage is managed from within
your application yaml files

25. Usecase :
Build AWS EBS or GCP PD
type platform On-Premise

26. Deploy EBS style platform On-Premise (Dedicated)
Your data center
Bare metal servers
Docker engines (Compute)
Bare metal storage
servers
Docker engines (OpenEBS)
Application
pod
yaml
A single application yaml
file controls both the
compute and storage
needs

27. Deploy EBS style platform On-Premise (Hyperconverged)
Your data center
Bare metal servers
Docker engines (Compute+OpenEBS)
Application
pod
yaml
A single application yaml
file controls both the
compute and storage
needs

28. Comparing AWS EBS & OpenEBS
Feature AWS EBS OpenEBS
Network Block Storage as a
service
Yes No. This is a technology
that can be used to build a
service or platform like AWS
EBS
Technology form factor Unknown Open Source
High Availability grade Five nines of availability Five nines of availability
Availability zones Yes Yes
Point in time Snapshots to
S3
Yes Yes, to S3 or any S3
equivalent (locally hosted
Minio for example)

29. Connectivity in AWS EBS (Optimized for Cloud service)
Login into AWS Console
Create an EBS volume
Attach to an instance
Format and put it to use
by application

30. Connectivity in OpenEBS (Optimized for Container service)
Login into Kubernetes
Set up an application
yaml with OpenEBS as
the storage provider
Volume is created on
OpenEBS and mounted
on K8S minion
Format and put it to use
by application
kubectl create -f demo-mysql-iscsi.yaml

31. Snapshots on AWS EBS vs OpenEBS
Incremental
Snapshot
Restore
Amazon
Cloud
Enterprise
data center
Incremental
Snapshot
Restore
Amazon
Cloud
Any other S3 Cloud
storage provider
S3 storage
in the data center
Scality
Minio
OpenEBS
volumes
Apps in
Containers
Incremental
Snapshot
Restore
Incremental
Snapshot
Restore
Incremental
Snapshot
Restore

32. Other features comparison of AWS EBS & OpenEBS
Feature AWS EBS OpenEBS
Volume migration across tiers
(Magnetic, General Purpose
SSD, Provisioned IOPS)
Yes Yes, VSMs can be migrated
seamlessly across tiers.
QoS control Yes (Provisioned IOPS) Yes, granular control of IOPS
on a per volume basis
Burst IOPS, credit bucket for
unused IOPS
Yes Yes. Min/Max IOPS, burst
IOPS, credit bucket
functionality
Auto scaling of the storage
nodes
Yes (Not sure. Internal to EBS
architecture)
Yes. OpenEBS storage nodes
are added to clusters to allow
auto scaling of either capacity
or performance

33. OpenEBS Roadmap
0.1
0.4
1.0
0.3
0.2
● Soft launch / Basic version
● Containerized controller
● Longhorn integration basics
0.2 - Basic k8s integration
0.3 - Full k8s integration
(Hyper-Converged)
0.4 - Tieringand distributedstorage
1.0 - First enterpriseedition
(Full functionality for basic
production usage)
K8s Provisioning via EBS-like driver
S3 building blocks
Complete longhorn integration
First usable release for community
consumption
Complete K8s integration
Fully hyper-converged
Support for local backing store
Building blocks of Q0S
Building blocks of Tiering
Tiering and QoS demonstrated
Building blocks of ML for storage analytics
DevOps operator use cases demonstrated
Tiering and QoS demonstrated
Building blocks of ML for storage analytics
DevOps operator use cases demonstrated
Nov, 2017
Sep, 2017
Jun, 2017
Mar, 2017
Jan, 2017

34. Take a peek at our github project and star us if you like
https://github.com/login/oauth/autho
rize?client_id=768d7617e686c9c2b
d81&scope=user%20public_repo&s
tate=openebs1234cloudbyte1234&
allow_signup=true
github.com/openebs

35. What do you think?
https://github.com/openebs
Join the
community
#slack
Slack.openebs.io
blog.openebs.io
@openebs