Quick overview on the journey of storage w.r.t containers and emerging trends in container storage. A look at the requirements and architectural patterns of the new storage startups. Presented at the Docker meetup#24 at #GoJekIndia

1. Emerging Storage
Trends
for Containers
@openebs
Docker Bangalore Meetup #24
@kiranmova
@CloudByteInc

2. Happy Docker!
Stateless Containers
Ephemeral Storage from
local disks.
Rebuilding is Easy
5 million software
downloads and 650,000
registered users

3. Containers in Production!
Scale-up
Long-running
Highly Networked Micro-
services
Downtime costs money

4. Rough tides...
Sydney storms on Sunday
knocked out the
operations of numerous
Amazon Web Services
customers.
The ferocious storms that
hit NSW, left AWS clients
including Domino's Pizza,
Foxtel, The Iconic, Stan
and Domain without
websites or key systems
for hours.
http://www.afr.com/technology/cloud-computing/amazon-web-services-storm-outages-serve-as-a-warning-
of-cloud-risk-to-businesses-20160606-gpcugk

5. Containers - Dev Vs Prod
https://twitter.com/mfdii/status/697532387240996864

6. Docker in Real-world
Clusters Everywhere!
Network and Storage -
Can’t be local!

7. Containers - Dev Vs Prod
https://twitter.com/mfdii/status/697532387240996864

8. Container - Where is my Data?
Flocker / ClusterHQ
Back in 2014 with first
DockerCon “ClusterHQ
were the first to say that
container data
management was essential
for containers…” - Mark
Davis CEO ClusterHQ

9. Long live Data Persistence
https://clusterhq.com/2016/07/17/dockercon-ecosystem-clusterhq/
"Containers have moved
beyond ephemeral storage
to persistent storage to
hold data and protect your
applications."- 451
Research
Mesosphere Declares
‘Container 2.0,’ the Stateful
Era
Kubernetes Version 1.3
include additional
resources for supporting
stateful containers

10. Docker Adoption by Storage
Vendors
Scott Sinclair, a storage analyst at Enterprise Strategy Group, said 2016
has ushered in a "significant adoption cycle" for Docker.
"We are starting to see more and more organizations taking a look at how
they can use containers," he said. "There's tremendous excitement by
vendors, too. It's rare for me to talk with a storage vendor that
doesn't have, or is planning to have, an offering for Docker."
http://searchstorage.techtarget.com/news/450303805/Vendors-target-storage-for-containers-with-
DevOps-in-mind
"Containers have moved
beyond ephemeral storage
to persistent storage to
hold data and protect your
applications."- 451
Research
Mesosphere Declares
‘Container 2.0,’ the Stateful
Era
Kubernetes Version 1.3
include additional
resources for supporting
stateful containers

11. Emerging Storage
Technologies
for Containers
@openebs
Connecting Storage to Containers
Orchestrating Storage for
Containers
Optimizing Underlying Storage
Technology for Containers

12. Connect - Volume Plugins
Connecting Storage
The advent of Volume
Plugins
External SAN/NAS
Pre-configured and auto-
provision
https://clusterhq.com/2015/12/09/difference-docker-volumes-flocker-volumes/

13. Connect - Volume Plugins
Connecting Storage
The advent of Volume
Plugins
External SAN/NAS
Pre-configured and auto-
provision

14. Connect - Volume Plugins
Connecting Storage
The advent of Volume
Plugins
External SAN/NAS
Pre-configured and auto-
provision

15. Connect - Volume Plugins
https://clusterhq.com/2015/06/17/flocker-1-0/
Connecting Storage
The advent of Volume
Plugins
External SAN/NAS
Pre-configured and auto-
provision
Flocker

16. Connect - Volume Plugins
http://rancher.com/introducing-convoy-a-docker-volume-driver-for-backup-and-recovery-of-persistent-data/
Connecting Storage
The advent of Volume
Plugins
External SAN/NAS
Pre-configured and auto-
provision
Flocker
Convoy

17. Connect - Volume Plugins
NetApp
Rook (Ceph)
Hedvig
RedHat
HP
AWS
GPE
EMC

18. Connect - Volume Plugins
Connecting Storage
EMC - libStorage

19. Connect - Orchestration Layer
The right place!
Volumes outlive the
containers. They are
configured at the pod.
Adoption on the rise.
Slowdown of convoy,
flocker, possibly docker

20. Growth vs Infrastructure
30% increase in Docker
adoption in one year.
The number of containers
running in production
quintuples (= 5x) 9
months after initial
deployment.
June 2016
https://blog.newrelic.com/2016/06/16/docker-container-infographic/

21. Containers Demand Robust Infra!
Capacity!
Data Protection
Elastic deployment (scale-
out/distributed)
Low latency predictable
performance
Faster and simpler provisioning
and upgrades
Hybrid Clouds
https://static-ssl.businessinsider.com/image/51560a4869bedd425f000003-960-720/pakistani-men-in-
islamabad-try-to-right-a-truck-transporting-wheat-straw.jpg

22. Trend#1 Elastic Deployment
Storage Orchestration
Hedvig
NetApp/SolidFire
Ceph
CoreOS Torus
Docker Infinit.sh
Rancher
LongHorn/OpenEBS
PortWorx
Distributed Storage (No
SPOFs)
ScaleOut
Share-Nothing

23. Trend#2 Ease of Access
Storage Orchestration
CoreOS Torus
Docker Infinit.sh
Rancher
LongHorn/OpenEBS
PortWorx
No Networking Provisioning
Overheads (Single
Volume/NBD)
Uses the docker networks
(overlay) and containers
Seamless Integration into
Orchestration Layers

24. Trend #3 Hyper Converged
Storage Orchestration
CoreOS Torus
Docker Infinit.sh
Rancher
LongHorn/OpenEBS
(3) Hyper Converged.
Developer Centric.

25. Trend #4 Hybrid Clouds
Storage Orchestration
Deploy Storage Clusters
along side K8s in GCP or
AWS.
Container in GCP talking
to volumes in AWS.
DP and DR
Rancher
LongHorn/OpenEBS
S3 Snapshots
Move data across clouds
EBS Compatible. Eliminate
need for new drivers

26. Trend#6 Containerized Storage
Storage Orchestration
Ease of Upgrades /
Isolation
Rancher
LongHorn/OpenEBS
DevOps Delivered

27. Trend#5 Tiered Storage
Storage Technology
Higher Capacity, Faster
and Cheaper$$ flash.
Rancher
LongHorn/OpenEBS
NVMe Flash Caching
2TB Flash with 20TB
Volumes

28. Watch out..
DataHub
Cloud Federated
Infrastructures
OpenSource - Sooner or
Later..

29. OpenEBS
Orchestration(K8s)
Remote Storage
Local Storage
OpenEBS Storage
Hosts
OpenEBS VSMs
iSCSI/TCMU/NBD
(data)
OpenEBS Maya Masters
Clients - Containers(Docker) / VMs
EBS Client
HTTPS
(EBS API
(manage)
https://github.com/openebs
www.openebs.io
Hiring Gopher Warriors!!

30. References
https://www.joyent.com/blog/persistent-storage-patterns
http://forums.theregister.co.uk/forum/1/2016/07/18/docker_and_storage_solving_the_problem_of_data_p
ersistence/
http://thenewstack.io/emc-pushes-alternative-dockers-persistent-storage-volumes/
https://www.sdxcentral.com/articles/news/emc-connects-storage-containers/2016/06/
https://kafka.apache.org/uses
http://www.dbta.com/DBTA-Downloads/WhitePapers/MOVING-TO-A-MODERN-DATA-
ARCHITECTURE-6743.aspx
https://docs.opensvc.com/agent.service.container.docker.enterprise_architecture.html
https://developer.ibm.com/urbancode/2016/03/21/docker-deployments-for-the-enterprise/
https://blog.docker.com/2016/06/docker-enterprise-hpe/
http://searchstorage.techtarget.com/feature/Hot-data-storage-technology-trends-for-2017

31. Thank You
@kiranmova
@openebs
https://github.com/openebs/openebs