ContainerDays NYC 2016: "State of the Persistence Art: Present Best Practices and Future Goals for Container Storage in Production" (Vinod Jayaraman)

DynamicInfraDays
DynamicInfraDaysDynamicInfraDays
1© 2016 PORTWORX | CONTAINERDAYS2016
State of the Art: Data Services
for Persistent Containerized Apps
VINOD JAYARAMAN, CO-FOUNDER AND CHIEF ARCHITECT
2© 2016 PORTWORX | CONTAINERDAYS2016
Agenda
§ Challenges
§ Best practices
§ Introducing Portworx
§ Demo
§ Use cases
§ Leading vendors
§ Q&A
3© 2016 PORTWORX | CONTAINERDAYS2016
Container Infrastructure: IT at Container Speed
X86 Commodity Servers
Monitoring
Orchestration/Scheduling
Minimal OS
Container Data Services
Container Infrastructure
Fixed Infrastructure
Security
Orchestration/Scheduling
Containers
HP
EMC
NetApp
IBM
Cisco
VMware
Dell
Oracle
RedHat
Microsoft
4© 2016 PORTWORX | CONTAINERDAYS2016
Problem for Enterprise IT with Container Deployments
Elastic Search Wordpress
Redis
Docker made apps portable and
enabled choice …
1. Compute spans servers, but data still silo’d
2. Data not effectively protected from failure
3. Ops + data are not portable
Developer IT Ops and DevOps
…but existing storage, connectors don’t work
for these containerized apps in production.
5© 2016 PORTWORX | CONTAINERDAYS2016
Traditional Storage Provisioning to Containers
SSD
HDD
volumenginx
Storage
Connector
python mysql
SAN
html5 rest sql
Containerized DevOps hits a wall in production Out-of-band infrastructure provisioning
DEVOPS IT
6© 2016 PORTWORX | CONTAINERDAYS2016
Challenges with SANs
LUN
Redis
No effective way to:
1. Snapshot + back up
2. Encrypt + ACL
3. Prioritize IO for users
4. Support high availability
... at the container level
7© 2016 PORTWORX | CONTAINERDAYS2016
Challenges with Physical Drives
Redis
SSD
HDD
No effective way to:
1. Expand storage capacity
2. Prioritize SSD/HDD for users
3. Perform maintenance (e.g. patch OS)
... at the container level
+
§ Snapshot+ backup
§ Encrypt and ACL
§ Support high availability
8© 2016 PORTWORX | CONTAINERDAYS2016
Challenges with VMs and Clouds
Redis
Public Cloud
Unnecessary VM overhead
+
No effective way to:
1. Snapshot + back up
2. Encrypt + ACL
3. Prioritize IO for users
4. Support high availability
... at the container level
9© 2016 PORTWORX | CONTAINERDAYS2016
Persistence
IOPs
Connectors
SSD
HDD
SAN
EBS
S3
Apps Don’t Want Volumes
1. Makes a science out of data protection
2. Churns IT playbook for every cloud
3. Does not modernize… why containerize?
10© 2016 PORTWORX | CONTAINERDAYS2016
SSD
HDD
nginx python mysql
SAN
html5 rest sql
Multi Cloud Data services
- Persistence
- Provisioning
- Security
- DR
Container Data
Services
StreamlinedDevOps Infrastructurethroughcontainer-granular self-provisioning
Apps Want Data Services
DEVOPS IT
11© 2016 PORTWORX | CONTAINERDAYS2016
Data Services for Containers
WordPress WordPress
PX
MySQL
MySQL
persistence
encryption
CoS
namespace
Auth/ACLs
HA Operational Experience
Delivered as a lightweight
container with support for
volumes and graph. Runs on any
server, any cloud.
Converged Performance
Data collocated with container
compute
Container Granular Controls
Encryption, CoS, replication,
snapshots are done at container
granularity
Hybrid Cloud Ops
Cloud-agnostic deployment and
data accessibility across multiple
clouds
12© 2016 PORTWORX | CONTAINERDAYS2016
Data Protection Data Access Performance Secure Volumes
App
Services
§ Replication, including regional
§ Snapshot read/writeable
§ File, block, object
§ Unified namespace
§ Auto expand capacity
§ Prioritize IOPs per container
§ Read parallelization
§ ACL volumes
§ Encryption inflight, at
rest
Data Protection Data Access Performance Secure Volumes
Infrastructure
Services
§ Protect from drive degradation
§ Async backup to S3
§ Global GUI control
§ Capacity plan
§ Edit existing capacity
§ IOPS tiering over SSD /
HDD
§ Monitoring integration
§ Audit access
§ BYOK encryption
Introducing Portworx Container Data Services
PORTWORX STORAGE CLUSTER
WordPress
Any
Containerized
App
Any
Cloud
Any
SAN
Any SSDAny HDDAny Server
Jenkins
13© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
Step 1: Install
SERVER SERVER EC2 NODESERVER/SAN SERVER/SAN EC2 NODE/SAN
14© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
Step 2: Fingerprint
SERVER/SAN SERVER/SAN EC2 NODE/SAN
8 SSD, 4 SATA 2 SSD, 10 SATA EBS
EXISTING
STORAGE
15© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
PORTWORX STORAGE CLUSTER
Step 3: Aggregate & Tier
FAST
MEDIUM
SLOW
PORTWORX STORAGE CLUSTER
16© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
+
+
+
Step 4: Elastic Storage
SERVER/SAN
5 SSD, 7 SATA
FAST
MEDIUM
SLOW
PORTWORX STORAGE CLUSTER
17© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
PORTWORX STORAGE CLUSTERVOL
MySQL
Step 5: Container Granular Storage
18© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
Step 6: High Availability
MySQL Data MySQL Data MySQL Data
VOL
MySQL
SERVER FAILURE
WordPress
VOL
WordPress
NFS
19© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE
Step 7: Snapshot
MySQL Data MySQL Data MySQL Data
VOL
MySQL
SNAP
MySQL
VOL
20© 2016 PORTWORX | CONTAINERDAYS2016
Portworx is infrastructure
automation at the data layer…
…so IT can offer the data
services that containers need.
SSD
HDD
SAN
EBS
S3
VOL VOLVOL VOL
21© 2016 PORTWORX | CONTAINERDAYS2016
Demo
22© 2016 PORTWORX | CONTAINERDAYS2016
On-Prem Case Study
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
STORAGE-HEAVY SERVER
SSDSSD
COMPUTE-HEAVY SERVERS
PORTWORX STORAGE CLUSTER
FAST
MEDIUM
SLOW
x 24 x 10 pools capacity,
automates infra,
same experience.
23© 2016 PORTWORX | CONTAINERDAYS2016
Portworx enables genomics institute to enter the container age.
+ Redistribute
Archive
On-Prem Case Study
FIXED INFRASTRUCTURE
Batch process
IRF
SAN
Sequencing
Pipeline:
IRF à BAM,
VCF
(3,000 cores)
Disks
Filer
Redistribute
+ Archive
PORTWORX
Fast
Slow
Medium
Sequencing
Pipeline
Batch
Disks SAN
PORTWORX
24© 2016 PORTWORX | CONTAINERDAYS2016
Container Infrastructure Landscape
ContainerLegacy/VM
Control Data
• Any hardware, OS, cloud
• All schedulers
• Data + services
25© 2016 PORTWORX | CONTAINERDAYS2016
Portworx: Storage and Container Expertise
Murli Thirumale Co-founder and CEO
Storage Executive, Dell
Co-founder and CEO, Ocarina
Group VP and GM, Citrix
Co-founder and CEO, Net6
GM, HP
Gou Rao, Co-founder and CTO
CTO/Executive, Dell
Co-founder and CTO, Ocarina
CTO, Citrix
Co-Founder and CTO, Net6
Vinod Jayaraman, Co-founder and Architect
Senior Principal Engineer/Architect, Dell
Principal Engineer, Ocarina Networks
Principal Software Engineer, F5 Networks
Software Engineer, NTT MCL
Leadership Team:24 Employees:
Startup veterans from Google, Ocarina,
Microsoft, Nimble Storage,
Pure Storage, NetApp, StorSimple
Los Altos, CA
Lead Investor:
26© 2016 PORTWORX | CONTAINERDAYS2016
Q&A
27© 2016 PORTWORX | CONTAINERDAYS2016
Learn More About Portworx
§ portworx.com for white papers,
analyst reports, videos
§ Portworx/Rancher webinar on
Wednesday, November 15
§ PX-Developer on GitHub
Sign up for more
information and a
chance to win an
Apple Watch!
1 of 27

More Related Content

What's hot(20)

Stratoscale Latest and GreatestStratoscale Latest and Greatest
Stratoscale Latest and Greatest
Zach Lanksbury893 views
Storage as a service OpenStackStorage as a service OpenStack
Storage as a service OpenStack
openstackindia3.4K views
Running Cassandra in AWSRunning Cassandra in AWS
Running Cassandra in AWS
DataStax Academy10.4K views
Filesystem as a service in OpenStackFilesystem as a service in OpenStack
Filesystem as a service in OpenStack
openstackindia7.3K views
StorageArchitecturesForCloudVDIStorageArchitecturesForCloudVDI
StorageArchitecturesForCloudVDI
Vinay Rao309 views
OpenStack Swift In the EnterpriseOpenStack Swift In the Enterprise
OpenStack Swift In the Enterprise
Hostway|HOSTING2.2K views
Running OpenStack on Amazon AWS, Alex FishmanRunning OpenStack on Amazon AWS, Alex Fishman
Running OpenStack on Amazon AWS, Alex Fishman
Cloud Native Day Tel Aviv10.4K views
Openstack swift - VietOpenStack 6thmeeetupOpenstack swift - VietOpenStack 6thmeeetup
Openstack swift - VietOpenStack 6thmeeetup
Vietnam Open Infrastructure User Group1.6K views

Viewers also liked(13)

Śniadanie Daje MocŚniadanie Daje Moc
Śniadanie Daje Moc
brandsupportpl173 views
Śniadanie Daje MocŚniadanie Daje Moc
Śniadanie Daje Moc
brandsupportpl318 views
Enrique Samson (Resume)Enrique Samson (Resume)
Enrique Samson (Resume)
Enrique Samson303 views
Ig5 brief 4_become_a_memberIg5 brief 4_become_a_member
Ig5 brief 4_become_a_member
kieranwoodcockmedia167 views
Kubernetes Scaling SIG (K8Scale)Kubernetes Scaling SIG (K8Scale)
Kubernetes Scaling SIG (K8Scale)
KubeAcademy3.4K views
Intro to cost accountingIntro to cost accounting
Intro to cost accounting
Kunal Singhal7K views
Introduction to DebenturesIntroduction to Debentures
Introduction to Debentures
LoanXpress4.2K views
Chicago Microservices Integration TalkChicago Microservices Integration Talk
Chicago Microservices Integration Talk
Christian Posta1.6K views
Docker for PHP Developers - JetbrainsDocker for PHP Developers - Jetbrains
Docker for PHP Developers - Jetbrains
Chris Tankersley9.1K views

Similar to ContainerDays NYC 2016: "State of the Persistence Art: Present Best Practices and Future Goals for Container Storage in Production" (Vinod Jayaraman)

EDB Postgres with ContainersEDB Postgres with Containers
EDB Postgres with ContainersEDB
2.8K views28 slides

Similar to ContainerDays NYC 2016: "State of the Persistence Art: Present Best Practices and Future Goals for Container Storage in Production" (Vinod Jayaraman)(20)

Cloud Opportunities with VirtualizationCloud Opportunities with Virtualization
Cloud Opportunities with Virtualization
Kellyn Pot'Vin-Gorman379 views
On Metal - The Future Of Hybrid CloudOn Metal - The Future Of Hybrid Cloud
On Metal - The Future Of Hybrid Cloud
Rackspace Asia589 views
Silicon Valley Workshop: Xanadu introduction Silicon Valley Workshop: Xanadu introduction
Silicon Valley Workshop: Xanadu introduction
Alex G. Lee, Ph.D. Esq. CLP358 views
Containerized Storage for ContainersContainerized Storage for Containers
Containerized Storage for Containers
Murat Karslioglu467 views
Customer Case : Citrix et NutanixCustomer Case : Citrix et Nutanix
Customer Case : Citrix et Nutanix
Digicomp Academy Suisse Romande SA2.3K views
Xanadu Big Data Platform Technology IntroductionXanadu Big Data Platform Technology Introduction
Xanadu Big Data Platform Technology Introduction
Alex G. Lee, Ph.D. Esq. CLP1.2K views

More from DynamicInfraDays(17)

ContainerDays NYC 2016: "State of the Persistence Art: Present Best Practices and Future Goals for Container Storage in Production" (Vinod Jayaraman)

  • 1. 1© 2016 PORTWORX | CONTAINERDAYS2016 State of the Art: Data Services for Persistent Containerized Apps VINOD JAYARAMAN, CO-FOUNDER AND CHIEF ARCHITECT
  • 2. 2© 2016 PORTWORX | CONTAINERDAYS2016 Agenda § Challenges § Best practices § Introducing Portworx § Demo § Use cases § Leading vendors § Q&A
  • 3. 3© 2016 PORTWORX | CONTAINERDAYS2016 Container Infrastructure: IT at Container Speed X86 Commodity Servers Monitoring Orchestration/Scheduling Minimal OS Container Data Services Container Infrastructure Fixed Infrastructure Security Orchestration/Scheduling Containers HP EMC NetApp IBM Cisco VMware Dell Oracle RedHat Microsoft
  • 4. 4© 2016 PORTWORX | CONTAINERDAYS2016 Problem for Enterprise IT with Container Deployments Elastic Search Wordpress Redis Docker made apps portable and enabled choice … 1. Compute spans servers, but data still silo’d 2. Data not effectively protected from failure 3. Ops + data are not portable Developer IT Ops and DevOps …but existing storage, connectors don’t work for these containerized apps in production.
  • 5. 5© 2016 PORTWORX | CONTAINERDAYS2016 Traditional Storage Provisioning to Containers SSD HDD volumenginx Storage Connector python mysql SAN html5 rest sql Containerized DevOps hits a wall in production Out-of-band infrastructure provisioning DEVOPS IT
  • 6. 6© 2016 PORTWORX | CONTAINERDAYS2016 Challenges with SANs LUN Redis No effective way to: 1. Snapshot + back up 2. Encrypt + ACL 3. Prioritize IO for users 4. Support high availability ... at the container level
  • 7. 7© 2016 PORTWORX | CONTAINERDAYS2016 Challenges with Physical Drives Redis SSD HDD No effective way to: 1. Expand storage capacity 2. Prioritize SSD/HDD for users 3. Perform maintenance (e.g. patch OS) ... at the container level + § Snapshot+ backup § Encrypt and ACL § Support high availability
  • 8. 8© 2016 PORTWORX | CONTAINERDAYS2016 Challenges with VMs and Clouds Redis Public Cloud Unnecessary VM overhead + No effective way to: 1. Snapshot + back up 2. Encrypt + ACL 3. Prioritize IO for users 4. Support high availability ... at the container level
  • 9. 9© 2016 PORTWORX | CONTAINERDAYS2016 Persistence IOPs Connectors SSD HDD SAN EBS S3 Apps Don’t Want Volumes 1. Makes a science out of data protection 2. Churns IT playbook for every cloud 3. Does not modernize… why containerize?
  • 10. 10© 2016 PORTWORX | CONTAINERDAYS2016 SSD HDD nginx python mysql SAN html5 rest sql Multi Cloud Data services - Persistence - Provisioning - Security - DR Container Data Services StreamlinedDevOps Infrastructurethroughcontainer-granular self-provisioning Apps Want Data Services DEVOPS IT
  • 11. 11© 2016 PORTWORX | CONTAINERDAYS2016 Data Services for Containers WordPress WordPress PX MySQL MySQL persistence encryption CoS namespace Auth/ACLs HA Operational Experience Delivered as a lightweight container with support for volumes and graph. Runs on any server, any cloud. Converged Performance Data collocated with container compute Container Granular Controls Encryption, CoS, replication, snapshots are done at container granularity Hybrid Cloud Ops Cloud-agnostic deployment and data accessibility across multiple clouds
  • 12. 12© 2016 PORTWORX | CONTAINERDAYS2016 Data Protection Data Access Performance Secure Volumes App Services § Replication, including regional § Snapshot read/writeable § File, block, object § Unified namespace § Auto expand capacity § Prioritize IOPs per container § Read parallelization § ACL volumes § Encryption inflight, at rest Data Protection Data Access Performance Secure Volumes Infrastructure Services § Protect from drive degradation § Async backup to S3 § Global GUI control § Capacity plan § Edit existing capacity § IOPS tiering over SSD / HDD § Monitoring integration § Audit access § BYOK encryption Introducing Portworx Container Data Services PORTWORX STORAGE CLUSTER WordPress Any Containerized App Any Cloud Any SAN Any SSDAny HDDAny Server Jenkins
  • 13. 13© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE Step 1: Install SERVER SERVER EC2 NODESERVER/SAN SERVER/SAN EC2 NODE/SAN
  • 14. 14© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE Step 2: Fingerprint SERVER/SAN SERVER/SAN EC2 NODE/SAN 8 SSD, 4 SATA 2 SSD, 10 SATA EBS EXISTING STORAGE
  • 15. 15© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE PORTWORX STORAGE CLUSTER Step 3: Aggregate & Tier FAST MEDIUM SLOW PORTWORX STORAGE CLUSTER
  • 16. 16© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE + + + Step 4: Elastic Storage SERVER/SAN 5 SSD, 7 SATA FAST MEDIUM SLOW PORTWORX STORAGE CLUSTER
  • 17. 17© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE PORTWORX STORAGE CLUSTERVOL MySQL Step 5: Container Granular Storage
  • 18. 18© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE Step 6: High Availability MySQL Data MySQL Data MySQL Data VOL MySQL SERVER FAILURE WordPress VOL WordPress NFS
  • 19. 19© 2016 PORTWORX | CONFIDENTIAL: DO NOT DISTRIBUTE Step 7: Snapshot MySQL Data MySQL Data MySQL Data VOL MySQL SNAP MySQL VOL
  • 20. 20© 2016 PORTWORX | CONTAINERDAYS2016 Portworx is infrastructure automation at the data layer… …so IT can offer the data services that containers need. SSD HDD SAN EBS S3 VOL VOLVOL VOL
  • 21. 21© 2016 PORTWORX | CONTAINERDAYS2016 Demo
  • 22. 22© 2016 PORTWORX | CONTAINERDAYS2016 On-Prem Case Study HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD HDD STORAGE-HEAVY SERVER SSDSSD COMPUTE-HEAVY SERVERS PORTWORX STORAGE CLUSTER FAST MEDIUM SLOW x 24 x 10 pools capacity, automates infra, same experience.
  • 23. 23© 2016 PORTWORX | CONTAINERDAYS2016 Portworx enables genomics institute to enter the container age. + Redistribute Archive On-Prem Case Study FIXED INFRASTRUCTURE Batch process IRF SAN Sequencing Pipeline: IRF à BAM, VCF (3,000 cores) Disks Filer Redistribute + Archive PORTWORX Fast Slow Medium Sequencing Pipeline Batch Disks SAN PORTWORX
  • 24. 24© 2016 PORTWORX | CONTAINERDAYS2016 Container Infrastructure Landscape ContainerLegacy/VM Control Data • Any hardware, OS, cloud • All schedulers • Data + services
  • 25. 25© 2016 PORTWORX | CONTAINERDAYS2016 Portworx: Storage and Container Expertise Murli Thirumale Co-founder and CEO Storage Executive, Dell Co-founder and CEO, Ocarina Group VP and GM, Citrix Co-founder and CEO, Net6 GM, HP Gou Rao, Co-founder and CTO CTO/Executive, Dell Co-founder and CTO, Ocarina CTO, Citrix Co-Founder and CTO, Net6 Vinod Jayaraman, Co-founder and Architect Senior Principal Engineer/Architect, Dell Principal Engineer, Ocarina Networks Principal Software Engineer, F5 Networks Software Engineer, NTT MCL Leadership Team:24 Employees: Startup veterans from Google, Ocarina, Microsoft, Nimble Storage, Pure Storage, NetApp, StorSimple Los Altos, CA Lead Investor:
  • 26. 26© 2016 PORTWORX | CONTAINERDAYS2016 Q&A
  • 27. 27© 2016 PORTWORX | CONTAINERDAYS2016 Learn More About Portworx § portworx.com for white papers, analyst reports, videos § Portworx/Rancher webinar on Wednesday, November 15 § PX-Developer on GitHub Sign up for more information and a chance to win an Apple Watch!