Radical innovations in storage for multi-
tenant infrastructure
John Griffith, Lead Open Source Developer & PTL,
OpenStack Block Storage
How did we get to this point?
•  First there was virtualization…and it was
good
•  For smaller scale use cases it still is good
•  But, when scaling virtual environments…
•  Adding and deploying hypervisors
•  Networking headaches
•  Management complexity
•  Storage performance degradation
•  Something had to change
Graphic Source: http://crystaltec.com.au/services/virtualization
Enter Cloud Computing
•  It’s no longer just about a collection of
virtual machines
•  It’s all about:
•  Abstraction
•  Automation
•  Resource utilization
•  Maximum efficiency
•  Utility consumption model
•  Dynamic and massive scale
•  Common interfaces / APIs
Graphic Source: http://www.intelligentitnyc.com
Enter OpenStack
•  Project launched in 2010
•  Open source software for building
and managing clouds
•  A collection of software projects that
tie into common management layer
•  Created to
•  Drive industry standards
•  Accelerate cloud adoption
•  Put an end to cloud “lock-in”
•  Deliver AWS-like functionality and
economics outside of AWS
Storage Infrastructure Implications
•  One of many abstracted resources tied into common management
layer
•  Virtual machines, Bare Metal, Storage, Networking, etc.
•  Must be…
•  Capable of granular & dynamic resource provisioning
•  Designed for multi-tenancy & scale
•  Easy to use REST-API
•  Enabling self-service virtual environments for end users
Storage in the OpenStack Context
Cinder / Block Storage Swift / Object Storage
Objectives
•  Storage for running VM disk
volumes on a host
•  Ideal for performance sensitive apps
•  Enables Amazon EBS-like service
•  Ideal for cost effective, scale-out storage
•  Fully distributed, API-accessible
•  Well suited for backup, archiving, data retention
•  Enables Dropbox-like service
Use Cases
•  Production Applications
•  Traditional IT Systems
•  Database Driven Apps
•  Messaging / Collaboration
•  Dev / Test Systems
•  VM Templates
•  ISO Images
•  Disk Volume Snapshots
•  Backup / Archive
•  Image / Video Repository
Workloads
•  High Change Content
•  Smaller, Random R/W
•  Higher / “Bursty” IO
•  Typically More Static Content
•  Larger, Sequential R/W
•  Lower IOPS
Cinder 101 •  Architected to provide traditional block level
storage resources to other OpenStack
services
•  Presents persistent block level storage
volumes for use with OpenStack Nova
compute instances
•  Manages the creation, attaching and
detaching of these volumes between a storage
system like SolidFire and different host
servers
How Cinder
Works
•  Plug-in architecture, use your own vendors
backend(s) or use the default
•  Default implementation built on LVM and iSCSI
•  Mix and match, add and remove
•  Back-end devices can be invisible to end-user/
tenant
•  Consistent API regardless of backend selection
•  Filter scheduling to auto place newly create volumes
•  More specific placement based on volume-type
selection
•  Expose differentiating features via custom volume-
types and extra-specs
Cinder Base Features
•  Create/delete volumes
•  Specify custom "types/extra-specs”
•  Clone
•  Copy image to volume and volume to image
•  Point in time copy (snapshots of volumes)
•  Create volume from snapshot
•  Backup volume (to object store, SWIFT and CEPH)
•  Transfer volume ownership
•  Customized scheduling filters
•  Per tenant usage quotas
Cinder
Design
View
Cinder
Benefits
•  Dynamic pool of block storage resources
•  Horizontally scalable
•  Well defined, easy to use API
•  Strict adherence to API compatibility regardless of backend
•  Back-End Storage Differentiators
•  To the Admin
•  Cost
•  Management
•  Reliability
•  To the End-user
•  Performance
•  Reliability
Vendor Unique Features In Cinder
•  Exposed through custom types or extensions
•  Different back-ends for different use cases
•  Back-End selected by filter scheduler
•  Back-End is setup based on desired capabilities and
characteristics
Striving for Self-
Service
•  Idea is for the end user to be able to request
resources on demand
•  “Give me 100 GB of block storage with XYZ
characteristics please”
•  “It’s mine, now what can I do with it”
•  Attach it, boot it, clone it, back it up
SolidFire’s Scale-Out Block Storage System
Designed for OpenStack and other cloud platforms
•  Scale-Out Design, 100 nodes, 3.4PB, 7.5M IOPS
•  Industry standard hardware, 10gb iSCSI
•  Linear non-disruptive growth
•  Automation top priority in API design
•  Built to deploy in an OpenStack environment
•  Extreme fault tolerance
•  Best-in-Class QoS Architecture
Eliminating Noisy Neighbors
The Noisy Neighbor Effect
•  Individual tenant impacts other applications
•  Unsuitable for performance sensitive apps
SolidFire QoS in Practice
•  Create fine-grained tiers of performance
•  Application performance is isolated
•  Performance SLAs enforced
Traditional
Multi-Tenant
Performance
Noisy
Neighbor
Traditional
Multi-Tenant
Performance
Application of
SolidFire QoS
controls
SolidFire &
Cinder
•  Full SolidFire driver integration with latest
OpenStack software release
•  Set and maintain true QoS levels on a per-
volume basis
•  Create/snapshot/clone/manage SolidFire
volumes using OpenStack clients and APIs
•  Run instances on a SolidFire volume
•  Enhanced boot from volume capabilities
•  Web-based API exposing all cluster
functionality
•  SolidFire/Cinder integration can be
configured in less than a minute
SolidFire &
Nebula
•  Brings together the founding engineers
and leaders of the Cinder project
•  Validated interoperability
•  Integrated user experience for customers
adding SolidFire storage appliances to
their Nebula One system
"In a cloud infrastructure, some applications require very high levels of consistent storage
performance. Adding a SolidFire appliance to the Nebula One system brings a level of
performance predictability not possible from spinning disks in our certified industry standard
servers.”
- Chris C. Kemp, Co-Founder & CEO, Nebula
Why SolidFire
& OpenStack?
•  No better block storage option for customers seeking
guaranteed performance, high availability and scale,
•  Designed from the ground up to integrate with
orchestration layers like OpenStack
•  Unrivaled integration & support
•  Breadth/depth of API creates significant flexibility
•  All core Cinder API features implemented in SolidFire
driver
•  No additional features/licenses required;
•  Driver included in OpenStack distribution
•  Don't have to buy license to enable snapshots, clones, etc.
•  Don't have to buy an SDK, or management server, etc.
Build your cloud…
Out of technology built for the cloud
Key Cloud Design Principles Relevant SolidFire Features
Abstraction Performance Virtualization
Automation All cluster operations emanate from API
Resource Utilization •  Fine grained QoS
•  Provision capacity independent of performance
Maximum Efficiency •  Built-In Data Reduction
•  Best-in-Class hardware density
Utility Consumption Model Dynamic Provisioning & Modifications
Dynamic & Massive Scale Linear, Non-Disruptive, Single Management Domain
Common Interface & APIs Enabled via REST-based APIs
1620 Pearl Street,
Boulder, Colorado 80302
Phone: 720.523.3278
Email: info@solidfire.com
www.solidfire.com
Questions?

Radical Innovations In Storage for Multi-Tenant Infrastructure

  • 1.
    Radical innovations instorage for multi- tenant infrastructure John Griffith, Lead Open Source Developer & PTL, OpenStack Block Storage
  • 2.
    How did weget to this point? •  First there was virtualization…and it was good •  For smaller scale use cases it still is good •  But, when scaling virtual environments… •  Adding and deploying hypervisors •  Networking headaches •  Management complexity •  Storage performance degradation •  Something had to change Graphic Source: http://crystaltec.com.au/services/virtualization
  • 3.
    Enter Cloud Computing • It’s no longer just about a collection of virtual machines •  It’s all about: •  Abstraction •  Automation •  Resource utilization •  Maximum efficiency •  Utility consumption model •  Dynamic and massive scale •  Common interfaces / APIs Graphic Source: http://www.intelligentitnyc.com
  • 4.
    Enter OpenStack •  Projectlaunched in 2010 •  Open source software for building and managing clouds •  A collection of software projects that tie into common management layer •  Created to •  Drive industry standards •  Accelerate cloud adoption •  Put an end to cloud “lock-in” •  Deliver AWS-like functionality and economics outside of AWS
  • 5.
    Storage Infrastructure Implications • One of many abstracted resources tied into common management layer •  Virtual machines, Bare Metal, Storage, Networking, etc. •  Must be… •  Capable of granular & dynamic resource provisioning •  Designed for multi-tenancy & scale •  Easy to use REST-API •  Enabling self-service virtual environments for end users
  • 6.
    Storage in theOpenStack Context Cinder / Block Storage Swift / Object Storage Objectives •  Storage for running VM disk volumes on a host •  Ideal for performance sensitive apps •  Enables Amazon EBS-like service •  Ideal for cost effective, scale-out storage •  Fully distributed, API-accessible •  Well suited for backup, archiving, data retention •  Enables Dropbox-like service Use Cases •  Production Applications •  Traditional IT Systems •  Database Driven Apps •  Messaging / Collaboration •  Dev / Test Systems •  VM Templates •  ISO Images •  Disk Volume Snapshots •  Backup / Archive •  Image / Video Repository Workloads •  High Change Content •  Smaller, Random R/W •  Higher / “Bursty” IO •  Typically More Static Content •  Larger, Sequential R/W •  Lower IOPS
  • 7.
    Cinder 101 • Architected to provide traditional block level storage resources to other OpenStack services •  Presents persistent block level storage volumes for use with OpenStack Nova compute instances •  Manages the creation, attaching and detaching of these volumes between a storage system like SolidFire and different host servers
  • 8.
    How Cinder Works •  Plug-inarchitecture, use your own vendors backend(s) or use the default •  Default implementation built on LVM and iSCSI •  Mix and match, add and remove •  Back-end devices can be invisible to end-user/ tenant •  Consistent API regardless of backend selection •  Filter scheduling to auto place newly create volumes •  More specific placement based on volume-type selection •  Expose differentiating features via custom volume- types and extra-specs
  • 9.
    Cinder Base Features • Create/delete volumes •  Specify custom "types/extra-specs” •  Clone •  Copy image to volume and volume to image •  Point in time copy (snapshots of volumes) •  Create volume from snapshot •  Backup volume (to object store, SWIFT and CEPH) •  Transfer volume ownership •  Customized scheduling filters •  Per tenant usage quotas
  • 10.
  • 11.
    Cinder Benefits •  Dynamic poolof block storage resources •  Horizontally scalable •  Well defined, easy to use API •  Strict adherence to API compatibility regardless of backend •  Back-End Storage Differentiators •  To the Admin •  Cost •  Management •  Reliability •  To the End-user •  Performance •  Reliability
  • 12.
    Vendor Unique FeaturesIn Cinder •  Exposed through custom types or extensions •  Different back-ends for different use cases •  Back-End selected by filter scheduler •  Back-End is setup based on desired capabilities and characteristics
  • 13.
    Striving for Self- Service • Idea is for the end user to be able to request resources on demand •  “Give me 100 GB of block storage with XYZ characteristics please” •  “It’s mine, now what can I do with it” •  Attach it, boot it, clone it, back it up
  • 14.
    SolidFire’s Scale-Out BlockStorage System Designed for OpenStack and other cloud platforms •  Scale-Out Design, 100 nodes, 3.4PB, 7.5M IOPS •  Industry standard hardware, 10gb iSCSI •  Linear non-disruptive growth •  Automation top priority in API design •  Built to deploy in an OpenStack environment •  Extreme fault tolerance •  Best-in-Class QoS Architecture
  • 15.
    Eliminating Noisy Neighbors TheNoisy Neighbor Effect •  Individual tenant impacts other applications •  Unsuitable for performance sensitive apps SolidFire QoS in Practice •  Create fine-grained tiers of performance •  Application performance is isolated •  Performance SLAs enforced Traditional Multi-Tenant Performance Noisy Neighbor Traditional Multi-Tenant Performance Application of SolidFire QoS controls
  • 16.
    SolidFire & Cinder •  FullSolidFire driver integration with latest OpenStack software release •  Set and maintain true QoS levels on a per- volume basis •  Create/snapshot/clone/manage SolidFire volumes using OpenStack clients and APIs •  Run instances on a SolidFire volume •  Enhanced boot from volume capabilities •  Web-based API exposing all cluster functionality •  SolidFire/Cinder integration can be configured in less than a minute
  • 17.
    SolidFire & Nebula •  Bringstogether the founding engineers and leaders of the Cinder project •  Validated interoperability •  Integrated user experience for customers adding SolidFire storage appliances to their Nebula One system "In a cloud infrastructure, some applications require very high levels of consistent storage performance. Adding a SolidFire appliance to the Nebula One system brings a level of performance predictability not possible from spinning disks in our certified industry standard servers.” - Chris C. Kemp, Co-Founder & CEO, Nebula
  • 18.
    Why SolidFire & OpenStack? • No better block storage option for customers seeking guaranteed performance, high availability and scale, •  Designed from the ground up to integrate with orchestration layers like OpenStack •  Unrivaled integration & support •  Breadth/depth of API creates significant flexibility •  All core Cinder API features implemented in SolidFire driver •  No additional features/licenses required; •  Driver included in OpenStack distribution •  Don't have to buy license to enable snapshots, clones, etc. •  Don't have to buy an SDK, or management server, etc.
  • 19.
    Build your cloud… Outof technology built for the cloud Key Cloud Design Principles Relevant SolidFire Features Abstraction Performance Virtualization Automation All cluster operations emanate from API Resource Utilization •  Fine grained QoS •  Provision capacity independent of performance Maximum Efficiency •  Built-In Data Reduction •  Best-in-Class hardware density Utility Consumption Model Dynamic Provisioning & Modifications Dynamic & Massive Scale Linear, Non-Disruptive, Single Management Domain Common Interface & APIs Enabled via REST-based APIs
  • 20.
    1620 Pearl Street, Boulder,Colorado 80302 Phone: 720.523.3278 Email: info@solidfire.com www.solidfire.com Questions?