SlideShare a Scribd company logo
1 of 23
Guaranteeing Storage Performance in 
CloudStack
Mike Tutkowski 
- Full-time CloudStack software engineer, CloudStack PMC member 
- Focused on CloudStack's storage component 
SolidFire (http://solidfire.com) 
- Based out of Boulder, CO 
- Develop a clustered, scale-out SAN technology (using industry-standard hardware) 
- Built from the ground up to support guaranteed Quality of Service (QoS) on a per-volume 
(LUN) basis (min, max, and burst IOPS per volume) 
- All SSD architecture (no spinning disks) 
- Leverage compression, de-duplication, and thin provisioning (all inline) on a 4-KB 
block boundary across the entire cluster to drive down cost/GB to be on par with 
traditional disk-based SANs 
- Rest-like API to enable automation of all aspects of the SAN
CloudStack from 
the top down 
(prior to 4.2)
Primary Storage Secondary Storage 
Objectives Storage for VMs (root and data disks) Data to be stored for future retrieval 
Use Cases • Production Applications 
• Traditional IT Systems 
• Database-Driven Apps 
• Messaging / Collaboration 
• Dev/Test Systems 
• VM Templates 
• ISO Images 
• Backups of Volumes 
Workloads • High-Change Content 
• Smaller, Random R/W 
• Higher / “Bursty” IO 
• Typically More Static Content 
• Larger, Sequential IO (more read 
than write) 
• Lower IOPS 
Storage Use Cases & Workloads
What is Primary Storage (Pre 4.2)? 
• Primary Storage is associated with a cluster 
• A cluster can access more than one Primary Storage 
• Primary Storage can be shared among hosts or local to a host 
● Primary Storage stores the disk volumes (both root and data disks) for all 
the VMs in that cluster 
• Depending on hypervisor type, there are several ways to configure 
Primary Storage (we shall take a look at XenServer)
Provisioning Primary Storage (Pre 4.2) 
• Admin allocates space ahead of time on the storage system 
(Example: Create a volume on the SolidFire SAN) 
• Admin defines a storage resource in the hypervisor 
(Example: Create a XenServer Storage Repository) 
• Admin defines a storage pool in CloudStack 
(Example: Create Primary Storage in CloudStack for a cluster) 
• Admin creates a Compute Offering using the Primary Storage 
(Example: 1 vCPU, 2 GB RAM, 50 GB)
Allocate 
Storage on the 
SolidFire SAN
Define the Storage Resource in the Hypervisor 
Select the type of the storage repository 
Name the storage repository 
Map the storage repository to the volume 
Storage repository is now available in the hypervisor
Define a Primary Storage Pool in CloudStack 
Add Primary Storage Define Primary Storage 
Primary Storage Available for Use
Create a Compute Offering in CloudStack 
Add Compute Offering Define Compute Offering 
Compute Offering Available for Use
Primary Storage in CloudStack >= 4.2 
• Fully automated 
provisioning through 
CloudStack 
• Dynamic volume creation 
for VM root disks and 
additional data disks 
• With SolidFire each 
volume receives 
guaranteed IOPS
My Specific Needs from the Plug-in 
Provide a way to expose vendor-unique features within CloudStack 
Eliminate the need for customers to create additional orchestration logic to 
provision storage 
Have the ability to defer the creation of a volume until the moment the end user 
elects to execute a Compute or Disk Offering
Creating Primary Storage Based on a Plug-in 
http://127.0.0.1:8080/client/api?command=createStoragePool& 
scope=zone& 
zoneId=cf4e6ddf-8ae7-4194-8270-d46733a52b55& 
name=SolidFire_121258566& 
url=MVIP%3D192.168.138.180%3A443%3BSVIP%3D192.168.56.7%3BclusterAdminUsername%3Dadmin%3BclusterAdminPassword 
%3Dpassword%3BclusterDefaultMinIops%3D200%3BclusterDefaultMaxIops%3D300%3BclusterDefaultBurstIopsPercentOfMaxIop%3D2.5& 
provider=SolidFire& 
tags=SolidFire_SAN_1& 
capacityIops=4000000& 
capacityBytes=2251799813685248& 
hypervisor=Any& 
response=json
Primary Storage Based on a Plug-in
Creating Disk Offerings with the Plug-in 
Admin-Defined QoS 
--OR-- 
Customer-Defined QoS 
Add Disk Offering 
• Orchestrated through CloudStack 
• Administrator-defined size (GBs) and QoS (IOPS) 
• Customer-defined size (GBs) and QoS (IOPS)
New Disk Offerings 
Admin-Defined QoS 
Customer-Defined QoS
Admin-Defined QoS 
--VS-- 
Customer Adds a Volume 
• Orchestrated through CloudStack 
• Based on Disk Offerings 
• Administrator-defined QoS (IOPS) 
• Customer-defined QoS (IOPS) 
Customer-Defined QoS 
Add a Volume
Customer Attaches the Volume to a VM 
• Choose the Volume to 
be Attached 
• Click Attach Disk 
• Select the VM 
Instance
What Happens on the SolidFire SAN? 
The Volume is Created 
The Volume's QoS 
Settings are Defined
What Happens on the Hypervisor? 
The Storage 
Repository is 
Created
Notes 
• ESX works in a similar fashion to XenServer. 
● Instead of a storage repository, a datastore is created. 
● Instead of a VDI inside of a storage repository, a VMDK file is created inside of a datastore. 
• Dynamic creation of backend volumes (LUNs) for data disks supported with KVM in CloudStack 4.3. 
● iscsiadm is used within the KVM agent to log in to the iSCSI target. 
● Libvirt is used to attach the new device to a VM. 
• Hypervisor snapshots (for data disks) are supported for XenServer and ESX in 4.3. 
● A new field has been introduced that allows admins to specify how much additional space, if any, 
should be set aside for hypervisor snapshots of the disk in question. 
● For SolidFire, there is very little overhead associated with creating, say, a 8 GB volume (LUN) 
versus, say, a 8 TB volume (LUN). 
● Dynamic creation of backend volumes (LUNs) for root disks supported for XenServer and ESX in 4.4 
(along with hypervisor snapshots). 
● Applicable template copied to new backend volume (LUN). 
● Due to SolidFire's de-duplication, this amounts to very little SSD interaction.
4.6 Backend-Snapshot Development 
• Adding support for backend snapshots (targeting XenServer and ESX first). 
● Due to the way a XenServer storage repository is structured, I cannot technically use SolidFire 
snapshots to implement this feature. 
● When a CloudStack snapshot of a volume is requested: 
● Create a new SolidFire volume with the same characteristics as the volume containing the VDI 
that CloudStack wants to snapshot. 
● Tell XenServer to create a VM snapshot of the VDI we're interested in. 
● Attach the new SolidFire volume (LUN) and have XenServer create a new SR on it. 
● Copy the snapshot VDI to the new SR (which is backed by the new SolidFire volume (LUN)). 
● Delete the snapshot VDI on the original SR. 
● Detach the new SR from XenServer (the SolidFire volume (LUN) now contains a copy of the 
correct VDI (with a unique UUID for the new SR and another unique UUID for the VDI)). 
(If the VDI CloudStack wants to take a snapshot of is on a detached volume, this code will attach the 
SR that contains the VDI before performing step 2. Once copying of the data from one SR to the other 
has completed, the source SR will be detached from XenServer.)
1620 Pearl Street, 
Boulder, Colorado 80302 
Phone: 720.523.3278 
Email: info@solidfire.com 
www.solidfire.com

More Related Content

What's hot

Hacking apache cloud stack
Hacking apache cloud stackHacking apache cloud stack
Hacking apache cloud stack
Nitin Mehta
 
Cloud stack overview
Cloud stack overviewCloud stack overview
Cloud stack overview
howie YU
 
Cloud stack design camp on jun 15
Cloud stack design camp on jun 15Cloud stack design camp on jun 15
Cloud stack design camp on jun 15
Isaac Chiang
 
Build public private cloud using openstack
Build public private cloud using openstackBuild public private cloud using openstack
Build public private cloud using openstack
Framgia Vietnam
 
Stratoscale Latest and Greatest
Stratoscale Latest and GreatestStratoscale Latest and Greatest
Stratoscale Latest and Greatest
Zach Lanksbury
 
Storage as a service and OpenStack Cinder
Storage as a service and OpenStack CinderStorage as a service and OpenStack Cinder
Storage as a service and OpenStack Cinder
openstackindia
 

What's hot (20)

Introduction to Apache CloudStack by David Nalley
Introduction to Apache CloudStack by David NalleyIntroduction to Apache CloudStack by David Nalley
Introduction to Apache CloudStack by David Nalley
 
CloudStack Hyderabad Meetup: How the Apache community works
CloudStack Hyderabad Meetup: How the Apache community worksCloudStack Hyderabad Meetup: How the Apache community works
CloudStack Hyderabad Meetup: How the Apache community works
 
Ceph and Apache CloudStack
Ceph and Apache CloudStackCeph and Apache CloudStack
Ceph and Apache CloudStack
 
Hacking apache cloud stack
Hacking apache cloud stackHacking apache cloud stack
Hacking apache cloud stack
 
Cloud stack overview
Cloud stack overviewCloud stack overview
Cloud stack overview
 
Muli Ben-Yehuda, Stratoscale - The Road to a Hyper-Converged OpenStack, OpenS...
Muli Ben-Yehuda, Stratoscale - The Road to a Hyper-Converged OpenStack, OpenS...Muli Ben-Yehuda, Stratoscale - The Road to a Hyper-Converged OpenStack, OpenS...
Muli Ben-Yehuda, Stratoscale - The Road to a Hyper-Converged OpenStack, OpenS...
 
Cloud stack design camp on jun 15
Cloud stack design camp on jun 15Cloud stack design camp on jun 15
Cloud stack design camp on jun 15
 
OpenStack Cinder
OpenStack CinderOpenStack Cinder
OpenStack Cinder
 
Build public private cloud using openstack
Build public private cloud using openstackBuild public private cloud using openstack
Build public private cloud using openstack
 
Stratoscale Latest and Greatest
Stratoscale Latest and GreatestStratoscale Latest and Greatest
Stratoscale Latest and Greatest
 
Openstack In Real Life
Openstack In Real LifeOpenstack In Real Life
Openstack In Real Life
 
Ceph with CloudStack
Ceph with CloudStackCeph with CloudStack
Ceph with CloudStack
 
Whats New in Apache CloudStack Version 4.5
Whats New in Apache CloudStack Version 4.5Whats New in Apache CloudStack Version 4.5
Whats New in Apache CloudStack Version 4.5
 
Storage as a service and OpenStack Cinder
Storage as a service and OpenStack CinderStorage as a service and OpenStack Cinder
Storage as a service and OpenStack Cinder
 
CloudStack vs Openstack
CloudStack vs OpenstackCloudStack vs Openstack
CloudStack vs Openstack
 
Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...
Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...
Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...
 
CloudStack vs OpenStack
CloudStack vs OpenStackCloudStack vs OpenStack
CloudStack vs OpenStack
 
On Docker and its use for LHC at CERN
On Docker and its use for LHC at CERNOn Docker and its use for LHC at CERN
On Docker and its use for LHC at CERN
 
Cloud Architect Alliance #15: Openstack
Cloud Architect Alliance #15: OpenstackCloud Architect Alliance #15: Openstack
Cloud Architect Alliance #15: Openstack
 
CloudStack Networking at CloudOpen Japan
CloudStack Networking at CloudOpen JapanCloudStack Networking at CloudOpen Japan
CloudStack Networking at CloudOpen Japan
 

Similar to Guaranteeing Storage Performance by Mike Tutkowski

Hyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and TricksHyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and Tricks
Amit Gatenyo
 

Similar to Guaranteeing Storage Performance by Mike Tutkowski (20)

Guaranteeing CloudStack Storage Performance
Guaranteeing CloudStack Storage Performance Guaranteeing CloudStack Storage Performance
Guaranteeing CloudStack Storage Performance
 
Primary Storage in CloudStack by Mike Tutkowski
Primary Storage in CloudStack by Mike TutkowskiPrimary Storage in CloudStack by Mike Tutkowski
Primary Storage in CloudStack by Mike Tutkowski
 
vSphere
vSpherevSphere
vSphere
 
VMworld Europe 2014: Virtual SAN Architecture Deep Dive
VMworld Europe 2014: Virtual SAN Architecture Deep DiveVMworld Europe 2014: Virtual SAN Architecture Deep Dive
VMworld Europe 2014: Virtual SAN Architecture Deep Dive
 
VMworld 2014: Virtual SAN Architecture Deep Dive
VMworld 2014: Virtual SAN Architecture Deep DiveVMworld 2014: Virtual SAN Architecture Deep Dive
VMworld 2014: Virtual SAN Architecture Deep Dive
 
Solid fire cloudstack storage overview - CloudStack European User Group
Solid fire cloudstack storage overview - CloudStack European User GroupSolid fire cloudstack storage overview - CloudStack European User Group
Solid fire cloudstack storage overview - CloudStack European User Group
 
CloudStack Meetup London - Primary Storage Presentation by SolidFire
CloudStack Meetup London - Primary Storage Presentation by SolidFire CloudStack Meetup London - Primary Storage Presentation by SolidFire
CloudStack Meetup London - Primary Storage Presentation by SolidFire
 
Nexenta Powered by Apache CloudStack from Iliyas Shirol
Nexenta Powered by Apache CloudStack from Iliyas ShirolNexenta Powered by Apache CloudStack from Iliyas Shirol
Nexenta Powered by Apache CloudStack from Iliyas Shirol
 
VMworld 2013: Successfully Virtualize Microsoft Exchange Server
VMworld 2013: Successfully Virtualize Microsoft Exchange Server VMworld 2013: Successfully Virtualize Microsoft Exchange Server
VMworld 2013: Successfully Virtualize Microsoft Exchange Server
 
VMware Virtual SAN Presentation
VMware Virtual SAN PresentationVMware Virtual SAN Presentation
VMware Virtual SAN Presentation
 
VCFPD4_M03_Domain Design-AR.pptx
VCFPD4_M03_Domain Design-AR.pptxVCFPD4_M03_Domain Design-AR.pptx
VCFPD4_M03_Domain Design-AR.pptx
 
Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1Open vStorage Road show 2015 Q1
Open vStorage Road show 2015 Q1
 
Road show 2015 triangle meetup
Road show 2015 triangle meetupRoad show 2015 triangle meetup
Road show 2015 triangle meetup
 
VMworld 2013: IBM Solutions for VMware Virtual SAN
VMworld 2013: IBM Solutions for VMware Virtual SAN VMworld 2013: IBM Solutions for VMware Virtual SAN
VMworld 2013: IBM Solutions for VMware Virtual SAN
 
3487570
34875703487570
3487570
 
DR-to-the-Cloud Best Practices
DR-to-the-Cloud Best PracticesDR-to-the-Cloud Best Practices
DR-to-the-Cloud Best Practices
 
VMworld 2013: Lowering TCO for Virtual Desktops with VMware View and VMware V...
VMworld 2013: Lowering TCO for Virtual Desktops with VMware View and VMware V...VMworld 2013: Lowering TCO for Virtual Desktops with VMware View and VMware V...
VMworld 2013: Lowering TCO for Virtual Desktops with VMware View and VMware V...
 
ProfessionalVMware BrownBag VCP5 Section3: Storage
ProfessionalVMware BrownBag VCP5 Section3: StorageProfessionalVMware BrownBag VCP5 Section3: Storage
ProfessionalVMware BrownBag VCP5 Section3: Storage
 
3. v sphere big data extensions
3. v sphere big data extensions3. v sphere big data extensions
3. v sphere big data extensions
 
Hyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and TricksHyper-V Best Practices & Tips and Tricks
Hyper-V Best Practices & Tips and Tricks
 

More from buildacloud

Policy Based SDN Solution for DC and Branch Office by Suresh Boddapati
Policy Based SDN Solution for DC and Branch Office by Suresh BoddapatiPolicy Based SDN Solution for DC and Branch Office by Suresh Boddapati
Policy Based SDN Solution for DC and Branch Office by Suresh Boddapati
buildacloud
 
Cloud Application Blueprints with Apache Brooklyn by Alex Henevald
Cloud Application Blueprints with Apache Brooklyn by Alex HenevaldCloud Application Blueprints with Apache Brooklyn by Alex Henevald
Cloud Application Blueprints with Apache Brooklyn by Alex Henevald
buildacloud
 
Managing infrastructure with Application Policy by Mike Cohen
Managing infrastructure with Application Policy by Mike CohenManaging infrastructure with Application Policy by Mike Cohen
Managing infrastructure with Application Policy by Mike Cohen
buildacloud
 
Intro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew KirchIntro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew Kirch
buildacloud
 
State of the cloud by reuven cohen
State of the cloud by reuven cohenState of the cloud by reuven cohen
State of the cloud by reuven cohen
buildacloud
 
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
buildacloud
 
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
buildacloud
 
CloudStack University by Sebastien Goasguen
CloudStack University by Sebastien GoasguenCloudStack University by Sebastien Goasguen
CloudStack University by Sebastien Goasguen
buildacloud
 
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian StadilBuilding Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
buildacloud
 
Cloudstack Continuous Delivery
Cloudstack Continuous DeliveryCloudstack Continuous Delivery
Cloudstack Continuous Delivery
buildacloud
 

More from buildacloud (20)

Policy Based SDN Solution for DC and Branch Office by Suresh Boddapati
Policy Based SDN Solution for DC and Branch Office by Suresh BoddapatiPolicy Based SDN Solution for DC and Branch Office by Suresh Boddapati
Policy Based SDN Solution for DC and Branch Office by Suresh Boddapati
 
L4-L7 services for SDN and NVF by Youcef Laribi
L4-L7 services for SDN and NVF by Youcef LaribiL4-L7 services for SDN and NVF by Youcef Laribi
L4-L7 services for SDN and NVF by Youcef Laribi
 
Intro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew KirchIntro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew Kirch
 
Cloud Application Blueprints with Apache Brooklyn by Alex Henevald
Cloud Application Blueprints with Apache Brooklyn by Alex HenevaldCloud Application Blueprints with Apache Brooklyn by Alex Henevald
Cloud Application Blueprints with Apache Brooklyn by Alex Henevald
 
Managing infrastructure with Application Policy by Mike Cohen
Managing infrastructure with Application Policy by Mike CohenManaging infrastructure with Application Policy by Mike Cohen
Managing infrastructure with Application Policy by Mike Cohen
 
Intro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew KirchIntro to Zenoss by Andrew Kirch
Intro to Zenoss by Andrew Kirch
 
Monitoring CloudStack in context with Converged Infrastructure by Mike Turnlund
Monitoring CloudStack in context with Converged Infrastructure by Mike TurnlundMonitoring CloudStack in context with Converged Infrastructure by Mike Turnlund
Monitoring CloudStack in context with Converged Infrastructure by Mike Turnlund
 
Rest api design by george reese
Rest api design by george reeseRest api design by george reese
Rest api design by george reese
 
Enterprise grade firewall and ssl termination to ac by will stevens
Enterprise grade firewall and ssl termination to ac by will stevensEnterprise grade firewall and ssl termination to ac by will stevens
Enterprise grade firewall and ssl termination to ac by will stevens
 
State of the cloud by reuven cohen
State of the cloud by reuven cohenState of the cloud by reuven cohen
State of the cloud by reuven cohen
 
Securing Your Cloud With the Xen Hypervisor by Russell Pavlicek
Securing Your Cloud With the Xen Hypervisor by Russell PavlicekSecuring Your Cloud With the Xen Hypervisor by Russell Pavlicek
Securing Your Cloud With the Xen Hypervisor by Russell Pavlicek
 
DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack
 
Cloud Network Virtualization with Juniper Contrail
Cloud Network Virtualization with Juniper ContrailCloud Network Virtualization with Juniper Contrail
Cloud Network Virtualization with Juniper Contrail
 
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
Ian rae panel cloud stack & cloud storage where are we at, and where do we ne...
 
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
Troubleshooting Strategies for CloudStack Installations by Kirk Kosinski
 
CloudStack University by Sebastien Goasguen
CloudStack University by Sebastien GoasguenCloudStack University by Sebastien Goasguen
CloudStack University by Sebastien Goasguen
 
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian StadilBuilding Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
Building Scalable, Resilient Infrastructure on CloudStack by Sebastian Stadil
 
Cloudstack Continuous Delivery
Cloudstack Continuous DeliveryCloudstack Continuous Delivery
Cloudstack Continuous Delivery
 
SDN in CloudStack
SDN in CloudStackSDN in CloudStack
SDN in CloudStack
 
Apalia/Amysta Cloud Usage Metering and Billing
Apalia/Amysta Cloud Usage Metering and BillingApalia/Amysta Cloud Usage Metering and Billing
Apalia/Amysta Cloud Usage Metering and Billing
 

Guaranteeing Storage Performance by Mike Tutkowski

  • 2. Mike Tutkowski - Full-time CloudStack software engineer, CloudStack PMC member - Focused on CloudStack's storage component SolidFire (http://solidfire.com) - Based out of Boulder, CO - Develop a clustered, scale-out SAN technology (using industry-standard hardware) - Built from the ground up to support guaranteed Quality of Service (QoS) on a per-volume (LUN) basis (min, max, and burst IOPS per volume) - All SSD architecture (no spinning disks) - Leverage compression, de-duplication, and thin provisioning (all inline) on a 4-KB block boundary across the entire cluster to drive down cost/GB to be on par with traditional disk-based SANs - Rest-like API to enable automation of all aspects of the SAN
  • 3. CloudStack from the top down (prior to 4.2)
  • 4. Primary Storage Secondary Storage Objectives Storage for VMs (root and data disks) Data to be stored for future retrieval Use Cases • Production Applications • Traditional IT Systems • Database-Driven Apps • Messaging / Collaboration • Dev/Test Systems • VM Templates • ISO Images • Backups of Volumes Workloads • High-Change Content • Smaller, Random R/W • Higher / “Bursty” IO • Typically More Static Content • Larger, Sequential IO (more read than write) • Lower IOPS Storage Use Cases & Workloads
  • 5. What is Primary Storage (Pre 4.2)? • Primary Storage is associated with a cluster • A cluster can access more than one Primary Storage • Primary Storage can be shared among hosts or local to a host ● Primary Storage stores the disk volumes (both root and data disks) for all the VMs in that cluster • Depending on hypervisor type, there are several ways to configure Primary Storage (we shall take a look at XenServer)
  • 6. Provisioning Primary Storage (Pre 4.2) • Admin allocates space ahead of time on the storage system (Example: Create a volume on the SolidFire SAN) • Admin defines a storage resource in the hypervisor (Example: Create a XenServer Storage Repository) • Admin defines a storage pool in CloudStack (Example: Create Primary Storage in CloudStack for a cluster) • Admin creates a Compute Offering using the Primary Storage (Example: 1 vCPU, 2 GB RAM, 50 GB)
  • 7. Allocate Storage on the SolidFire SAN
  • 8. Define the Storage Resource in the Hypervisor Select the type of the storage repository Name the storage repository Map the storage repository to the volume Storage repository is now available in the hypervisor
  • 9. Define a Primary Storage Pool in CloudStack Add Primary Storage Define Primary Storage Primary Storage Available for Use
  • 10. Create a Compute Offering in CloudStack Add Compute Offering Define Compute Offering Compute Offering Available for Use
  • 11. Primary Storage in CloudStack >= 4.2 • Fully automated provisioning through CloudStack • Dynamic volume creation for VM root disks and additional data disks • With SolidFire each volume receives guaranteed IOPS
  • 12. My Specific Needs from the Plug-in Provide a way to expose vendor-unique features within CloudStack Eliminate the need for customers to create additional orchestration logic to provision storage Have the ability to defer the creation of a volume until the moment the end user elects to execute a Compute or Disk Offering
  • 13. Creating Primary Storage Based on a Plug-in http://127.0.0.1:8080/client/api?command=createStoragePool& scope=zone& zoneId=cf4e6ddf-8ae7-4194-8270-d46733a52b55& name=SolidFire_121258566& url=MVIP%3D192.168.138.180%3A443%3BSVIP%3D192.168.56.7%3BclusterAdminUsername%3Dadmin%3BclusterAdminPassword %3Dpassword%3BclusterDefaultMinIops%3D200%3BclusterDefaultMaxIops%3D300%3BclusterDefaultBurstIopsPercentOfMaxIop%3D2.5& provider=SolidFire& tags=SolidFire_SAN_1& capacityIops=4000000& capacityBytes=2251799813685248& hypervisor=Any& response=json
  • 14. Primary Storage Based on a Plug-in
  • 15. Creating Disk Offerings with the Plug-in Admin-Defined QoS --OR-- Customer-Defined QoS Add Disk Offering • Orchestrated through CloudStack • Administrator-defined size (GBs) and QoS (IOPS) • Customer-defined size (GBs) and QoS (IOPS)
  • 16. New Disk Offerings Admin-Defined QoS Customer-Defined QoS
  • 17. Admin-Defined QoS --VS-- Customer Adds a Volume • Orchestrated through CloudStack • Based on Disk Offerings • Administrator-defined QoS (IOPS) • Customer-defined QoS (IOPS) Customer-Defined QoS Add a Volume
  • 18. Customer Attaches the Volume to a VM • Choose the Volume to be Attached • Click Attach Disk • Select the VM Instance
  • 19. What Happens on the SolidFire SAN? The Volume is Created The Volume's QoS Settings are Defined
  • 20. What Happens on the Hypervisor? The Storage Repository is Created
  • 21. Notes • ESX works in a similar fashion to XenServer. ● Instead of a storage repository, a datastore is created. ● Instead of a VDI inside of a storage repository, a VMDK file is created inside of a datastore. • Dynamic creation of backend volumes (LUNs) for data disks supported with KVM in CloudStack 4.3. ● iscsiadm is used within the KVM agent to log in to the iSCSI target. ● Libvirt is used to attach the new device to a VM. • Hypervisor snapshots (for data disks) are supported for XenServer and ESX in 4.3. ● A new field has been introduced that allows admins to specify how much additional space, if any, should be set aside for hypervisor snapshots of the disk in question. ● For SolidFire, there is very little overhead associated with creating, say, a 8 GB volume (LUN) versus, say, a 8 TB volume (LUN). ● Dynamic creation of backend volumes (LUNs) for root disks supported for XenServer and ESX in 4.4 (along with hypervisor snapshots). ● Applicable template copied to new backend volume (LUN). ● Due to SolidFire's de-duplication, this amounts to very little SSD interaction.
  • 22. 4.6 Backend-Snapshot Development • Adding support for backend snapshots (targeting XenServer and ESX first). ● Due to the way a XenServer storage repository is structured, I cannot technically use SolidFire snapshots to implement this feature. ● When a CloudStack snapshot of a volume is requested: ● Create a new SolidFire volume with the same characteristics as the volume containing the VDI that CloudStack wants to snapshot. ● Tell XenServer to create a VM snapshot of the VDI we're interested in. ● Attach the new SolidFire volume (LUN) and have XenServer create a new SR on it. ● Copy the snapshot VDI to the new SR (which is backed by the new SolidFire volume (LUN)). ● Delete the snapshot VDI on the original SR. ● Detach the new SR from XenServer (the SolidFire volume (LUN) now contains a copy of the correct VDI (with a unique UUID for the new SR and another unique UUID for the VDI)). (If the VDI CloudStack wants to take a snapshot of is on a detached volume, this code will attach the SR that contains the VDI before performing step 2. Once copying of the data from one SR to the other has completed, the source SR will be detached from XenServer.)
  • 23. 1620 Pearl Street, Boulder, Colorado 80302 Phone: 720.523.3278 Email: info@solidfire.com www.solidfire.com

Editor's Notes

  1. <number>