Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Guaranteeing Storage Performance in 
CloudStack
Mike Tutkowski 
- Full-time CloudStack software engineer, CloudStack PMC member 
- Focused on CloudStack's storage compone...
CloudStack from 
the top down 
(prior to 4.2)
Primary Storage Secondary Storage 
Objectives Storage for VMs (root and data disks) Data to be stored for future retrieval...
What is Primary Storage (Pre 4.2)? 
• Primary Storage is associated with a cluster 
• A cluster can access more than one P...
Provisioning Primary Storage (Pre 4.2) 
• Admin allocates space ahead of time on the storage system 
(Example: Create a vo...
Allocate 
Storage on the 
SolidFire SAN
Define the Storage Resource in the Hypervisor 
Select the type of the storage repository 
Name the storage repository 
Map...
Define a Primary Storage Pool in CloudStack 
Add Primary Storage Define Primary Storage 
Primary Storage Available for Use
Create a Compute Offering in CloudStack 
Add Compute Offering Define Compute Offering 
Compute Offering Available for Use
Primary Storage in CloudStack >= 4.2 
• Fully automated 
provisioning through 
CloudStack 
• Dynamic volume creation 
for ...
My Specific Needs from the Plug-in 
Provide a way to expose vendor-unique features within CloudStack 
Eliminate the need f...
Creating Primary Storage Based on a Plug-in 
http://127.0.0.1:8080/client/api?command=createStoragePool& 
scope=zone& 
zon...
Primary Storage Based on a Plug-in
Creating Disk Offerings with the Plug-in 
Admin-Defined QoS 
--OR-- 
Customer-Defined QoS 
Add Disk Offering 
• Orchestrat...
New Disk Offerings 
Admin-Defined QoS 
Customer-Defined QoS
Admin-Defined QoS 
--VS-- 
Customer Adds a Volume 
• Orchestrated through CloudStack 
• Based on Disk Offerings 
• Adminis...
Customer Attaches the Volume to a VM 
• Choose the Volume to 
be Attached 
• Click Attach Disk 
• Select the VM 
Instance
What Happens on the SolidFire SAN? 
The Volume is Created 
The Volume's QoS 
Settings are Defined
What Happens on the Hypervisor? 
The Storage 
Repository is 
Created
Notes 
• ESX works in a similar fashion to XenServer. 
● Instead of a storage repository, a datastore is created. 
● Inste...
4.6 Backend-Snapshot Development 
• Adding support for backend snapshots (targeting XenServer and ESX first). 
● Due to th...
1620 Pearl Street, 
Boulder, Colorado 80302 
Phone: 720.523.3278 
Email: info@solidfire.com 
www.solidfire.com
Upcoming SlideShare
Loading in …5
×

Guaranteeing Storage Performance by Mike Tutkowski

1,127 views

Published on

This session will introduce the basics of primary storage in CloudStack. Additionally, I discuss the challenges of guaranteeing storage performance in a cloud and how by leveraging the latest enhancements to CloudStack, storage administrators can deliver consistent, repeatable performance to 10s, 100s or 1,000s of application workloads in parallel. I'll review the CloudStack enhancements in detail, outline the management benefits they provide and discuss common go-to-market approaches.

About Mike Tutkowski

Mike Tutkowski, a member of the CloudStack PMC, develops software for the Apache Software Foundation's CloudStack project to help drive improvements in its storage component and to integrate SolidFire more deeply into the product.

  • Be the first to comment

  • Be the first to like this

Guaranteeing Storage Performance by Mike Tutkowski

  1. 1. Guaranteeing Storage Performance in CloudStack
  2. 2. Mike Tutkowski - Full-time CloudStack software engineer, CloudStack PMC member - Focused on CloudStack's storage component SolidFire (http://solidfire.com) - Based out of Boulder, CO - Develop a clustered, scale-out SAN technology (using industry-standard hardware) - Built from the ground up to support guaranteed Quality of Service (QoS) on a per-volume (LUN) basis (min, max, and burst IOPS per volume) - All SSD architecture (no spinning disks) - Leverage compression, de-duplication, and thin provisioning (all inline) on a 4-KB block boundary across the entire cluster to drive down cost/GB to be on par with traditional disk-based SANs - Rest-like API to enable automation of all aspects of the SAN
  3. 3. CloudStack from the top down (prior to 4.2)
  4. 4. Primary Storage Secondary Storage Objectives Storage for VMs (root and data disks) Data to be stored for future retrieval Use Cases • Production Applications • Traditional IT Systems • Database-Driven Apps • Messaging / Collaboration • Dev/Test Systems • VM Templates • ISO Images • Backups of Volumes Workloads • High-Change Content • Smaller, Random R/W • Higher / “Bursty” IO • Typically More Static Content • Larger, Sequential IO (more read than write) • Lower IOPS Storage Use Cases & Workloads
  5. 5. What is Primary Storage (Pre 4.2)? • Primary Storage is associated with a cluster • A cluster can access more than one Primary Storage • Primary Storage can be shared among hosts or local to a host ● Primary Storage stores the disk volumes (both root and data disks) for all the VMs in that cluster • Depending on hypervisor type, there are several ways to configure Primary Storage (we shall take a look at XenServer)
  6. 6. Provisioning Primary Storage (Pre 4.2) • Admin allocates space ahead of time on the storage system (Example: Create a volume on the SolidFire SAN) • Admin defines a storage resource in the hypervisor (Example: Create a XenServer Storage Repository) • Admin defines a storage pool in CloudStack (Example: Create Primary Storage in CloudStack for a cluster) • Admin creates a Compute Offering using the Primary Storage (Example: 1 vCPU, 2 GB RAM, 50 GB)
  7. 7. Allocate Storage on the SolidFire SAN
  8. 8. Define the Storage Resource in the Hypervisor Select the type of the storage repository Name the storage repository Map the storage repository to the volume Storage repository is now available in the hypervisor
  9. 9. Define a Primary Storage Pool in CloudStack Add Primary Storage Define Primary Storage Primary Storage Available for Use
  10. 10. Create a Compute Offering in CloudStack Add Compute Offering Define Compute Offering Compute Offering Available for Use
  11. 11. Primary Storage in CloudStack >= 4.2 • Fully automated provisioning through CloudStack • Dynamic volume creation for VM root disks and additional data disks • With SolidFire each volume receives guaranteed IOPS
  12. 12. My Specific Needs from the Plug-in Provide a way to expose vendor-unique features within CloudStack Eliminate the need for customers to create additional orchestration logic to provision storage Have the ability to defer the creation of a volume until the moment the end user elects to execute a Compute or Disk Offering
  13. 13. Creating Primary Storage Based on a Plug-in http://127.0.0.1:8080/client/api?command=createStoragePool& scope=zone& zoneId=cf4e6ddf-8ae7-4194-8270-d46733a52b55& name=SolidFire_121258566& url=MVIP%3D192.168.138.180%3A443%3BSVIP%3D192.168.56.7%3BclusterAdminUsername%3Dadmin%3BclusterAdminPassword %3Dpassword%3BclusterDefaultMinIops%3D200%3BclusterDefaultMaxIops%3D300%3BclusterDefaultBurstIopsPercentOfMaxIop%3D2.5& provider=SolidFire& tags=SolidFire_SAN_1& capacityIops=4000000& capacityBytes=2251799813685248& hypervisor=Any& response=json
  14. 14. Primary Storage Based on a Plug-in
  15. 15. Creating Disk Offerings with the Plug-in Admin-Defined QoS --OR-- Customer-Defined QoS Add Disk Offering • Orchestrated through CloudStack • Administrator-defined size (GBs) and QoS (IOPS) • Customer-defined size (GBs) and QoS (IOPS)
  16. 16. New Disk Offerings Admin-Defined QoS Customer-Defined QoS
  17. 17. Admin-Defined QoS --VS-- Customer Adds a Volume • Orchestrated through CloudStack • Based on Disk Offerings • Administrator-defined QoS (IOPS) • Customer-defined QoS (IOPS) Customer-Defined QoS Add a Volume
  18. 18. Customer Attaches the Volume to a VM • Choose the Volume to be Attached • Click Attach Disk • Select the VM Instance
  19. 19. What Happens on the SolidFire SAN? The Volume is Created The Volume's QoS Settings are Defined
  20. 20. What Happens on the Hypervisor? The Storage Repository is Created
  21. 21. Notes • ESX works in a similar fashion to XenServer. ● Instead of a storage repository, a datastore is created. ● Instead of a VDI inside of a storage repository, a VMDK file is created inside of a datastore. • Dynamic creation of backend volumes (LUNs) for data disks supported with KVM in CloudStack 4.3. ● iscsiadm is used within the KVM agent to log in to the iSCSI target. ● Libvirt is used to attach the new device to a VM. • Hypervisor snapshots (for data disks) are supported for XenServer and ESX in 4.3. ● A new field has been introduced that allows admins to specify how much additional space, if any, should be set aside for hypervisor snapshots of the disk in question. ● For SolidFire, there is very little overhead associated with creating, say, a 8 GB volume (LUN) versus, say, a 8 TB volume (LUN). ● Dynamic creation of backend volumes (LUNs) for root disks supported for XenServer and ESX in 4.4 (along with hypervisor snapshots). ● Applicable template copied to new backend volume (LUN). ● Due to SolidFire's de-duplication, this amounts to very little SSD interaction.
  22. 22. 4.6 Backend-Snapshot Development • Adding support for backend snapshots (targeting XenServer and ESX first). ● Due to the way a XenServer storage repository is structured, I cannot technically use SolidFire snapshots to implement this feature. ● When a CloudStack snapshot of a volume is requested: ● Create a new SolidFire volume with the same characteristics as the volume containing the VDI that CloudStack wants to snapshot. ● Tell XenServer to create a VM snapshot of the VDI we're interested in. ● Attach the new SolidFire volume (LUN) and have XenServer create a new SR on it. ● Copy the snapshot VDI to the new SR (which is backed by the new SolidFire volume (LUN)). ● Delete the snapshot VDI on the original SR. ● Detach the new SR from XenServer (the SolidFire volume (LUN) now contains a copy of the correct VDI (with a unique UUID for the new SR and another unique UUID for the VDI)). (If the VDI CloudStack wants to take a snapshot of is on a detached volume, this code will attach the SR that contains the VDI before performing step 2. Once copying of the data from one SR to the other has completed, the source SR will be detached from XenServer.)
  23. 23. 1620 Pearl Street, Boulder, Colorado 80302 Phone: 720.523.3278 Email: info@solidfire.com www.solidfire.com

×