CloudStack Meetup London - Primary Storage Presentation by SolidFire


Published on

CloudStack Meetup in London. Primary Storage CloudStack by SolidFIre

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CloudStack Meetup London - Primary Storage Presentation by SolidFire

  1. 1. Primary Storage in CloudStack
  2. 2. Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines as a highly available, highly scalable Infrastructure-as-a-Service (IaaS) cloud computing platform. Compute, Network, Storage Automation What is CloudStack?
  3. 3. CloudStack from the top down
  4. 4. Primary Storage Secondary Storage Objectives Storage for running VM disk volumes on a host Data to be stored for future retrieval Use Cases • Production Applications • Traditional IT Systems • Database Driven Apps • Messaging / Collaboration • Dev/Test Systems • VM Templates • ISO Images • Disk Volume Snapshots • Backup / Archive • Image Repository Workloads • High Change Content • Smaller, Random R/W • Higher / “Bursty” IO • Typically More Static Content • Larger, Sequential R/W • Lower IOPS Storage Use Cases & Workloads
  5. 5. • Primary Storage stores the disk volumes (both root and data disks) for all the VMs in that cluster • Primary Storage is associated with a cluster • A cluster can access more than one Primary Storage pool • Primary Storage can be shared among hosts or local to a host • Depending on hypervisor type, there are several ways to configure Primary Storage (we shall take a look at XenServer) What is Primary Storage?
  6. 6. • Admin allocates space ahead of time on the storage system (Example: Create a volume on the SolidFire SAN) • Admin defines a storage resource in the hypervisor (Example: Create a XenServer Storage Repository) • Admin defines a storage pool in CloudStack (Example: Create Primary Storage in CloudStack for a cluster) • Admin creates a compute offering using the Primary Storage (Example: 1 vCPU, 2 GB RAM, 50GB) Provisioning Primary Storage Today (CloudStack Version <=4.1)
  7. 7. Allocate Storage on the SolidFire SAN
  8. 8. Define the Storage Resource in the Hypervisor Select the type of the storage resource Name the storage resource Resource is now available in the Hypervisor Map the resource to the storage volume
  9. 9. Define a Primary Storage Pool in CloudStack Add Primary Storage Define the Storage Pool Primary Storage Available for Use
  10. 10. Create a Compute Offering in CloudStack New Compute Offering Available Add a Compute Offering Define the Compute Offering
  11. 11. A Glimpse into the Future of Primary Storage • Fully automated provisioning through CloudPlatform / CloudStack • Dynamic volume creation for VM root disks and additional data disks • With SolidFire each volume receives guaranteed IOPS
  12. 12. Provide a way to expose vendor unique features within CloudStack Eliminate the need for customer’s to create additional orchestration logic to provision storage Have the ability to defer the creation of a volume until the moment the end user elects to execute a Compute or Disk Offering My Specific Needs from the Plug-in
  13. 13. Creating Primary Storage with the Plug-in --OR-- Admin Defined QoS Customer Defined QoS • Orchestrated through CloudStack • Administrator defined size (GBs) and QoS (IOPS) • Customers defined capacity (GBs) and QoS (IOPS) Add Primary Storage in CloudStack
  14. 14. New Disk Offerings Admin Defined QoS Customer Defined QoS
  15. 15. --VS-- Admin Defined QoS • Orchestrated through CloudStack • Based on Disk Offerings • Administrator defined QoS (IOPS) • Customers defined QoS (IOPS) Customer Adds a Volume Customer Defined QoS Add a Volume to a VM
  16. 16. Customer Attaches the Volume to a VM • Choose the Volume to be Attached • Click Attach Disk • Select the Instance
  17. 17. What Happens on SolidFire? The Volume is Created on the SolidFire SAN The Volume QoS Settings are Defined
  18. 18. What Happens on the Hypervisor? The Storage Resource is Created
  19. 19. A CloudStack storage plug-in is divided into three components: Provider: Logic related to the plug-in in general (ex: name of plug-in). Life Cycle: Logic related to life cycle (ex: creation) of a given storage system (ex: a single SolidFire SAN). Driver: Logic related to creating and deleting volumes on the storage system. Must add a dependency in the client/pom.xml file as such: <dependency> <groupId>org.apache.cloudstack</groupId> <artifactId>cloud-plugin-storage-volume-solidfire</artifactId> <version>${project.version}</version> </dependency> So…how do you actually make a plug-in?
  20. 20.  Must implement the PrimaryDataStoreProvider interface  Provides CloudStack with the plug-in's name as well as the Life Cycle and Driver objects the storage system uses  Must be listed in the file (Spring Framework related)  A single instance of this class is created for CloudStack About the Provider
  21. 21. public interface PrimaryDataStoreProvider extends DataStoreProvider { } public interface DataStoreProvider { public static enum DataStoreProviderType { PRIMARY, IMAGE } public DataStoreLifeCycle getDataStoreLifeCycle(); public DataStoreDriver getDataStoreDriver(); public HypervisorHostListener getHostListener(); public String getName(); public boolean configure(Map<String, Object> params); public Set<DataStoreProviderType> getTypes(); } The Provider Interface
  22. 22. public class SolidfirePrimaryDataStoreProvider implements PrimaryDataStoreProvider { private final String providerName = "SolidFire"; protected PrimaryDataStoreDriver driver; protected HypervisorHostListener listener; protected DataStoreLifeCycle lifecyle; @Override public String getName() { return providerName; } @Override public DataStoreLifeCycle getDataStoreLifeCycle() { return lifecyle; } @Override public boolean configure(Map<String, Object> params) { lifecyle = ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class); driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.class); listener = ComponentContext.inject(DefaultHostListener.class); return true; } Provider Implementation
  23. 23. Notes:  client/tomcatconf/  Each provider adds a single line.  “id” is only used by Spring Framework (not by CS Management Server). Recommend just providing a descriptive name. Example: <bean id="ClassicalPrimaryDataStoreProvider" class="" /> <bean id="solidFireDataStoreProvider" class="" /> Provider Configuration
  24. 24.  Must implement the PrimaryDataStoreLifeCycle interface.  Handles the creation, deletion, etc. of a storage system (ex: SAN) in CloudStack.  The initialize method of the Life Cycle object adds a row into the cloud.storage_pool table to represent a newly added storage system. About the Life Cycle
  25. 25. public interface PrimaryDataStoreLifeCycle extends DataStoreLifeCycle { } public interface DataStoreLifeCycle { public DataStore initialize(Map<String, Object> dsInfos); public boolean attachCluster(DataStore store, ClusterScope scope); public boolean attachHost(DataStore store, HostScope scope, StoragePoolInfo existingInfo); boolean attachZone(DataStore dataStore, ZoneScope scope); public boolean dettach(); public boolean unmanaged(); public boolean maintain(DataStore store); public boolean cancelMaintain(DataStore store); public boolean deleteDataStore(DataStore store); } The Life Cycle Interface
  26. 26. @Override public DataStore initialize(Map<String, Object> dsInfos) { String url = (String)dsInfos.get("url"); String uuid = getUuid(); // maybe base this off of something already unique Long zoneId = (Long)dsInfos.get("zoneId"); String storagePoolName = (String) dsInfos.get("name"); String providerName = (String)dsInfos.get("providerName"); PrimaryDataStoreParameters parameters = new PrimaryDataStoreParameters(); parameters.setHost(""); // really get from URL parameters.setPort(3260); // really get from URL parameters.setPath(url); parameters.setType(StoragePoolType.IscsiLUN); parameters.setUuid(uuid); parameters.setZoneId(zoneId); parameters.setName(storagePoolName); parameters.setProviderName(providerName); return dataStoreHelper.createPrimaryDataStore(parameters); } Life Cycle Implementation
  27. 27.  Must implement the PrimaryDataStoreDriver interface.  Your opportunity to create or delete a volume and to add a row to or delete a row from the cloud.volumes table.  A single instance of this class is responsible for creating and deleting volumes on all storage systems of the same type. About the Driver
  28. 28. public interface PrimaryDataStoreDriver extends DataStoreDriver { public void takeSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CreateCmdResult> callback); public void revertSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CommandResult> callback); } public interface DataStoreDriver { public String grantAccess(DataObject data, EndPoint ep); public boolean revokeAccess(DataObject data, EndPoint ep); public Set<DataObject> listObjects(DataStore store); public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback); public void deleteAsync(DataObject data, AsyncCompletionCallback<CommandResult> callback); public void copyAsync(DataObject srcdata, DataObject destData, AsyncCompletionCallback<CopyCommandResult> callback); public boolean canCopy(DataObject srcData, DataObject destData); public void resize(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback); } The Driver Interface
  29. 29. public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback) { String iqn = null; try { VolumeInfo volumeInfo = (VolumeInfo)data; iqn = createSolidFireVolume(volumeInfo); VolumeVO volume = new VolumeVO(volumeInfo); volume.setPath(iqn); volumeDao.persist(volume); } catch (Exception e) { s_logger.debug("Failed to create volume (Exception)", e); } CreateCmdResult result = new CreateCmdResult(iqn, errMsg == null ? data.getSize() : null); result.setResult(errMsg); callback.complete(result); } Driver Implementation
  30. 30. Ask the CS MS to provide a list of all storage providers Ask the CS MS to add a Primary Storage (a row in the cloud.storage_pool table) based on your plug-in (ex: make CloudStack aware of a SolidFire SAN) 4afc-a9ee- 8cba82b43474&name=SolidFire_831569365&url=MVIP%3A192.168.138.180%3BSVIP%3A10.10.7.1& provider=SolidFire&response=json Ask the CS MS to provide a list of all Primary Storages API Calls
  31. 31. 1620 Pearl Street, Boulder, Colorado 80302 Phone: 720.523.3278 Email: