Mike from Solidfire talks about storage in Cloudstack at the European CloudStack usergroup

1. Primary Storage in CloudStack

2. Apache CloudStack is open source software designed to
deploy and manage large networks of virtual machines as a
highly available, highly scalable Infrastructure-as-a-Service
(IaaS) cloud computing platform.
Compute, Network, Storage
Automation
What is CloudStack?

3. CloudStack from
the top down

4. Primary Storage Secondary Storage
Objectives Storage for running VM disk volumes
on a host
Data to be stored for future retrieval
Use Cases • Production Applications
• Traditional IT Systems
• Database Driven Apps
• Messaging / Collaboration
• Dev/Test Systems
• VM Templates
• ISO Images
• Disk Volume Snapshots
• Backup / Archive
• Image Repository
Workloads • High Change Content
• Smaller, Random R/W
• Higher / “Bursty” IO
• Typically More Static Content
• Larger, Sequential R/W
• Lower IOPS
Storage Use Cases & Workloads

5. • Primary Storage stores the disk volumes (both root and data disks) for all
the VMs in that cluster
• Primary Storage is associated with a cluster
• A cluster can access more than one Primary Storage pool
• Primary Storage can be shared among hosts or local to a host
• Depending on hypervisor type, there are several ways to configure
Primary Storage (we shall take a look at XenServer)
What is Primary Storage?

6. • Admin allocates space ahead of time on the storage system
(Example: Create a volume on the SolidFire SAN)
• Admin defines a storage resource in the hypervisor
(Example: Create a XenServer Storage Repository)
• Admin defines a storage pool in CloudStack
(Example: Create Primary Storage in CloudStack for a cluster)
• Admin creates a compute offering using the Primary Storage
(Example: 1 vCPU, 2 GB RAM, 50GB)
Provisioning Primary Storage Today
(CloudStack Version <=4.1)

7. Allocate
Storage on the
SolidFire SAN

8. Define the Storage Resource in the Hypervisor
Select the type of the storage resource
Name the storage resource
Resource is now available in the Hypervisor
Map the resource to the storage volume

9. Define a Primary Storage Pool in CloudStack
Add Primary Storage Define the Storage Pool
Primary Storage Available for Use

10. Create a Compute Offering in CloudStack
New Compute Offering Available
Add a Compute Offering Define the Compute Offering

11. A Glimpse into the Future of Primary Storage
• Fully automated
provisioning through
CloudPlatform / CloudStack
• Dynamic volume creation
for VM root disks and
additional data disks
• With SolidFire each volume
receives guaranteed IOPS

12. Provide a way to expose vendor unique features within CloudStack
Eliminate the need for customer’s to create additional orchestration logic to
provision storage
Have the ability to defer the creation of a volume until the moment the end user
elects to execute a Compute or Disk Offering
My Specific Needs from the Plug-in

13. Creating Primary Storage with the Plug-in
--OR--
Admin Defined QoS
Customer Defined QoS
• Orchestrated through CloudStack
• Administrator defined size (GBs) and QoS (IOPS)
• Customers defined capacity (GBs) and QoS (IOPS)
Add Primary Storage in CloudStack

14. New Disk Offerings
Admin Defined QoS
Customer Defined QoS

15. --VS--
Admin Defined QoS
• Orchestrated through CloudStack
• Based on Disk Offerings
• Administrator defined QoS (IOPS)
• Customers defined QoS (IOPS)
Customer Adds a Volume
Customer Defined QoS
Add a Volume to a VM

16. Customer Attaches the Volume to a VM
• Choose the Volume to
be Attached
• Click Attach Disk
• Select the Instance

17. What Happens on SolidFire?
The Volume is Created
on the SolidFire SAN
The Volume QoS
Settings are Defined

18. What Happens on the Hypervisor?
The Storage
Resource is
Created

19. A CloudStack storage plug-in is divided into three components:
Provider: Logic related to the plug-in in general (ex: name of plug-in).
Life Cycle: Logic related to life cycle (ex: creation) of a given storage system (ex: a single
SolidFire SAN).
Driver: Logic related to creating and deleting volumes on the storage system.
Must add a dependency in the client/pom.xml file as such:
<dependency>
<groupId>org.apache.cloudstack</groupId>
<artifactId>cloud-plugin-storage-volume-solidfire</artifactId>
<version>${project.version}</version>
</dependency>
So…how do you actually make a plug-in?

20.  Must implement the PrimaryDataStoreProvider interface
 Provides CloudStack with the plug-in's name as well as the Life Cycle and Driver
objects the storage system uses
 Must be listed in the applicationContext.xml.in file (Spring Framework related)
 A single instance of this class is created for CloudStack
About the Provider

21. public interface PrimaryDataStoreProvider extends DataStoreProvider {
}
public interface DataStoreProvider {
public static enum DataStoreProviderType {
PRIMARY,
IMAGE
}
public DataStoreLifeCycle getDataStoreLifeCycle();
public DataStoreDriver getDataStoreDriver();
public HypervisorHostListener getHostListener();
public String getName();
public boolean configure(Map<String, Object> params);
public Set<DataStoreProviderType> getTypes();
}
The Provider Interface

22. public class SolidfirePrimaryDataStoreProvider implements PrimaryDataStoreProvider {
private final String providerName = "SolidFire";
protected PrimaryDataStoreDriver driver;
protected HypervisorHostListener listener;
protected DataStoreLifeCycle lifecyle;
@Override
public String getName() { return providerName; }
@Override
public DataStoreLifeCycle getDataStoreLifeCycle() { return lifecyle; }
@Override
public boolean configure(Map<String, Object> params) {
lifecyle = ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class);
driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.class);
listener = ComponentContext.inject(DefaultHostListener.class);
return true;
}
Provider Implementation

23. Notes:
 client/tomcatconf/applicationContext.xml.in
 Each provider adds a single line.
 “id” is only used by Spring Framework (not by CS Management Server). Recommend just providing a descriptive name.
Example:
<bean id="ClassicalPrimaryDataStoreProvider"
class="org.apache.cloudstack.storage.datastore.provider.CloudStackPrimaryDataStoreProviderImpl" />
<bean id="solidFireDataStoreProvider"
class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider" />
Provider Configuration

24.  Must implement the PrimaryDataStoreLifeCycle interface.
 Handles the creation, deletion, etc. of a storage system (ex: SAN) in CloudStack.
 The initialize method of the Life Cycle object adds a row into the cloud.storage_pool
table to represent a newly added storage system.
About the Life Cycle

25. public interface PrimaryDataStoreLifeCycle extends DataStoreLifeCycle {
}
public interface DataStoreLifeCycle {
public DataStore initialize(Map<String, Object> dsInfos);
public boolean attachCluster(DataStore store, ClusterScope scope);
public boolean attachHost(DataStore store, HostScope scope, StoragePoolInfo existingInfo);
boolean attachZone(DataStore dataStore, ZoneScope scope);
public boolean dettach();
public boolean unmanaged();
public boolean maintain(DataStore store);
public boolean cancelMaintain(DataStore store);
public boolean deleteDataStore(DataStore store);
}
The Life Cycle Interface

26. @Override
public DataStore initialize(Map<String, Object> dsInfos) {
String url = (String)dsInfos.get("url");
String uuid = getUuid(); // maybe base this off of something already unique
Long zoneId = (Long)dsInfos.get("zoneId");
String storagePoolName = (String) dsInfos.get("name");
String providerName = (String)dsInfos.get("providerName");
PrimaryDataStoreParameters parameters = new PrimaryDataStoreParameters();
parameters.setHost("10.10.7.1"); // really get from URL
parameters.setPort(3260); // really get from URL
parameters.setPath(url);
parameters.setType(StoragePoolType.IscsiLUN);
parameters.setUuid(uuid);
parameters.setZoneId(zoneId);
parameters.setName(storagePoolName);
parameters.setProviderName(providerName);
return dataStoreHelper.createPrimaryDataStore(parameters);
}
Life Cycle Implementation

27.  Must implement the PrimaryDataStoreDriver interface.
 Your opportunity to create or delete a volume and to add a row to or delete a row from
the cloud.volumes table.
 A single instance of this class is responsible for creating and deleting volumes on all
storage systems of the same type.
About the Driver

28. public interface PrimaryDataStoreDriver extends DataStoreDriver {
public void takeSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CreateCmdResult> callback);
public void revertSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CommandResult> callback);
}
public interface DataStoreDriver {
public String grantAccess(DataObject data, EndPoint ep);
public boolean revokeAccess(DataObject data, EndPoint ep);
public Set<DataObject> listObjects(DataStore store);
public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback);
public void deleteAsync(DataObject data, AsyncCompletionCallback<CommandResult> callback);
public void copyAsync(DataObject srcdata, DataObject destData, AsyncCompletionCallback<CopyCommandResult> callback);
public boolean canCopy(DataObject srcData, DataObject destData);
public void resize(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback);
}
The Driver Interface

29. public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback) {
String iqn = null;
try {
VolumeInfo volumeInfo = (VolumeInfo)data;
iqn = createSolidFireVolume(volumeInfo);
VolumeVO volume = new VolumeVO(volumeInfo);
volume.setPath(iqn);
volumeDao.persist(volume);
} catch (Exception e) {
s_logger.debug("Failed to create volume (Exception)", e);
}
CreateCmdResult result = new CreateCmdResult(iqn, errMsg == null ? data.getSize() : null);
result.setResult(errMsg);
callback.complete(result);
}
Driver Implementation

30. Ask the CS MS to provide a list of all storage providers
http://127.0.0.1:8080/client/api?command=listStorageProviders&type=primary&response=json
Ask the CS MS to add a Primary Storage (a row in the cloud.storage_pool table) based on your
plug-in (ex: make CloudStack aware of a SolidFire SAN)
http://127.0.0.1:8080/client/api?command=createStoragePool&scope=zone&zoneId=a7af53b4-ec15-
4afc-a9ee-
8cba82b43474&name=SolidFire_831569365&url=MVIP%3A192.168.138.180%3BSVIP%3A10.10.7.1&
provider=SolidFire&response=json
Ask the CS MS to provide a list of all Primary Storages
http://127.0.0.1:8080/client/api?command=listStoragePools&response=json
API Calls

31. 1620 Pearl Street,
Boulder, Colorado 80302
Phone: 720.523.3278
Email: info@solidfire.com
www.solidfire.com