Storage Plug-ins!!!Plug-in preview for the new storage frameworks with SolidFire
Mike Tutkowski• Lead Open Source Developer @SolidFire• Dedicated to CloudStack DevelopmentMcClain Buggle• Strategic Allian...
The CloudStack Opportunity
 Cloud is happening NOW Pain points are REAL LACK of viable alternatives Opportunity is MASSIVE AWS is not standing s...
ApplicationValue/MarginHighMedLowIOPSWhat is theopportunity?Low HighPerformance Sensitive AppsCRM / ERP / DatabaseMessagin...
We’veseen thismoviebefore...$-$100$200$300$400$500$600$700$800$900$1,0002002 2003 2004 2005 2006 2007x86 Virtualiza on -- ...
Is this are-run?0.00100.00200.00300.00400.00500.00600.00700.00800.00900.002006 2007 2008 2009 2010 2011AWS S3 Objects (Mil...
Why yes,it is!0100200300400500600700800900$0$100$200$300$400$500$600$700$800$900$1,000YR1 YR2 YR3 YR4 YR5 YR6x86 Virtualiz...
x86 VirtualizationThis movieended well…x86 Virtualization – From Test/Dev to Production02004006008001000120014001600180020...
CloudComputingThis endingis still beingwritten The Test/Dev EraThe Produc on EraThe Production Era Opportunity: x86 Virtua...
How do weinfluence theoutcome?MarginHighMedLowIOPSLow HighPerformance Sensitive AppsCRM / ERP / DatabaseMessaging / Produc...
• Storage is a major pain-point in most early-cloud deployments• Unpredictable Performance• Not designed for Multi-tenancy...
Where we are today with a storage plug-in
Primary Storage in CloudStack
CloudStack was not designed for dynamic provisioning and does not leverage vendorunique storage features within the framew...
Ability to defer the creation of a volume until the moment the end user elects to executea Compute or Disk Offering.Still ...
A CloudStack storage plug-in is divided into three components:Provider: Logic related to the plug-in in general (ex: name ...
 Must implement the PrimaryDataStoreProvider interface. Provides CloudStack with the plug-ins name as well as the Life C...
public interface PrimaryDataStoreProvider extends DataStoreProvider {}public interface DataStoreProvider {public static en...
public class SolidfirePrimaryDataStoreProvider implements PrimaryDataStoreProvider {private final String providerName = "S...
Notes: client/tomcatconf/applicationContext.xml.in Each provider adds a single line. “id” is only used by Spring Framew...
 Must implement the PrimaryDataStoreLifeCycle interface. Handles the creation, deletion, etc. of a storage system (ex: S...
public interface PrimaryDataStoreLifeCycle extends DataStoreLifeCycle {}public interface DataStoreLifeCycle {public DataSt...
@Overridepublic DataStore initialize(Map<String, Object> dsInfos) {String url = (String)dsInfos.get("url");String uuid = g...
 Must implement the PrimaryDataStoreDriver interface. Your opportunity to create or delete a volume and to add a row to ...
public interface PrimaryDataStoreDriver extends DataStoreDriver {public void takeSnapshot(SnapshotInfo snapshot, AsyncComp...
public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback) {String iqn = null;try {Volume...
Ask the CS MS to provide a list of all storage providershttp://127.0.0.1:8080/client/api?command=listStorageProviders&type...
 Need support for root disks. At the moment, the framework is mainly focused on datadisks. Need code to create datastore...
TriviaQuestionThe framework treats the defaultstorage behavior as a plug-inWhy?
Upcoming SlideShare
Loading in …5
×

CloudStack Meetup Santa Clara

322 views

Published on

Creating storage plug-ins for CloudStack by SolidFire

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
322
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CloudStack Meetup Santa Clara

  1. 1. Storage Plug-ins!!!Plug-in preview for the new storage frameworks with SolidFire
  2. 2. Mike Tutkowski• Lead Open Source Developer @SolidFire• Dedicated to CloudStack DevelopmentMcClain Buggle• Strategic Alliance Manager @SolidFireWho are these guys?
  3. 3. The CloudStack Opportunity
  4. 4.  Cloud is happening NOW Pain points are REAL LACK of viable alternatives Opportunity is MASSIVE AWS is not standing stillWhy the Urgency Around CloudStack?
  5. 5. ApplicationValue/MarginHighMedLowIOPSWhat is theopportunity?Low HighPerformance Sensitive AppsCRM / ERP / DatabaseMessaging / ProductivityDesktopPerformanceSensitivity$$$$$$ApplicationsDev / TestBackup / Archive
  6. 6. We’veseen thismoviebefore...$-$100$200$300$400$500$600$700$800$900$1,0002002 2003 2004 2005 2006 2007x86 Virtualiza on -- VMware License Revenue ($M)x86 Virtualization – The Test/Dev Era
  7. 7. Is this are-run?0.00100.00200.00300.00400.00500.00600.00700.00800.00900.002006 2007 2008 2009 2010 2011AWS S3 Objects (Millions)Cloud Computing – The Test/Dev Era
  8. 8. Why yes,it is!0100200300400500600700800900$0$100$200$300$400$500$600$700$800$900$1,000YR1 YR2 YR3 YR4 YR5 YR6x86 Virtualiza on -- VMware License Revenue ($M) Cloud -- Amazon S3 Objects (Billions)The Test/Dev Era – x86 Virtualization vs. Cloud Computing
  9. 9. x86 VirtualizationThis movieended well…x86 Virtualization – From Test/Dev to Production02004006008001000120014001600180020002002 2003 2004 2005 2006 2007 2008 2009 2010 2011x86 Virtualiza on -- VMware License Revenue ($M)
  10. 10. CloudComputingThis endingis still beingwritten The Test/Dev EraThe Produc on EraThe Production Era Opportunity: x86 Virtualization vs. Cloud
  11. 11. How do weinfluence theoutcome?MarginHighMedLowIOPSLow HighPerformance Sensitive AppsCRM / ERP / DatabaseMessaging / ProductivityDesktopPerformanceSensitivity$$$$$$ApplicationsDev / TestBackup / ArchiveKey CloudInfrastructureInnovations• Availability• Performance• Quality-of-Service• Scalability• Automation
  12. 12. • Storage is a major pain-point in most early-cloud deployments• Unpredictable Performance• Not designed for Multi-tenancy• Storage a key underpinning to successful application deployments• Today = Backup/Archive, Dev/Test• Tomorrow = Mission & Business Critical ApplicationsWhat does this have to do with CloudStack?
  13. 13. Where we are today with a storage plug-in
  14. 14. Primary Storage in CloudStack
  15. 15. CloudStack was not designed for dynamic provisioning and does not leverage vendorunique storage features within the framework.For SolidFire we are interested in features that allow users to select minimum,maximum, and burst IOPS for a given volume.Use Cases for a CloudStack Plug-In
  16. 16. Ability to defer the creation of a volume until the moment the end user elects to executea Compute or Disk Offering.Still have CS Admin configure the Primary Storage, but now it is based on a plug-ininstead of on a pre-existing storage volume.No requirement on part of the CSP to write orchestration logic.My Specific Needs from the Plug-in
  17. 17. A CloudStack storage plug-in is divided into three components:Provider: Logic related to the plug-in in general (ex: name of plug-in).Life Cycle: Logic related to life cycle (ex: creation) of a given storage system (ex: a single SolidFireSAN).Driver: Logic related to creating and deleting volumes on the storage system.Must add a dependency in the client/pom.xml file as such:<dependency><groupId>org.apache.cloudstack</groupId><artifactId>cloud-plugin-storage-volume-solidfire</artifactId><version>${project.version}</version></dependency>So…how do you actually make a plug-in?
  18. 18.  Must implement the PrimaryDataStoreProvider interface. Provides CloudStack with the plug-ins name as well as the Life Cycle and Driverobjects the storage system uses. Must be listed in the applicationContext.xml.in file (Spring Framework related). A single instance of this class is created for CloudStack.Provider – About
  19. 19. public interface PrimaryDataStoreProvider extends DataStoreProvider {}public interface DataStoreProvider {public static enum DataStoreProviderType {PRIMARY,IMAGE}public DataStoreLifeCycle getDataStoreLifeCycle();public DataStoreDriver getDataStoreDriver();public HypervisorHostListener getHostListener();public String getName();public boolean configure(Map<String, Object> params);public Set<DataStoreProviderType> getTypes();}Provider – Interface
  20. 20. public class SolidfirePrimaryDataStoreProvider implements PrimaryDataStoreProvider {private final String providerName = "SolidFire";protected PrimaryDataStoreDriver driver;protected HypervisorHostListener listener;protected DataStoreLifeCycle lifecyle;@Overridepublic String getName() { return providerName; }@Overridepublic DataStoreLifeCycle getDataStoreLifeCycle() { return lifecyle; }@Overridepublic boolean configure(Map<String, Object> params) {lifecyle = ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class);driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.class);listener = ComponentContext.inject(DefaultHostListener.class);return true;}Provider – Implementation
  21. 21. Notes: client/tomcatconf/applicationContext.xml.in Each provider adds a single line. “id” is only used by Spring Framework (not by CS Management Server). Recommend just providing a descriptivename.Example:<bean id="ClassicalPrimaryDataStoreProvider"class="org.apache.cloudstack.storage.datastore.provider.CloudStackPrimaryDataStoreProviderImpl" /><bean id="solidFireDataStoreProvider"class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider" />Provider – Configuration
  22. 22.  Must implement the PrimaryDataStoreLifeCycle interface. Handles the creation, deletion, etc. of a storage system (ex: SAN) in CloudStack. The initialize method of the Life Cycle object adds a row into the cloud.storage_pooltable to represent a newly added storage system.Life Cycle – About
  23. 23. public interface PrimaryDataStoreLifeCycle extends DataStoreLifeCycle {}public interface DataStoreLifeCycle {public DataStore initialize(Map<String, Object> dsInfos);public boolean attachCluster(DataStore store, ClusterScope scope);public boolean attachHost(DataStore store, HostScope scope, StoragePoolInfo existingInfo);boolean attachZone(DataStore dataStore, ZoneScope scope);public boolean dettach();public boolean unmanaged();public boolean maintain(DataStore store);public boolean cancelMaintain(DataStore store);public boolean deleteDataStore(DataStore store);}Life Cycle – Interface
  24. 24. @Overridepublic DataStore initialize(Map<String, Object> dsInfos) {String url = (String)dsInfos.get("url");String uuid = getUuid(); // maybe base this off of something already uniqueLong zoneId = (Long)dsInfos.get("zoneId");String storagePoolName = (String) dsInfos.get("name");String providerName = (String)dsInfos.get("providerName");PrimaryDataStoreParameters parameters = new PrimaryDataStoreParameters();parameters.setHost("10.10.7.1"); // really get from URLparameters.setPort(3260); // really get from URLparameters.setPath(url);parameters.setType(StoragePoolType.IscsiLUN);parameters.setUuid(uuid);parameters.setZoneId(zoneId);parameters.setName(storagePoolName);parameters.setProviderName(providerName);return dataStoreHelper.createPrimaryDataStore(parameters);}Life Cycle – Implementation
  25. 25.  Must implement the PrimaryDataStoreDriver interface. Your opportunity to create or delete a volume and to add a row to or delete a rowfrom the cloud.volumes table. A single instance of this class is responsible for creating and deleting volumes on allstorage systems of the same type.Driver – About
  26. 26. public interface PrimaryDataStoreDriver extends DataStoreDriver {public void takeSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CreateCmdResult> callback);public void revertSnapshot(SnapshotInfo snapshot, AsyncCompletionCallback<CommandResult> callback);}public interface DataStoreDriver {public String grantAccess(DataObject data, EndPoint ep);public boolean revokeAccess(DataObject data, EndPoint ep);public Set<DataObject> listObjects(DataStore store);public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback);public void deleteAsync(DataObject data, AsyncCompletionCallback<CommandResult> callback);public void copyAsync(DataObject srcdata, DataObject destData, AsyncCompletionCallback<CopyCommandResult> callback);public boolean canCopy(DataObject srcData, DataObject destData);public void resize(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback);}Driver – Interface
  27. 27. public void createAsync(DataObject data, AsyncCompletionCallback<CreateCmdResult> callback) {String iqn = null;try {VolumeInfo volumeInfo = (VolumeInfo)data;iqn = createSolidFireVolume(volumeInfo);VolumeVO volume = new VolumeVO(volumeInfo);volume.setPath(iqn);volumeDao.persist(volume);} catch (Exception e) {s_logger.debug("Failed to create volume (Exception)", e);}CreateCmdResult result = new CreateCmdResult(iqn, errMsg == null ? data.getSize() : null);result.setResult(errMsg);callback.complete(result);}Driver – Implementation
  28. 28. Ask the CS MS to provide a list of all storage providershttp://127.0.0.1:8080/client/api?command=listStorageProviders&type=primary&response=jsonAsk the CS MS to add a Primary Storage (a row in the cloud.storage_pool table) based on yourplug-in (ex: make CloudStack aware of a SolidFire SAN)http://127.0.0.1:8080/client/api?command=createStoragePool&scope=zone&zoneId=a7af53b4-ec15-4afc-a9ee-8cba82b43474&name=SolidFire_831569365&url=MVIP%3A192.168.138.180%3BSVIP%3A10.10.7.1&provider=SolidFire&response=jsonAsk the CS MS to provide a list of all Primary Storageshttp://127.0.0.1:8080/client/api?command=listStoragePools&response=jsonAPI Calls
  29. 29.  Need support for root disks. At the moment, the framework is mainly focused on datadisks. Need code to create datastores on ESX hosts and shared mount points on KVM hosts(we already have logic to create storage repositories on XenServer hosts). Speaking in terms of XenServer (but true for other hypervisors), when a volume isattached or detached, we need logic in place that handles zone-wide storage. No GUI support yet to add a provider...must be done with the API.What’s left to do?
  30. 30. TriviaQuestionThe framework treats the defaultstorage behavior as a plug-inWhy?

×