CloudStack storage subsystem introductionEdison Su
● Agenda– CloudStack storage subsystem introduction– New storage API introduction
Storages in CloudStack● Primary storage– Root disks and data disks● Secondary storage– Images– Volumes and snapshots backup
Primary storage● Different scopes– Local storage, cluster-wide storage, zone-widestorage● Different types– NFS/iSCSI/FC/Ceph
Life Cycle of a primary storageStart Initialized ReadyInitializeAttachMaintenanceMaintenanceRemovedRemoveUnMaintenance
Features highlight● Storage as a service– Storage tags– Disk offering with tags
Features highlight● Mix setup of different storages– Multiple scope of storages can be coexist in onezone– Multiple types of storages can be coexist in onezone– Different hypervisors can be coexist in one zone
Secondary storage● Different scopes– Region/zone wide● Different types– Nfs/S3/Swift
What is secondary storage VM?● Dont mislead by its name, its more powerfulthan just dealing with secondary storage● A bootstrap VM created by CloudStack● Its the place where storage provisionhappened● VMs cpu/ram/network throughput is finegrained controlled
A typical setupCluster storagestoragestorageNfs Secondary storageCluster storagestoragestorageZone
Multiple zones setupClustersstoragesNfs as cache storageZone BZone ANfs as cache storageS3/SwiftstoragesClusters
What you can do with storage?● Volumes– Create root/data volume on primary storage– Import data volume– Create template from volume– Create snapshot from volume
What you can do with storage?● Snapshot– Create volume snapshot on primary storage– Backup snapshot into secondary storage– Create template from snapshot– Xenserver supports delta snapshot
What you can do with storage?● Template– Import template– Create vm from template– Export template– Copy template between zones
Part2: storage frameworkMgt server storage orchestration engineImageServiceVolumeServiceSnapshotServiceDataMotionServiceStoragePoolAllocatorDataStore pluginDataStore pluginDataStore pluginStorage provisionHypervisor agentSSVM agentAny other provision serviceStorage box
Design considerations● Separate orchestration and provision– Orchestration decides which storage to use, whereto conduct storage provision, what resourcesneeded for storage provision– Orchestration only working on the control panel– Provision working on the data panel, actuallytalking to storage, either through hypervisor API orstorage vendors API, or common storageprotocols.
Orchestration is complicated● Different hypervisors, different storagesolutions have different requirements, canwork differently● CloudStack has different plugins to hookupinto CloudStack management server, changethe way how to orchestrate storage relatedoperations.
Orchestration is complicated● For example, whats the requirements todownload a template from s3 into primarystorage?– For file system based storage, its easy, directlydownload from s3 to primary storage– For block based storage, you may need to downloadtemplate into a temporary file system, then importthe file primary storage through hypervisors API– So during the orchestration, it decides for thisstorage, need a file based storage as cache storage,download it into cache storage, then import intoprimary storage. While for other storage, dont needthis cache storage.
Orchestration is complicated● For example, where is the place to createvolume?– For KVM/Xenserver, one can login into hypervisorhost, do a lot of things to create a volume– For Vmware, has to go through vCenter API– If storage vendor has its own APIs on the controlpanel, it can be directly called on the mgt server– If there are other storage services running out ofcloudstack mgt server, cloudstack should be able totalk to them
Orchestration is complicated● For example, whats the requirements forstorage migration?– If hypervisor supports, one can directly callhypervisors API, no additional resource needed– If no, then may need a temporary storage to copythe storage back and forth– If other storage vendor has its own clever way todo storage migration, may have otherrequirements
Orchestration is complicated● For example, whats the best way to take andbackup snapshot?– Full snapshot vs delta snapshot– Hypervisor based snapshot vs storage snapshot– Do you need a temporary storage as cachestorage, during snapshot backup?
Whats the plugins for orchestration● CloudStack 4.2 will provide a flexible storage framework fordeep integration with different storage solutions● One thing cant fit all, so we orchestrate differently● The extension points for orchestration:– Pluggable UI– DataStoreDriver– DataMotionStrategy– SnapshotStrategy● Its actively developed on object_store branch, should bemerged into 4.2
Provision● Mainly related to data motion– Download template from s3 to primary storage– Back up snapshot from primary storage to s3– Move disk from one storage to another– Etc● Its on the data path● Provision usually happened inside SSVM or onthe hypervisor host– SSVM is the place to scale, by default