Uploaded byShapeBlue
Using next gen storage in Cloudstack
The document discusses using next-generation storage to enhance performance, scalability, and efficiency in CloudStack infrastructure, highlighting key design considerations for primary and secondary storage. It emphasizes the importance of automation in scaling storage solutions and provides real-world examples of storage requirements and performance needs. Additionally, it introduces SolidFire's storage solutions, showcasing features such as guaranteed performance, automated management, and efficient capacity utilization.
More Related Content
Similar to Using next gen storage in Cloudstack
Using next gen storage in Cloudstack
- 1. Using Next GenerationStorage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure Giles Sirett CEO Giles.sirett@shapeblue.com Twitter: @ShapeBlue
- 2. Agenda Key Design Considerationsfor a CloudStack IaaS Cloud Primary & Secondary Storage Scaling Storage The IOPS Dilemma Real World Storage Scenarios Solidfire’s Next Generation Storage Platform Demo of Solidfire / CloudStack Integration Q&A @ShapeBlue
- 3. Todays Speakers Giles Sirett CEO ShapeBlue MikeTutkowski Senior CloudStack Developer SolidFire Geoff Higginbottom CTO ShapeBlue McClain Buggle Strategic Alliance Manager SolidFire @ShapeBlue
- 4. Questions Please use theGTM question feature. Don’t wait until the end ! @ShapeBlue
- 5. About ShapeBlue “ShapeBlue areexpert builders of public & private clouds. They are the leading global independent CloudStack / CloudPlatform integrator & consultancy” @ShapeBlue
- 6.
- 7.
- 8. How to buildan IaaS cloud PaaS Billing Developer Tooling Multi-cloud Management eCommerce Platform Ecommerce Management orchestration API CloudStack API CMP - Orchestration layer Apache CloudStack Choice of Hypervisor Hypervisor KVM, VMware, XenServer, Hyper-V Networking Commodity Compute Compute Storage @ShapeBlue
- 9.
- 10. Secondary Storage Configured atZone level Has to be accessible by Management Servers, Hosts and SSVMs Stores all Templates, ISOs and Snapshots Has to be NFS Swift and S3 can be used to replicate data between Zones Zones can have more than one Secondary Storage Volume Public Templates & ISOs are copied to every store within a Zone @ShapeBlue
- 11. Scaling Secondary Storage Capacitydetermined by Number of Public ISOs Number of Public Templates Data Consumed by Users Volume Snapshots Private ISOs Private Templates Uploaded Volumes @ShapeBlue
- 12. Primary Storage Configured atCluster or Zone level Stores all Disk Volumes and VM Snapshots for VMs Options are: NFS iSCSI FC Local Storage High Performance is critical @ShapeBlue
- 13. Geoff Higginbottom CTO ShapeBlue www.shapeblue.com CloudStackCollaboration Conference 2012
- 14. Considerations for ScalingPrimary Storage Workload is a key factor Optimal LUN Size for iSCSI ~ 500GB Capacity required for each VM Disk performance required for each VM 20 VMs per iSCSI LUN for optimal performance Significant admin overhead to create and manage Admins tend to create large LUNs for ease of management Performance Suffers Scale-Out is a manual process @ShapeBlue
- 15. Cloud Scale RequiresAutomation Cloud scale requires automation of all components CloudStack can orchestrate Hypervisors Networks (SDN) Network Appliances – F5, LB, Netscaler etc Virtual Machines Storage is a very significant piece of the infrastructure Choose a storage vendor who integrates with CloudStack @ShapeBlue
- 16. Scaling Primary Storage- Example 4 XenServer Clusters, each with 4 Hosts 28 usable cores (after Hypervisor overhead) 764GB usable RAM (after Hypervisor overhead) Mapping of 6vCPU to 1 core @ShapeBlue
- 17. Scaling Primary Storage- Example Average VM Specification is 1vCPU / 4GB of RAM Based on total host capacity 2528 Guest VMs 75,840 IOPs (30 IOPS per VM) 50.5 TB of Storage (20GB per VM) 1264 VRs 12640 (10 IOPS per VR) 2.5 TB of Storage (2 GB per VM) @ShapeBlue
- 18. Storage Array IOPSvs Capacity - Example What we need What can a traditional storage array provide (real world figures) 53 TB capacity 88,480 IOPS 50TB Capacity 9,500 IOPS Therefore we need 10 Storage Arrays 500TB Capacity 95,000 IOPS 10x the required capacity to achieve required IOPS @ShapeBlue
- 19. Extract from CloudStackDesign Document @ShapeBlue
- 20. Capacity Overhead –Real World Example Requirements: 12TB of Storage 19,500 IOPS Solution: 100TB Storage Capacity 19,000 IOPS x6 Storage Arrays = 528TB of wasted capacity @ShapeBlue
- 21. Extract from CloudStackDesign Document @ShapeBlue
- 22. Other Factors AffectingCapacity Linked Clones Each VM within a LUN/Volume is linked to a ‘Template’ image Template may have 20GB allocated but only 2GB on disk Linked VMs only contain the differences – starts at 200MB Thin Provisioning Deduplication Actual storage space required is much lower than you think! @ShapeBlue
- 23. IOPS vs CapacityProblem Total IOPS per Storage Array – not per VM Traditional Multi-Tiered Storage has its limits Forced to add excess capacity to achieve IOPS Deduplication actually makes matters worse @ShapeBlue
- 24. McClain Buggle Strategic AllianceManager SolidFire
- 25. Key Primary Storage Considerations Guaranteed Performance Fine-grainperformance management with Guaranteed Quality of Service (QoS) Scale-out Simultaneous scaling of both capacity and performance In-line efficiency In-line data reduction reduces data footprint, requires less purchased capacity High Availability RAID-less data protection maintains data availability, security and performance regardless of failure condition. Management Automation Intuitive UI, CLI, and REST-based API for ease of management complete automation
- 26. Scale-Out Architecture Linear Scalability 96TB 5,SF3010 Nodes • 60 TB • 250,000 IOPS Starter 5 node SF3010 configuration Capacity 84TB 400,000 IOPS 350,000 IOPS 72TB 300,000 IOPS 60TB 250,000 IOPS Performance • New nodes are added as demand dictates • Performance and capacity instantly available to all volumes • Nodes added on the fly without down time
- 27. High Availability • Cluster wideRAID-less data protection, SolidFire Helix™ • Self-healing – Automatic and complete restoration of redundancy after failure • Complete HA – No single points of failure • Non-disruptive upgrades 9 Maintains all QoS settings in the face of component and/or system level failure • 7 Non-controller based architecture • 6 4 3 Failure Isolation – without performance impact • 1 2 A 7 5 5 8 8 4 6 3 2 1 A 9
- 28. Guaranteed Performance • Allocate:Storage performance independent of capacity • Manage: performance real-time without impacting other volumes • Guarantee: performance to every volume with fine-grain QoS settings
- 29. Efficiency, In-line • Always-on, in-lineData Reduction – Deduplication – Compression – Thin Provisioning • Executed across entire data store without performance impact • Delivers drastic reduction in power, cooling, and floor space
- 30. System Automation • IntegratedREST-based API • Enables complete automation of any SolidFire function • Supports development of user-facing storage controls • Seamless integration into current billing/charge-back systems and management stacks • API Integration Support from SolidFire
- 31. Avoid Sprawling Capacityto Gain Performance FAS6250AE FAS6250AE 96RU - 391 TB 117,000 IOPS 84RU - 328 TB – 97,500 IOPS SF9010 - 5RU - 174 TB - 375,000 IOPS 76RU - 296 TB – 84,500 IOPS Power = Average 220W/RU 5U = 1,100W 68RU - 224TB – 71,500 IOPS 6250AE - 44RU - 174 TB - 52,000 IOPS Power = Average 1000W/RU 44RU = 44,000W
- 32. Demo Mike Tutkowski Senior CloudStackDeveloper SolidFire
- 33.
- 34. Further Information orquestions slide Shapeblue.com/blog Slides can be found at www.slideshare.net/shapeblue Giles.sirett@shapeblue.com Geoff.higginbottom@shapeblue.com @shapeblue @cloudstackgurou @ShapeBlue
- 35. Using Next GenerationStorage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure Giles Sirett CEO Giles.sirett@shapeblue.com Twitter: @ShapeBlue
Editor's Notes
- #10 RegionA grouping of Availability Zones within a geographic areaManagement Server Infrastructure to manage the RegionAvailability ZoneTypically one Zone per DCContains at least 1 POD, 1 Cluster and Secondary StoragePodTypically a Rack containing one or more Clusters and NetworkingClusterGroup of identical Hosts running a common HypervisorPrimary StoragePrimary StorageTraditionally Unique to each ClusterKVM and VMware now support Zone-Wide Primary StorageHosts the Guest Virtual Machines and VM SnapshotsCan be any format the Hypervisor SupportsSecondary StorageZone WideNFS + S3 or Swift for Region Wide ReplicationStores Templates, ISOs and Volume Snapshots (backups)
- #15 Highlight the lack of Automation so traditional use of large LUNs = lower performanceHorizontalScale out requires lots of Human interaction
- #16 Cloud Automation is Key to achieving ScaleHypervisors, CISCO UCS, SDN, Load Balancers, Firewalls, Netscalersetc can all be Orchestrated
- #17 Example specs of a 16 Host PODNote – 4 Socket Hosts, but could equally be 32 Host POD using lower Dual SocketHosts – same total RAMXenServer and VMware have limits on max number of LUNs per Cluster
- #18 Capacity based on Linux VMs with 1vCPU, 4GB RAM and 20GB Disk,Using Redundant VR Feature – Public Cloud
- #20 Real World Design – not an exampleOriginal vendor replaced with alternate due to lack of performance
- #21 6 Arrays Deployed – ½ Peta-Byte of excess capacity600VMs20GB per VM30 IOPS per Vm150 VRs2GB per VR10 IOPS per VR
- #22 Highlight the last ParagraphStorage was being designed by Enterprise Storage Team based on IOPS requirements
- #32 Customer Example vs. NetAppCustomer needed to deploy 11,000 VM over 8 months.Took advantage of high deduplication rates and SF cloning technology and achieved the following results.SolidFire CapEx Spend is $2.45M less than NetAppSolidFire is 93% more dense than NetApp14 racks are not needed110 sq. ft. of data center space available for other useEquates to $264,000 in floor space savings97% fewer kWh used resulting in cost avoidance of $2.14 million over 5 years in power costsThis comparison is with a FAS6240AEShelf ConfigsFAS6240AE with 10GB UTADisk Shelves Disk Type:900GB 10k SAS Controllers4 (2 per controller) NIC #1 - HBA SAS 4-Port Copper 3/6 Gb QSFP PCIe,-C 2 (1 per controller) 512GB Flashcache4 (2 per controller) NIC #2-ADPT 2-Pt Unified Tgt 10GbE SFP+ PCIe,EN,-C Cabinet Type 42U NetApp Cabinet 32amp single phaseShelves = 2 U 24 x 900GB 10k SAS Controllers = 3 U with 512 GB FlashCache 1 per controller
- #36 eSkyCityBroker BinSunGardCiscoOrangeT-Mobile