Using next gen storage in Cloudstack

1,411 views

Published on

Using Next Generation Storage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure

Published in: Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,411
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • RegionA grouping of Availability Zones within a geographic areaManagement Server Infrastructure to manage the RegionAvailability ZoneTypically one Zone per DCContains at least 1 POD, 1 Cluster and Secondary StoragePodTypically a Rack containing one or more Clusters and NetworkingClusterGroup of identical Hosts running a common HypervisorPrimary StoragePrimary StorageTraditionally Unique to each ClusterKVM and VMware now support Zone-Wide Primary StorageHosts the Guest Virtual Machines and VM SnapshotsCan be any format the Hypervisor SupportsSecondary StorageZone WideNFS + S3 or Swift for Region Wide ReplicationStores Templates, ISOs and Volume Snapshots (backups)
  • Highlight the lack of Automation so traditional use of large LUNs = lower performanceHorizontalScale out requires lots of Human interaction
  • Cloud Automation is Key to achieving ScaleHypervisors, CISCO UCS, SDN, Load Balancers, Firewalls, Netscalersetc can all be Orchestrated
  • Example specs of a 16 Host PODNote – 4 Socket Hosts, but could equally be 32 Host POD using lower Dual SocketHosts – same total RAMXenServer and VMware have limits on max number of LUNs per Cluster
  • Capacity based on Linux VMs with 1vCPU, 4GB RAM and 20GB Disk,Using Redundant VR Feature – Public Cloud
  • Real World Design – not an exampleOriginal vendor replaced with alternate due to lack of performance
  • 6 Arrays Deployed – ½ Peta-Byte of excess capacity600VMs20GB per VM30 IOPS per Vm150 VRs2GB per VR10 IOPS per VR
  • Highlight the last ParagraphStorage was being designed by Enterprise Storage Team based on IOPS requirements
  • Customer Example vs. NetAppCustomer needed to deploy 11,000 VM over 8 months.Took advantage of high deduplication rates and SF cloning technology and achieved the following results.SolidFire CapEx Spend is $2.45M less than NetAppSolidFire is 93% more dense than NetApp14 racks are not needed110 sq. ft. of data center space available for other useEquates to $264,000 in floor space savings97% fewer kWh used resulting in cost avoidance of $2.14 million over 5 years in power costsThis comparison is with a FAS6240AEShelf ConfigsFAS6240AE with 10GB UTADisk Shelves Disk Type:900GB 10k SAS Controllers4 (2 per controller) NIC #1 - HBA SAS 4-Port Copper 3/6 Gb QSFP PCIe,-C 2 (1 per controller) 512GB Flashcache4 (2 per controller) NIC #2-ADPT 2-Pt Unified Tgt 10GbE SFP+ PCIe,EN,-C Cabinet Type 42U NetApp Cabinet 32amp single phaseShelves = 2 U 24 x 900GB 10k SAS Controllers = 3 U with 512 GB FlashCache 1 per controller
  • eSkyCityBroker BinSunGardCiscoOrangeT-Mobile
  • Using next gen storage in Cloudstack

    1. 1. Using Next Generation Storage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure Giles Sirett CEO Giles.sirett@shapeblue.com Twitter: @ShapeBlue
    2. 2. Agenda         Key Design Considerations for a CloudStack IaaS Cloud Primary & Secondary Storage Scaling Storage The IOPS Dilemma Real World Storage Scenarios Solidfire’s Next Generation Storage Platform Demo of Solidfire / CloudStack Integration Q&A @ShapeBlue
    3. 3. Todays Speakers Giles Sirett CEO ShapeBlue Mike Tutkowski Senior CloudStack Developer SolidFire Geoff Higginbottom CTO ShapeBlue McClain Buggle Strategic Alliance Manager SolidFire @ShapeBlue
    4. 4. Questions   Please use the GTM question feature. Don’t wait until the end ! @ShapeBlue
    5. 5. About ShapeBlue “ShapeBlue are expert builders of public & private clouds. They are the leading global independent CloudStack / CloudPlatform integrator & consultancy” @ShapeBlue
    6. 6. @ShapeBlue
    7. 7. About ShapeBlue @ShapeBlue
    8. 8. How to build an IaaS cloud PaaS Billing Developer Tooling Multi-cloud Management eCommerce Platform Ecommerce Management orchestration API CloudStack API CMP - Orchestration layer Apache CloudStack Choice of Hypervisor Hypervisor KVM, VMware, XenServer, Hyper-V Networking Commodity Compute Compute Storage @ShapeBlue
    9. 9. CloudStack Architecture Region Zone POD Secondary Storage Cluster Host(s) Primary Storage @ShapeBlue
    10. 10. Secondary Storage        Configured at Zone level Has to be accessible by Management Servers, Hosts and SSVMs Stores all Templates, ISOs and Snapshots Has to be NFS Swift and S3 can be used to replicate data between Zones Zones can have more than one Secondary Storage Volume Public Templates & ISOs are copied to every store within a Zone @ShapeBlue
    11. 11. Scaling Secondary Storage  Capacity determined by    Number of Public ISOs Number of Public Templates Data Consumed by Users     Volume Snapshots Private ISOs Private Templates Uploaded Volumes @ShapeBlue
    12. 12. Primary Storage    Configured at Cluster or Zone level Stores all Disk Volumes and VM Snapshots for VMs Options are:      NFS iSCSI FC Local Storage High Performance is critical @ShapeBlue
    13. 13. Geoff Higginbottom CTO ShapeBlue www.shapeblue.com CloudStack Collaboration Conference 2012
    14. 14. Considerations for Scaling Primary Storage  Workload is a key factor    Optimal LUN Size for iSCSI ~ 500GB    Capacity required for each VM Disk performance required for each VM 20 VMs per iSCSI LUN for optimal performance Significant admin overhead to create and manage Admins tend to create large LUNs for ease of management   Performance Suffers Scale-Out is a manual process @ShapeBlue
    15. 15. Cloud Scale Requires Automation   Cloud scale requires automation of all components CloudStack can orchestrate       Hypervisors Networks (SDN) Network Appliances – F5, LB, Netscaler etc Virtual Machines Storage is a very significant piece of the infrastructure Choose a storage vendor who integrates with CloudStack @ShapeBlue
    16. 16. Scaling Primary Storage - Example  4 XenServer Clusters, each with 4 Hosts    28 usable cores (after Hypervisor overhead) 764GB usable RAM (after Hypervisor overhead) Mapping of 6vCPU to 1 core @ShapeBlue
    17. 17. Scaling Primary Storage - Example   Average VM Specification is 1vCPU / 4GB of RAM Based on total host capacity  2528 Guest VMs    75,840 IOPs (30 IOPS per VM) 50.5 TB of Storage (20GB per VM) 1264 VRs   12640 (10 IOPS per VR) 2.5 TB of Storage (2 GB per VM) @ShapeBlue
    18. 18. Storage Array IOPS vs Capacity - Example  What we need    What can a traditional storage array provide (real world figures)    53 TB capacity 88,480 IOPS 50TB Capacity 9,500 IOPS Therefore we need 10 Storage Arrays   500TB Capacity 95,000 IOPS 10x the required capacity to achieve required IOPS @ShapeBlue
    19. 19. Extract from CloudStack Design Document @ShapeBlue
    20. 20. Capacity Overhead – Real World Example  Requirements:    12TB of Storage 19,500 IOPS Solution:   100TB Storage Capacity 19,000 IOPS x6 Storage Arrays = 528TB of wasted capacity @ShapeBlue
    21. 21. Extract from CloudStack Design Document @ShapeBlue
    22. 22. Other Factors Affecting Capacity  Linked Clones    Each VM within a LUN/Volume is linked to a ‘Template’ image Template may have 20GB allocated but only 2GB on disk Linked VMs only contain the differences – starts at 200MB  Thin Provisioning Deduplication  Actual storage space required is much lower than you think!  @ShapeBlue
    23. 23. IOPS vs Capacity Problem     Total IOPS per Storage Array – not per VM Traditional Multi-Tiered Storage has its limits Forced to add excess capacity to achieve IOPS Deduplication actually makes matters worse @ShapeBlue
    24. 24. McClain Buggle Strategic Alliance Manager SolidFire
    25. 25. Key Primary Storage Considerations Guaranteed Performance Fine-grain performance management with Guaranteed Quality of Service (QoS) Scale-out Simultaneous scaling of both capacity and performance In-line efficiency In-line data reduction reduces data footprint, requires less purchased capacity High Availability RAID-less data protection maintains data availability, security and performance regardless of failure condition. Management Automation Intuitive UI, CLI, and REST-based API for ease of management complete automation
    26. 26. Scale-Out Architecture Linear Scalability 96TB 5, SF3010 Nodes • 60 TB • 250,000 IOPS Starter 5 node SF3010 configuration Capacity 84TB 400,000 IOPS 350,000 IOPS 72TB 300,000 IOPS 60TB 250,000 IOPS Performance • New nodes are added as demand dictates • Performance and capacity instantly available to all volumes • Nodes added on the fly without down time
    27. 27. High Availability • Cluster wide RAID-less data protection, SolidFire Helix™ • Self-healing – Automatic and complete restoration of redundancy after failure • Complete HA – No single points of failure • Non-disruptive upgrades 9 Maintains all QoS settings in the face of component and/or system level failure • 7 Non-controller based architecture • 6 4 3 Failure Isolation – without performance impact • 1 2 A 7 5 5 8 8 4 6 3 2 1 A 9
    28. 28. Guaranteed Performance • Allocate: Storage performance independent of capacity • Manage: performance real-time without impacting other volumes • Guarantee: performance to every volume with fine-grain QoS settings
    29. 29. Efficiency, In-line • Always-on, in-line Data Reduction – Deduplication – Compression – Thin Provisioning • Executed across entire data store without performance impact • Delivers drastic reduction in power, cooling, and floor space
    30. 30. System Automation • Integrated REST-based API • Enables complete automation of any SolidFire function • Supports development of user-facing storage controls • Seamless integration into current billing/charge-back systems and management stacks • API Integration Support from SolidFire
    31. 31. Avoid Sprawling Capacity to Gain Performance FAS6250AE FAS6250AE 96RU - 391 TB 117,000 IOPS 84RU - 328 TB – 97,500 IOPS SF9010 - 5RU - 174 TB - 375,000 IOPS 76RU - 296 TB – 84,500 IOPS Power = Average 220W/RU 5U = 1,100W 68RU - 224TB – 71,500 IOPS 6250AE - 44RU - 174 TB - 52,000 IOPS Power = Average 1000W/RU 44RU = 44,000W
    32. 32. Demo Mike Tutkowski Senior CloudStack Developer SolidFire
    33. 33. Questions @ShapeBlue
    34. 34. Further Information or questions slide  Shapeblue.com/blog  Slides can be found at      www.slideshare.net/shapeblue Giles.sirett@shapeblue.com Geoff.higginbottom@shapeblue.com @shapeblue @cloudstackgurou @ShapeBlue
    35. 35. Using Next Generation Storage to Improve Performance, Scalability & Efficiency in your CloudStack Infrastructure Giles Sirett CEO Giles.sirett@shapeblue.com Twitter: @ShapeBlue

    ×