OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan


Published on

  • Be the first to comment

  • Be the first to like this

OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan

  1. 1. VMworld 2011:Running the Hands On Labs (HOL) and How We Got There Andy Bennett, Craig Morgan Nexenta Systems
  2. 2. VMware Trial Cloud - SuperRack– Background • Approached February 2011 • Co-design for a commodity white box solution • To Be Comparable to Vpod and Vblock– Architecture Team • Pulled together with VMware Integration Engineering Team (IE) • Responsible for bleeding edge deployments and PoCs • Brains behind the HoL as a testbed for early deployments of VMware and partner technologies
  3. 3. VMware Trial Cloud - SuperRack
  4. 4. VMware Trial Cloud - SuperRack• Requirements – run High Density Generic Cloud Workloads – Cost per TB primary design consideration but good IOP’s performance important – commodity x86 hardware via technology partners – Nexenta HA design and scale out for future with NFS referrals – Classic POD Cloud Design
  5. 5. SuperRack
  6. 6. Nexenta Storage ModulePart Description Total QtySystem 2u 6 HDD Chassis 4Motherboard X8DAH+F 4Processors X5670 2.93G 8Memory 8GB NETLIST 72HDD Seagate 2TB SAS 7.2k 360JBODs DataOn 60x bay 6Controllers LSI SAS 810GbE NIC STD Dual-port w. SFP+ 4SATA DOM Innodisk 64GB 4License 720TB raw 1SSD 2x STEC ZeusRAM 8GB 8SSD STEC ZeusIOPS 200GB 8
  7. 7. Nexenta Storage Module• Nexenta Layout – multiple pools (active/active cluster) – 2x data pools• Per Pool – vdev layout is 6x RAID Z2 – 30x vdevs per pool – 2x SLOG devices (STEC ZeusRAM mirrored)• Totals (Rack) – 360 drives, 720TB raw, 480TB usable
  8. 8. Other Storage Service Providers 2x FAS 3270 Appliances 144 x 15K Hard Disks 4 PAM Cards per Appliance2x EMC VNX 7500 platforms3x File Blades per VNX1 VNX was all SSD 96 SSD’s in total1 VNX 140+ 15K Disks FAST Cache, FAST VP SSD Tier
  9. 9. Hands-on-Labs Workload– Some highlights • each lab provision – 26x distinct labs – 4x to 25x VMs per lab – max VM size was 26GB – create, deploy, destroy per login – lab choice random (student choice), no pre-pop possible! • nested VMs workload, highly latency sensitive– Takeaways • 148,103 VM’s created during Vegas show • 1x VM created every 1.215 seconds
  10. 10. HoL StatisticsElement Las Vegas Copenhagen TotalLab Time 50 38 100 (12@Sydney)Labs Taken 13415 4797 18912 (700@Sydney)Lab Hours 12388 ?Lab Seats available 487 280 767VMs per Lab 4-25x 4-22xUnique Attendees 5042 1045 6207 (120@Sydney)VMs Created 148103 53358 201461
  11. 11. Nexenta Statistics• Ran 4 of 8 VMware verticals in Vegas• 10.3 billion NFS IOPs served – 7.9 billion in Vegas, 2.4 billion in Copenhagen• 3 billion NFS IOPs from one head in Vegas• Peak controller load – 154,000x 4K NFS ops/sec at sub 1 ms latency – 38,590x 16K NFS ops/sec on a single controller
  12. 12. Nexenta Statistics (cont.)– Highest Bandwidth (single head, 16K average I/O) • 1,305 MB/sec total • 928MB/sec read • 376MB/sec write • ... less than 2ms latency throughout above!
  13. 13. Nexenta Operational Issues– DRAM failure • DRAM failure in one head triggered HA failover • partner head ran the workload of “both” for 6hrs until evening maintenance window • NS called out failed DIMM serial number • DIMM replaced head back in service 12mins– High Availability RSF-1 HA plugin worked flawlessly • VMware saw no loss of service • monitoring informed NOC before they attributed issue • head over provisioning in design meant solution didn’t glitch with extra workload
  14. 14. Nexenta VMworld Custom Analytics– Tech demo (give us feedback!) • Sketch concept to delivered in 1 1/2 days • Custom written with distance visualisation in mind • 30ft screen (jumbotron) at 100ft distance • Convey as much useful info as densely as possible • High level, useful statistics • NFS IOPs (not raw I/O or bytes) • VM & VMDK latency and OPS (appropriate objects) • Response to VMworld NOC team input • Deliver • bandwidth, flow, read, write, top 10x, server, latency, alignment
  15. 15. Nexenta VMworld Custom Analytics
  16. 16. Nexenta VMworld Custom Analytics VMworld Vegas 2011 PrototypeVideo Clip available at: http://bit.ly/aura-vegas
  17. 17. Nexenta VMworld Custom Analytics VMworld Copenhagen 2011 PrototypeVideo Clip available at: http://bit.ly/aura-copenhagen
  18. 18. Thanks! Question and Answer Andy Bennett (andy@nexenta.com) Craig Morgan (craig@nexenta.com)Richard Elling (richard@nexenta.com)