VMware Vsan vtug 2014


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

VMware Vsan vtug 2014

  1. 1. © 2014 VMware Inc. All rights reserved. VMware Virtual SAN VSAN – Radically Simple Storage! Jim LaFollette (jiml@vmware.com) VMware VTUG Spring Ahead - April 24, 2014
  2. 2. Agenda 1 SDDC & Importance of Storage 2 VSAN Product Overview 3 VSAN Technical Deep Dive 4 Additional Resources 5 Q&A 2
  3. 3. Why Storage is Important SDDC & Hyper-convergence 33
  4. 4. The Software-Defined Data Center Transform storage by aligning it with app demands Management tools give way to automation Expand virtual compute to all applications Virtualize the network for speed and efficiency 4
  5. 5. The Software-Defined Data Center Transform storage by aligning it with app demands 5
  6. 6. Storage Market in Midst of Disruption 6 Key Drivers Server flash Falling storage prices Abundant CPU cycles Hypervisor-converged infrastructure Cloud economics Server Storage 20-30 years ago Shared Storage 10-15 years ago New Forms Today
  7. 7. New Storage Tiers Are Rapidly Growing Flash: Enables New Storage Architectures • Flash is 50x – 2,000x faster than HDD – 110K/140K IOPs R/W from 360GB MLC PCIe card1 – Less than $0.10 per IOP • Eliminates the need to stripe across 100s of HDDs • Enables high performance server-side storage Cloud: Enables Cost-Effective Storage • Highly scalable, pay-as-you-go • Access through standard APIs • Low cost for capacity – $0.05 per GB per month2 • Forecasted to grow at 40% annually to 20183 7 Cloud Storage 1. Source: FuisionIO ioDrive2, Feb 2014 2. Source: Amazon S3, Feb 2014 3. Source: MarketsandMarkets Cloud Storage report - http://www.marketsandmarkets.com/PressReleases/cloud-storage.asp
  8. 8. Today’s Challenge: Massive Increase in Storage Demand & Complexity Source: IDC, Yezhkova, Worldwide Enterprise Storage Systems Forecast, November 2013, #244293 Storage Growth 41% YoY Source: IDC, Storage Predictions 2014, January 2014, General Storage QuickPoll, #243511, n=307 8
  9. 9. The Hypervisor Opens Up New Opportunities 9 SAN / NAS x86 Servers Cloud Storage vSphere The virtualization platform: • Knows the needs of all apps in real time • Sits directly in the I/O path • Global view of underlying infrastructure • Hardware agnostic
  10. 10. Object-based Pool SAN/NAS Pool Hypervisor Converged Pool Leveraging The Hypervisor We Can Transform Storage Today Software-defined Storage 10 LUN Array A LUN LUN Array B LUN LUN Abstract and pool (Virtualized Data Plane) Automate SLAs via VM-centric policies (Policy-based Control Plane) VM level Data services (Virtual Data Services) SAN / NASx86 Servers Cloud Object Storage vSphere Replication Snapshots
  11. 11. 11 2011-2013 2008-2010 2005-2007 VI 3.x ● VMFS ● Snapshots ● Storage vMotion ● NAS & iSCSI support vSphere 5.x ● Storage DRS ● Profile-driven Storage ● VASA ● vSphere Storage Appliance ● vSphere Data Protection ● vSphere Replication ● vSphere Flash ReadvSphere 4.x ● Thin Provisioning ● Storage I/O control ● Boot from SAN ● VAAI ● Linked mode 2014+ Software-defined Storage vSphere 5.5 & VSAN VMware Leads the Way to A New Approach Software-Defined Storage represents the next step in Storage Evolution
  12. 12. Software-Defined Storage 12 Bringing the efficient operational model of virtualization to storage Virtual Data Services Data Protection Mobility Performance Policy-driven Control Plane SAN / NAS SAN/NAS Pool Virtual Data Plane x86 Servers Hypervisor-converged Storage pool Object Storage Pool Cloud Object Storage Virtual SAN
  13. 13. VSAN Product Overview Radically simple hypervisor-converged storage 13
  14. 14. Virtual SAN: Radically Simple Hypervisor-Converged Storage 14 vSphere + Virtual SAN … • Software-defined storage embedded in vSphere • Runs on any standard x86 server • Pools HDD/flash into a shared datastore • Managed through storage policy-based management framework • High performance through flash acceleration • Highly resilient - zero data loss in the event of hardware failures • Deeply integrated with the VMware stack The Basics Hard disks SSD Hard disks SSD Hard disks SSD Virtual SAN Shared Datastore
  15. 15. 15 ● Installs in two clicks ● Managed from vSphere Client ● Policy-based management ● Self-tuning and elastic ● Deep integration with VMware stack Radically Simple ● Embedded in vSphere kernel ● Flash-accelerated ● Up to 915K IOPs from 16 nodes cluster ● Matches the VDI density of all flash array ● Best price/performance High Performance Lower TCO ● Eliminates large upfront investments (CAPEX) ● Grow-as-you-go (OPEX) ● Flexible choice of industry standard hardware ● Does not require specialized skills Virtual SAN Key Benefits
  16. 16. Virtual SAN is Deeply Integrated with VMware Stack 16 Ideal for VMware Environments vMotion vSphere HA DRS Storage vMotion vSphere Snapshots Linked Clones VDP Advanced vSphere Replication Data Protection VMware View Virtual Desktop vCenter Operations Manager vCloud Automation Center IaaS Cloud Ops and Automation Site Recovery Manager Disaster Recovery Site A Site B Storage Policy-Based Management
  17. 17. vSphere + VSAN 19 Virtual SAN resiliency: • Simple to set up via policy • Delivered on per VM basis • zero data loss in case of disk, network or host failures • Ensures zero downtime from disk or network failures • Interoperable with vSphere HA and Maintenance Mode • Modularizes infrastructure for efficient data center operations through break-fix model Virtual SAN Is Highly Resilient Against Any Hardware Failure Virtual SAN is designed to ensure data is never lost in case of failures
  18. 18. High Performance with Elastic and Linear Scalability 20 Notes: based on IOmeter benchmark Mixed = 70% Read, 4K 80% random Notes: Based on View Planner benchmark Up to 2M IOPs in 32 Node Cluster Comparable VDI density to an All Flash Array
  19. 19. 21 Embedded in the Hypervisor • No virtual appliance needed • Streamlines data path • Makes optimal data placement and I/O optimizations for enhanced performance Storage Virtual Appliance Read/Write Caching Read Cache VSAN Datastore Flash Accelerated Architecture vSphere + VSAN • Write buffer accelerates write performance • SSD accelerate read performance • Data persists on HDD Persistency Layer Architecture Delivers Superior Performance Virtual SAN is embedded into the vSphere kernel to minimize the I/O data path
  20. 20. Scale UP Add more Disks IOPSCapacity 40 TB 400 TB 4.4 PB Scale OUT Add more nodes • Elastic Grow or shrink on demand • Granular Add single nodes or disks • Non-disruptive No app downtime Virtual SAN Enables Elastic Linear Scaling of Performance and Capacity No More Complex Forecasting & Large Upfront Investments “Virtual SAN enables us to scale our storage infrastructure and while providing the necessary redundancy. This allows us to be more agile and bring our solutions to market faster.” — Frans Van Rooyen Cloud Architect, Adobe 22
  21. 21. Virtual SAN Delivers Enterprise-Grade Scale 2M IOPS 3,200 VMs 4.4 Petabytes Maximum Scalability per Virtual SAN Cluster 32 Hosts
  22. 22. Virtual SAN Reduces CAPEX and OPEX for Better TCO 24 CAPEX • Server-side economics • No Fibre Channel network • Pay-as-you-grow OPEX • Simplified storage configuration • No LUNs • Managed directly through vSphere Web Client • Automated VM provisioning • Simplified capacity planning As Low as $0.50/GB2 As Low as $0.25/IOPS 5X Lower OPEX4 Up to 50% TCO Reduction As Low as $50/Desktop1 1. Full clones 2. Usable capacity 3. Estimated based on 2013 street pricing, Capex (includes storage hardware + Software License costs) 4. Source: Taneja Group
  23. 23. • Compared to external storage at scale • Estimated based on 2013 street pricing, Capex (includes storage hardware + Software License costs) • Additional savings come from reduced Opex through automation • Virtual SAN configuration: 9 VMs per core, with 40GB per VM, 2 copies for availability and 10% SSD for performance Granular Scaling Eliminates Overprovisioning Delivers Predictable Scaling and ability to Control Costs VSAN enables predictable linear scaling Spikes correspond to scaling out due to IOPs requirements 25
  24. 24. Two Ways to Build a Virtual SAN Node 26 Completely Hardware Independent 1. Virtual SAN Ready Node …with multiple options available at GA + 30 Preconfigured server ready to use Virtual SAN… 2. Build Your Own …using the Virtual SAN Compatibility Guide* Choose individual components … SSD or PCIe SAS/NL-SAS/ SATA HDDs Any Server on vSphere Hardware Compatibility List HBA/RAID Controller ⃰ Note: For additional details, please refer to Virtual SAN VMware Compatibility Guide Page ⃰ Components for Virtual SAN must be chosen from Virtual SAN HCL, using any other components is unsupported
  25. 25. Flexibly Configure For Performance And Capacity 27 Performance 2xCPU – 8-core 128GB Memory 2xCPU – 8-core 128GB Memory 2xCPU – 8-core 128GB Memory 1x 400GB MLC SSD (~15% of usable capacity) 1x 400GB MLC SSD (~10% of usable capacity) 2x 400GB MLC SSD (~4% of usable capacity) 5x 1.2TB 10K SAS 7x 2TB 7.2K NL-SAS 10x 4TB 7.2K NL-SAS IOPS1 Raw Capacity ~20-15K 6TB ~15-10K 14TB ~10-5K 40TB Capacity 1. Mix workload 70% Read, 80% Random Estimated based on 2013 street pricing, Capex (includes storage hardware + Software License costs)
  26. 26. Virtual SAN Pricing and Packaging 28 Virtual SAN with Data Protection $3,590 / CPU Virtual SAN (1 CPU) vSphere Data Protection Advanced (1 CPU) Standalone • Fully featured, no scale limit. • Includes vSphere Distributed Switch • Applies only to internal drives (no JBOD/ext. storage) • Must license all CPU in a cluster Standalone • 10 license package size aligned with View For Any Workload For Virtual Desktop Only Launch Promos VSA to VSAN upgrade $11,475 / bundle Virtual SAN (6 CPUs per bundle) Register and download promo (Minimum 10 license purchase) Virtual SAN (1 CPU) Beta Promo Virtual SAN for Desktop $50 / CCU Bundle Promos Virtual SAN $2,495 / CPU 20% Discount 20% Discount 20% Discount Note: Virtual SAN also available on regional price lists on VMware supported local currencies.
  27. 27. VSAN Technical Deep Dive Under the hood … 29
  28. 28. Virtual SAN is NOT a Virtual Storage Appliance 30 – Virtual SAN is fully integrated with vSphere (ESXi & vCenter) – Drivers embedded in ESXi 5.5 contain the Virtual SAN smarts – Kernel modules: • Provide the shortest path for I/O • Remove unnecessary management overheads when dealing with an appliance • Do not consume resources unnecessarily Virtual SAN – Embedded into vSphereVirtual SAN – Not a VSA + VSA
  29. 29. 31 Management Clusters Typical use cases for VSAN Backup and DR Target DMZ / Isolated Tier 2 / Tier 3 Test / Dev / Staging Private cloud Virtual Desktop ROBO VDI Site A Site B vSphere VSAN
  30. 30. Hardware Requirements 32 Any Server on the VMware Compatibility Guide • SSD, HDD, and Storage Controllers must be listed on the VMware Compatibility Guide for VSAN http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan • ESXi 5.5 Hosts: Minimum 3, Maximum 32 1Gb/10Gb NIC SAS/SATA Controllers (RAID Controllers must work in “pass-through” or RAID0” mode SAS/SATA/PCIe SSD SAS/NL-SAS/SATA HDD At least 1 of each 4GB to 8GB USB, SD Cards
  31. 31. Boot Devices ● What installation device to use: – Depends on amount of host memory – Up to 512 GB – Use SD/USB devices as the installation media. – 512 GB or greater – Use a separate magnetic disk or solid stated disk as the installation device 33
  32. 32. Flash Based Devices In Virtual SAN ALL read and write operations always go directly to the Flash tier. Flash based devices serve two purposes in Virtual SAN 1. Non-volatile Write Buffer (30%) – Writes are acknowledged when they enter prepare stage on SSD. – Reduces latency for writes 2. Read Cache (70%) – Cache hits reduces read latency – Cache miss – retrieve data from HDD Choice of hardware is the #1 performance differentiator between Virtual SAN configurations. 34
  33. 33. Magnetic Disks (HDD) • SAS/NL-SAS/SATA HDDs supported – 7200 RPM for capacity – 10000 RPM for performance – 15000 RPM for additional performance • NL SAS will provide higher HDD controller queue depth at same drive rotational speed and similar price point – NL SAS recommended if choosing between SATA and NL SAS • Differentiate performance between clusters with SSD selection, and SSD:HDD ratio. • Flash rule of thumb guideline is 10% of anticipated capacity usage 35
  34. 34. Storage Controllers • SAS/SATA Storage Controllers – Pass-through or “RAID0” mode supported • Performance using RAID0 mode is controller dependent – Check with your vendor for SSD performance behind a RAID-controller • Storage Controller Queue Depth matters – Higher storage controller queue depth will increase performance • Validate number of drives supported for each controller 36
  35. 35. Network • 1Gb / 10Gb supported – 10Gb shared with NIOC for QoS will support most environments – If 1GB then recommend dedicated links for Virtual SAN • Jumbo Frames will provide nominal performance increase – Enable for greenfield deployments • Virtual SAN supports both VSS & VDS – NIOC requires VDS – Nexus 1000v – Should work but hasn't been fully tested • Network bandwidth performance has more impact on host evacuation, rebuild times than on workload performance 37
  36. 36. Firewalls • Virtual SAN Vendor Provider (VSANVP) – Inbound and outbound - TCP 8080 • Cluster Monitoring, Membership, and Monitoring Services (CMMDS) – Inbound and outbound UDP 12345 - 23451 • Reliable Datagram Transport (RDT) – Inbound and outbound TCP 2233 38
  37. 37. Technical Characteristics Virtual SAN is a cluster level feature similar to: – vSphere DRS – vSphere HA – Virtual SAN Deployed, configured and manage from vCenter through the vSphere Web Client (ONLY!). – Radically simple • Configure VMkernel interface for Virtual SAN • Enable Virtual SAN by clicking Turn On 39
  38. 38. Virtual SAN Implementation Requirements • Virtual SAN requires: – Minimum of 3 hosts in a cluster configuration – First 3 host MUST!!! contribute storage • vSphere 5.5 U1 or later – Maximum of 32 hosts • Not all hosts must contribute storage (hosts #4 - #32) – Locally attached disks • Magnetic disks (HDD) • Flash-based devices (SSD) – Network connectivity • 1GB Ethernet • 10GB Ethernet (preferred) 40 esxi-01 local storage local storage local storage vSphere 5.5 U1 Cluster esxi-02 esxi-03 cluster HDDHDD HDD
  39. 39. Virtual SAN Policies • Virtual SAN currently surfaces five unique storage capabilities to vCenter. 43
  40. 40. Number of Failures to Tolerate • Number of failures to tolerate – Defines the number of hosts, disk or network failures a storage object can tolerate. For “n” failures tolerated, “n+1” copies of the object are created and “2n+1” host contributing storage are required. 44 vsan network vmdkvmdk witness esxi-01 esxi-02 esxi-03 esxi-04 ~50% of I/O ~50% of I/O Virtual SAN Policy: “Number of failures to tolerate = 1” raid-1
  41. 41. Number of Disk Stripes Per Object • Number of disk stripes per object – The number of HDDs across which each replica of a storage object is distributed. Higher values may result in better performance. 45 vsan network stripe-2b witness esxi-01 esxi-02 esxi-03 esxi-04 stripe-1b stripe-1a stripe-2a raid-0raid-0 VSAN Policy: “Number of failures to tolerate = 1” + “Stripe Width =2” raid-1
  42. 42. Virtual SAN Storage Capabilities • Force provisioning – if yes, the object will be provisioned even is the policy specified in the storage policy is not satisfiable with the resources currently available. • Flash read cache reservation (%) – Flash capacity reserved as read cache for the storage object. Specified as a percentage of logical size of the object. • Object space reservation (%) – Percentage of the logical size of the storage object that will be reserved (thick provisioned) upon VM provisioning. The rest of the storage object is thin provisioned. 46
  43. 43. Storage Capabilities Recommended Practices 47 Storage Capability Use Case Value Number of failures to tolerate (RAID 1 – Mirror) Redundancy Default 1 Max 3 Number of disk stripes per object (RAID 0 – Stripe) Performance Default 1 Max 12 Object space reservation Thick Provisioning Default 0 Max 100% Flash read cache reservation Performance Default 0 Max 100% Force provisioning Override policy Disabled
  44. 44. Virtual SAN Constructs and Artifacts New Virtual SAN constructs, artifacts and terminologies: • Disk Groups. • VSAN Datastore. • Objects. • Components. • Virtual SAN Network. 48
  45. 45. Virtual SAN Disk Groups • Virtual SAN uses the concept of disk groups to pool together flash devices and magnetic disks as single management constructs. • Disk groups are composed of at least 1 flash device and 1 magnetic disk. – Flash devices are use for performance (Read cache + Write buffer). – Magnetic disks are used for storage capacity. – Disk groups cannot be created without a flash device. 49 disk group disk group disk group disk group Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs disk group HDD HDDHDDHDDHDD
  46. 46. Virtual SAN Datastore • Virtual SAN is an object store solution that is presented to vSphere as a file system. • The object store mounts the VMFS volumes from all hosts in a cluster and presents them as a single shared datastore. – Only members of the cluster can access the Virtual SAN datastore – Not all hosts need to contribute storage, but its recommended. 50 disk group disk group disk group disk group Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs disk group VSAN network VSAN network VSAN network VSAN networkVSAN network vsanDatastore HDD HDDHDDHDDHDD
  47. 47. Virtual SAN Objects • Virtual SAN manages data in the form of flexible data containers called objects. virtual machine files are referred to as objects. • Virtual machines files are referred to as objects. – There are four different types of virtual machine objects: • VM Home • VM swap • VMDK • Snapshots • Virtual machine objects are split into multiple components based on performance and availability requirements defined in VM Storage profile. 51 disk group disk group disk group disk group Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs disk group VSAN network VSAN network VSAN network VSAN networkVSAN network vsanDatastore HDD HDD HDD HDD HDD
  48. 48. Virtual SAN Components • Virtual SAN components are chunks of objects distributes across multiple hosts in a cluster in order to tolerate simultaneous failures and meet performance requirements. • Virtual SAN utilizes a Distributed RAID architecture to distribute data across the cluster. • Components are distributed with the use of two main techniques: – Striping (RAID0) – Mirroring (RAID1) • Number of component replicas and copies created is based on the object policy definition. 52 disk group disk group disk group disk group disk group VSAN network VSAN network VSAN network VSAN networkVSAN network vsanDatastore replica-1 replica-2 RAID1 HDD HDD HDD HDD HDD
  49. 49. Object and Components Layout 53 VSAN network VSAN network VSAN network VSAN networkVSAN network Virtual SAN Storage Objects R1 R0 R0 R0 Availability defined as number of copies Low level storage objects would reside on different hosts VMFS VMFS VMFS rolo2.vmdk The VM Home directory object is formatted with VMFS to allow a VM’s configuration files to be stored on it. Performance may include a stripe width VMFS rolo1.vmdk rolo.vmx, .log, etc /vmfs/volumes/vsanDatastore/rolo/rolo.vmdk disk group HDD disk group HDD disk group HDD disk group HDD disk group HDD
  50. 50. Virtual SAN Network • New Virtual SAN traffic VMkernel interface. – Dedicated for Virtual SAN intra-cluster communication and data replication. • Supports both Standard and Distributes vSwitches – Leverage NIOC for QoS in shared scenarios • NIC teaming – used for availability and not for bandwidth aggregation. • Layer 2 Multicast must be enabled on physical switches. – Much easier to manage and implement than Layer 3 Multicast 54 Management Virtual Machines vMotion Virtual SAN Distributed Switch 20 shares 30 shares 50 shares 100 shares uplink1 uplink2 vmk1 vmk2vmk0
  51. 51. Virtual SAN Scalable Architecture 56 • Scale up and Scale out architecture – granular and linearly storage, performance and compute scaling capabilities – Per magnetic disks – for capacity – Per flash based device – for performance – Per disk group – for performance and capacity – Per node – for compute capacity disk group disk group disk group VSAN network VSAN networkVSAN network vsanDatastore HDD disk group HDD HDD HDD disk group VSAN network HDD sc al e u p scale out
  52. 52. Understanding Failure Events ● Virtual SAN recognized two different types of hardware device events in order to define the type of failed scenario: – Absent – Degraded ● Absent events are responsible to trigger the 60 minutes recovery operations. – Virtual SAN will wait 60 minutes before starting the object and component recovery operations – 60 minutes is the default setting for all absent events – Configurable value via hosts advanced settings 57
  53. 53. Understanding Failure Events ● Degraded events are responsible to trigger the immediate recovery operations. – Triggers the immediate recovery operation of objects and components – Not configurable ● Any of the following detected I/O errors are always deemed degraded: – Magnetic disk failures – Flash based devices failures – Storage controller failures ● Any of the following detected I/O errors are always deemed absent: – Network failures – Network Interface Cards (NICs) – Host failures 58
  54. 54. Managing Failure Scenarios ● Through policies, VM’s on Virtual SAN can tolerate multiple failures – Disk Failure – degraded event – SSD Failure – degraded event – Controller Failure – degraded event – Network Failure – absent event – Server Failure – absent event ● VM’s continue to run ● Parallel rebuilds minimize performance pain – SSD Fail – immediately – HDD Fail – immediately – Controller Fail – immediately – Network Fail – 60 minutes – Host Fail – 60 minutes 59
  55. 55. Maintenance Mode – planned downtime ● 3 Maintenance mode options: ● Ensure accessibility ● Full data migration ● No data migration
  56. 56. Additional Resources – Sales Team sdssales@vmware.com 61 Product Page http://www.vmware.com/products/virtual-san/ VSAN Community https://communities.vmware.com/community/vmtn/vsan VSAN Feature Walk Through – Demos common operations http://featurewalkthrough.vmware.com/VSAN/ Hands-On-Lab – Free, Interactive Lab showcasing VSAN http://vmware.com/go/vsanlab Virtual SAN 60-day Free Evaluation http://www.vmware.com/go/try-vsan-en VSAN Design & Sizing Guide http://www.vmware.com/files/pdf/products/vsan/VSAN_Design_and_Si zing_Guide.pdf
  57. 57. Thank You Jim LaFollette jiml@vmware.com CONFIDENTIAL – NDA ONLY
  58. 58. Appendix Optional subtitle
  59. 59. ● VMUG Offerings (membership is FREE) • 200 Local Groups • 44 User Conferences • Special Interest Groups organized around industry and technology • Monthly e-Newsletter: VMUG Voice • Virtual education, including webcasts, session recordings and more • VMUG Advantage: exclusive discount package Visit www.vmug.com to join! The VMware User Group (VMUG) is a global independent customer-led organization which maximizes members’ use of VMware and partner solutions through knowledge sharing, training, collaboration and events.