Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Software Defined Storage at
VMware Primer
Duncan Epping (@DuncanYB)
Lee Dilworth (@LeeDilworth)
An introduction in to the ...
Agenda
1 Introduction
2 Virtual SAN
3 Virtual Volumes
4 vSphere APIs for I/O Filters
5 Summary
2
The Software Defined Data Center
Compute Networking Storage
Management
• All infrastructure services virtualized:
compute,...
Hardware
evolution
started the
infrastructure
revolution
Hyper-Converged Infrastructure: new IT model
“A lazy admin is the
best admin”
5
Simplicity: Operational / Management
6
The Hypervisor is the Strategic High Ground
SAN/NASx86 - HCI Object Storage
VMware vSphere
Cloud Storage
7
Storage Policy-Based Management – App centric automation
8
• Intelligent placement
• Fine control of services at VM level
...
Introducing
Virtual SAN
9
Virtual SAN, what is it?
10
Hyper-Converged Infrastructure
Distributed, Scale-out Architecture
Integrated with vSphere pla...
But what does that really mean?
VSAN network
Generic x86 hardware
VMware vSphere & Virtual SAN Integrated with your Hyperv...
VSAN is the Most Widely Adopted HCI Product
In my experience VMware
solutions are rock solid…we’re
ready to nearly double ...
VSAN is the Most Widely Adopted HCI Product
13
Virtual SAN Use Cases
14
VMware vSphere + Virtual SAN
End User
Computing Test/Dev
ROBOStagingManagementDMZ
Business
Critic...
Tiered Hybrid vs All-Flash
All-Flash
100K IOPS per Host
+
sub-millisecond latency
Caching
Writes cached first,
Reads from ...
Really Simple Setup
16
Deduplication
and
Compression
Enable?
Fault Domains,
2 node or
stretched
cluster?
Provisioning a VM? Define a policy first…
Virtual SAN currently surfaces multiple storage capabilities to vCenter Server
1...
Enterprise Availability in a few clicks
18
Today
vSphere & Virtual SAN
Overview
• Set “failures to tolerate” for high avai...
Deduplication and Compression for Space Efficiency
• Nearline deduplication and compression per disk group level.
– Enable...
RAID-5/6 (Inline Erasure Coding)
• When Number of Failures to Tolerate = 1 and Failure Tolerance Method = Capacity  RAID-...
Single interface for all Day 2 operations
21
VMware Virtual SAN: Generic Object Storage Platform
VMware vSphere
Virtual SAN
VMFS Block File Rest
22
Introducing
Virtual Volumes
23
But what about traditional storage?
Goals
• Make VMs a first class citizen on traditional storage
• Provide customers the ...
Software-Defined Storage and Availability
25
vSphere
Storage Capabilities
Snapshot Every Hour
Replication Synchronous
Avai...
vSphere Virtual Volumes
• VVols
– Virtual machine objects stored natively on the
array
– No Filesystem on-disk formatting ...
Virtual Volumes Primer
The Basics
• Virtualize SAN and NAS devices
• Virtual disks are natively represented on arrays
• En...
Storage Container
Storage Containers
• Logical storage constructs for grouping of
virtual volumes.
• Typically defined and...
VASA Provider (VP)
• Software component developed by
Storage Array Vendors
• ESXi and vCenter Server connect
to VASA Provi...
Protocol Endpoint
Protocol Endpoints
• Access points that enables communication
between ESXi hosts and storage array syste...
VM Provisioning Workflow
vSphere Admin
1. Create Virtual Machines
2. Assign a VM Storage Policy
3. Choose a suitable Datas...
VVoL replication
32
Goals
• Replicates Virtual Volumes instead of entire lun / exports / datastores
• Ability to group VVo...
Virtual Volumes – Continued Support from the Storage
Ecosystem
33
“[…] storage operations will be
fundamentally simplified...
Introducing
Virtual Data Services
(vSphere APIs for I/O Filters)
34
vSphere APIs for IO Filtering
• Add new 3rd party software-
based data services seamlessly
in vSphere
• Virtual (software ...
Why vSphere APIs for IO Filtering?
• To Enable our Ecosystem
– Empower 3rd parties to add functionality to vSphere
– Use c...
IO Path without vSphere APIs for IO Filtering
37
H2 2015/Q1 2016
Physical Device
Kernel World
vSCSI Backend
File System La...
IO Path with vSphere APIs for IO Filtering
38
H2 2015/Q1 2016
Physical Device
Kernel World
vSCSI Backend
File System Layer...
Strong Ecosystem
Today’s use cases:
• Caching
– Write thru and write back Cache
– Distributed Cache Management
• Replicati...
Center of the universe
40
Join the revolution
41
42
Upcoming SlideShare
Loading in …5
×

VMworld - sto7650 -Software defined storage @VMmware primer

10,929 views

Published on

VMworld - sto7650 -Software defined storage @VMmware primer

Presenters: Lee Dilworth, Duncan Epping

Published in: Technology
  • Be the first to comment

VMworld - sto7650 -Software defined storage @VMmware primer

  1. 1. Software Defined Storage at VMware Primer Duncan Epping (@DuncanYB) Lee Dilworth (@LeeDilworth) An introduction in to the world of Software Defined Storage Twitter: #STO7650
  2. 2. Agenda 1 Introduction 2 Virtual SAN 3 Virtual Volumes 4 vSphere APIs for I/O Filters 5 Summary 2
  3. 3. The Software Defined Data Center Compute Networking Storage Management • All infrastructure services virtualized: compute, networking, storage • Underlying hardware abstracted, resources are pooled • Control of data center automated by software (management, security) • Virtual Machines are first class citizens of the SDDC • Today’s session will focus on one aspect of the SDDC - storage 3
  4. 4. Hardware evolution started the infrastructure revolution
  5. 5. Hyper-Converged Infrastructure: new IT model “A lazy admin is the best admin” 5
  6. 6. Simplicity: Operational / Management 6
  7. 7. The Hypervisor is the Strategic High Ground SAN/NASx86 - HCI Object Storage VMware vSphere Cloud Storage 7
  8. 8. Storage Policy-Based Management – App centric automation 8 • Intelligent placement • Fine control of services at VM level • Automation at scale through policy • Need new services for VM? • Change current policy on-the-fly • Attach new policy on-the-fly Virtual Machine Storage policy Reserve Capacity 10 GB Availability 2 Failures to tolerate Limit IOPS 200 IOPS Snapshot Every Hour Replication Synchronous Deduplication Enabled Local / SAN / NAS Devices Storage Policy-Based Management vSphere Virtual SAN Virtual Volumes VAIO I/O Filters
  9. 9. Introducing Virtual SAN 9
  10. 10. Virtual SAN, what is it? 10 Hyper-Converged Infrastructure Distributed, Scale-out Architecture Integrated with vSphere platform Ready for today’s vSphere use cases Software-Defined Storage vSphere & Virtual SAN
  11. 11. But what does that really mean? VSAN network Generic x86 hardware VMware vSphere & Virtual SAN Integrated with your Hypervisor Leveraging local storage resources Exposing a single shared datastore Virtual SAN 11
  12. 12. VSAN is the Most Widely Adopted HCI Product In my experience VMware solutions are rock solid…we’re ready to nearly double our Virtual SAN deployment. It really did work as advertised…the fact that I have been able to set it and forget it is huge! 5000 Customers choose VMware HCS 12
  13. 13. VSAN is the Most Widely Adopted HCI Product 13
  14. 14. Virtual SAN Use Cases 14 VMware vSphere + Virtual SAN End User Computing Test/Dev ROBOStagingManagementDMZ Business Critical Apps DR / DA
  15. 15. Tiered Hybrid vs All-Flash All-Flash 100K IOPS per Host + sub-millisecond latency Caching Writes cached first, Reads from capacity tier Capacity Tier Flash Devices Reads go directly to capacity tier SSD PCIe Ultra DIMM Data Persistence Hybrid 40K IOPS per Host Read and Write Cache Capacity Tier SAS / NL-SAS / SATA SSD PCIe Ultra DIMM Virtual SAN 15
  16. 16. Really Simple Setup 16 Deduplication and Compression Enable? Fault Domains, 2 node or stretched cluster?
  17. 17. Provisioning a VM? Define a policy first… Virtual SAN currently surfaces multiple storage capabilities to vCenter Server 17 What If APIs New Capabilities in VSAN 6.2
  18. 18. Enterprise Availability in a few clicks 18 Today vSphere & Virtual SAN Overview • Set “failures to tolerate” for high availability • Virtual SAN provides rack awareness – Allowing for full rack failures through smart placement mechanism • Or ”simply” add a second site and stretch your Virtual SAN across – Of course within the defined boundaries • And if that isn’t sufficient, you can always replicate to a 3rd site! – With or without the use of Site Recovery Manager Rack 1 Rack 2 Rack 3 Rack 4 witness 5ms RTT, 10GbE vSphere & Virtual SAN vmdk vmdk witness vSphere & VSAN Site Recovery Manager vmdk
  19. 19. Deduplication and Compression for Space Efficiency • Nearline deduplication and compression per disk group level. – Enabled on a cluster level – Deduplicated when de-staging from cache tier to capacity tier – Fixed block length deduplication (4KB Blocks) • Compression after deduplication – If block is compressed <= 2KB – Otherwise full 4KB block is stored Beta esxi-01 esxi-02 esxi-03 vmdk vmdk vSphere & Virtual SAN vmdk All Flash 19
  20. 20. RAID-5/6 (Inline Erasure Coding) • When Number of Failures to Tolerate = 1 and Failure Tolerance Method = Capacity  RAID-5 – 3+1 (4 host minimum) – 1.33x overhead for RAID-5 instead of 2x compared to FTT=1 with RAID-1 • When Number of Failures to Tolerate = 2 and Failure Tolerance Method = Capacity  RAID-6 – 4+2 (6 host minimum) – 1.5x overhead for for RAID-6 instead of 3x compared to FTT=2 with RAID-1 RAID-5 ESXi Host parity data data data ESXi Host data parity data data ESXi Host data data parity data ESXi Host data data data parity All Flash 20
  21. 21. Single interface for all Day 2 operations 21
  22. 22. VMware Virtual SAN: Generic Object Storage Platform VMware vSphere Virtual SAN VMFS Block File Rest 22
  23. 23. Introducing Virtual Volumes 23
  24. 24. But what about traditional storage? Goals • Make VMs a first class citizen on traditional storage • Provide customers the option to use per-VM data operations on storage systems • Build framework to offload ANY per-VM data operations to the storage system • Minimal disruption to existing processes or infrastructure 24 I would like per VM data services for that as well… VS
  25. 25. Software-Defined Storage and Availability 25 vSphere Storage Capabilities Snapshot Every Hour Replication Synchronous Availability RAID-1 IOPS Limit 150 Local / SAN / NAS Devices Storage Policy-Based Management vSphere Virtual SAN Virtual Volumes VAIO I/O Filters
  26. 26. vSphere Virtual Volumes • VVols – Virtual machine objects stored natively on the array – No Filesystem on-disk formatting required • There are five types of VVols: – CONFIG – vmx, logs, nvram, log files, etc – DATA – VMDKs – MEM – Snapshots – SWAP – Swap files – Other – Vendor solution specific vSphere Web Client View 26
  27. 27. Virtual Volumes Primer The Basics • Virtualize SAN and NAS devices • Virtual disks are natively represented on arrays • Enables VM granular storage operations using array-based data services • Policy enables automated consumption at scale • Supports existing storage I/O protocols • Included with vSphere Standard and up virtual datastore(s) protocol endpoint(s) protocol endpoint(s) storage container(s) VASA Provider – Published Capabilities Snapshots Deduplication Quality of ServiceStorage System VASA Data Path VMware vSphere 27
  28. 28. Storage Container Storage Containers • Logical storage constructs for grouping of virtual volumes. • Typically defined and setup by storage administrators on the array in order to define: – Storage capacity allocations – Define capabilities for a pool • Logically partition or isolate VMs with diverse storage needs and requirement (or security) • Minimum one storage container per array • Maximum depends on the array 28 virtual datastore(s) protocol endpoint(s) protocol endpoint(s) storage container(s) Storage System Data Path VMware vSphere
  29. 29. VASA Provider (VP) • Software component developed by Storage Array Vendors • ESXi and vCenter Server connect to VASA Provider • Provides storage awareness services • VASA Provider can be implemented within the array’s management server or firmware – Can be deployed in HA mode, when vendor has implemented this! • Responsible for creating Virtual Volumes – Required for powering on VMs! 29 virtual datastore(s) protocol endpoint(s) protocol endpoint(s) storage container(s) Storage System VASA/SPBM Control Path VMware vSphere VASAProvider Control Path
  30. 30. Protocol Endpoint Protocol Endpoints • Access points that enables communication between ESXi hosts and storage array systems. – Part of the physical storage fabric – Created by Storage administrators Scope of Protocol Endpoints • Compatible with all SAN and NAS Protocols: - iSCSI - NFS v3 - FC - FCoE • A Protocol Endpoint can support any one of the protocols at a given time Why Protocol Endpoints? • Single access point to avoid LUN limits 30 virtual datastore(s) protocol endpoint(s) protocol endpoint(s) storage container(s) Storage System Data Path VMware vSphere
  31. 31. VM Provisioning Workflow vSphere Admin 1. Create Virtual Machines 2. Assign a VM Storage Policy 3. Choose a suitable Datastore Under the Covers Provisioning operations are translated into VASA API calls in order to create the individual virtual volumes. Under the Covers Provisioning operations are offloaded to the array for the creation of virtual volumes on the storage container that match the capabilities defined in the VM Storage Policies 31 virtual datastore(s) protocol endpoint(s) protocol endpoint(s) storage container(s) Storage System VASA/SPBM Data Path VMware vSphere VASAProvider
  32. 32. VVoL replication 32 Goals • Replicates Virtual Volumes instead of entire lun / exports / datastores • Ability to group VVols into Replication Groups • Array-based replication used to replicate VVoLs / Replication Groups • Leverages VASA 3 APIs, expose storage replication capabilities, match containers to policies VS VM VM VM VM
  33. 33. Virtual Volumes – Continued Support from the Storage Ecosystem 33 “[…] storage operations will be fundamentally simplified.” Laura Guio VP and Business Line Executive, Storage Systems VMware Alliance Executive “[…] as a key design partner, we’ve worked very closely with VMware…” Tim Russell VP of Product Management for Data Center Solutions “[…] one of the most important storage technology advancements[…]” Ravi Chalaka VP Solutions Marketing “This is a huge shift from the LUN- centric model of today” Peter Waugh Marketing Director of Storage and Servers “[…] will transform the way you consume storage in your VMware environment.” Craig Nunes VP of Storage Marketing “[…] brings a relevant solution ...reducing cost and complexity in virtual environments” Christopher Ratcliffe Senior VP of Marketing, Core Technologies
  34. 34. Introducing Virtual Data Services (vSphere APIs for I/O Filters) 34
  35. 35. vSphere APIs for IO Filtering • Add new 3rd party software- based data services seamlessly in vSphere • Virtual (software based) data services controlled by Policy • Enables secure filtering of a VM’s IO • Caching and replication initial use cases • Storage agnostic to different architecture 35 New Storage Capabilities Replication Synchronous IOPS Limit 150 Local / SAN / NAS Devices vSphere Storage Policy-Based Management vSphere Virtual SAN Virtual Volumes VAIO I/O Filters
  36. 36. Why vSphere APIs for IO Filtering? • To Enable our Ecosystem – Empower 3rd parties to add functionality to vSphere – Use cases that VMware cannot address on our own • To Provide Customer Choice – Customers want more choice for their infrastructure – Enabling scenarios that they cannot get from VMware – Add (Virtual) Data Services to storage systems that may not offer them inbox 36
  37. 37. IO Path without vSphere APIs for IO Filtering 37 H2 2015/Q1 2016 Physical Device Kernel World vSCSI Backend File System Layer File Device Layer User World vSCSI Device
  38. 38. IO Path with vSphere APIs for IO Filtering 38 H2 2015/Q1 2016 Physical Device Kernel World vSCSI Backend File System Layer File Device Layer User World IO Filter(s) VAIO Framework vSCSI Device 3rd Party Software Data Services Data service executes the filter against the IO – no data copying needed 2 IO return directly to be committed to physical device 3 VAIO framework detects a filter policy before IO committed 1 Storage Policy Based Management
  39. 39. Strong Ecosystem Today’s use cases: • Caching – Write thru and write back Cache – Distributed Cache Management • Replication – Synchronous access to VM IO Event Queue – Full IP Sockets Interface for Replication Tomorrow’s use cases: • Encryption? – vSphere VMCrypt • Quality of Service? – Storage IO Control • and.... 39
  40. 40. Center of the universe 40
  41. 41. Join the revolution 41
  42. 42. 42

×