Your SlideShare is downloading. ×
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Storage Virtualization Introduction
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Storage Virtualization Introduction

7,894

Published on

Published in: Technology
2 Comments
9 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,894
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
615
Comments
2
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Taneja “Next-Generation FC Arrays”:Clustered controller designSub-disk virtualizationSelf-configuring and self-tuning storageAutomated storage tieringThin technologies
  • Up to 256 FC or iSCSI LUNsESX multipathingLoad balancingFailoverFailover between FC and iSCSI*Beware of block sizes greater than 256 KB!If you want virtual disks greater than 256 GB, you must use a VMFS block size larger than 1 MBAlign your virtual disk starting offset to your array (by booting the VM and using diskpart, Windows PE, or UNIX fdisk)*
  • Link Aggregate Control Protocol (LACP) for trunking/EtherChannel - Use “fixed” path policy, not LRUUp to 8 (or 32) NFS mount pointsTurn off access time updatesThin provisioning? Turn on AutoSize and watch out
  • www.netapp.com/library/tr/3428.pdf
  • Transcript

    • 1. Storage Virtualization Seminar
      Stephen Foskett
      Director of Data Practice, Contoural
    • 2. Part 1:Breaking the Connections
      Storage virtualization is here, breaking the connection between physical storage infrastructure and the logical way we use it
    • 3. Agenda
      What is storage virtualization?
      Volume management
      Advanced file systems
      Virtualizing the SAN
      Virtual NAS
    • 4. Poll: Who is Already Using Storage Virtualization?
      • We talk about virtualization like it is new or strange…
      • 5. …but your storage is already virtualized!
      • 6. Disk drives map blocks
      • 7. RAID is as old as storage (conceived 1978-1988)
      • 8. Modern OSes include volume management and path management
      • 9. Network-attached storage (NAS) redirectors and DFS
      • 10. Storage arrays are highly virtualized (clustering, LUN carving, relocation, tiering, etc…)
      • 11. According to ESG, 52% have already implemented storage virtualization and 48% plan to! (ESG 2008)
    • The act of abstracting, hiding, or isolating the internal function of a storage (sub)system or service from applications, compute servers or general network resources for the purpose of enabling application and network independent management of storage or data.
      The application of virtualization to storage services or devices for the purpose of aggregating, hiding complexity or adding new capabilities to lower level storage resources. Storage can be virtualized simultaneously in multiple layers of a system, for instance to create HSM like systems.
      SNIA Defines Storage Virtualization
      The act of abstracting, hiding, or isolating the internalfunction of a storage (sub)system or service from applications, compute servers or general network resources for the purpose of enabling application and network independentmanagement of storage or data.
      The application of virtualization to storage services or devices for the purpose of aggregating, hiding complexity or adding new capabilities to lower level storage resources. Storage can be virtualized simultaneously in multiple layers of a system, for instance to create HSM like systems.
    • 12. What and Why?
      Virtualization removes the hard connection between storage hardware and users
      Address space is mapped to logical rather than physical locations
      The virtualizing service consistently maintains this meta-data
      I/O can be redirected to a new physical location
      We gain by virtualizing
      Efficiency, flexibility, and scalability
      Stability, availability, and recoverability
    • 13. The Non-Revolution:Storage Virtualization
      Software
      • We’ve been talking about storage virtualization for 15 years!
      • 14. Virtualization exists for both block and file storage networks
      • 15. Can be located in server-based software, on network-based appliances, SAN switches, or integrated with the storage array
      Switch
      Appliance
      Array
    • 16. Introducing Volume Management
      • Volume management = server-based storage virtualization
      • 17. Volume managers abstract block storage (LUNs, disks, partitions) into virtual “volumes”
      • 18. Very common – all* modern OSes have volume managers built in
      • 19. Windows Logical Disk Manager, Linux LVM/EVMS, AIX LVM, HP-UX LVM, Solaris Solstice, Veritas Volume Manager
      • 20. Mostly used for flexibility
      • 21. Resize volumes
      • 22. Protect data (RAID)
      • 23. Add capacity (concatenate or expand stripe or RAID)
      • 24. Mirror, snapshot, replicate
      • 25. Migrate data
    • Logical Volume Managers
    • 26. ZFS: Super File System!
      • ZFS (originally “zettabyte file system”) is a combined file system, volume manager, disk/partition manager
      • 27. Open source (CDDL) project managed by Sun
      • 28. Will probably replace UFS (Sun), HFS+ (Apple OS X Snow Leopard Server)
      • 29. ZFS creates a truly flexible, extensible, and full-featured pool of storage across systems and disks
      • 30. Filesystems contained in “zpools” on “vdevs” with striping and optional RAID-Z/Z2
      • 31. 128-bit addresses mean near-infinite capacity (in theory)
      • 32. Blocks are “copy-on-write” with checksums for snapshots, clones, authentication
      • 33. …but there are some limitations
      • 34. Adding (and especially removing) vdevs is hard/impossible
      • 35. Stacked RAID is impossible
      • 36. There is no clustering (until Sun adds Lustre)
    • Path Management Software
      Path management virtualizes the connection from a server to a storage system
      Failover
      Load balancing strategies
      A few choices
      Veritas DMP (cross-platform, with Storage Foundation)
      EMC PowerPath (supports EMC, HDS, IBM, HP)
      IBM SDD (free for IBM)
      HDS (HDLM)
      Microsoft MPIO (Windows, supports iSCSI and most FC)
      VMware Failover Paths
    • 37. Virtualizing the SAN
      • The storage area network (SAN) is a popular location for virtualization
      • 38. Can require less reconfiguration and server work
      • 39. Works with all servers and storage (potentially)
      • 40. Resides on appliance or switch placed in the storage network
      • 41. Some are in the data path, others are less so
      • 42. Brocade and Cisco switches have application blades
      • 43. Some use dedicated storage services modules (SSMs)
    • In-Band vs. Out-of-Band
      In-band devices intercept traffic
      Out-of-band devices redirect traffic
      Where’s my data?
      Where’s my data?
      It’s over there!
      I got yer data right here!
    • 44. SAN Virtualization Products
    • 45. Virtual NAS
      • File-based network-attached storage (NAS) lends itself to virtualization
      • 46. IP network connectivity and host processing possibilities
      • 47. Multitude of file servers? Virtualize!
      • 48. Global namespace across all NAS and servers
      • 49. Share excess capacity
      • 50. Transparently migrate data (easier than redirecting users!)
      • 51. Tier files on large “shares” with variety of data
      • 52. Create multiple virtual file servers
    • NAS Virtualization Products
    • 53. Transformed Storage Systems
      • Virtualization technology is common in storage array controllers
      • 54. Arrays create large RAID sets and “carve out” virtual LUNs for use by servers
      • 55. Controller clusters (and grids) redirect activity based on workload and availability
      • 56. Snapshots/mirrors and replication are common features
      • 57. A new generation arrays with virtualization features is appearing, with tiered storage, thin provisioning, migration, de-duplication
      • 58. Sub-disk RAID = the end of RAID as we know it?
    • Virtual Tiered Storage
      • Array controllers can transparently move data from low-cost to high-performance disk
      • 59. Most arrays support multiple drive types
      • 60. “Bulk” SATA or SAS drives are common (500 GB - 1 TB)
      • 61. Solid-state drives are the latest innovation
      • 62. Some arrays can dynamically load balance
      • 63. A few can “hide” other arrays “behind”
      • 64. SAN: HDS USP-V and similar from Sun, HP
      • 65. NAS: Network Appliance vFiler, ONStor Bobcat
    • Thin Provisioning
      • Storage is commonly over-allocated to servers
      • 66. Some arrays can “thinly” provision just the capacity that actually contains data
      • 67. 500 GB request for new project, but only 2 GB of initial data is written – array only allocates 2 GB and expands as data is written
      • 68. Symantec API, thin-unprovisioning capabilities
      • 69. What’s not to love?
      • 70. Oops – we provisioned a petabyte and ran out of storage
      • 71. Chunk sizes and formatting conflicts
      • 72. Can it thin unprovision?
      • 73. Can it replicate to and from thin provisioned volumes?
      • 74. Thin provisioning is an abdication of our responsibilities!
    • De-Duplication
      • The next frontier – efficiently storing duplicate content
      • 75. More appropriate to some applications than others
      • 76. Software or appliance (and now array!) analyzes files or blocks, saving duplicates just once
      • 77. Block-based reduce capacity more by looking inside files
      • 78. Once common only for archives, now available for production data
      • 79. Serious implications for performance and capacity utilization
      • 80. In-line devices process all data before it is written
      • 81. Post-processing systems scan written data for duplicates
    • “Cloud” Storage
      Many companies are choosing managed services for servers and storage
      Lots of managed archive and backup providers
      Zantaz, Google Postini, EMC Mozy, Symantec SPN, etc
      Managed storage services is coming into its own (finally!)
      Amazon S3 and Nirvanix
      EMC “Fortress”
    • 82. The Next-Generation Data Center
      Virtualization of server and storage will transform the data center
      Clusters of capability host virtual servers
      Cradle to grave integrated management
      SAN/network convergence is next
      InfiniBand offers converged virtual connectivity today
      iSCSI and FCoE become datacenter Ethernet (DCE) with converged network adapters (CNAs)
    • 83. Audience Response
      Question?
    • 84. Break sponsored by
    • 85. Part 2:Storage in the Virtual World
      Responding to the demands of server, application, and business users with new flexible technologies
    • 86. Agenda
      Why virtual storage for virtual servers?
      The real world impact and benefits
      Best practices for implementation
    • 87. Poll: Who Is Using VMware?
    • 88. Poll: Does Server Virtualization Improve Storage Utilization?
    • 89. Why Use Virtual Storage For Virtual Servers?
      Mobility of virtual machines between physical servers for load balancing
      Improved disaster recovery
      Higher availability
      Enabling physical server upgrades
      Operational recovery of virtual machine images
    • 90. Server Virtualization = SAN and NAS
      • Server virtualization has transformed the data center and storage requirements
      • 91. VMware is the #1 driver of SAN adoption today!
      • 92. 60% of virtual server storage is on SAN or NAS (ESG 2008)
      • 93. 86% have implemented some server virtualization (ESG 2008)
      • 94. Server virtualization has enabled and demanded centralization and sharing of storage on arrays like never before!
    • Three Pillars of VM Performance
    • 95. Server Virtualization Recoil
      • Dramaticallyincreased I/O
      • 96. Patchwork of support, few standards
      • 97. “VMware mode” on storage arrays
      • 98. Virtual HBA/N_Port ID Virtualization (NPIV)
      • 99. Everyone is qualifying everyone and jockeying for position
      • 100. Can be “detrimental” to storage utilization
      • 101. Befuddled traditional backup, replication, reporting
    • VMware Storage Options:Shared Storage
      Shared storage - the common/ workstation approach
      Stores VMDK image in VMFS datastores
      DAS or FC/iSCSI SAN
      Hyper-V VHD is similar
      Why?
      Traditional, familiar, common (~90%)
      Prime features (Storage VMotion, etc)
      Multipathing, load balancing, failover*
      But…
      Overhead of two storage stacks (5-8%)
      Harder to leverage storage features
      Often shares storage LUN and queue
      Difficult storage management
      VM
      Host
      Guest
      OS
      VMFS
      VMDK
      DAS or SAN
      Storage
    • 102. VMware Storage Options:Shared Storage on NFS
      Shared storage on NFS – skip VMFS and use NAS
      NTFS is the datastore
      Wow!
      Simple – no SAN
      Multiple queues
      Flexible (on-the-fly changes)
      Simple snap and replicate*
      Enables full Vmotion
      Use fixed LACP for trunking
      But…
      Less familiar (3.0+)
      CPU load questions
      Default limited to 8 NFS datastores
      Will multi-VMDK snaps be consistent?
      VM
      Host
      Guest
      OS
      NFS
      Storage
      VMDK
    • 103. VMware Storage Options:Raw Device Mapping (RDM)
      Raw device mapping (RDM) - guest VM’s access storage directly over iSCSI or FC
      VM’s can even boot from raw devices
      Hyper-V pass-through LUN is similar
      Great!
      Per-server queues for performance
      Easier measurement
      The only method for clustering
      But…
      Tricky VMotion and DRS
      No storage VMotion
      More management overhead
      Limited to 256 LUNs per data center
      VM
      Host
      Guest
      OS
      I/O
      Mapping File
      SAN Storage
    • 104. Physical vs. Virtual RDM
      Virtual Compatibility Mode
      Appears the same as a VMDK on VMFS
      Retains file locking for clustering
      Allows VM snapshots, clones, VMotion
      Retains same characteristics if storage is moved
      Physical Compatibility Mode
      Appears as a LUN on a “hard” host
      Allows V-to-P clustering,a VMware locking
      No VM snapshots, VCB, VMotion
      All characteristics and SCSI commands (except “Report LUN”) are passed through – required for some SAN management software
    • 105. Physical vs. Virtual RDM
    • 106. Poll: Which VMware Storage Method Performs Best?
      Mixed Random I/O
      CPU Cost Per I/O
      VMFS,
      RDM (p), or RDM (v)
      Source: “Performance Characterization of VMFS and RDM Using a SAN”, VMware Inc., 2008
    • 107. Which Storage Protocol is For You?
      FC, iSCSI, NFS all work well
      Most production VM data is on FC
      Either/or? - 50% use a combination (ESG 2008)
      Leverage what you have and are familiar with
      For IP storage
      Use TOE cards/iSCSI HBAs
      Use a separate network or VLAN
      Is your switch backplane fast?
      No VM Cluster support with iSCSI*
      For FC storage
      4 Gb FC is awesome for VM’s
      Get NPIV (if you can)
    • 108. Poll: Which Storage Protocol Performs Best?
      Throughput by I/O Size
      CPU Cost Per I/O
      Fibre Channel,
      NFS,
      iSCSI (sw),
      iSCSI (TOE)
      Source: “Comparison of Storage Protocol Performance”, VMware Inc., 2008
    • 109. Storage Configuration Best Practices
      Separate operating system and application data
      OS volumes (C: or /) on a different VMFS or LUN from applications (D: etc)
      Heavy apps get their own VMFS or raw LUN(s)
      Optimize storage by application
      Consider different tiers or RAID levels for OS, data, transaction logs - automated tiering can help
      No more than one VMFS per LUN
      Less than 16 production ESX .VMDKs per VMFS
      Get thin
      Deduplication can have a huge impact on VMDKs created from a template!
      Thin provisioning can be very useful – Thin disk is in Server, not ESX!?!
    • 110. Why NPIV Matters
      Without NPIV
      N_Port ID Virtualization (NPIV) gives each server a unique WWN
      Easier to move and clone* virtual servers
      Better handling of fabric login
      Virtual servers can have their own LUNs, QoS, and zoning
      Just like a real server!
      When looking at NPIV, consider:
      How many virtual WWNs does it support? T11 spec says “up to 256”
      OS, virtualization software, HBA, FC switch, and array support and licensing
      Can’t upgrade some old hardware for NPIV, especially HBAs
      Virtual Server
      Virtual Server
      Virtual Server
      21:00:00:e0:8b:05:05:04
      With NPIV
      Virtual Server
      Virtual Server
      Virtual Server
      …05:05:05
      …05:05:06
      …05:05:07
    • 111. Virtualization-Enabled Disaster Recovery
      DR is a prime beneficiary of server and storage virtualization
      Fewer remote machines idling
      No need for identical equipment
      Quicker recovery (RTO) through preparation and automation
      Who’s doing it?
      26% are replicating server images, an additional 39% plan to (ESG 2008)
      Half have never used replication before (ESG 2008)
      News: VMware Site Recovery Manager (SRM) integrates storage replication with DR
    • 112. Enhancing Virtual Servers with Storage Virtualization
      Mobility of server and storage images enhances load balancing, availability, and maintenance
      SAN and NAS arrays can snap and replicate server images
      VMotion moves the server, Storage VMotion (new in 3.5) moves the storage between shared storage locations
      Virtualization-optimized storage
      Pillar and HDS claim to tweak allocation per VM
      Many vendors announcing compatibility with VMware SRM
      Most new arrays are NPIV-capable
      Virtual storage appliances
      LeftHand VSA – A virtual virtualized storage array
      FalconStor CDP – a virtual CDP system
    • 113. Enabling Virtual Backup
      Virtual servers cause havoc for traditional client/server backups
      I/O crunch as schedules kick off – load is consolidated instead of balanced
      Difficult to manage and administer (or even comprehend!)
      Storage virtualization can help
      Add disk to handle the load (VTL)
      Switch to alternative mechanisms (snapshots, CDP)
      Consider VMware consolidated backup (VCB)
      Snapshot-based backup of shared VMware storage
      Block-based backup of all VMDKs on a physical server
    • 114. Audience Response
      Question?
    • 115. Break sponsored by
    • 116. Part 3:Should You Virtualize?
      A look at the practical benefits of virtualized storage
    • 117. Agenda
      • Pooling for efficiency, flexibility, and scalability
      • 118. Performance
      • 119. Stability, availability, and recoverability
      • 120. The down side
      • 121. Cost benefit analysis
      • 122. Where will you virtualize?
    • Pooling:Flexibility, and Scalability
      • Effective allocation of resources
      • 123. The right amount of storage for the application
      • 124. The right type (tiered storage)
      • 125. Quickly add and remove on demand
      • 126. Move storage from device to another
      • 127. Tiering, expansion, retirement
      • 128. Larger systems have fewer capacity limitations
    • How Green Am I?
      Server virtualization can dramatically reduce power, cooling, and space requirements
      Fewer physical servers
      Better (any) power management
      Storage virtualization offers fewer green benefits
      Does not normally reduce equipment footprint
      Enterprise storage systems not very energy efficient
      Transformed storage systems might help
      De-duplication, tiered storage, and archiving can slow growth
      New MAID and spin-down devices offer power/cooling savings
    • 129. Performance
      A battle royale between in- and out-of-band!
      In-band virtualization can improve performance with caching
      Out-of-band stays out of the way, relying on caching at the device level
      Split-path adds scalability to in-band
      Large arrays perform better (usually) than lots of tiny RAIDs or disks
      First rule of performance: Spindles
      Second rule of performance: Cache
      Third rule of performance: I/O Bottlenecks
    • 130. Solid State Drives (and Myths)
      The new (old) buzz
      RAM vs. NAND flash vs. disk
      EMC added flash drives to the DMX (CX?) as “tier-0”, CEO Joe Tucci claims flash will displace high-end disk after 2010
      Sun, HP adding flash to the server as a cache
      Gear6 caches NAS with RAM
      But…
      Are they reliable?
      Do they really perform that well?
      Will you be able to use them?
      Is the 10x-30x cost justified?
      Do they really save power?
      Notes: 1 – No one writes this fast 24x7
      2 – Manufacturers claim 2x to 10x better endurance
    • 131. Stability, Availability, and Recoverability
      Replication creates copies of storage in other locations
      Local replicas (mirrors and snapshots) are usually frequent and focused on restoring data in daily use
      Remote replicas are used to recover from disasters
      Virtualization can ease replication
      Single point of configuration and monitoring
      Can support different hardware at each location
    • 132. We Love It!
      Efficiency, scalability, performance, availability, recoverability, etc…
      Without virtualization, none of this can happen!
    • 133. The Down Side
      • Consolidation and centralization creates bigger baskets for your precious data
      • 134. Downtime and performance affect more systems
      • 135. Harder to back out if unsatisfied
      • 136. Additional complexity and interoperability concerns
      • 137. Scalability issues - ever-bigger systems
    • Implementation Issues
      Many virtualization systems require additional software loaded on servers
      Device drivers, path managers, agents, “shims”
      Additional maintenance and configuration can offset “single pane” benefits
      Organizational issues can crop up
      Virtualization blurs the lines between who owns what
      Future datacenter combines server, storage, network
      What about application?
    • 138. Cost Benefit Analysis
      Benefits
      Improved utilization
      Tiering lowers per-GB cost
      Reduced need for proprietary technologies
      Potential reduction of administrative/ staffing costs
      Flexibility boosts IT response time
      Performance boosts operational efficiency
      Costs
      Additional hardware and software cost
      Added complexity, vendors
      Training and daily management
      Reporting and incomprehensibility
      Possible negative performance impact
      Stability and reliability concerns
    • 139. Where Will You Virtualize?
    • 140. Closing Thought:What Is Virtualization Good For?
      Virtualization is a technology not a product
      What will you get from using it?
      Better DR?
      Improved service levels and availability?
      Better performance?
      Shortened provisioning time?
      The cost must be justified based on business benefit, not cool technology
    • 141. Audience Response
      Questions?
      Stephen Foskett
      Contoural, Inc.
      sfoskett@contoural.com
      http://blog.fosketts.net

    ×