Uploaded byRandy Weis
Storage Virtualization Challenges
This document discusses data management strategies in a virtualized environment. It covers topics such as storage design impacts on reliability, availability and scalability. It also discusses VMware backup challenges and solutions like VMware Consolidated Backup (VCB), vStorage APIs for Data Protection (VADP), and vStorage APIs for Array Integration (VAAI). Specific solutions mentioned include data deduplication, thin provisioning, replication and snapshots.
Storage Virtualization Challenges
- 1. Data Management StrategiesIn A Virtualized EnvironmentRandy Weis, Sr. Solutions Architect, Data Management January 20, 2011
- 2. AgendaWhat is DataManagement?
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15. DeduplicationWhat is DataManagement?What is Data management?
- 16. A critical businessstrategy for data growth, storage costs, data protection and long-term data.
- 17.
- 18. Increasingly consolidated andcomplex infrastructure
- 19. Impact of virtualizedcomputing on storage design
- 20. The right dataprotection strategy
- 21.
- 22.
- 23. Fundamental design principlesfor shared storage
- 24. Reliability, Availability, andScalabilityStorage Design ImpactsReliability
- 25.
- 26. ScalabilityPhases of CloudAdoptionPhase IIIPhase IIPhase IVPhase IPrivateCloudPublicCloudHybrid
- 27. What’s New invSphere: vStorage APIvStorage API for Data ProtectionvStorage APIs for Site Recovery ManagervStorage APIs for Array Integration (VAAI).vStorage APIs for Multipathing (Pluggable Storage Architecture)
- 28. VADPvStorage API forData Protection: a set of APIs focused on local backup/recovery use cases.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34. CommVault 9*VAAIvStorage APIfor Array Integration:
- 35. a set ofAPIs allow array vendors to “offload” various tasks from the ESX host
- 36. Copy offload (thinkVDI cloning)
- 37. “I/O Dedupe”, orBlock Zeroing
- 38. Scalable Lock Management= "Hardware Assisted Locking“
- 39.
- 40. The missing fourthprimitive: Thin Provisioning StunESX serverHardwareVMware Backup ChallengesVirtualization changes server, application, and information paradigm… Backup must evolve to deliver even greater consolidation and valueOld ParadigmPhysical environment: Low overall server utilization and plenty of bandwidth for backupNew ParadigmVirtual environment: High overall server utilization and littlebandwidth for backup100%100%80%80%60%60%CPU UtilizationCPU Utilization40%40%Shared physical resources20%20%0%0%VirtualServer AVirtual Server BVirtual Server CServer AServer BServer C20% resource utilization80% resource utilization
- 41. vStorage APIs for Data ProtectionBackup Partner SolutionsVMware enables scalable, non-disruptive backup and simple, reliable restore to any hardwareAPPAPPBACKUP APPAPPTraditional backupDisruptive to applications and users
- 42. Slow, complex processfor full restore
- 43. Hardware dependencies complicaterestoreBackup with VMware vSphereNon-disruptive to applications & users
- 44. Enables off-host, off-LANbackup with standard backup software via vStorage APIs for Data Protection
- 45. Enables image andfile-level backup of virtual machines
- 46. VSS quiescing forapp consistency (OS level consistency in Windows 2008)ArcServeOSOSOSOSNetBackup & Backup ExecESXBackup JobvRangerTivoli Storage ManagerSimpanaAvamarTape or diskVM Snapshot
- 47. VMware Backup MethodsTwobackup methods for VMwareVCB (VMware Consolidated Backup –VI3) - proxyVADP (vStorage API for Data Protection-vSphere) – no agents or proxy
- 48. Second Friday FullBackupBCDEFLGHData Deduplication: Technology OverviewThurs IncrementalACKFriday Full BackupMon IncrementalABHABCDAEFGTues IncrementalWeds IncrementalCBIEGJABCDEFG Backup Estimated Data Logical Reduction PhysicalFRIDAY FULL 1 TB 2–4x 250 GBMonday Incremental 100 GB 7–10x 10 GBTuesday Incremental 100 GB 7–10x 10 GBWednesday Incremental 100 GB 7–10x 10 GBThursday Incremental 100 GB 7–10x 10 GBSecond FRIDAY FULL 1 TB 50–60x 18 GBTOTAL 2.4 TB 7.8x 308 GBHIJKL
- 49. Retain: Store Morefor Longer with LessOver one year of retention in 3U of Data Domain deduplication storageBackup Cumulative Estimated PhysicalData Logical ReductionFirst Full 1 TB 4x 250 GBWeek 1April 7 2.4 TB 8x 308 GBWeek 2April 14 3.8 TB 10x 366 GBWeek 3April 21 5.2 TB 12x 424 GBMonth 1April 28 6.6 TB 14x 482 GBMonth 2May 31 12.2 TB 17x 714 GBMonth 3June 30 17.8 TB 19x 946 GBMonth 4July 31 23.4 TB 20x 1,178 GBTOTAL 23.4 TB 20x 1,178 GB
- 50. Shift to VirtualTheData Protection ChallengeRTO/RPO Data Protection ResultsEventTime LossRPOData LossRTO30% DR testing success rate within SLAs76% back up success rate with no formal tools to monitor jobs 40% cite difficulty in validating back up / recovery process Multi-Protection Silos
- 51.
- 52.
- 53. Zoning Implementation Strategiesfor Brocade SAN FabricsVMwareFibre Channel SAN Configuration Guide (v3.5 and v4.0)
- 54.
- 55. SAN System Designand Deployment Guide
- 56. ESX Server 3Configuration Guide
- 57.
- 58.
- 59. What Is Newin VMware vSphere™ 4: StorageGreat Demohttp://www.commvault.com/virtually-perfect-protection/demo.asp
Editor's Notes
- #4 Data management is a critical business strategy that tackles data growth, storage costs, data protection and long-term data. Datacenter virtualization presents new challenges to manage business data, in the consolidated and increasingly complex infrastructure resulting from server and storage virtualization. How should an organization design and implement the right data protection strategy for the increasingly consolidated and virtualized datacenter, recognizing that virtualization has completely changed the rules for backup and high availability? What is the impact of virtualized computing on storage design?This presentation will review how the fundamental design principles for shared storage (Reliability, Availability, and Scalability) are applied to virtualized computing infrastructure, and then dive into the ramifications of server virtualiztion on data protection and high availability solutions.
- #5 Our say to do ratio is high. We execute technically. We allocate time out for training. You don’t have to pay us to learn. We’ve already done this and can apply to real life situations.No one else has the cross-trained expertise across all technology silos.
- #7 VMDirectPath I/O for Storage: http://www.youtube.com/watch?v=jmQ5Ej8r-aAParavirtualized SCSIPluggable Storage Architecture (PSA)In vSphere, leveraging third party storage vendor multi-path software capabilities has been introduced through a modularstorage architecture that allows storage partners to write a plug-in for their specific capabilities. These modules cancommunicate with the intelligence running in the storage array to determine the best path selection, as well as leveragingparallel paths to increase performance and reliability of the I/O from the ESX to the storage array. By default, the nativemultipath driver (NMP) supplied by VMware will be used. It can be configured to support round-robin multipath as well.However, if storVMware has used SCSI Reservation Locks with its Virtual Machine File System (VMFS) to allow multiple ESX hosts access to the same datastore. These locks allow for multiple access of many ESX hosts to the same datastore and are critical for preventing corruption, such as if two VMware admins both tried to create the same VM at the same time on the same volume. When a VM is powered-on its files become locked, much the same way as a document opened in Word becomes locked. This prevents silly administrative accidents by making it impossible to delete a VM (or document) while it is powered on (or in use).
- #8 a set of APIs focused on local backup/recovery use casesBackup vSphere virtual machines from a central backup server or virtual machine without requiring backup agents or requiring backup processing to be done inside each guest virtual machine on the ESX host