• Share
  • Email
  • Embed
  • Like
  • Private Content
Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage
 

Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage

on

  • 1,639 views

This white paper describes how a multi-tiered, multi-site backup solution can be deployed to protect data in a VMware vSphere environment using vSphere data protection APIs along with Symantec ...

This white paper describes how a multi-tiered, multi-site backup solution can be deployed to protect data in a VMware vSphere environment using vSphere data protection APIs along with Symantec NetBackup 7.5 and EMC Isilon SyncIQ. This paper also outlines the setup, configuration, and functional testing of a disk-to-disk backup environment.

Statistics

Views

Total Views
1,639
Views on SlideShare
1,637
Embed Views
2

Actions

Likes
2
Downloads
37
Comments
0

1 Embed 2

http://www.slashdocs.com 2

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage Document Transcript

    • White PaperBACKUP AND RECOVERY SOLUTION FORVMWARE VSPHERE ON EMC ISILON STORAGEUsing Symantec NetBackup 7.5, VMware vSphere VADP and EMCIsilon SyncIQ Data Protection Software Abstract This white paper describes how a multi-tiered, multi-site backup solution can be deployed to protect data in a VMware vSphere environment using vSphere data protection APIs along with Symantec NetBackup 7.5 and EMC Isilon SyncIQ. This paper also outlines the setup, configuration, and functional testing of a disk-to-disk backup environment. July 2012
    • Copyright © 2012 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware, ESX, vMotion, VMware vCenter, and VMware vSphere are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number H10879Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 2
    • Table of ContentsExecutive summary.................................................................................................. 5 Business case .................................................................................................................... 5 Solution overview ............................................................................................................... 6 Key results and recommendations...................................................................................... 6Introduction ............................................................................................................ 7 Purpose .............................................................................................................................. 7 Scope ................................................................................................................................. 8 Audience ............................................................................................................................ 8 Terminology ....................................................................................................................... 8 Assumptions .................................................................................................................... 10Data protection strategies and use cases ............................................................... 10 Data protection ................................................................................................................ 10 Use cases ......................................................................................................................... 11Technology overview ............................................................................................. 12 Overview .......................................................................................................................... 12 VMware VADP ................................................................................................................... 12 Symantec NetBackup 7.5 ................................................................................................. 13 Isilon SyncIQ 3.0 .............................................................................................................. 13 vSphere Management Assistant ....................................................................................... 13Body of work ......................................................................................................... 14 Configuration ................................................................................................................... 14 Process overview .......................................................................................................... 14 Solution architecture .................................................................................................... 14 Physical environment ................................................................................................... 15 Validation......................................................................................................................... 16 Scenario #1: image-level backups ................................................................................ 16 Scenario #2: recovery of a VM from local backup.......................................................... 17 Scenario #3: recovery of individual files from a VM backup image ................................ 18 Scenario #4: recovery of a VM from remote backup ...................................................... 19 Component hardware and software .................................................................................. 22 Storage configuration ................................................................................................... 22 VMware ESXi host hardware configuration.................................................................... 23 Environment resources and configurations ................................................................... 25Conclusion ............................................................................................................ 26 Summary .......................................................................................................................... 26 Findings ........................................................................................................................... 26 Next steps ........................................................................................................................ 26 Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 3
    • References ............................................................................................................ 27 White papers .................................................................................................................... 27 Product documentation .................................................................................................... 27About EMC Isilon ................................................................................................... 28 Contact EMC Isilon ........................................................................................................... 28 Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 4
    • Executive summaryAs enterprise server technology becomes increasingly powerful with each newgeneration of hardware, virtualization technologies and capabilities grow at the samepace. The end result for most organizations is an ever-increasing virtualizedinfrastructure. As more and larger workloads are migrated to a virtualizedenvironment, the data sets associated with those workloads grow in both size andcriticality.At the same time, organizations with 24x7 service level expectations and obligationsare forced to consider strategies to keep services available at all times while ensuringthat their data is protected. While a single backup solution is sufficient for manyorganizations, the ever-increasing complexity and criticality of an organization’s data,coupled with the need to recover data against multiple failure scenarios, oftenrequires organizations to consider a multiple layer approach to data backup andrecovery. In the case of virtualized environments hosted on EMC® Isilon® storage,EMC offers both native features and interoperability with enterprise-class dataprotection solutions to assist customers in meeting their backup and recoveryobjectives.This white paper steps through several of those solutions to assist organizations increating a comprehensive data protection strategy. By using EMC Isilon SyncIQ® andSnapshotIQ® data protection software, as well as by deploying virtualization-awarebackup and recovery solutions, customers can maintain the continuous availability ofvirtualized workloads and services, while ensuring their ability to recover criticalenterprise functionality and services in the event of data loss or corruption.Business caseFor most organizations, server virtualization has yielded both significant cost savingsand service improvements. The downside in many cases is a proliferation of virtualservers and virtualized data. Often, organizations with a high percentage of theirworkloads virtualized find their traditional approach to backup and recovery is nolonger adequate. Consider the following:• Using traditional backup agents installed in each VM results in a significant amount of overhead on the virtualized infrastructure, as each VM runs nearly identical processes to back up nearly identical operating system data.• As virtualized infrastructures continue to scale up, a traditional tape-backup system may not be able to scale at the same pace. Often, organizations find themselves constrained by traditional tape solutions that make full backups impossible to complete within the designated backup window.• At the same time, as service demands and business operational requirements increase, those same organizations are seeing their backup windows continue to shrink. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 5
    • • Application consistency can be an issue if an organization’s backup solution is not VM-aware.• Using multiple backup solutions independently—e.g. separate backups for operational recovery, application consistency and disaster recoverability—often exceed the allotted backup window, and consume excessive backup media.• In the event of a large-scale loss of data, such as multiple concurrent hardware failures leading to the loss of multiple LUNs—a traditional backup approach would require a lengthy restore process to recover all VMs affected by the outage.Solution overviewThe solution described in this white paper features a multi-tiered approach tosafeguard the integrity and the recoverability of critical organizational and systemdata. It takes advantage of data protection features from VMware, Symantec and EMCto ensure that, in the event of a system or site failure, organizations who adopt abackup and recovery strategy similar to the one enumerated in this paper can quicklyand reliably recover impacted data and services.This solution is based on NetBackup 7.5 with deduplication from Symantec,combined with VMware vSphere APIs for Data Protection (VADP) feature and EMCIsilon SyncIQ. It provides a multi-layered strategy to protect vSphere data, includingonsite, disk-based data, replication to an offsite disk cluster, and multi-tieredrecovery strategies.Validation of the solution was achieved at multiple steps along the way as well. Thesteps of verifying the integrity of the overall solution included:• Monitoring the success of the VADP-based backup policies as configured and scheduled in NetBackup 7.5• Checking the state of data deduplication during the backup process and monitoring the size of backup jobs relative to the original data footprint.• Ensuring successful replication as scheduled between the primary Isilon storage cluster and the offsite disk cluster using Isilon SyncIQ.• Monitoring replication size, duration and success of the SyncIQ daily synchronization.• Verifying the successful recovery process of a VM using the locally-stored backup data.• Performing a successful recovery of the same VM using the remote backup dataset as the basis for restoring the data.Key results and recommendationsEMC recommends leveraging the latest developments in virtualization-aware backuptechnologies, and applying them to create a comprehensive data protection strategythat is designed to protect data at multiple levels and multiple locations. The intent is Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 6
    • to safeguard corporate and organizational data at multiple points, including file-level,VM-level, and site-level protection.Additionally, while the solution itself is a straightforward design, the criticality of thedata being protected requires careful planning, implementation and testing prior togo-live. Confirmation of the solution’s ability to deliver a complete backup solution,one that fits within operational and organizational constraints, is necessary tovalidate a stable backup environment that provides sufficient protection for theorganization’s data, and the services and processes that depend on it.Finally, in order to ensure the continued reliability and performance of the solution,regular monitoring of the jobs and processes that comprise the solution is essential,as is the practice of performing regular data restoration exercises. Doing so willvalidate both the data and the procedures necessary to successfully recover fromdata loss or corruption.IntroductionPurposeThe purpose of this white paper is to describe how a multi-tiered, multi-site backupsolution can be deployed to protect data in a VMware vSphere environment usingvSphere data protection APIs along with Symantec NetBackup 7.5 and EMC IsilonSyncIQ. This Guide outlines the setup, configuration, and functional testing of a disk-to-disk backup environment for this solution. The guide features the latest generationof virtualization-aware backup technology, coupled with native snapshot andreplication functionality from EMC Isilon.While the solution modeled in this guide specifically leverages Symantec NetBackup,this is only one example of an enterprise VM data protection platform. Although thisis a specific approach to vSphere backups, EMC recognizes that the specificrequirements that drove the solution in this guide are not universal.Many organizations have disparate industry-level and/or regulatory requirements fordata protection, so every organization will have unique requirements and challenges.The data protection solution described in this paper, while designed to meet aspecific set of protection criteria, may not meet every need for another organization.Rather, this guide should be used to illustrate a multi-tiered approach to dataprotection, which simplifies operations while minimizing the computing overheadfrequently associated with traditional enterprise backup solutions.The solution presented here recognizes that enterprise backup strategies need tomake use of the advancing technology that virtualization provides, while still needingto rely on more traditional methods like multiple layers of protection across multiplelocations. A VADP-aware backup solution simplifies the backup platform byeliminating the need to install backup agents on every VM, and the change-blocktracking feature means that only changed data needs to be backed up. At the sametime, deduplication reduces the amount of data that SyncIQ needs to replicate via Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 7
    • WAN link to the offsite cluster. Finally, the use of disk-based backup as the primaryoffsite storage medium increases the speed at which data is transferred offsite. Takentogether, the approach described in this paper results in simplified management,increased data protection, improved efficiency, and reduced backup windows formost organizations that adopt this strategy.ScopeThe scope of this white paper is to describe a backup solution for capturing,deduplicating and archiving enterprise VM data. It does not cover specificconfiguration steps of the entire solution, nor does it address the need for monitoringor verifying successful backup status. Every organization will have unique tools andrequirements for those componentsAudienceThis white paper is intended for backup administrators, systems engineers andconsultants to couple the next generation of virtualization-aware backup solutionswith Isilon storage for their VMware environments. It describes how Symantec backupand deduplication tools combine with Isilon snapshot and replication software tosimplify the backup and recovery of virtual machines and accompanying data. Itassumes the reader is familiar and proficient with the following:• Virtualization using VMware vSphere 5.0• EMC Isilon storage design and management• EMC SnapshotIQ and SyncIQ data protection software• Command-line syntax for administration of VMware vSphere and the EMC Isilon OneFS® operating system. The automation of some tasks in this guide requires the use of the VMware vSphere Management Assistant and SSH connectivity to the Isilon storage cluster.• Backup and recovery software and concepts• Microsoft Windows Server 2003 or later, including hardware-level management, NTFS permissions, and Active Directory authenticationTerminologyThis paper includes the following terminology.Table 1: Terminology Term Definition Backup policy The procedures and rules used to ensure adequate data backups can be made. As configured within most backup software, backup policies typically include parameters such as acceptable backup windows, the systems and data to be backed up, the type of backup to be performed, and the destination media. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 8
    • Term DefinitionBackup window The time period required to complete a backup procedure. Backup procedures typically require the same computing and network resources as production workloads, so if a backup procedure exceeds its allotted backup window it can have detrimental effects on production system performance. Additionally, backups that fail to complete in the time available can put production data at risk if a system failure occurs before data can be successfully backed up.Change Block Tracking A VMware feature that helps perform incremental backups by tracking which disk sectors within a VMDK file have changed since the last backup. This feature can be leveraged by 3rd party backup and recovery software.Cumulative Incremental A backup of all changes made since the last full backup.backup Compared to differential incremental backups, cumulative incremental backups require a longer backup window to complete successfully, but shorten the time necessary to restore data after a failure or loss.Data deduplication A data-reduction technique in which redundant data within a given dataset is reduced to a single instance, and subsequent occurrences of the same data are replaced with a link back to the first instance. This technique is used to improve storage utilization, or reduce the amount of data to be transferred via network link.Differential Incremental A backup that contains only the data that has changed since thebackup more recent backup. Compared to cumulative incremental backups, differential incremental backups require a shorter backup window, but lengthen the recovery time after a loss of data.Full backup A complete backup of all selected data on a given system.Recovery Point Objective Abbreviated as RPO, the maximum amount of data loss, expressed in units of time, that an organization can afford to sustain. For example, a 12-hour RPO for a given organization means that the organization can afford up to lose up to 12 hours of data in the event of a major incident. In designing a data protection strategy, the RPO determines the frequency of the backup cycle that an organization must plan for, the frequency and size of the offsite synchronization process and the computing and network infrastructure necessary to support the designated RPO.Recovery Time Objective The RTO is the duration of time allowed for system and data recovery in order to restore business functions after a service disruption. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 9
    • Term Definition Snapshot A point-in-time capture of the state of a file or file system. Snapshots streamline access to stored data and can speed up the process of data recovery. Synchronization The process of making sure that two or more locations contain the same up-to-date files.AssumptionsThe following conditions were applied to the test setting. Results may vary if thesesame conditions are not in effect in the customer’s environment.• This guide assumes the use of Windows Server 2003 or later guest operating systems. Results may not be consistent with Linux or earlier versions of Windows.• An Active Directory domain authentication source is necessary for the NetBackup Master and Media servers to communicate with one another.• Symantec NetBackup 7.5, VMware vSphere 5 and EMC Isilon OneFS 6.5.x with Isilon SyncIQ 3.0 were used in this environment. Other versions of these platforms may yield different results.• Specific components leveraged in this solution may require additional licensing to enable and use the full feature set of capabilities necessary for the multi-tiered data protection strategy outlined in this document.Additionally, the implementation and test procedures outlined in this document donot give exact instructions for each step. It is assumed that the audience is familiarenough with the tools and interfaces of the different products in this solution toperform them without explicit guidance.Data protection strategies and use casesData protectionAs the name implies, ‘data protection’ as a term refers to the safeguarding ofimportant data against corruption, alteration, or loss. A well-planned and well-executed data protection strategy can protect critical organizational data against anyor all of these scenarios. It can also serve as a defense against a disaster event, inwhich critical infrastructure and data is permanently inaccessible or destroyed. Formany organizations, in a number of industries, an effective data protection strategy isrequired by law, and organizations must ensure that their overall approach iscompliant with appropriate legal and regulatory requirements.The elements necessary for sufficiently thorough data protection are driven in part bythe size and nature of the organization. It requires a multifaceted approach thatincludes a combination of technology, business processes and overall best practices.The specific mix of practices, processes and technologies necessary for an Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 10
    • appropriate strategy is driven in part by the size of the organization, and in part by itsnature. For any organization, the objective of a data protection strategy is to minimizethe risk of adverse business impact or loss resulting from inaccessible or unverifiablebusiness data.The practices to consider in planning and implementing a data protection strategyinclude:• Backup and recovery• Backup integrity verification• Offsite data replication• Storage system security• Data Lifecycle ManagementBackup and recovery refers to the protection of data by making one or more offlinecopies of the data to be restored in the event of disaster or data loss. Backup integrityverification requires the regular recovery of backup datasets to an alternate locationto ensure the validity and accessibility of system and data backups.Offsite data replication is the real-time or near-real-time moving of data to a remotesite, external to the primary storage system, to enable recoverability in the event ofphysical damage to systems and facilities. The two most common forms of thistechnique are remote copy and replication. These techniques duplicate data from onesystem to another, in a different location.Storage system security refers to taking appropriate measures to ensure theprotection of the storage system itself against improper access. Minimallyappropriate measures include the following: physical security of the facility in whichthe storage is housed, network access control to safeguard against improper dataaccess, alteration, or theft, and antivirus software to protect against data corruption.Data Lifecycle Management is the automated movement of critical data to online andoffline storage. Potential aspects of include moving data considered to be in a finalstate onto read-only storage, where it cannot be changed, and moving data todifferent tiers of storage based on its age.While this document focuses primarily on the technologies necessary to support acomprehensive data protection solution, it is not intended to serve as a universalguide for all organizations or all requirements.Use casesThe solution described in this guide is designed to address the following dataprotection scenarios and objectives:• Disaster recoverability• Business continuity• Disk-to-disk backup Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 11
    • • Disk-to-tape backup• Offsite data protectionDisaster recoverability requires the implementation of processes and mechanisms toenable the recovery and continuation of technology infrastructure and criticalbusiness data in the event of a large-scale disaster. At a minimum, disasterrecoverability requires a reliable, accessible offsite data repository located at asufficient distance from the primary site to be unimpacted by the initial disasterevent. Business continuity refers to the necessary steps to ensure that criticalbusiness functions will remain available in the event of a disaster.Disk-to-disk (D2D) backup is the backing up of data from one disk to another, ratherthan to removable media such as tape. Relative to tape-based backup, D2D offershigher backup speeds and capacities, which can reduce backup times. Because D2Dprovides nonlinear access to data, it can significantly decrease recovery times aswell. Disk-to-tape (D2T) backup typically refers to the transfer of data from disk toremovable media, which can then be disconnected and stored offline.Offsite data protection refers to the sending of critical data from the primary site aspart of a disaster recovery plan. Data is either transported via tape or optical media,or replicated to an offsite storage array through a wide-area-network link. Thisensures the recoverability of critical data in the event of a disaster or system failure.Technology overviewOverviewThe solution described in this white paper leverages the following technologies aspart of a comprehensive data protection strategy for a vSphere infrastructure.• VMware vStorage APIs for Data Protection• Symantec NetBackup 7.5• EMC Isilon SyncIQVMware VADPBeginning with vSphere 4.0, VMware introduced the vStorage APIs for Data Protection(VADP) as a means for offloading backup processing from the host. Not only doesVADP allow for easier access to a VMs virtual disk file, but it also contains featuresthat can improve backup speeds, such as Changed Block Tracking (CBT). CBT keepstrack of any blocks that have changed since the last backup. As a result, a backupapplication can simply query the VMkernel to identify the specific blocks needing tobe backed up. This means the backup application no longer needs to track this,resulting in much quicker incremental backups. VADP provides a much more efficientmechanism for backing up when used with backup applications that take fulladvantage of them. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 12
    • Symantec NetBackup 7.5NetBackup for VMware provides backup and restore of the VMware virtual machinesthat run on vSphere servers. NetBackup leverages VADP to offload backup processesfrom the ESXi hosts to a designated vSphere backup server. It includes the followingcapabilities:• Off-host backup of virtual machines. This reduces the backup processing load on the ESXi host(s).• Increased backup speeds relative to standard file-order backup methods.• Automatic creation of quiesced snapshots of Windows virtual machines using Windows Volume Shadow Copy Services (VSS). It can also create quiesced snapshots of Linux virtual machines if an appropriate utility (e.g. SYMCquiesce) is installed.• Use of snapshot technology to keep virtual machines 100% available to users.• Full and incremental backups, including block-level incremental backups, of virtual machines regardless of the VM’s power state.• VM- and file-level restore capabilities.• NetBackup offers deduplication at a number of levels.Isilon SyncIQ 3.0EMC Isilon SyncIQ replicates data in an Isilon storage environment from a primary siteto a secondary, local or remote site, creating an offline copy of the data for disasterrecovery, business continuance, disk-to-disk backup, or remote archiving purposes.SyncIQ delivers high-performance, asynchronous replication of data over both short(LAN) or long (WAN) distances, providing protection from site-specific and regionaldisasters. It includes a robust, policy-driven engine that allows for the customizationof replication datasets to minimize system impact while still satisfying dataprotection requirements.vSphere Management AssistantThe vSphere Management Assistant (vMA) from VMware is a Linux-based virtualmachine that includes prepackaged software such as the vSphere command-lineinterface, and the vSphere SDK for Perl. vMA allows administrators to run scripts oragents that interact with ESXi hosts and the vCenter server through a single portal.Using the vMA, administrators can manage vCenter services and hosts, as well asconnect via SSH to other command-line administrative consoles, including the Isilonstorage cluster, either for automating management tasks via script or for single-console access to multiple systems. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 13
    • Body of workConfigurationThis section describes the solution as configured. Specific hardware and softwareconfiguration details are provided in Component Hardware and Software section ofthis guide.Process overviewBackup of the vSphere infrastructure in this solution uses Symantec NetBackupSymantec and EMC Isilon SyncIQ according to the following processes:1. The Master Server coordinates with the vSphere infrastructure, using VADP, to create snapshots of the VMs in the cluster for backup to one or more Media Servers. Specific backup settings are defined in one or more backup policies on the Master Server. Examples of settings defined in NetBackup policies may include: a. Backup schedule b. Scope of backup, e.g. all drives, specific volumes, exclusion of pagefiles, etc. c. Type(s) of backup, such as Full, Incremental, or Differential. d. Staging of backup data to the Media Server(s), including location, deduplication settings, etc.2. Data from the snapshot(s) created in Step 1 is then streamed to the Media Server.3. The Media Server stages the backup data to one or more storage repositories as defined in backup policies on the Master Server. In this solution, backup data is also deduplicated at the time of staging in order to reduce the storage footprint of the backup data.4. Using SyncIQ, the deduplicated data is replicated to an offsite Isilon storage cluster for archival and potential disaster recovery purposes.5. Additional offline archival takes place as data is backed up to tape. This can also be configured from the primary cluster.Solution architectureFigure 1 below illustrates the physical and process architecture of the backupsolution described in this guide. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 14
    • Figure 1: vSphere Infrastructure Solution Architecture with NetBackup Data ProtectionPhysical environmentThis section describes the physical components of the environmentHardware configuration:• At the primary site, the backup solution utilizes five vSphere ESXi hosts, running on a 9-node Isilon cluster composed of three Isilon S-Series nodes, three Isilon X- Series nodes, and three Isilon NL-Series nodes.• At the secondary site, three vSphere ESXi hosts connect to five NL-Series nodes which replicate with the primary Isilon cluster using SyncIQ.• The NetBackup 7.5 infrastructure in this solution consists of two Windows Server 2008 R2 VMs at the primary site, one configured as the Master Server, and the other set up as the Media Server.Software configuration• The Master Server manages backup policy creation along with the creation, configuration, scheduling, and execution of backup jobs. It communicates with the vCenter server for VADP tasks, and with the Media Server on the actual streaming of VM data for backup. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 15
    • • The Media Server manages connections to the individual ESXi hosts in the vSphere infrastructure to receive VM data for backup and deduplication. It also provides connections to the storage pools that are used for staging backup data.ValidationThe following operational test cases were used as part of the validation process toverify the reliability and performance of the solution design as described in thisdocument. These tests were intended to verify and document the ability of thesolution to protect organizational data against multiple failure scenarios:• Image-level backups• Recovery of an individual VM from local backup• Recovery of individual files from within a backed-up virtual machine• Recovery of individual server from offsite copyThis section describes the test procedures as executed against the design for all ofthe failure and recovery scenarios described above.Scenario #1: image-level backupsUsing VADP to connect to vSphere, NetBackup 7.5 creates a VMware-level snapshotof the virtual machine, and then presents it to the Media Server which backs up theVMDK files that make up the VM. Properly configured, this enables the option ofeither performing image-level recovery or file-level recovery from a single backupprocess.Test objectivesThe purpose of this test is to ensure the reliable backup of all VMs configured in thetest backup policy according to the policy settings. These include a full weeklybackup and a daily incremental backup per VM within the designated backupwindow. It also validates the deduplication of the VM data at the time of backup tothe Media Server in order to minimize disk space used and the necessary bandwidthto replicate the backup data to the remote site.In this scenario, a policy was created in which the VMs on the test vSphere clusterwere configured for daily backups. The policy was configured to perform full backupsonce per week, with incremental backups for each VM occurring on the other days. Tominimize disk space, the NetBackup Media Server was configured to deduplicatebackup data as it streamed from the individual VMs. This reduced both the amount ofdisk space necessary at the primary site to store the local backup data, as well as theamount of data that needed to replicate via WAN link to the secondary site.Test procedures1. Create a 1TB VMDK file and attach it to the Media Server. From within the Media Server’s guest operating system, mount the volume as E:. Using the NetBackup Administration Console on the Master Server, create a storage unit on the Media Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 16
    • Server, assign it a name, set the storage unit type to Disk, and set the disk type to PureDisk. Configure disk deduplication from the NetBackup Administrative Console.2. From the NetBackup Master Server, create a new VMware Backup Policy. Within the policy, configure the Policy Name, VMware backup host, backup options (e.g. block-level backups, exclusions, transport modes, and application protection settings as applicable). Select the virtual machine(s) to be backed up as part of the policy. Enable full backups and differential incremental backups as part of the policy, then set the full-backup frequency to weekly, and the incremental-backup frequency to daily, and the retention policy for two weeks. Set the backup window to run during off hours. Save the policy.3. Allow the policy to run through two full weekly backup cycles to ensure full and differential incremental backups completed successfully.4. Check Activity status on Master Server, check log reports to verify backup success.Test resultsBackup policies successfully executed against all in-scope VMs over a period of twoweeks. Activity logs on Master Server showed backups occurring on a daily basis forall VMs configured in the test policy, with Full and Differential-Incremental backupshappening as scheduled. Disk deduplication rates per server varied.Scenario #2: recovery of a VM from local backupThis scenario is a validation of the ability of the solution to recover an entire failedVM. Situations in which this might become necessary include inadvertent VM deletionwithin vCenter, file-system corruption, or development environment reset.Test objectivesConfirm the recoverability of individual VMs from the backup images. Ensure that VMscan be restored from any backup point over the retention cycle and that all VM dataand settings are preserved in the restored copy.In this test, the target VM was deleted from within vCenter, and the NetBackupBackup, Archive, and Restore console was used to recover the VM back to its originalconfiguration.Test procedures1. Identify a target VM for the restore test in vCenter. Confirm successful backups of the test VM through the NetBackup Backup, Archive and Restore application. Confirm also that the backup history lists restore points for every day over the two- week lifecycle of the retention policy.2. Power down the target VM within vCenter. Delete the VM from disk.3. Run the Restore process, choose a recovery point from the available backups, and recover the VM to its original location and retain the original settings. Configure Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 17
    • the restore job to power the VM up upon completion. Run the Pre-Recovery Check, and initiate the recovery process. Monitor recovery status through the Administration Console.4. When restore process completes, log into recovered VM and verify network, disk and configuration settings.Test resultsRestore process completed successfully. Upon boot, VM displayed the Windows ErrorRecovery Screen indicating its last shutdown had not been successful. After logginginto the guest operating system console, the Shutdown Event Tracker appeared. TheVMs configuration, settings and files were otherwise completely recovered.Scenario #3: recovery of individual files from a VM backup imageThis scenario is a variation on the recovery of a full VM image from local backup. Itextends the test in Scenario #2 with additional procedures designed to restoreindividual files from a VMs backup image. The test added the process for returningfiles to their original location on the VMs file system while leaving the rest of the VMintact, i.e., rather than a wholesale replacement of the entire VM to recover one ormore individual files, this test validated a procedure for recovering files by extractingthem from the backup image.Test objectivesVerify that individual files can be recovered from the backup image and restored totheir original location on the target VM.Test procedures1. Identify a target VM for the restore test in vCenter. Confirm successful backups of the test VM through the NetBackup Backup, Archive and Restore application. Confirm also that the backup history lists restore points for every day over the two- week lifecycle of the retention policy.2. Identify target files on the VM for recovery. Note the files’ names, size, contents and path. Delete the files from within the guest operating system by moving them to the Windows Recycle Bin.3. Identify another VM which can be used to stage the recovered virtual disk(s) for restoring the deleted files. This VM should be running the same guest operating system as the target VM, and should be a member of the same authenticating domain.4. Launch the Backup, Archive and Restore application on the Master Server. Run the Restore process and choose a recovery point from the available backups. Configure the VM to be recovered to an alternate datastore, and note the new location. Enter an alternate name (e.g. “<VM-name-Files>”and an alternate datastore. To reduce the recovery time, choose Thin Provisioning for the restored virtual disks. Clear all network connections to avoid potential network and domain Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 18
    • conflicts when the VM powers on after recovery completes. Run the Pre-Recovery Check, and initiate the recovery process. Monitor recovery status through the Administration Console.5. When restore process completes, power on the VM and verify recovery of target files as noted in step 2 above, including folder path, file name(s), size(s), and contents. If the file(s) to be recovered may have NTFS permissions different from those of the parent folder, note these as well. Power down the VM and remove it from the vCenter inventory, but do not delete its files.6. From within vCenter, edit the settings of the staging VM identified in step 3 above. Add a hard disk to the staging VM and choose the “Use an existing virtual disk” option, then browse to the appropriate datastore selected in step 4 above and locate the appropriate VMDK file(s) to attach to the staging VM.7. Log on to the staging VMs console. From Disk Manager, mount the newly attached disk and assign it a new drive letter. Open Windows Explorer and browse to the correct path within the newly-assigned drive letter to locate the file(s) to be recovered.8. Copy the file(s) from the staging VM back to their original location on the target VM. If the NTFS permissions on the restore(d) files differ from the parent directory, these will need to be manually restored, since copying the files back to their original location will cause the restored file(s) to inherit permissions from the destination parent folder.9. After verifying the restore back to the original location(s) on the target VM, unmount the drive from within the guest operating system of the staging VM.10. From vCenter, detach the VMDK file from the restore guest. The VMDK file can either be left on the datastore or deleted at this time.Test resultsRestore process completed successfully. Individual files were completely recoveredback to the original state at the time of backup.Note: While this process works for individual files, it can also be used to restore acomplete virtual disk back to the target guest machine, without having to reset theentire VM back to its last backup state.Scenario #4: recovery of a VM from remote backupThis final test is intended to demonstrate the ability of the solution design to replicatethe entire backup dataset to an offsite location using SyncIQ, then recover individualor multiple servers to the offsite location.Test objectivesReplicate the Master and Media servers to an alternate site, on a second Isilon clusterand accessible via a second vSphere cluster. Use the replicated NetBackup Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 19
    • environment to recover VMs to an alternate location and bring services and/or dataonline in the new site.Note: This procedure assumes that one or more domain controllers from the samedomain are online at the recovery site. Using this process to backup and restoredomain controllers may lead to domain-level corruption. Active Directory domaincontrollers should be backed up using one or more supported alternate ActiveDirectory-aware methods.Test procedures1. From the OneFS WebUI on the primary (source) Isilon cluster, create a new directory. Create an NFS export that references the new directory. This will be the source directory for SyncIQ replication that will be configured in Step 3 below.Note: For best results, create the new directory and NFS export using the proceduresoutlined in the “Deployment Guide – EMC Isilon Scale-Out Storage and VMwarevSphere 5” white paper. A link to that document is included in the References sectionof this guide.2. From the OneFS WebUI on the secondary (target) Isilon cluster, create a new directory and NFS export, using the same processes as in Step 1 above. This will be the target directory for SyncIQ replication that will be configured in Step 3 below.Note: As with Step 1, follow the procedures outlined in the “Deployment Guide – EMCIsilon Scale-Out Storage and VMware vSphere 5” white paper to complete this step.3. From the OneFS WebUI on the primary (source) Isilon cluster, create a new SyncIQ policy. Give the new policy a name and description, and set the job to synchronize data between the Isilon clusters, and configure the job to run on a daily schedule. Set the source cluster’s root directory to the directory created in Step 1 above. Set the target cluster’s name or address, and set the target path to the directory created in Step 2 above. Save the new policy.Note: If the target cluster is specified using an IP address in this step, then SyncIQ willconnect to only one node on the target cluster to replicate the source directory.Multiple nodes can be connected if a SmartConnect zone on the target cluster isconfigured in this step.4. From the vMA, mount a new datastore on all ESXi hosts in the primary-site vSphere cluster to the NFS export created in Step 1 above. This is necessary for proper vSphere HA and DRS functionality.5. From the vMA, connect to the ESXi hosts in the secondary site and mount a new datastore to the NFS export created in Step 2 above.6. From the Activity Monitor of the NetBackup Administration Console, verify that all backup jobs have completed. Connect to the vCenter client and shut down the Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 20
    • Master Server and the Media Server. Migrate both NetBackup servers to the new NFS datastore created in Step 3 above.7. When the migration of both NetBackup servers to the new datastore is complete, start the initial replication process by launching the SyncIQ job configured in Step 3 above. When the SyncIQ job has successfully launched, power both NetBackup VMs back on and continue running backup jobs as scheduled. The synchronization period will depend on the size of the NetBackup servers’ disks and the speed of the WAN connection. Continue to monitor the SyncIQ job from the WebUI of the source cluster until the directory has been entirely synchronized to the target Isilon cluster.8. Allow the SyncIQ job to run as scheduled for one week before continuing with the next step.9. Ensure that all backups have completed on the Master Server through the Activity Monitor of the NetBackup Administration Console. From within the guest operating system of the NetBackup servers, shut down both servers and ensure they are powered off before continuing.10. Run the SyncIQ job manually. Replication should update only those blocks changed since the last synchronization job and will ensure full symmetry between source and destination data. Monitor the SyncIQ replication job until it completes.Note: The SyncIQ job will create a read-only directory on the target cluster. If viewingthe target cluster’s file system using the WebUI, a red padlock icon will appear on thetarget directory.11. From the WebUI on the target cluster, break SyncIQ policy created in Step 3 above. This will enable read-write access on the target directory and allow the ESXi hosts at the secondary site to mount the NetBackup VMs replicated from the primary site.12. Shut down both NetBackup servers at the primary site. From a vSphere client connected to the target vSphere cluster, remove both the Master Server and the Media Server from the vSphere inventory. Then, browse the datastore added in Step 5 to the ESXi hosts at the target site until the folders for the two NetBackup servers are located. Add both servers to the local vSphere cluster’s inventory.13. Ensure both servers have network connections to the virtual network on the target vSphere cluster. Boot both NetBackup servers and change the IP address to match the local subnet, and update the DNS server information as necessary. Verify connectivity between the restored NetBackup servers at the secondary site, and with the Active Directory domain.14. Identify a target VM from the backup catalog to restore to the secondary site. Remove the target VM from vCenter. From the Backup, Archive, and Restore console on the Master Server at the target site, launch the VM restore process. Select an alternate location to recover the image. Change the vCenter server if necessary, as well as selecting an appropriate local ESXi host, Datacenter, Folder and datastore/datastore cluster. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 21
    • 15. Configure the restore job to power the VM up upon completion. Connect the VM to the appropriate local network. Run the Pre-Recovery Check, and start the recovery process. Monitor recovery status through the Administration Console.16. When restore process completes, log into recovered VM. Change the IP address to match the local subnet settings and DNS services. Verify connectivity to the network and to Active Directory.Test resultsReplication of approximately 1TB of data across the WAN link between the primaryand secondary sites took 23 hours. Subsequent daily synchronization jobs tookroughly one hour each. Target VM restored to the new destination and successfullyfound the network and Active Directory upon reboot.Test notes• Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, this workload should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated.• All performance data contained in this report was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly.• EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute.Component hardware and softwareThis section lists the hardware used to validate the solution.Storage configurationThe storage configuration as described in this solution is intended to include a fullrepresentation of the available node types in a typical Isilon storage array. Table 2describes the hardware and network specifications for each node type in the primary-site cluster, which consists of a total of nine nodes. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 22
    • Table 2: Primary-Site Storage Node Hardware Specifications Node # Node Type Quantity Capacity per Node RAM Network Interfaces per Node 1-3 36NL 3 33,256 GB HDD 4GB 2x1Gb/s Ethernet 2x10Gb/s Ethernet 4-6 X200 3 184 GB SSD 24GB 2x1Gb/s Ethernet 9,626 GB HDD 2x10Gb/s Ethernet 7-9 S200 3 184 GB SSD 48GB 2x1Gb/s Ethernet 7,066 GB HDD 2x10Gb/s EthernetThe secondary-site storage cluster consists entirely of NL-series nodes, as shown inTable 3:Table 3: Secondary-Site Storage Node Hardware Specifications Node # Node Type Quantity Capacity per Node RAM Network Interfaces per Node 1-5 36NL 3 33,256 GB HDD 4GB 2x1Gb/s Ethernet 2x10Gb/s EthernetVMware ESXi host hardware configurationThe vSphere cluster consists of eight identically configured host servers, five in theprimary site, and three additional hosts, running additional localized workloads atthe secondary site. Table 4 describes the hardware configuration of each ESXi hostserver. The host count and configuration were selected based on the workload profile,and on the expected CPU and memory resource requirements. The host count andconfiguration are also based on the need to maintain sufficient vSphere clustercapacity in the event of a host hardware failure. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 23
    • Table 4: Host Hardware Specifications Host Hardware Hardware Configuration Host Count 5 at primary site 3 at secondary site CPUs per host 2 Number of cores per CPU 8 Number of cores per host 16 CPU type Intel® Xeon® E5606 CPU speed 2.13GHz Memory per host 48GB Network connections 1Gb/s Ethernet interfaces, per host 2 10Gb/s Ethernet interfaces, per host 2 Datastore connections NFS v3 Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 24
    • Environment resources and configurationsTable 5 lists the environment components, including component software, softwareversions, and configuration settings used in this solution.Table 5: Environment Components and Configurations Software Configuration Isilon Array OneFS operating system v6.5.4.15 at primary site v6.5.4.10 at secondary site SyncIQ v3 NetBackup 7.5 Servers Operating System Windows Server 2008 R2 Standard Edition, Service Pack 1 vCPU, per VM 4 RAM, per VM 4GB SCSI Controller LSI Logic SAS VMware Tools for Windows v8.6.0, build-515842 VMware Hardware Version 8 Symantec NetBackup v7.5 Master Server Disk capacity 1x40GB VMDK Media Server Disk Capacity 1x40GB VMDK 1x1TB VMDK vSphere Management Assistant appliance v5.0.0.1 Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 25
    • ConclusionSummaryBy combining Symantec’s VADP-aware NetBackup 7.5 backup and deduplicationsoftware solution with EMC Isilon scale-out storage platform and SyncIQ dataprotection software, organizations can provide a comprehensive data protection andrecovery strategy. NetBackup provides efficient off-host backup that integrates withvSphere to protect VMDK images and critical VM data against loss or damage. SyncIQprovides an efficient mechanism for replicating data between Isilon storage clustersacross multiple sites. Finally, by deduplicating VM data at the time of backup with theNetBackup PureDisk capability, data consistency is preserved at all levels withoutoverwhelming disk or network capacity resources.For organizations with workloads beyond those in a vSphere cluster, NetBackup andIsilon both offer simplicity and scale-out capabilities to ensure that organizations canexpand their data footprint and backup capabilities to other platforms and serviceswhile maintaining the same level of simplicity that customers have come to expectfrom EMC’s Isilon storage.FindingsNetBackup 7.5, combined with Isilon SyncIQ, can be used for both local and offsitebackups. If the vMA is deployed in an organization, the process of shutting down theNetBackup VMs and launching a SyncIQ policy can be scripted and automated,eliminating much of the dependency on human administrators. The overall success ofsuch a solution in any given organization will depend on multiple factors, including:• The organization’s designated Recover Point Objective• How much data an organization needs to recover within the target Recovery Point Objective. The rate of data change per day• The type and format of the organization’s data. Some data types can be deduplicated at a much better rate than others• The speed of the organization’s WAN link between sitesFinally, before deploying any backup solution into a production environment, acomprehensive testing and pilot program is should be undertaken to ensure both theadequacy and the cost-effectiveness of the desired backup solution.Next stepsEMC can help accelerate the assessment, design and implementation of a vSpherebackup solution on an Isilon storage platform while lowering the implementationrisks.To learn more about this and other solutions, contact an EMC representative. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 26
    • ReferencesWhite papersFor additional information, see the white papers listed below.• Best Practices for Data Replication with EMC Isilon SyncIQ• Best Practice Guide – EMC Isilon Storage and VMware vSphere 5• Deployment Guide – EMC Isilon Scale-Out Storage and VMware vSphere 5• Best Practices Guide – Symantec NetBackup for VMwareProduct documentationFor additional information, see the specific product documents listed below.• Symantec NetBackup for VMware Administrator’s Guide• vSphere Command-Line Interface Documentation• vSphere Management Assistant Guide Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 27
    • About EMC IsilonEMC Isilon is the global leader in scale-out NAS. We provide powerful yet simplesolutions for enterprises that want to manage their data, not their storage. Isilonproducts are simple to install, manage and scale, at any size and, unlike traditionalenterprise storage, Isilon stays simple no matter how much storage is added, howmuch performance is required, or how business needs change in the future. We’rechallenging enterprises to think differently about their storage, because when theydo, they’ll recognize there’s a better, simpler way. Learn what we mean atwww.emc.com/isilon.U.S. Patent Numbers 7,146,524; 7,346,720; 7,386,675. Other patents pending.Contact EMC Isilonwww.emc.com/isilon505 1st Avenue South, Seattle, WA 98104Toll-free: 877-2-ISILON | Phone: +1-206-315-7602Fax: +1-206-315-7501 | Email: sales@isilon.com© 2012 EMC. All rights reserved. Isilon, Isilon Systems, OneFS, and SyncIQ areregistered trademarks of Isilon Systems LLC. Isilon IQ, SmartConnect, AutoBalance,FlexProtect, SmartCache, SmartPools, InsightIQ, and the Isilon logo are trademarks ofEMC. Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage 28