IBM Tivoli Storage Manager Data Protection for VMware - PCTY 2011

Tivoli Storage ManagerData Protection for VMwareJim SmithTSM Architectsmithjp@us.ibm.comMay 24, 2011 © IBM Svenska AB

Disclaimer• The information on the new product is intended to outline our general product direction and it should not be relied on in making a purchasing decision.• The information on the new product is for informational purposes only and may not be incorporated into any contract.• The information on the new product is not a commitment, promise, or legal obligation to deliver any material, code or functionality.• The development, release, and timing of any features or functionality described for our products remains at our sole discretion. 2

Agenda – Session 1• Backup in virtual environments – in-guest, on-host, and off-host• Why you should be excited about vStorage APIs for Data Protection – VMware Consolidated Backup vs. vStorage APIs• TSM Solution Overview• Potential Future Work Items 3

Agenda – Session 2• Backup – How VMware backup data is stored in nodes and filespaces – Virtual vs. Physical deployment – Incremental backup – Versioning and Policy – Scheduling – Tape considerations – Permissions – Deployment considerations• Recovery – File-level recovery options• Reporting• Demo 4

SESSION 1 5

Analyst PredictionsHow do you manage data protection and recovery in your virtual servers? Sources – Gartner, IDC, ESG 6

VMware Infrastructure C: D:GuestMachines vCenterHostMachines(ESX, ESXi)VMFS VMDK NAS/local SAN TSM Storage datastore datastore datastore Hierarchy RDM TSM Server 7

In-Guest Backup Philosophy: treat virtual machines as physical machines TSM Agents placed in eachGuestMachines guest; traditional backup over LAN a vCenterHost Cumulative Result: Individual filesMachines backup load stored in TSM server for(ESX, ESXi) on host hard operational recovery to measure LA N on lyVMFS NAS/local a files SAN TSM Storage datastore datastore datastore Hierarchy This method is generally supported by TSM for VMware ESX/ESXi, Microsoft Hyper-V, Linux KVM TSM Server 8

On-Host Backup Philosophy: treat hypervisor as general purpose OS Optional: manual procedure to backupGuest data from a snapshot;Machines no integrated solution a for making data consistent vCenter ESX only; VMware TSM Linux agent Result: Individual filesHost representing virtual disks and moving away placed on ESX host;Machines from ESX user selectively backs control files stored in TSM(ESX, ESXi) up data stores at server for DR of virtual directory level machineVMFS VMDK NAS/local LA No a rS AN vmdk(s) SAN TSM Storage datastore datastore datastore Hierarchy This method is generally supported by TSM for VMware ESX and Microsoft Hyper-V TSM Server 9

Off-Host Backup Philosophy: exploit access to shared storage from another hostGuestMachines a vCenter Backup Agent orchestrates Result: blocks representingHost snapshots of virtual virtual machine stored inMachines machines and moves data(ESX, ESXi) server for DR of virtual from snapshot to backup machine storage VMDKVMFS NAS/local a SAN LAN or SAN TSM Storage datastore datastore datastore Hierarchy Backup Agent Can also exploit hardware-based snapshots as backup versions TSM Server 10

Virtual Machine Backup MethodsSummaryMethod Philosophy Advantages DisadvantagesIn-Guest Treat virtual machines as •Re-use existing •Typically cannot physical machines processes and exploit SAN products without •Hard to quantify modification cumulative load on hostOn-Host Treat hypervisor as •Re-use existing •Not scalable to general purpose OS processes and large number of vms products with little per host modification •Hypervisor vendors •SAN access moving away from general purpose OS as hypervisorOff-Host Exploit access to shared •Off-load backup •Need to modify storage from another host from hosts backup products •SAN access 11

How VCB Works - Backup C: D: 1. VCB interactedGuest with infrastructure toMachines vCenter vm1 take snapshot (vSphere API)HostMachines(ESX, ESXi) 3. TSM Backup-Archive client moved flat files to TSM ServerVMFS VMDK SAN TSM Storage datastore datastore Hierarchy datastore VC B Proxy = TSM + VCB + 2. VCB “clones” machine to disk staging (VDDK) Disk staging TSM Server 12

How VCB Works - Recovery C: D: 1. TSM Backup-Guest Archive clientMachines vCenter vm1 restored flat files from TSM ServerHostMachines(ESX, ESXi) 2. VMware Converter used to convert recovered files to virtual machineVMFS VMDK SAN TSM Storage datastore datastore Hierarchy datastore VC B Proxy = TSM + VCB + Disk staging TSM Server 13

VCB Limitations• Extra product to install, configure, and maintain separate from backup product• Needed extra disk space to “clone” copies of virtual machine for backup• VCB doesn’t do restore (B=Backup); need to use VMware Converter• Any new function provided in virtual disk development kit (VDDK) may not be available through VCB implementation• VMware tried to move away from VCB in vSphere 4.1; probably will not be supported in future releases 14

vStorage APIs for Data Protection - Discovery and Mapping Based on backup request,Guest TSM agent queries host orMachines vCenter to find virtual guests that are backup candidates, e.g., all guests on an ESX vCenter host or all guests in a specific folderHostMachines(ESX, ESXi)VMFS VMDK NAS/local vSphere SDK SAN datastore datastore datastore vStorage Server (TSM agent) TSM Server 15

vStorage APIs for Data Protection - Snapshot and PreparationGuestMachines 2. Virtual machines given control for backup event to put vCenter data in a consistent state either through Microsoft VolumeHost Shadow Copy Services (VSS),Machines custom scripts or VMware(ESX, ESXi) synch driverVMFS VMDK NAS/local 1. TSM invokes snapshots of vSphere guest machines through SDK vSphere SDK; saves SAN machine configuration information datastore datastore datastore vStorage Server 3. VMware takes software snapshot, new .vmdk created, (TSM agent) virtual machine passed backup complete event TSM Server 16

vStorage APIs for Data Protection – Data Movement TSM agent reads data directly from snapshotGuest VMDK and sends data toMachines TSM server Change block tracking vCenter feature allows to backup only changed blocks sinceHost last backup OR only usedMachines blocks on first backup(ESX, ESXi)VMFS VMDK NAS/local SAN Virtual Disk Development Kit (VDDK) datastore datastore datastore vStorage Server Snapshot VMDK accessed via SAN, LAN, or HotAdd transport (TSM agent) TSM Server 17

vStorage APIs for Data Protection -– TerminationGuestMachines A vCenterHost TSM Server contains block-Machines level image of guest(ESX, ESXi) machine(s) which can be used for operational recovery of individual files or DR of entire machineVMFS VMDK NAS/local A SAN TSM Storage datastore datastore datastore Hierarchy vStorage Server (TSM agent) Software snapshot removed TSM Server 18

vStorage APIs for Data Protection – Recovery 1. Guest machine is redefined by creatingGuest new machine with sameMachines properties (vSphere SDK) 2. VMDK files written back vCenter to virtual machine (VDDK)HostMachines(ESX, ESXi)VMFS VMDK NAS/local 1 vSphere SDK SAN 2 datastore datastore datastore VDDK vStorage Server (TSM agent) TSM Server 19

vStorage APIs for Data ProtectionComparison vStorage APIs for DataVCB Protection• Extra product to install, configure, and • Underlying APIs shipped with backup maintain separate from backup product product • No staging area; data streamed• Needed extra disk space to “clone” directly to/from VMware storage copies of virtual machine for backup • Supports both backup and recovery;• VCB doesn’t do restore (B=Backup); no need for VMware Converter need to use VMware Converter • Can fully exploit underlying VDDK• Any new function provided in virtual function disk development kit (VDDK) may not • Not dependent on host (ESX) software be available through VCB implementation• VMware tried to move away from VCB in vSphere 4.1; probably will not be supported in future releases 20

TSM Data Protection for VMwareBackup-Archive Client DP for VMware• Full-VM backup and restore of • Uses B-A client for base any guest OS using vStorage function at left APIs for Data Protection • Full + Incremental backup• Backup of used blocks only (also via CBT) through VMware’s changed – B-A client provides this block tracking function through license file • Individual file recovery for Windows and Linux guests • Near-instant volume recovery for Windows and Linux guests 21

Components of DP for VMware TSM Backup-Archive ClientGuest responsibilities:Machines 1. Full-VM backups and vCenter Full-VM restoresHost 2. Main UI for DP forMachines VMware(ESX, ESXi) 3. Can be placed on physical or virtual machineVMFS VMDK 4. License file enables incremental backup capability SAN TSM Storage datastore datastore Hierarchy datastore RDM vStorage Server TSM Server 22

Components of DP for VMware TSM DP for VMwareGuest Recovery AgentMachines 1. Mount for individual file vCenter recovery and near-instant volume restoreHostMachines 2. Can be placed on(ESX, ESXi) vStorage Server or inside target virtual machineVMFS VMDK SAN TSM Storage datastore datastore Hierarchy datastore RDM vStorage Server TSM Server 23

POTENTIAL FUTUREENHANCEMENTS 24

Potential FuturevSphere Client Plug-in Enhancement 25

Potential FuturevSphere Client Plug-in Enhancement Datastores What is FlashCopy Manager? IBM Tivoli Storage FlashCopy Manager software provides fast application-aware backups and restores using the snapshot technologies of IBM storage systems. What do you want to do? Define a backup task ... Run a restore ... View current backup status ... Understanding backups Understanding restores 26

Potential FutureFlashCopy Manager – Full-VM Backup Enhancement 1. FCM initiates a software snapshot of virtual guest volumes VM1 (vSphere API) C: D: Linux vStorage Server (physical or virtual machine) /a VM2 FlashCopy ManagerSAN ESXESXi Server VMFS Snaps vmdk vmdk vmdk VM1 TSM Server SAN Storage Subsystem 27

Potential FutureFlashCopy Manager – Full-VM Backup Enhancement 1. FCM initiates a software snapshot of virtual guest volumes VM1 (vSphere API) C: D: Linux vStorage 2. FCM determines which LUN(s) Server are associated with virtual machines (physical or virtual machine) /a VM2 FlashCopy ManagerSAN ESXESXi Server VMFS LUN 2 LUN 1 Snaps vmdk vmdk vmdk VM1 TSM Server SAN Storage Subsystem 28

Potential FutureFlashCopy Manager – Full-VM Backup Enhancement 1. FCM initiates a software snapshot of virtual guest volumes VM1 (vSphere API) C: D: Linux vStorage 2. FCMdetermines which LUN(s) Server are associated with virtual machines (physical or 3. FCM invokes hardware copy virtual machine) /a VM2 services to create a persistent snapshot copy of the LUN(s) hosting the .vmdk and software snapshot FlashCopy ManagerSAN ESXESXi Server VMFS LUN 2 LUN 1 Snaps vmdk vmdk vmdk VM1 LUN 2’ LUN 1’ Snaps vmdk vmdk vmdk VM1 TSM Server SAN Storage Subsystem 29

Potential FutureFlashCopy Manager – Full-VM Backup Enhancement 1. FCM initiates a software snapshot of virtual guest volumes VM1 (vSphere API) C: D: Linux vStorage 2. FCMdetermines which LUN(s) Server are associated with virtual machines (physical or 3. FCM invokes hardware copy virtual machine) /a VM2 services to create a persistent snapshot copy of the LUN(s) hosting the .vmdk and software snapshot FlashCopy ManagerSAN ESXESXi Server VMFS LUN 2 LUN 1 4. Hardware snapshot is Snaps persisted for use as vmdk vmdk vmdk VM1 source for recovery operation, software snapshots are deleted. LUN 2’ LUN 1’ Snaps vmdk vmdk vmdk VM1 TSM Server SAN Storage Subsystem 30

Potential FutureUnified Recovery for VMware Enhancement vStorage API for Data Protection DP for FlashCopy VMware or Manager Common UI using VMware vSphere Client plug-in Hardware Snapshot Tivoli Storage Manager Storage Pool 31

DP for VMware – Official Wiki Pagehttp://www.ibm.com/developerworks/wikis/display/tivolistoragemanager/IBM+Tivoli+Storage+Manager+for+Virtual+Environments 32

SESSION 2 33

Disclaimer• The information on the new product is intended to outline our general product direction and it should not be relied on in making a purchasing decision.• The information on the new product is for informational purposes only and may not be incorporated into any contract.• The information on the new product is not a commitment, promise, or legal obligation to deliver any material, code or functionality.• The development, release, and timing of any features or functionality described for our products remains at our sole discretion. 34

Agenda – Session 2• Backup – How VMware backup data is stored in nodes and filespaces – Virtual vs. Physical deployment – Incremental backup – Versioning and Policy – Scheduling – Tape considerations – Permissions – Deployment considerations• Recovery – File-level recovery options• Reporting• Demo 35

Reference Deployment dcsmc – node=vmbldesx1 – vm1 vm2 vm3 asnode=dc1 dcsmc – vm4 vm5 vm6 node=vmbldesx2 – asnode=dc1 NODENAME DC1 dcsmc –node=vmep1Datacenter = DC1 –asnode=dc2 vm10 vm11 vm12 vm13 vm14 vm15 dcsmc –node=vmep2 –asnode=dc2 NODENAME DC2 vStorage Server TSM Server (TSM agent) Each data mover is a proxy Each data mover stores data agent which moves data on behalf of a proxy target from a ESX host and has its which represents aDatacenter = DC2 own node name datacenter. 36

Data Paths •Transport is independent of Backup Path •VDDK picks best transport in order of SAN, Hotadd, nbdssl, nbdGuestMachines •Can override VMware transport with vmvstortransport Backup-Archive client option •Hotadd transport not available for physical vStorage ServerHost •SAN Backup Path not available whenMachines using virtual vStorage Server(ESX, ESXi) TSM Storage HierarchyVMFS VMDK Transport Backup Path Datastore -> vStorage vStorage Server -> TSM Server SAN Servecr datastore datastore SAN, SAN Hotadd, (LAN-free), nbdssl, TCP/IP nbd vStorage Server TSM Server (TSM agent) 37

Virtual vs. Physical DeploymentPhysical vStorage Server Virtual vStorage Server• Off-loads backup • Backup workload on host workload from host • Supports Hotadd• Supports IP and SAN transport data path to TSM Server • Only supports IP data storage path to TSM Server• No client-side data storage deduplication • Client-side data deduplication 38

Incremental Backup •Current model = traditional full + incrementalGuest Disk Storage PoolMachines •Recovery = restore full + incremental in chronological order, 0 1 2 1 2 e.g., D1 + D2 + D3 + D4Host 3 3 4Machines •VMware Change(ESX, ESXi) Day 1 (Full) Day 3 Block Tracking (CBT) used to track incr. changes 2 3 1 4 2VMFS VMDK •Must use VMware Day 2 HW 7 for CBT Day 4 SAN datastore datastore TSM Storage Hierarchy 39

Versioning • Version defined as the set of 1 full + supporting incremental backups • Honors VEREXISTS and RETAINEXTRA parameters • Example at left – VEREXISTS=2; third full backup will cause the first S M T W T F S week of backups to be expired F I I I I I • No concept of deleting/expiring F I I I I I backup versions if a virtual F machine is excluded from backup domain • Client option VMMC to specify non-default management class 40

Scheduling – Recommended StartingPoint• Set backup domain for each vStorage Server (data mover) to protect an ESX host – Use Domain.vmfull Backup-Archive client option or via Preferences Editor – Can specify “all vms”, Folder, Host, or individual virtual machine – Can remove individual virtual machines from backup processing• Full backups “batched” together – All VM’s full backup during extended time (e.g., weekend) – All VM’s incremental backup during weekdays• Advantages: – This is the easiest approach to configure/schedule – Least number of TSM schedule definitions required (one weekend schedule + one weekday schedule) TSM Data Protection for VWware “Schedule Recommendations” Whitepaper:https://www.ibm.com/developerworks/wikis/display/tivolistoragemanager/Recommendations+for+Scheduling+with+TSM+for+Virtual+Environments 41

Data Representation File Types •BITMAP – masterGuest MBLK0000 Disk Storage Pool bitmap for image (1)Machines •OVF = machine MBLK0001 definition (1) MBLK0002 •CTL/DAT = control DISK 0 JOB1 JOB1 JOB1 + 128 MB data MBLK0000.CTL MBLKnnnn.CTL BITMAP.DATHostMachines JOB2 JOB2 JOB2(ESX, ESXi) MBLK0000.CTL MBLK0003.CTL BITMAP.DAT MBLKnnnn …vmname.OVF JOB1. JOB1 JOB2 = incr. backup MBLK0000.DAT MBLKnnnn.DAT (fewer CTL/DAT files)VMFS VMDK JOB2 JOB2 …vmname.OVF MBLK0000.DAT MBLK0003.DAT SAN datastore datastore TSM Storage Hierarchy 42

Tape Configuration Disk Storage Pool = Meta-DataGuest JOB1 JOB1 JOB1Machines MBLK0000 MBLK0000.CTL MBLKnnnn.CTL BITMAP.DAT VMCTLMC MBLK0001 JOB2 JOB2 JOB2 MBLK0000.CTL MBLK0003.CTL BITMAP.DAT MBLK0002 DISK 0HostMachines(ESX, ESXi) MBLKnnnn Tape Storage Pool = Data JOB1. JOB1 …vmname.OVF MBLK0000.DAT MBLKnnnn.DAT VMMCVMFS VMDK JOB2 JOB2 …vmname.OVF MBLK0000.DAT MBLK0003.DAT SAN datastore datastoreBest Practice Recommendation when using disk / tape - Setthe MIGDELAY (e.g. 10 days) parameter for the disk storage pool TSM Storageto enable the majority of mount requests to be satisfied with Hierarchysnapshots that are on disk. 43

VMware Permissions Windows ActiveDirectory Linux User - or + Set of Privileges Object = Permission > Group•DP for VMware can perform backup and recovery with restricted set of VMware permissions•Recommended to use Active Directory to manage users•For recovery, need permissions at datacenter which encompasses datastore, virtual machines, andnetwork•In general, do not need permissions to interact with virtual machine (e.g., Power on, Power off)•Technote describing minimal permissions:https://www-304.ibm.com/support/docview.wss?uid=swg21497028 44

Summary – Backup-Archive ClientPreference Editor for VMware 45

Other Deployment Considerations• Virtual machines should be migrated to hardware version 7 to take advantage of change block tracking/incremental backup• Plan on 1 – 1.5 processors per instance of the Backup-Archive client (1.5 if you are planning on using TSM client-side data deduplication) 46

Estimated Per-Process Throughput Estimated Per-Process Throughput for Proxy Host Sizing NOTE: These values are estimates intended for proxy host sizing only and are not intended to be a guarantee of actual performance. Deduplication? Low Estimate Medium Estimate High Estimate No 65 GB/Hour 120 GB/Hour 170 GB/Hour Yes 40 GB/Hour 60 GB/Hour 85 GB/Hour Factors to consider when selecting an estimate: • Number of concurrent backup processes within proxy • Datastore I/O capacity • SAN and LAN bandwidth speed and utilization • TSM Server throughput capacity • Deduplication and compression results • Selection of secure transport (nbdssl) • Physical vs. virtual vStorage Server (data mover) 47

RECOVERY 48

General Disk Terminology KEY: C: D: VMDK Virtual representation of physical disk as stored in VMware data store E: F: G: disk Physical disk or block device or LUN as viewed from VMware /boot / guest operating system, e.g., DISK 0 or /dev/sda partition /dvlp /test /prod Logical partition where file system is mounted 49

Individual File Recovery – Windows Partition (via Windows DP for VMware Mount) Supported: Windows 2003 Windows 2008 Windows XP Windows VistavStorage Backup Server TSMor Windows Guest Machine APISoftware required: TSM ServerWindows DP for VMware Mount TSM Storage Hierarchy X: (snapshot of E: at time t3) 3 C: D: Desired partition -> E: F: G: 50

Individual File Recovery – Linux Partition (via Linux DP for VMware Mount) Software required: TSM Supported: Windows DP for VMware Mount API RHEL 5.2-5.4 CygWin (provides SSH) SLES 10 SP2 vStorage Backup Server iSCSI target or Windows Guest (snapshot of /sdb at time t3) Machine 3 TSM Server SSH – TCP/IP iSCS I TSM Storage HierarchyLinux Guest Machine /boot / Software required: Linux DP for VMware Mount iSCSI initiator iSCSI initiator, SSH, Perl, (/dpmount = mdadm snapshot of /dvlp at time t3) 3 /dvlp /test /prod Desired partition -> 51

Individual File Recovery – Linux Disk (via manual iSCSI Initiator) Supported: Platforms supported in previous slides TSM API Best Effort Support Several Linux distributions andvStorage Backup Server Windows 7 -or Windows Guest Machine iSCSI target Search “TSM (snapshot of /sdb at time t3) 3 LINUX ISCSI BEST TSM Server EFFORT” iSCS I TSM Storage HierarchyLinux Guest Machine iSCSI initiator /boot / (/dev/sdc = snapshot of /dev/sdb at time t3) 3 User issues: mount /dev/sdc1 /dpmount /dvlp /test /prod Desired partition -> 52

Individual File Recovery – Linux Disk (viamanual iSCSI Initiator) Windows Linux 53

Linux File Recovery - ComparisonPartition Mount Disk Mount• Interface contained to Linux DP for • Two interfaces: Windows DP for VMware Mount VMware Mount and system tools in• In addition to Windows DP for VMware Linux guest Mount, requires Cygwin (Windows), • Requires only Windows DP for Linux DP for VMware Mount (Linux VMware package on Windows guest), iSCSI initiator (Linux guest) machine; no Tivoli software installed in• iSCSI configuration is not externalized Linux guest to customer • Manual iSCSI target and initiatior• Support for RHEL 5.2-5.4; SLES 11 configuration• Support for major fs types (e.g., ext3, • Same support limitations as partition reiserfs) mount• No support for mounting LVM, dynamic volumes, etc. 54

REPORTING 55

Reporting – Backup Summary Activity log Messages 4141I -4144I and 4149I provide summary information on virtual machines processed 56

Reporting – Identifying Errors Check activity log for other failures by virtual machine name 57

Reporting – Identifying Missed Backups Check filespace table for virtual machines that don’t have current backups 58

Reporting – Other Notes• In general, use “asnodename” for queries• QUERY OCCUPANCY for storage used per vm• Notes on summary records – Summary record created for proxynode agent (not target or “asnodename” like previous queries) – Currently no filespace name in summary records – Restore summary records cut for incremental backups 59

DEMO 60

IBM Tivoli Storage Manager Data Protection for VMware - PCTY 2011

by IBM Sverige

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 3

Statistics

IBM Tivoli Storage Manager Data Protection for VMware - PCTY 2011 Presentation Transcript