Your SlideShare is downloading. ×
Business Continuity and Disaster Recovery for Oracle11g Enabled by EMC Symmetrix VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Business Continuity and Disaster Recovery for Oracle11g Enabled by EMC Symmetrix VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager

994

Published on

This white paper describes a data protection and disaster recovery solution for Virtualized Oracle Database 11gOLTP environments, enabled by EMC Symmetrix VMAXe with Enginuity for VMAXe, EMC …

This white paper describes a data protection and disaster recovery solution for Virtualized Oracle Database 11gOLTP environments, enabled by EMC Symmetrix VMAXe with Enginuity for VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager. It covers both local data protection and automated failover and failback between remote sites.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
994
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
77
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. White PaperBUSINESS CONTINUITY AND DISASTERRECOVERY FOR ORACLE 11g ENABLED BYEMC SYMMETRIX VMAXe, EMCRECOVERPOINT, AND VMWARE vCENTERSITE RECOVERY MANAGERAn Architectural Overview EMC SOLUTIONS GROUP Abstract This white paper describes a data protection and disaster recovery solution for virtualized Oracle Database 11g OLTP environments, enabled by EMC® Symmetrix® VMAXe™ with Enginuity™ for VMAXe, EMC RecoverPoint, and VMware® vCenter™ Site Recovery Manager. It covers both local data protection and automated failover and failback between remote sites. July 2011
  • 2. Copyright © 2011 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware, ESX, VMware vCenter, and VMware vSphere are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number H8207Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 2 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 3. Table of contents Executive summary ............................................................................................................... 6 Introduction to EMC Symmetrix VMAXe ............................................................................................ 6 Business case .................................................................................................................................. 7 Solution overview ............................................................................................................................ 7 Key results ....................................................................................................................................... 7 Introduction .......................................................................................................................... 8 Purpose ........................................................................................................................................... 8 Scope .............................................................................................................................................. 8 Audience.......................................................................................................................................... 8 Terminology ..................................................................................................................................... 8 Key technology components ................................................................................................ 10 Solution components ..................................................................................................................... 10 Oracle application .......................................................................................................................... 10 EMC Symmetrix VMAXe with Enginuity 5875 .................................................................................. 10 Overview ................................................................................................................................... 10 Virtual Provisioning ................................................................................................................... 11 Symmetrix Management Console .............................................................................................. 11 EMC VNX5700 ................................................................................................................................ 11 VMware vSphere ............................................................................................................................ 11 EMC RecoverPoint .......................................................................................................................... 12 Overview ................................................................................................................................... 12 RecoverPoint data protection options ........................................................................................ 12 RecoverPoint appliance ............................................................................................................. 13 RecoverPoint splitter ................................................................................................................. 13 RecoverPoint journals ................................................................................................................ 13 RecoverPoint consistency groups .............................................................................................. 13 VMware vCenter Site Recovery Manager ......................................................................................... 14 RecoverPoint and SRM integration ................................................................................................. 14 RecoverPoint CDP and CRR – how they work ................................................................................... 15 Continuous data protection ....................................................................................................... 15 Continuous remote replication .................................................................................................. 16 Solution architecture and design ......................................................................................... 17 Solution architecture...................................................................................................................... 17 Environment profile........................................................................................................................ 18 Hardware environment ................................................................................................................... 18 Software environment .................................................................................................................... 19 Symmetrix VMAXe storage allocation ............................................................................................. 20 Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 3 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 4. Virtual machine resources .............................................................................................................. 21Oracle database configuration............................................................................................. 22 Oracle database deployment on Symmetrix VMAXe ....................................................................... 22 Virtual Provisioning for Oracle ASM disk groups ........................................................................ 22 Database schema .......................................................................................................................... 22Configuring EMC RecoverPoint ............................................................................................. 24 Preparing the VMAXe for RecoverPoint connectivity ........................................................................ 24 RPA repository and gatekeeper provisioning .............................................................................. 24 Enabling write protect bypass for RPA initiators ......................................................................... 26 RecoverPoint configuration for this solution ................................................................................... 27 Journal size ............................................................................................................................... 27 Installing RecoverPoint................................................................................................................... 28 Configuring CLR .............................................................................................................................. 28 Overview ................................................................................................................................... 28 Step 1: Present the storage to be replicated to the RPA clusters ............................................... 29 Step 2: Tag the protected devices for RecoverPoint use ............................................................ 29 Step 3: Configure RecoverPoint access to the RecoverPoint splitters ......................................... 30 Step 4: Create the consistency group ........................................................................................ 30 Configuring the consistency group for management by SRM .......................................................... 32Automating site recovery with VMware vCenter Site Recovery Manager ................................. 33 Overview ........................................................................................................................................ 33 Prerequisites .................................................................................................................................. 34 Step 1: Install and configure SRM .................................................................................................. 34 Step 2: Connect the protected and recovery sites ........................................................................... 35 Step 3: Configure RecoverPoint array managers ............................................................................. 35 Step 4: Configure protection groups ............................................................................................... 36 Step 5: Configure inventory mappings............................................................................................ 37 Step 6: Customize virtual machine recovery options ...................................................................... 37 Step 7: Customize recovery site IP addresses ................................................................................. 38 Step 8: Create the recovery plan..................................................................................................... 39Testing the Oracle Failover recovery plan ............................................................................. 40 Overview ........................................................................................................................................ 40 Testing the recovery plan ............................................................................................................... 40 Verifying the success of the recovery test ....................................................................................... 42 The recovery test report .................................................................................................................. 43 Benefits of testing recovery plans .................................................................................................. 44RecoverPoint CRR site failover process................................................................................. 45 Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 4 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 5. Overview ........................................................................................................................................ 45 Running the Oracle Failover recovery plan ...................................................................................... 45 Restarting the Oracle database ...................................................................................................... 47 Verifying data integrity at the recovery site ..................................................................................... 47 Benefits of RecoverPoint site failover with SRM .............................................................................. 48RecoverPoint CRR site failback process ................................................................................ 49 Overview ........................................................................................................................................ 49 Housekeeping ................................................................................................................................ 49 Configuring SRM for failback .......................................................................................................... 50 Testing the failback recovery plan .................................................................................................. 50 Failing back to the production site ................................................................................................. 50 Verifying failback to the production site ......................................................................................... 51 Benefits of RecoverPoint site failback with SRM ............................................................................. 51RecoverPoint CDP database recovery ................................................................................... 52 Overview ........................................................................................................................................ 52 Preparing the test scenario ............................................................................................................ 52 Recovering the database from the CDP image ................................................................................ 53 Benefits of database protection with RecoverPoint......................................................................... 56RecoverPoint operation when VMAXe is operating in a degraded mode................................. 57 Overview ........................................................................................................................................ 57 Setting up VMAXe degraded mode ................................................................................................. 57 Testing degraded mode.................................................................................................................. 57Conclusion ......................................................................................................................... 59 Summary ....................................................................................................................................... 59 Findings ......................................................................................................................................... 59References .......................................................................................................................... 60 White papers ................................................................................................................................. 60 Product documentation.................................................................................................................. 60 Other documentation ..................................................................................................................... 60 Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 5 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 6. Executive summaryIntroduction to EMC® Symmetrix® is the world’s most trusted storage platform and enterpriseEMC Symmetrix customers have been deploying large mission-critical applications with Symmetrix forVMAXe over 20 years. Symmetrix VMAX™ is the world’s most scalable storage array, with the richest feature set, best availability, and best performance in the industry. The Symmetrix VMAXe™ series is a new class of enterprise storage, which delivers an efficient and cost-effective hardware design combined with built-in software and simplified installation, configuration, and management. VMAXe bring high-end storage array capabilities to service providers, healthcare organizations, and others with demanding virtual computing environments but limited storage expertise and IT resources. • The VMAXe employs an implementation of the Symmetrix Virtual Matrix Architecture™ that is optimized for rapid deployment and easier management. The system can scale from a single-bay, single-engine configuration to a six- bay, four-engine system with 960 drives and up to 1.3 PB of usable capacity. This enables customers to cost-effectively grow and upgrade their system to accommodate application and data growth. • The Enginuity™ operating environment—the intelligence that controls all VMAXe array components—is delivered preconfigured with EMC Virtual Provisioning™ and EMC Fully Automated Storage Tiering for Virtual Pools (FAST VP). The 100 percent virtually provisioned environment improves storage utilization and reduces costs and administration time. FAST VP provides automatic and highly granular movement of sub-LUN data between storage tiers. This optimizes performance and reduces cost, while radically simplifying management and increasing storage efficiency. • The VMAXe uses 100 percent internal redundancy to deliver enterprise-class reliability, availability, and serviceability (RAS). • EMC RecoverPoint is replication technology that uses sophisticated journaling techniques and write splitting to provide local and remote replication with DRV- like recovery to any point in time. The VMAXe has an integrated write splitter to support RecoverPoint replication. The VMAXe series is also designed for fast and efficient deployment and includes features that are particularly useful in small or crowded data centers: • VMAXe systems are delivered preconfigured, 100 percent virtually provisioned, and ready for same-day installation and startup. • With the VMAXe array dispersion capability, bays can be separated up to 10 meters apart, enabling very flexible deployments. • High storage densities can also be achieved, with a single-rack, single-engine VMAXe capable of supporting 120 drives. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 6 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 7. Business case Enterprise customers of all sizes and across different vertical segments experience similar challenges—downtime, application and data growth, demanding SLAs, and budget restraints. Symmetrix VMAXe with Enginuity delivers multi-controller scale-out architecture, availability, and efficiency to enterprise customers to address these challenges. Data protection and disaster recovery are critical to all enterprises and are among the most important aspects of administration in Oracle database environments. EMC RecoverPoint is proven technology for high-availability Oracle environments, providing local and remote replication with no impact in asynchronous environments and very limited application degradation in synchronous environments: • RecoverPoint’s journaled replication architecture enables recovery to any point in time, which reduces the recovery point objectives (RPOs) for Oracle data. • RecoverPoint maintains transaction-consistent images at the recovery site, which reduces the recovery time objectives (RTOs). The integrated RecoverPoint splitter for VMAXe simplifies database replication and supports operational and disaster recovery of virtualized environments. VMware® vCenter™ Site Recovery Manager works with RecoverPoint to coordinate and automate the recovery process, and enables administrators to test their disaster recovery plans without impacting the production environment or ongoing replication.Solution overview This solution focuses on disaster recovery between heterogeneous arrays—a Symmetrix VMAXe and an EMC VNX5700™—enabled by RecoverPoint continuous remote replication (CRR) and the RecoverPoint splitter. Local point-in-time database recovery using RecoverPoint continuous data protection (CDP) is also documented. The solution demonstrates the data protection and disaster recovery capabilities of these technologies in the context of a virtualized Oracle Database 11g OLTP environment. The virtualization platform for the solution is enabled by VMware vSphere™ and VMware vCenter Site Recovery Manager (SRM). Integration of RecoverPoint CRR and VMWare SRM enables automated failover of the Oracle database from the production site to the recovery site and ensures that data replicated to the recovery site is available to the recovery site servers.Key results The solution offers the following key benefits: • Simplified replication of Oracle OLTP database environments between heterogeneous arrays by using the integrated RecoverPoint splitter for VMAXe and the integrated RecoverPoint splitter for VNX • Rapid recovery from unplanned disasters by using RecoverPoint in conjunction with Symmetrix VMAXe and EMC VNX arrays • Automated failover and failback for planned and unplanned disasters, enabled by integration of RecoverPoint and VMware vCenter SRM • Nondisruptive disaster recovery rehearsal using RecoverPoint and VMware vCenter SRM • Point-in-time database recovery from bookmarked RecoverPoint images Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 7 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 8. IntroductionPurpose This white paper describes a data protection and disaster recovery solution for virtualized Oracle Database 11g R2 OLTP environments, enabled by RecoverPoint, Symmetrix VMAXe and VNX RecoverPoint splitters, and VMware vCenter Site Recovery Manager (SRM).Scope The scope of this paper is to describe: • The storage and virtualization infrastructure for the solution • RecoverPoint and vCenter SRM configuration for automated failover and failback • Nondisruptive disaster recovery rehearsal • Failover to the recovery site • Failback to the production site • Local point-in-time recovery of the database • RecoverPoint operation when VMAXe is operating in a degraded modeAudience This white paper is intended for Oracle database administrators, storage administrators, VMware administrators, EMC customers, and field personnel who want to understand the data protection and disaster recovery capabilities of Symmetrix VMAXe, RecoverPoint, and SRM in the context of virtualized Oracle OLTP databases.Terminology Table 1 defines terms used in this white paper. Table 1. Terminology Term Definition ASM Automatic Storage Management. Oracle logical volume manager. CDP Continuous data protection (see RecoverPoint data protection options). CLR Concurrent local and remote replication (see RecoverPoint data protection options). CRR Continuous remote replication (see RecoverPoint data protection options). Data device Virtual Provisioning term for devices (not mapped to the host) that provide physical storage for thin devices. Data devices must be contained in a virtual pool before they can be used. DR Disaster recovery. Enginuity The operating environment that provides the intelligence that controls all components in a VMAXe array. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 8 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 9. Term DefinitionFAST VP Fully Automated Storage Tiering for Virtual Pools. A feature of EMC Enginuity 5875 that provides automatic storage tiering at the sub-LUN level. VP denotes virtual pools, which are Virtual Provisioning thin pools.RDM Raw device mapping. A method of presenting a SAN/data device directly to a virtual machine. An alternative to using VMware VMFS.RPA RecoverPoint appliance (see RecoverPoint appliance).RPO Recovery point objective. The maximum acceptable time period between the last available consistent image and a disaster or failure.RTO Recovery time objective. The maximum acceptable time to bring a system or application back to operational state after a failure or disaster.SMC Symmetrix Management Console. A browser-based interface for managing EMC Symmetrix storage.SRM VMware vCenter Site Recovery Manager. An extension to VMware vCenter that enables integration with array-based replication.SYMCLI Symmetrix Solutions Enabler command line interface.Thin device (TDev) A cache-only device that is presented to a host. TDevs are pointers to units of physical storage contained in Virtual Provisioning thin pools.vCPU Virtual CPU. A processor within a virtual machine. VMware ESX® 4.1 currently supports up to eight vCPUs per virtual machine.Virtual pool A Virtual Provisioning thin pool, consisting of a collection of data devices that provides storage capacity for the thin devices that are bound to the pool.VMFS VMware Virtual Machine File System. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 9 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 10. Key technology componentsSolution This section provides an overview of the key components of the solution, as listed incomponents Table 2. Table 2. Solution components Function Components Business application Oracle Database 11g R2 Enterprise Edition, with Oracle Grid Infrastructure and Oracle ASM Storage platforms Production site • EMC Symmetrix VMAXe with Enginuity for VMAXe • EMC Symmetrix Management Console Recovery site • EMC VNX5700 • EMC Unisphere™ Virtualization platform VMware vSphere VMware vCenter Replication and recovery EMC RecoverPoint VMware vCenter Site Recovery Manager EMC RecoverPoint Storage Replication Adapter for VMware vCenter Site Recovery ManagerOracle application The solution is designed to provide local protection and disaster recovery for consolidated Oracle Database 11g OLTP environments. It demonstrates these capabilities for a single-instance OLTP database deployed on a VMware ESX virtual machine.EMC Symmetrix OverviewVMAXe with The solution demonstrates the local protection and disaster recovery capabilities ofEnginuity 5875 Symmetrix VMAXe with Enginuity 5875, which provides the storage platform at the production site. The Symmetrix VMAXe system is built on the highly scalable EMC Virtual Matrix Architecture, which enables it to grow seamlessly and cost-effectively from an entry- level, single-bay and single-engine configuration to a six-bay, four-engine system with 384 GB of cache memory, 960 drives, and up to 1.3 PB of usable capacity. Built with simplicity and ease-of-use in mind, the VMAXe is 100 percent virtually provisioned, with storage tiering managed automatically by EMC Fully Automated Storage Tiering for Virtual Pools (FAST VP). Flash, FC, and SATA drives are all supported, as well as RAID 1, 5, and 6 protection options. VMAXe systems are delivered preconfigured, ready for same-day installation and startup. They support all EMC Symmetrix monitoring and management tools, including the latest enhanced Symmetrix Management Console, which provides simpler installation and management with smart wizards. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 10 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 11. The integrated RecoverPoint splitter for VMAXe enables local and/or remote replication for flexible and efficient RPOs and RTOs. Virtual Provisioning EMC Virtual Provisioning simplifies storage configuration and management, improves capacity utilization, and enhances performance by enabling the creation of thin devices that present applications with more capacity than is physically allocated in the storage array. The physical storage used to supply disk space to the thin devices comes from a thin pool, which is comprised of data devices that provide the actual physical storage. The thin pools can be grown or shrunk nondisruptively by adding or removing data devices. Virtual Provisioning supports local and remote replication with RecoverPoint on VMAXe. Virtual Provisioning is described in detail in the EMC Solutions Enabler Symmetrix Array Controls CLI Version 7.3 Product Guide. Symmetrix Management Console The Symmetrix Management Console (SMC) is a powerful, browser-based interface that simplifies management of EMC Symmetrix storage, from device creation to advanced Symmetrix features such as FAST VP, Virtual Provisioning, Auto- provisioning Groups, replication configuration, and monitoring.EMC VNX5700 For the solution, storage at the recovery site is provided by an EMC VNX5700 storage array, demonstrating RecoverPoint support for heterogeneous storage platforms. The VNX5700 is a member of the VNX series next-generation storage platform, powered by Intel quad-core Xeon 5600 series processors. It is designed to deliver maximum performance and scalability for midtier enterprises, enabling them to grow, share, and cost-effectively manage multiprotocol file and block systems. VNX arrays incorporate the RecoverPoint splitter, which supports unified file and block replication for local data protection and disaster recovery. EMC Unisphere is the central management platform for the EMC VNX series, providing a single combined view of file and block systems, with all features and functions available through a common interface. Unisphere is optimized for virtual applications and provides industry-leading VMware integration, automatically discovering virtual machines and ESX servers and providing end-to-end, virtual-to-physical mapping.VMware vSphere VMware vSphere provides the virtualization platform for the solution, with VMware ESX virtual machines hosting the database and management systems at both the production and recovery sites. VMware vSphere abstracts applications and information from the complexity of underlying infrastructure, through comprehensive virtualization of server, storage, and networking hardware. It is the industry’s most complete and robust virtualization platform, virtualizing business-critical applications with dynamic resource pools for unprecedented flexibility and reliability. VMware vCenter provides the centralized management platform for vSphere environments, enabling control and visibility at every level of the virtual infrastructure, Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 11 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 12. including integration with RecoverPoint to enable discovery of the protection status for virtual machines in the environment.EMC RecoverPoint Overview RecoverPoint is an advanced enterprise-class disaster recovery solution designed with the performance, reliability, and flexibility required for enterprise applications in heterogeneous storage and server environments. It provides bi-directional local and remote data replication, without distance limits and with minimal performance degradation. EMC RecoverPoint/EX is a product variant that is optimized for the EMC VMAXe and VNX series and the CLARiiON® CX3 and CX4 series of storage arrays. EMC RecoverPoint/EX is the product used in this solution. RecoverPoint data protection options RecoverPoint provides the following replication options for both physical and VMware virtualized environments: Figure 1. RecoverPoint replication options • Continuous data protection (CDP): CDP continuously captures and stores data modifications locally, enabling local recovery from any point in time, with no data loss. Both synchronous and asynchronous replication are supported. • Continuous remote replication (CRR): CRR supports synchronous and asynchronous replication between remote sites over FC and a WAN. Synchronous replication is supported when the remote sites are connected through FC and provides a zero RPO. Asynchronous replication provides crash- consistent protection and recovery to specific points in time, with a small RPO. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 12 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 13. • Concurrent local and remote replication (CLR): CLR is a combination of CRR and CDP and provides concurrent local and remote data protection.In RecoverPoint, CDP is normally used for operational recovery, while CRR is normallyused for disaster recovery—the solution demonstrates a RecoverPoint CLRconfiguration.RecoverPoint applianceRecoverPoint is appliance-based, which enables it to better support large amounts ofinformation stored across heterogeneous environments. This out-of-band approachenables RecoverPoint to deliver continuous replication with minimal impact to anapplication’s I/O operations.RecoverPoint appliances (RPAs) run the RecoverPoint software and manage allaspects of data replication. For local replication, a cluster configuration of two ormore active RPAs is deployed—this supports immediate switchover to anotherappliance if one of the RPA nodes in a cluster goes down. For remote replication, aRecoverPoint cluster is deployed at both sites.RPAs use powerful deduplication, compression, and bandwidth reductiontechnologies to minimize the use of bandwidth and dramatically reduce the time lagbetween writing data to storage at the source and target sites.RecoverPoint splitterRecoverPoint uses lightweight write splitting technology, on the application server, inthe fabric, or in the array, to mirror application writes to the RecoverPoint cluster.VMAXe and VNX arrays have integrated RecoverPoint splitters that operate in eachfront-end adapter (FA)—this ensures that the RPA receives a copy of each write.The array-based splitter is the most effective write splitter for VMware replication,enabling replication of VMFS and RDM volumes without the cost or complexity ofadditional hardware. The splitter supports both FC and iSCSI volumes presented bythe VMAXe or VNX arrays to any host, including to an ESX server.RecoverPoint journalsRecoverPoint journals store timestamped application writes for later recovery toselected points in time. Three journals are provisioned for local and remotereplication—one production journal at the production site, and a history journal atboth the production and recovery sites.For synchronous replication, every write is retained in the history journal for recoveryto any point in time. For asynchronous replication, several writes are grouped beforedelivery to the history journal—this supports recovery to significant points in time.Bookmarked points-in-time can be created automatically or manually to enablerecovery to specific application or system events.RecoverPoint consistency groupsThe consistency and write-order fidelity of point-in-time images are assured byRecoverPoint’s use of replication sets and consistency groups. A replication setdefines an association between a production volume and the local and/or remotevolumes to which it is replicating. A consistency group logically groups replicationsets that must be consistent with one another. The consistency group ensures thatBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 13 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 14. updates to the production volumes are written to the replicas in consistent write order and that the replicas can always be used to continue working from or to restore the production source. RecoverPoint replication is policy-driven. A replication policy, based on a particular business need, can be uniquely specified for each consistency group. This policy governs the replication parameters for the consistency group—for example, the RPO and RTO for the consistency group, and its deduplication, data compression, and bandwidth reduction settings.VMware vCenter VMware vCenter Site Recovery Manager (SRM) is a disaster recovery framework thatSite Recovery integrates with EMC RecoverPoint to automate recovery of VMware datastores so thatManager it becomes as simple as pressing a single button. SRM is an extension to VMware vCenter that enables integration with array-based replication, discovery and management of replicated datastores, and automated migration of inventory from one vCenter to another. SRM does not replicate any data. For this, it leverages an external replication solution such as RecoverPoint. SRM servers coordinate the operations of the replicated storage arrays and vCenter servers at the production and recovery sites so that, as virtual machines at the production site are shut down, virtual machines at the recovery site start up and assume responsibility for providing the same services, using the data replicated from the production site. Migration of protected inventory and services from one site to the other is controlled by a recovery plan that specifies the order in which virtual machines are shut down and started up, the compute resources that are allocated, and the networks they can access. SRM integrated with RecoverPoint also enables recovery plans to be tested, using a temporary copy of the replicated data, in a way that does not disrupt ongoing operations at either site.RecoverPoint and Integration of SRM with RecoverPoint is provided by the EMC RecoverPoint StorageSRM integration Replication Adapter (SRA) for VMware vCenter Site Recovery Manager. The RecoverPoint SRA supports discovery of arrays attached to RecoverPoint and of consistency groups that are enabled for management by SRM. It also supports SRM functions such as failover and failover testing by mapping SRM recovery plans to the appropriate RecoverPoint actions. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 14 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 15. RecoverPoint CDP Continuous data protectionand CRR – how Local protection of the production environment is performed by RecoverPointthey work continuous data protection (CDP), using the integrated RecoverPoint splitter. Figure 2 illustrates the flow of a write in CDP. Figure 2. RecoverPoint CDP data flow 1. The application server issues a write to a LUN that is being protected by RecoverPoint. The write is intercepted by the RecoverPoint splitter. 2. The splitter “splits” the write and sends it to the production volume and to the RPA simultaneously. 3. When the write is received by the RPA, it is acknowledged back to the splitter. 4. In parallel, the RPA moves the data into the local journal volume, along with a timestamp and any application, event, or user-generated bookmarks for the write. 5. After the data is safely stored in the journal, it is distributed to the target local volumes through the RPA FC connections—write order is preserved during distribution. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 15 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 16. Continuous remote replicationReplication of the production environment to the recovery environment is performedby RecoverPoint continuous remote replication (CRR) using the RecoverPoint splitter.Figure 3 illustrates the flow of a write in CRR.Figure 3. RecoverPoint CRR data flow 1. The application server issues a write to a LUN that is being protected by RecoverPoint. The write is intercepted by the RecoverPoint splitter. 2. The splitter “splits” the write and sends it to the production volume and to the local RPA simultaneously, the same as in a CDP deployment. 3. When the RPA receives the write, it immediately acknowledges it back to the splitter, unless synchronous remote replication is in effect. With synchronous replication, the ACK is delayed until the write has been received by the RPA at the remote site. 4. When the write is received by the local RPA, it is bundled with other writes, deduplicated to remove redundant blocks, sequenced, and timestamped. The package is then compressed and transmitted with a checksum for delivery to the remote RPA cluster. 5. When the package is received at the remote site, the remote RPA verifies the checksum, to ensure the package was not corrupted in transmission, and uncompresses the data. 6. The RPA then writes the data to the journal volume at the remote site. 7. After the data has been written to the journal volume, it is distributed to the remote replica volumes through the RPA FC connections—write order is preserved during this distribution.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 16 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 17. Solution architecture and designSolution EMC solutions are designed to reflect and validate real-world deployments. Figure 4architecture depicts the physical architecture of the solution described in this white paper. Figure 4. Solution architecture The solution demonstrates the local protection and disaster recovery capabilities of the EMC Symmetrix VMAXe with Enginuity, which provides the storage platform at the production site. Storage at the recovery site is provided by an EMC VNX5700 array, demonstrating RecoverPoint support for heterogeneous storage platforms. VMware vSphere provides the virtualization platform for the solution. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 17 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 18. Environment Table 3 details the environment profile for the solution.profile Table 3. Solution profile Profile characteristic Details Database type OLTP Database size 500 GB Number of databases 1 Workload profile Swingbench Order Entry (TPC-C-like) workload Database read/write ratio 60/40 Network connectivity FC 8 Gb and 10 GbEHardware Table 4 details the hardware environment for the solution.environment Table 4. Solution hardware environment Purpose Quantity Configuration Storage 1 EMC Symmetrix VMAXe with: (production site) • Single engine • 96 GB cache memory • 88 x 450 GB, 15k FC drives, plus spare • 16 x 2000 GB SATA drives, plus spare • 4 x 200 GB Flash drives, plus spare • Enginuity for VMAXe 5875 2011Q2SR Storage 1 EMC VNX5700 with: (recovery site) • 8 Gb FC connectivity • 1 Gb network connectivity • 15 x 200 GB SATA Flash • 15 x 300 GB SAS drives • 45 x 2 TB NL-SAS drives • 2 x Data Movers • 8 x GbE NICs • 1 x Control Station • VNX Operating Environment VMware ESX servers 2 • 2 x quad-core Xeon 5560 CPUs, 2.80 GHz, 98 GB RAM • 2 x 10 GB CNA adapters Network switches 2 Gigabit Ethernet switches FC switches 4 8 GB/s FC switches, 2 per site RecoverPoint appliances 4 GEN 4, 2 RPAs per site Host adapters 4 10 GB CNA adapter (2 per physical server) Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 18 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 19. Software Table 5 details the software environment for the solution.environment Table 5. Solution software environment Software Version Purpose EMC Symmetrix Management 7.3 Symmetrix VMAXe configuration and Console management tool EMC Unisphere 1.1 VNX management software EMC Solutions Enabler 7.3 Symmetrix VMAXe management software EMC RecoverPoint 3.4.1 EMC replication software, installed on each of the 4 RPAs EMC RecoverPoint Adapter for 4.1.1 EMC software for integrating VMware vCenter Site Recovery RecoverPoint and SRM Manager VMware vSphere 4.1 GA B260247 Hypervisor hosting all virtual machines VMware vCenter 4.1 GA B259021 Management of VMware vSphere VMware vCenter Site Recovery 4.1.1 Managing failover and failback of Manager virtual machines Oracle Database 11g R2 Enterprise Oracle database software for grid Edition 11.2.0.2 computing Red Hat Enterprise Linux 5.5 Server OS for Oracle database server Microsoft Windows Server 2008 R2 x64 Server OS for Swingbench load generator Swingbench 2.4.0.723 Database workload generation tool Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 19 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 20. Symmetrix VMAXe The Symmetrix VMAXe array is factory-configured, based on information provided bystorage allocation the customer at the time of ordering. Storage pools are preconfigured based on the protection schemes requested by the customer. To provision storage to the ESX hosts, simply run the Add New Host And Provision Virtual Storage wizard from the SMC dashboard. This guides you through the steps involved, as shown in Figure 5. Figure 5. Add New Host and Provision Virtual Storage wizard Table 6 details the VMFS volumes provisioned for the solution from the Symmetrix VMAXe array. All these volumes were replicated with RecoverPoint. Table 6. Symmetrix VMAXe storage allocation VMFS volume Capacity Meta members Thin pools OracleVM_DataStore 100 GB 1 VM_ FC_3R5 DATA 800 GB 16 FC_3R5 REDO 100 GB 2 FC_3R5 TMP 200 GB 4 FC_3R5 FRA 1,440 GB 8 SATA_6R6 Oracle_Binaries 20 GB 1 FC_3R5 Note: Volumes replicated with RecoverPoint require a target volume of the same size on the local array and remote array. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 20 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 21. Virtual machine Table 7 details the virtual allocation of hardware resources for the virtual machines atresources the production site. Table 7. Virtual allocation of hardware resources for virtual machines Virtual machine role Quantity Configuration Oracle database node 1 6 vCPUs, 24 GB RAM, RHEL 5.5 VMware vCenter/SRM 1 2 vCPUs, 8 GB RAM, Windows Server 2008 R2 x64 server* EMC SMC/SPA server* 1 2 vCPUs, 4 GB RAM, Windows Server 2008 R2 x64 Swingbench server* 1 4 vCPUs, 8 GB RAM, Windows Server 2008 R2 x64 Application server 1 1 vCPU, 8 GB RAM, Windows Server 2008 R2 x64 * These virtual machines reside on an existing ESX server used only for management purposes. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 21 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 22. Oracle database configurationOracle database Virtual Provisioning for Oracle ASM disk groupsdeployment on The Oracle database for the solution has separate ASM disk groups for data files,Symmetrix VMAXe redo logs, fast recovery area (FRA), and temp files. Separate VMFS volumes are presented to the virtual machines on dedicated datastores for each ASM disk group. The +DATA, +TEMP, and +REDO ASM disk groups are deployed on thin provisioned devices bound to a virtual pool composed of FC devices with RAID 5 (3+1) protection. This pool provides the high performance required by these devices. The thin device provisioned for the +REDO logs was fully provisioned at the time of deployment— that is, all physical storage was assigned up front. The VMFS datastore for the +REDO log file was also provisioned to request all storage from the outset. The +FRA device is bound to a virtual pool composed of high-capacity SATA devices with RAID 6 (6+2) protection. FRA devices typically do not have the same high- performance demands as data files and redo logs, so SATA devices provide an efficient deployment for this data. Figure 6 shows the relationships from the ASM disk groups through to the thin pools. +DATA +REDO +TMP +FRA ASM disk group Oracle_DATA Oracle_REDO Oracle_TMP Oracle_FRA VMFS datastore Thin device (TDEV) Thin pool FC Pool RAID5(3+1) SATA Pool RAID6 (6+2) Figure 6. Relationship between ASM disk groups and storage poolsDatabase schema A single instance of the Swingbench Order Entry PL/SQL (SOE) schema was used to deliver the OLTP workloads required by the solution. The Swingbench SOE schema models a traditional OLTP database, with tables and indexes residing in separate tablespaces. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 22 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 23. The schema for the solution contains the tables and indexes listed in Table 8.Table 8. Schema tables and indexes Table name Index CUSTOMERS CUSTOMERS_PK (UNIQUE), CUST_ACCOUNT_MANAGER_IX, CUST_EMAIL_IX, CUST_LNAME_IX, CUST_UPPER_NAME_IX INVENTORIES INVENTORY_PK (UNIQUE), INV_PRODUCT_IX, INV_WAREHOUSE_IX ORDERS ORDER_PK (UNIQUE), ORD_CUSTOMER_IX, ORD_ORDER_DATE_IX, ORD_SALES_REP_IX, ORD_STATUS_IX ORDER_ITEMS ORDER_ITEMS_PK (UNIQUE), ITEM_ORDER_IX, ITEM_PRODUCT_IX PRODUCT_DESCRIPTIONS PRD_DESC_PK (UNIQUE), PROD_NAME_IX PRODUCT_INFORMATION PRODUCT_INFORMATION_PK (UNIQUE), PROD_SUPPLIER_IX WAREHOUSES WAREHOUSES_PK (UNIQUE) LOGONTo verify the virtual database environment, the Swingbench Order Entry workload wasrun against the database, with a user count of 100, as shown in Figure 7.Figure 7. Swingbench workload running against the databaseBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 23 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 24. Configuring EMC RecoverPointPreparing the RPA repository and gatekeeper provisioningVMAXe for Configuring the RecoverPoint splitter on the Symmetrix VMAXe array requiresRecoverPoint provisioning of the following volumes:connectivity • A repository volume (3 GB minimum) for the RPA cluster. This stores configuration information about the RPAs and RecoverPoint consistency groups, which enables a properly functioning RPA to seamlessly assume the replication activities of a failing RPA from the same cluster. A repository volume of the same size was also provisioned on the VNX5700 for the RPA cluster at the recovery site. • Eight unique gatekeeper volumes each for RPA1 and RPA2. These volumes are provisioned by creating Auto-provisioning masking views that present the volumes to the RPAs. For the solution, three masking views were created for the RecoverPoint cluster, as it consists of two RPAs (RPA1 and RPA2): • RecoverpointConfig – this presents the repository volume, the journal volumes, and the replica volumes to all nodes in the RPA cluster • RPA1_GK – this presents the relevant gatekeepers to RPA1 • RPA2_GK – this presents the relevant gatekeepers to RPA2 Figure 8 shows creation of the RecoverpointConfig masking view using the Masking View Management – Create dialog box in Symmetrix Management Console. For further information on Auto-provisioning masking views, consult Deploying RecoverPoint with Symmetrix—Technical Notes. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 24 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 25. Figure 8. Configuring masking views for a RecoverPoint clusterBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 25 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 26. Enabling write protect bypass for RPA initiatorsThe RecoverPoint splitter for VMAXe requires the RPA initiators to have special accessthat enables them to write to write-protected devices. To grant this access, the writeprotect bypass initiator flag—WP_Bypass(WPBP)—must be set for all RPA initiators.Figure 9 shows the SMC procedure for doing this. 1 2 3 1. Choose Enable Recover Point command for initiator group. 2. SMC confirms initiator group enabled for RecoverPoint. 3. Initiator properties show WPBP flag enabled.Figure 9. Enabling write protect bypass for RPAsBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 26 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 27. RecoverPoint RecoverPoint was configured as follows for the solution:configuration for • CLR was the replication method used for the solution. This combines both CDPthis solution and CRR. • Two consistency groups were created:  Oracle_SRM—this encompasses all the replication sets for the Oracle database, Oracle binaries, and the datastore containing the running operating system for the virtual machine that hosts the Oracle software.  App_SRM—this contains the replication sets for the datastore containing the Swingbench and application server virtual machines (see Table 7). By separating replicated virtual machines into multiple consistency groups, it was possible to plan for different disaster recovery scenarios. For example, the Oracle environment could be failed over separately from the other virtual machines, or the entire production environment could be failed over as a single process. • For each consistency group, three journals were set up, two at the production site to support the production volumes and their local replicas, and one at the recovery site to support the remote replica. All journals on the VMAXe were bound to the FC_3R5 pool. For further details on configuring RecoverPoint, consult the EMC RecoverPoint Release 3.4.1 Administrator’s Guide. Journal size The size of the journal volumes should reflect your RPO. Determining the journal size requires administrators to calculate the expected peak change rate in their environment. (new data writes per second) x (required rollback time in seconds) The following is the Journal Volume Sizing formula: Journal size = x 1.05 (1 − target side log size) Twenty percent of the journal must be reserved for the target side log and 5 percent for internal system needs. For example, to support a 24-hour rollback requirement (86,400 seconds), with 5 Mb/s of new data writes to the replication volumes in a consistency group, the (5 x 86400) calculation would be as follows: x 1.05 = 567000 Mb = 69.213 GB (~70 GB) (1 − 0.2) For the solution, all journals were sized to 500 GB. With a change rate of 5 Mb/s, this enables RecoverPoint to roll back at least seven days. It may be possible to increase the recover point by bookmark consolidation and journal compression. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 27 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 28. Installing The RecoverPoint Installer Wizard, which is part of the RecoverPoint DistributionRecoverPoint Manager, guides you through the procedures for installing RecoverPoint and setting up the RPA clusters at the production and recovery sites. Before running the wizard, it is essential that the steps described in Preparing the VMAXe for RecoverPoint connectivity have been completed, as the wizard prompts you to identify the repository volumes created then. Figure 10 shows the Prerequisites screen from the wizard, listing the conditions that must be met before using the wizard. It also shows the Summary screen, which lists the tasks completed during installation. Figure 10. RecoverPoint Installer WizardConfiguring CLR Overview When RecoverPoint installation has successfully completed, you can configure the RecoverPoint consistency group(s) required for local and remote replication of your application data. To do this, perform the following steps for each consistency group: 1. Present the storage to be replicated to the RPA clusters at both sites. 2. Tag all VMAXe devices that are to be protected by RecoverPoint. 3. Configure RecoverPoint access to the RecoverPoint splitters on the VMAXe and VNX arrays. 4. Create the RecoverPoint consistency group. This section describes the configuration process for the solution’s Oracle_SRM consistency group. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 28 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 29. Step 1: Present the storage to be replicated to the RPA clustersLocal arrayThe application production devices that are to be protected by RecoverPoint must bepresented to the RPA cluster. To do this: 1. Launch Symmetrix Management Console for the local array. 2. Open the initiator group for the existing auto-provisioning view that contains the devices to be protected. 3. Add the RPA initiator groups for all RPAs, as shown in Figure 11.Figure 11. Adding RPA initiator groupsRemote arrayThe remote journal and the target devices for the CRR copy must be presented to theremote RPA cluster. How this is accomplished depends on the target array type. Thetarget array for the solution is a VNX5700. In this case, a storage group containing thejournal and target devices was set up and presented to the remote RPAs. A secondstorage group was created and presented to the ESX host at the remote site toprovide access to the replica storage.For more detailed information on configuring VNX storage for replication withRecoverPoint, consult EMC RecoverPoint Deploying with VNX/CLARiiON Arrays andSplitter—Technical Notes.Step 2: Tag the protected devices for RecoverPoint useIn RecoverPoint splitter for VMAXe configurations, it is necessary to tag devices forRecoverPoint use. This makes the devices accessible to the splitter and also makes iteasy to identify which VMAXe volumes have been set up for RecoverPoint use.Figure 12 shows the Symmetrix Management Console procedure for tagging thedevices in the storage group for the ESX server. Use the same procedure to tag thereplica volumes in the RecoverpointConfig storage group.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 29 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 30. Figure 12. Tagging devices for RecoverPoint useNote: You must repeat this step whenever you add devices to the configuration, if they are to be protected by RecoverPoint.Step 3: Configure RecoverPoint access to the RecoverPoint splittersTo configure RecoverPoint access to the RecoverPoint splitters on the VMAXe and VNXarrays, use the New Splitter Wizard, as shown in Figure 13.Figure 13. Adding new splittersStep 4: Create the consistency groupCreating a consistency groups involves: 1. Defining the replication policy settings for the consistency group, the production copy (that is, the data to be protected), and the local and remote replica copies. 2. Adding replication sets to the consistency group by selecting the production volumes to be replicated and assigning the corresponding replica volumes at the local and/or remote copies. 3. Selecting the journal volumes for the production and replica journals.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 30 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 31. The New Consistency Group Wizard, in the RecoverPoint Management Application,guides you through the required steps, as shown in Figure 14. The replication policysettings are all optional as the default settings provide a practical configuration.Figure 14. New Consistency Group Wizard stepsThe consistency groups for the solution were created with the wizard and the defaultvalues were accepted for all optional settings.Figure 15 shows the final summary screen for the Oracle_SRM consistency group,which was configured with a local copy (Oracle_CDP) and a remote copy (Oracle_CRR)to support both local data protection and remote replication. The summary screenincludes the replication sets defined for the consistency group.Figure 15. Summary of consistency group settingsBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 31 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 32. Configuring the After the consistency group has been created and SRM has been installed, you needconsistency group to configure the consistency group for management by SRM. You do this by using thefor management by Policy settings in the RecoverPoint Management Application, as shown in Figure 16.SRM consistency group policy settings Figure 16. Configuring the consistency group for management by SRM Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 32 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 33. Automating site recovery with VMware vCenter Site Recovery ManagerOverview Installing and configuring VMware vCenter Site Recovery Manager (SRM) for automated site recovery using RecoverPoint involves these steps: 1. Install and configure SRM. 2. Configure the connection between the protected and recovery sites. 3. Configure RecoverPoint array managers. 4. Configure protection groups. 5. Configure inventory mappings. 6. Customize virtual machine recovery options. 7. Customize recovery site IP addresses. 8. Create the recovery plan. Most of these steps are executed from the SRM interface in vCenter, where intelligent wizards support quick and easy configuration. For full details of these steps, consult the VMware vCenter Site Recovery Manager Administration Guide 4.1. It is possible to create multiple recovery plans to cater for many different recovery scenarios. In the example shown in Figure 17, three recovery plans have been created—one to fail over the entire environment, and the other two to fail over the application and database layers independently. Figure 17. Multiple recovery plans All these recovery plans were successfully executed as part of the solution testing. For the purpose of demonstrating the failover process using SRM and RecoverPoint, this white paper documents failover and failback of the entire production environment. This section describes how SRM was configured for failover from the production site to the recovery site. The RecoverPoint CRR site failback process section describes how SRM was configured for failback from the recovery site to the production site. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 33 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 34. Prerequisites SRM has several requirements for the vSphere configuration at each site: • Each site must include a vCenter server containing at least one vSphere data center. • The recovery site must support array‐based replication with the protected (production) site, and must have hardware and network resources that can support the same virtual machines and workloads as the protected site. • At least one virtual machine must be located on a datastore that is replicated by RecoverPoint at the protected site. • The protected and recovery sites must be connected by a reliable IP network. Storage arrays may have additional network requirements. • The recovery site must have access to the same public and private networks as the protected site, though not necessarily access to the same range of network addresses. • VMware tools must be installed on all virtual machines.Step 1: Install and Installing and configuring SRM includes the following tasks:configure SRM 1. Configure SRM databases at both sites—these store the recovery plans, inventory information, and so on. 2. Install the SRM server at both sites. 3. Install the RecoverPoint Storage Replication Adapter on the SRM server at both sites—this enables SRM and RecoverPoint integration. 4. Install the SRM client plug-in into one or more vSphere Clients at either or both sites. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 34 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 35. Step 2: Connect With the SRM database and SRM server installed at each site, you then need tothe protected and configure the connection from the protected site to the recovery site. You do this byrecovery sites specifying the remote SRM server IP details in the Connect to Remote Site wizard at the protected site, as shown in Figure 18. 1 2 3 Figure 18. Connecting the SRM and vCenter servers at the protected and recovery sitesStep 3: Configure For SRM to integrate with RecoverPoint, RecoverPoint array managers must beRecoverPoint array configured at both the protected and recovery sites. You do this by using the SRMmanagers Configure Array Managers wizard. The wizard discovers the replicated storage devices at the protected and recovery sites, and identifies the VMFS datastores that they support. When finished, it presents a list of replicated datastore groups. Figure 19 shows the wizard in progress, with RecoverPoint specified as the manager type and Production as the protected site. The connection information for RecoverPoint at the protected site is also specified. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 35 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 36. Figure 19. Configuring the RecoverPoint array managersStep 4: Configure A protection group is a collection of virtual machines that all use the same datastoreprotection groups group (the same set of replicated LUNs) and that all fail over together. To create a protection group, you select the datastore group(s) to protect, and specify a datastore group at the recovery site where SRM can create placeholders for members of the protection group. You use the Create Protection Group wizard to do this. The wizard automatically detects the datastores currently protected by RecoverPoint and allows you to select the one(s) to include in the protection group. Two protection groups were created for the use case—one for the Oracle_SRM consistency group and the other for the App_SRM consistency group. Figure 20 shows the Oracle protection group being created. Datastore group selected for inclusion in protection group Virtual machines supported by selected datastore group Figure 20. Creating a protection group Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 36 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 37. When a protection group has been created, shadow virtual machines are automatically created in the recovery site vCenter inventory. These act as placeholders for the virtual machines that can be failed over with SRM. The shadow virtual machines cannot be started independently of SRM and are removed if their protection group is deleted.Step 5: Configure Network, compute, and virtual machine folder resources must be configured at theinventory recovery site in order for SRM to know which resources to use in the event of failover.mappings The Inventory Mappings screen lists the resources at the protected site and allows you to select the corresponding resources to use at the recovery site. Figure 21 shows the inventory mappings configured for the solution. Figure 21. Creating inventory mappings Note: For ease of management, it is useful to group protected virtual machines into a folder. For the solution, the protected virtual machines were grouped in the SRM Protected VMs folder.Step 6: Customize After the protection groups have been created, the recovery options for individualvirtual machine virtual machines can be customized to suit specific requirements. The customizationrecovery options options available provide a powerful tool for ensuring that complex recovery plans are implemented with ease. For example, if there are multiple virtual machines in a protection group, their startup sequence can be customized by assigning them different recovery priorities. In an Oracle database environment, this means that a recovery plan can be configured to boot the database virtual machines before the application server virtual machines. For the solution, the Alicanto-PV1 virtual machine was assigned a recovery priority of High, as shown in Figure 22. This machine would be recovered first, before the Swingbench virtual machine (Normal priority), and the AppServer virtual machine (Low priority). Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 37 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 38. Figure 22. Modifying the startup priority of a virtual machineStep 7: Customize When failing over to a different data center, typically some adjustments are requiredrecovery site IP to host IP settings due to infrastructure differences. When failing over an entireaddresses configuration, this can involve updating settings for multiple virtual machines. SRM provides a bulk IP customization utility (dr-ip-customizer.exe) for automatically updating IP settings for recovered virtual machines. The utility generates a CSV file containing the IP settings for all the virtual machines that are configured for SRM failover. You can edit this file to specify the recovery site IP settings and then run the utility again to upload the new settings to the recovery site vCenter server. For the solution, the utility was used as follows to update the recovery site IP settings: 1. Log on to the vCenter server at the recovery site. 2. Run the dr-ip-customizer.exe utility, specifying the name and location for the CSV file, as shown in Figure 23. Figure 23. Running the IP utility to generate a CSV file 3. Edit the CSV file to provide the IP settings for the virtual machines at the recovery site. Figure 24 shows the edited file for the solution. Figure 24. CSV file edited for recovery site IP settings Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 38 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 39. 4. Run the utility to upload the new settings to the recovery site vCenter server, as shown in Figure 25. Figure 25. Uploading the IP settings for the recovery site Note: If you delete or re-create a protection group, you must repeat this process to reapply the IP customizations.Step 8: Create the A recovery plan determines the automated steps that SRM executes when starting uprecovery plan a protected environment at the remote site. All recovery plans include a set of prescribed steps that are executed in a prescribed order. Any customizations you have configured prior to creating the recovery plan, such as recovery priority and IP settings, are also implemented. To create a plan, you run the Create Recovery Plan wizard at the recovery site and simply specify the recovery plan name and the protection group to be recovered. Figure 26 shows the Oracle Failover recovery plan being created for the solution. This recovery plan fails over the entire production environment to the recovery site. Figure 26. Creating a recovery plan Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 39 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 40. Testing the Oracle Failover recovery planOverview An SRM recovery plan can be tested at any time without disrupting replication or ongoing operations at the protected site. The test is carried out in an isolated environment at the recovery site, using a temporary copy of the replicated data. It runs all the steps in the recovery plan except powering down of virtual machines at the protected site and assumption of control of replicated data by devices at the recovery site. If the plan requests suspension of local virtual machines at the recovery site, this happens during test recovery as well. This facility enables you to rehearse your recovery plans thoroughly and easily in order to verify their timings and reliability. This section describes a test recovery of the Oracle Failover recovery plan. The Automating site recovery with VMware vCenter Site Recovery Manager section of the white paper describes how this recovery plan was created.Testing the First of all, 4,000 records were added to the production database. A script was thenrecovery plan run to output the record count and the timestamp of the last entry, as shown in Figure 27. This information was later used to verify the success of the test recovery. Figure 27. Production site record count and timestamp The test was then initiated simply by selecting the Oracle Failover recovery plan in SRM and clicking the Test button, as shown in Figure 28. Figure 28. Starting the recovery test Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 40 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 41. When running a test plan, SRM creates an isolated TEST network on the recovery siteand enables image access on the RecoverPoint consistency group. It also starts upthe virtual machines replicated by RecoverPoint and performs any reconfigurationneeded to access the disks on the remote array.Figure 29 shows the recovery steps performed during testing of the Oracle Failoverrecovery plan. SRM first created an SRM bookmark and enabled image access on theCRR copy. It then recovered the high-priority Alicanto-PV1 virtual machine, followedby the normal-priority Swingbench virtual machine, and then the low-priorityAppServer virtual machine. The virtual machines at the production site were not shutdown as part of the test process. Create SRM bookmark and enable image access on CRR copy Recover High priority VM Recover Normal priority VM Recover Low priority VM Figure 29. Test recovery stepsAt this stage, the environment is available on the remote site and administrators canverify application functionality in the secure environment created as part of the test.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 41 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 42. Verifying the For the Oracle Failover recovery test, the status view in the RecoverPoint Managementsuccess of the Application verified that RecoverPoint image access was enabled, as shown in Figurerecovery test 30. } Image access enabled Figure 30. RecoverPoint image access enabled To verify that IP address customizations had been applied correctly, the IP settings for the failed-over virtual machines were checked in the recovery site vCenter instance, as shown in Figure 31. Figure 31. Verifying IP address customizations To verify data integrity at the recovery site, a script was run on the failed-over machine to output the record count and the timestamp of the last entry. As shown in Figure 32, these matched the record count and timestamp at the protected site (see Figure 27). Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 42 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 43. Figure 32. Recovery site record count and timestamp To verify that the remote environment was able to process transactions, a Swingbench load, with 100 users, was run against the database at the recovery site.The recovery test After verifying the success of the recovery test, the test was ended. At this point, SRMreport disabled image access on the CRR copy and the summary test report shown in Figure 33 was generated. The time taken for the test recovery was just over 24 minutes. This included the verification steps performed to ensure that the remote image was accessible and that the Oracle environment could process transactions. Figure 33. Recovery test report This test report can be used to demonstrate accurate timing of recovery plans and recovery reliability for auditing, compliance, system administration, and other business purposes. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 43 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 44. Benefits of testing Having a disaster recovery plan in place is the first step to ensuring you are preparedrecovery plans for the worst-case scenario. However, the only way to truly know if your disaster recovery plan works is to test it. Site Recovery Manager’s ability to execute recovery plans in test mode allows you to do just that. • Recovery plans can be tested at any time without interrupting replication or the business’s RPO. • Accurate timing of recovery plan tests enables administrators to predict how long recovery of the virtual environment will take. • Administrators can quickly access a replica of their entire production environment for testing purposes. • Automatically generated summary reports can be used to demonstrate compliance with regulatory requirements. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 44 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 45. RecoverPoint CRR site failover processOverview For the use case, remote site recovery was enabled by RecoverPoint continuous remote replication (CRR), integrated with VMware vCenter SRM. SRM automates the recovery process so that executing failover becomes as simple as pressing a single button. There is no need for the user to interact with the RecoverPoint console. You simply execute the recovery plan from the SRM plug-in within the recovery site vCenter instance. This section describes how the Oracle environment was failed over to the recovery site by executing the Oracle Failover recovery plan. The Automating site recovery with VMware vCenter Site Recovery Manager section of the white paper describes how this recovery plan was created. The Testing the Oracle Failover recovery plan section describes a test run of the recovery plan.Running the Oracle Prior to performing the failover, some records were added to the productionFailover recovery database. A script was then run to output the record count and the timestamp of theplan last entry—this information was later used to verify the success of the failover. Figure 34 shows a total of 15,000 records in the database at this time. Figure 34. Adding records to the production database Failing over the production environment to the recovery site involved simply running the Oracle Failover recovery plan on the recovery site vCenter server, as shown in Figure 35. Figure 35. Running the recovery plan Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 45 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 46. When the failover process finished, SRM displayed a summary report of the recoverysteps, as shown in Figure 36.Figure 36. Oracle Failover summary reportSRM automatically executed the failover steps in the recovery plan, including anyadministration tasks needed to import the virtual machines to the remote VCenterinstance. The recovery priority and IP setting customizations that were configuredprior to creating the recovery plan were also implemented. This meant that the high-priority Alicanto-PV1 virtual machine was started before the normal-prioritySwingbench virtual machine and the low-priority AppServer virtual machine.The status view in the RecoverPoint Management Application (see Figure 37) showsthe failed-over state of the Oracle_SRM consistency group after the recovery plan hadbeen executed. Note that the replication direction has been reversed. This meansthat any changes at the DR site are replicated to the production site using the policiesset up for consistency group Oracle_SRM. DR site is now the sourceFigure 37. Oracle_SRM consistency group in failoverBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 46 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 47. Restarting the When an Oracle database virtual machine is failed over to a disaster recovery site,Oracle database Oracle automatically recovers the crash-consistent CRR image that is presented to the recovery site by RecoverPoint. Figure 38 shows Oracle recovery of the crash- consistent image of the Alicanto-PV1 database host. Figure 38. Oracle crash-consistent image recoveryVerifying data To verify data integrity at the recovery site, a script was run on the failed-overintegrity at the Alicanto-PV1 virtual machine to output the record count and the timestamp of the lastrecovery site entry. As shown in Figure 39, these matched the record count and timestamp at the production site (see Figure 34). Figure 39. Verifying data integrity after failover Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 47 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 48. Benefits of The benefits of automating failover and recovery of Oracle environments withRecoverPoint site RecoverPoint and SRM include:failover with SRM • By virtualizing the operating system and applications, and replicating these with RecoverPoint, deployment time for the disaster recovery environment is eliminated. The entire operating environment is deployed once at the production site and is replicated to the remote site. The only installation required at the remote site is the VMware ESX servers, with vCenter, and the SRM servers required to manage failover. • The tests performed showed that SRM and RecoverPoint can replicate entire Oracle environments between heterogeneous storage arrays (Symmetrix VMAXe and VNX5700). • By implementing VMware ESX to virtualize server operating environments, the server environments at the production and recovery sites do not need to be identical. • Automating site recovery with SRM takes the fear factor out of executing a disaster recovery plan. SRM recovery plans can be tested in advance, which means that there are no surprises when a real disaster recovery occurs. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 48 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 49. RecoverPoint CRR site failback processOverview When the issues that caused an outage at the production site have been resolved, production can be returned to the production site for normal processing. The simplest way to fail back the environment is to repeat the failover process, but in the opposite direction. This section outlines how the Oracle environment was failed back to the production site by creating and executing the Oracle Failback recovery plan. The individual steps are described in detail in the Automating site recovery with VMware vCenter Site Recovery Manager section of the white paper.Housekeeping Before you can configure SRM for failback, the following housekeeping steps must be carried out: 1. Save a summary report of the failover recovery plan. This can then serve as a guide for configuring failback. (This is recommended but not mandatory.) 2. Delete the failed-over virtual machines from the vCenter inventory at the production site using the command shown in Figure 40. Figure 40. Deleting failed over virtual machines 3. Delete the existing protection groups and recovery plans at both the production and recovery sites. Figure 41 shows a recovery plan being deleted. Figure 41. Deleting a recovery plan 4. Rescan the storage at the production site from the vCenter instance for each ESX server in the configuration—this is recommended but not mandatory. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 49 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 50. Configuring SRM For the use case, configuring SRM for failback involved these steps:for failback 1. Use the SRM Configure Array Managers wizard to configure array managers at both sites. This time, supply the IP address of the DR RPA as the protected site. 2. Create a protection group for the datastore group to be failed back. 3. Configure inventory mappings between the protected and recovery sites. The procedure is the same as that used to configure the inventory mappings for failover. However, this time the original recovery site served as the protected site for SRM. 4. Customize the recovery priority settings for the virtual machines. 5. On the SRM server at the production site, use the dr-ip-customizer.exe utility to customize the IP addresses for the production virtual machines. 6. Use the Create Recovery Plan wizard to create the Oracle Failback recovery plan at the production site—this plan is exactly the same as the plan created for failover.Testing the After SRM had been configured, the recovery plan was tested to ensure that failbackfailback recovery worked as expected. The procedure was the same as that described in the Testing theplan Oracle Failover recovery plan section of the white paper.Failing back to the After verifying that the failback test was successful, the Oracle Failback recovery planproduction site was executed to fail back the environment to the production site and resume normal processing. The procedure was the same as that described in the RecoverPoint CRR site failover process section of the white paper. Figure 42 shows the SRM summary report that was generated when the recovery plan had finished. This demonstrates that the entire process for failing back three virtual machines to the production array took less than 15 minutes. Figure 42. Recovery plan summary Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 50 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 51. Verifying failback Prior to failing back to the production site, some processing was performed at theto the production recovery site. A script was then run to output the record count and the timestamp ofsite the last entry. There were 22,000 records in the database and the timestamp was 31-May-11 11.32 am, as shown in Figure 43. VMs are running at DR-VCENTER Figure 43. Record count and timestamp at recovery site After executing the Oracle Failback recovery plan, the script was run at the original production site, as shown in Figure 44. There were 22,000 records in the database and the timestamp was exactly the same as on the recovery site. VMs are running at PR-VCENTER Figure 44. Record count and timestamp at original production site, after failbackBenefits of The benefits of automating failback of Oracle environments with RecoverPoint andRecoverPoint site SRM are similar to the benefits of failing over Oracle environments with thesefailback with SRM technologies. • Administrators can create and customize failback recovery plans to automate the failback process. • Administrators can test their failback recovery plans in advance to verify that they work correctly. • After a recovery plan has been set up, administrators can test or run the plan with a single click. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 51 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 52. RecoverPoint CDP database recoveryOverview In RecoverPoint CLR configurations, you can recover your Oracle database from an image at either the local site or the remote site. You can also manually bookmark particular points in time at either site, such as an event in an application, or a point in time to which you wish to fail over. You can then recover the database from any of these bookmarks. For the purpose of this solution, manual bookmarks were created at both the CDP and CRR sites, and recovery from both images was tested. This section describes recovery from the CDP copy. After a point-in-time image had been manually bookmarked in the local journal, the database was intentionally corrupted by deleting the data files. Oracle was unable to start because of the missing files, so the database was recovered from the pre- corruption bookmarked image and then started.Preparing the test The following steps were performed to set up the test scenario, before recovering thescenario database: 1. Configure the database consistency group (Oracle_SRM) for management by RecoverPoint. You do this using the Policy settings in the RecoverPoint Management Application, as shown in Figure 45. This is necessary because the consistency group is being used by both RecoverPoint CDP and SRM. Figure 45. Configuring the consistency group for management by RecoverPoint 2. Create a pre-corruption manual bookmark, as shown in Figure 46. Figure 46. Creating a bookmark Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 52 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 53. 3. Delete the data files from the production database. Figure 47 shows Oracle unable to start the database because of the missing files. Figure 47. Oracle message indicating database corruptionRecovering the The following steps were performed to recovery the database from the pre-corruptiondatabase from the bookmarked image:CDP image 1. Enable image access for the bookmarked image. You do this by using the Enable Image Access option for the CDP replica, as shown in Figure 48. Image Access menu Select pre-corruption bookmarked image Figure 48. Enabling image access to the CDP replica 2. Prior to recovering the production image, shut down the production virtual machine and any other protected virtual machines in the same consistency group. Note: Instead of recovering immediately to production, you can mount the CDP image to another ESX server in order to verify that it contains the correct data. You do not need to shut down the production virtual machine to do this, but you do need to undo any writes before recovering to production. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 53 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 54. 3. Select the Recover Production option at the CDP replica, as shown in Figure 49, and initiate the production restore process. Recovery menu Figure 49. Initiate the restore from the bookmarked image 4. Before resuming processing at the production site, enable image access for the image immediately following the Synchronization Completed image, as shown in Figure 50. Figure 50. Enabling image access 5. Resume processing on the production site. After recovery has been completed on the production image, the Resume Production option is enabled, as shown in Figure 51. When you click this, write splitting resumes on the production volumes.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 54 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 55. Resume Production Figure 51. Resuming production 6. Rescan the storage at the production site from the vCenter instance for each ESX server in the configuration. 7. Restart the production virtual machine and any other protected virtual machines in the same consistency group. Oracle now performs its own recovery to the crash-consistent image that has been restored to the production volumes. If the selected image does not contain the data that was missing, you can repeat steps 1 to 6 for an earlier image. Figure 52 shows the recovered Oracle instance following the Oracle restart. . Figure 52. Recovered Oracle instance Figure 53 shows the record count and timestamp for the recovered database, which matches the record count in the bookmarked image. Figure 53. Record count and timestamp for the recovered databaseBusiness Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 55 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 56. Benefits of The benefits of using RecoverPoint for database protection include the following:database • RecoverPoint is constantly recording application-consistent snapshot imagesprotection with that are accessible through its intuitive GUI interface. In the event of operatingRecoverPoint system or database corruption, administrators can roll back through these images or restore the database from an image that has been manually bookmarked for a particular purpose. • RecoverPoint provides a zero RPO, which means that administrators can quickly roll back to the point in time just before corruption occurred. • RecoverPoint images can also be used to mount a copy of the database to a different host in order to verify it or to simply restore part of a database file for surgical type repairs. The testing documented in this section outlines the steps required to restore an Oracle database to a consistent known good state, enabling faster recovery from a situation that could potentially involve media recovery, lead to lengthy downtime, and require manual intervention to recover to a consistent state. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 56 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 57. RecoverPoint operation when VMAXe is operating in a degraded modeOverview With Symmetrix VMAXe arrays, all directors operate in an active state at all times. This means that there is no delay in processing I/O in the event of a director failure—the failed I/O is simply transmitted down an alternate path. When the failed path comes back online, it resumes processing I/O as normal. The RecoverPoint splitter deals with the loss of a director in the same way, by redirecting split writes down alternate channels. This provides a highly available disaster recovery solution.Setting up VMAXe To test the operation of RecoverPoint when VMAXe is operating in a degraded state,degraded mode one of the VMAXe front-end director ports was forced offline. SMC indicated that the director status was Dead, as shown in Figure 54. Figure 54. SMC view of offline director status The dead director was also detected and logged as a problem by RecoverPoint, as shown in Figure 55. X indicates problem communicating with storage array Figure 55. RecoverPoint indicating a problem communicating with the storage arrayTesting degraded While the director was offline, the Swingbench load generator was running tomode generate load. At this stage, the Oracle Failover recovery plan was run in test mode. Figure 56 shows the summary report from this test. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 57 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 58. Figure 56. Oracle Failover recovery plan summary reportWhen the test recovery was complete, the database was started on the remote site toverify consistency. Figure 57 shows an extract from the Oracle alert log. Thisdemonstrates that the database successfully restarted from the crash-consistentimage at the recovery site.Figure 57. Oracle alert log showing the database successfully restartedWhile the director was offline, RecoverPoint replication continued by using theremaining paths. When the director came back online, RecoverPoint automaticallyreturned to normal state, with no need for manual intervention.Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 58 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 59. ConclusionSummary This white paper describes a data protection and disaster recovery solution for virtualized Oracle Database11g OLTP environments, using EMC RecoverPoint and VMware vCenter Site Recovery Manager. Write splitting was implemented by RecoverPoint splitters at the production site (Symmetrix VMAXe array) and recovery site (VNX5700 array). EMC RecoverPoint is a mature replication technology that provides granular local and remote point-in-time failover and recovery protection for data centers. In an Oracle environment, RecoverPoint can reliably protect the database by providing multiple, consistent point-in-time recovery points that are maintained by sophisticated journaling technology. The automated bookmarking and journaling enable selective rollback to a crash-consistent image of the Oracle database and for very granular recovery of the environment. This provides a great deal of flexibility when recovering from a disaster scenario and provides DVR-like recovery capability. With VMware vSphere virtualization of the database hosts, and VMware vCenter management of the virtual infrastructure, administrators can automate and test their disaster recovery plans using VMware vCenter Site Recovery Manager (SRM). SRM, integrated with RecoverPoint, enables creation and automation of sophisticated recovery plans that ensure managed and documented site recovery in the event of a disaster. SRM also enables rehearsal and modification of these recovery plans, with no impact to production or replication. This ensures reliable and predictable recovery in the event of a true disaster.Findings The key findings of the solution testing include: • Integration of RecoverPoint splitter technology into the Symmetrix VMAXe and VNX arrays simplifies RecoverPoint configuration and also reduces costs, as no additional write-splitting hardware is required. • The RecoverPoint splitter supports replication across heterogeneous storage platforms. • RecoverPoint enables replication of entire virtualized Oracle environments between data centers for disaster recovery. • RecoverPoint is a robust technology, with an intuitive GUI, that enables faster recovery from a DR scenario, with a granularity not possible with other replication technologies. • RecoverPoint enables disaster recovery scenarios to be tested without affecting production or interrupting replication, and while maintaining consistency across protected consistency groups. • Integration of RecoverPoint with vCenter Site Recovery Manager enables DR testing to be carried out in isolated environments on the recovery site so that production can remain active and replication can continue uninterrupted. SRM also documents the recovery process. Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 59 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview
  • 60. ReferencesWhite papers For additional information, see the white papers and technical notes listed below. • Maximize Operational Efficiency for Oracle RAC with EMC Symmetrix FAST VP (Automated Tiering) and VMware vSphere—An Architectural Overview • Improving VMware Disaster Recovery With EMC RecoverPoint—Applied Technology • Storage Provisioning with Symmetrix Auto-provisioning Groups —Technical Notes • EMC RecoverPoint/EX for the VMAXe Series—Applied Technology • Rapid Deployment and Scale Out for Oracle E-Business Suite Enabled by EMC RecoverPoint, EMC Replication Manager, and VMware vSphere—A Detailed Review • EMC RecoverPoint Deploying with VNX/CLARiiON Arrays and Splitter—Technical Notes • Deploying RecoverPoint with Symmetrix—Technical NotesProduct For additional information, see the product documents listed below.documentation • EMC RecoverPoint Site Recovery Manager Failback Plug-in—Technical Notes • EMC Solutions Enabler Symmetrix Array Controls CLI Version 7.3 Product Guide • EMC RecoverPoint Release 3.4.1 Administrator’s GuideOther For additional information, see the VMware documents listed below.documentation • Getting Started with VMware vCenter Site Recovery Manager Site Recovery Manager 4.0 and later • VMware vCenter Site Recovery Manager Administration Guide 4.1 and later • Adding a DNS Update Step to a Recovery Plan—Technical Note Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC Symmetrix 60 VMAXe, EMC RecoverPoint, and VMware vCenter Site Recovery Manager An Architectural Overview

×