Dfw   ims cloning-tool_dfwug
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Dfw ims cloning-tool_dfwug

  • 1,224 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,224
On Slideshare
1,224
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
11
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Ron Bisceglia 15 June 2010 IMS Cloning Tool Faster, Automated, IMS Cloning and Refresh Operations Information Management © 2010 IBM Corporation © 2010 Rocket Software
  • 2. Information Management Agenda  Trends and directions  Cloning terminology  Use cases  Traditional Copy/Cloning Methods  Options for copying data  Clone IMS systems by volume  Refresh IMS Databases by data set 2 © 2010 IBM Corporation © 2010 Rocket Software
  • 3. Information Management Database and Storage Administration Trends and Directions  Large DB2 and IMS systems require high availability – Fast and non-intrusive backup and cloning facilities are required – Fast recovery and cloning capabilities are required to minimize downtime and promote high availability – Most backup, recovery and cloning solutions do not leverage storage-based fast-replication facilities  Storage-based fast-replication facilities are under-utilized – Tend to be used by storage organizations – Tend not to be used by database administrators (DBAs)  Storage aware database products – Allow DBAs to use fast-replication in a safe and transparent manner – Provide fast and non-intrusive backup and cloning operations – Simplify recovery operations and reduces recovery time – Simplify disaster recovery procedures  The amount of data is increasing – More data needs to be processed 3 © 2010 IBM Corporation © 2010 Rocket Software
  • 4. Information Management Database and Storage Integration Mainframe Application and Database Database Management Systems Domain • Organizational Integration • New Backup Methods Storage Aware • New Recovery Strategies Database Tools • Business Recovery Monitoring • Cloning Automation • Disaster Restart Solutions Storage Administration Source and Database Business Continuity Backup, Domain Clone, DR 4 © 2010 IBM Corporation © 2010 Rocket Software
  • 5. Information Management Database and Storage Integration Operational Advantages  Reduce backup, recovery, and cloning administration costs  Reduce host CPU and I/O resource utilization  Perform backups and create clone copies instantly  Fast restore and parallel recovery reduces recovery time  Simplify disaster recovery operations and procedures  DBMS and storage-based fast-replication integration – Leverage storage processors and fast-replication investments • IBM, EMC, HDS, STK – Expose fast-replication capabilities to the DBAs safely and transparently using “storage-aware” database utilities  Provide a sophisticated infrastructure and metadata to manage the DBMS and storage processor coordination 5 © 2010 IBM Corporation © 2010 Rocket Software
  • 6. Information Management Database and Storage Integration New Solutions for DBAs to Consider  DBA’s use traditional database backup and recovery tools for cloning – Hard to integrate new cloning methodologies – Uncomfortable with solutions for system cloning and database refresh operations  Lack of database and storage administration coordination – Storage processor fast-replication facilities are not well understood by application and DBA personnel • DBA’s don’t trust storage technologies – Database applicability of fast-replication not well understood by storage administrators • Storage groups don’t trust DBA’s  Storage-aware database utilities resolve these issues 6 © 2010 IBM Corporation © 2010 Rocket Software
  • 7. Information Management Cloning Terminology  A clone is an exact replica – Clone an IMS system by volume – Clone an IMS database by data set  IMS system cloning and database refresh – The act of replicating the data, making the replica accessible, and then using the replica in lieu of the original data  IMS system cloning automation – Clones a complete IMS system including all its databases  IMS database refresh automation – Refreshes specific databases 7 © 2010 IBM Corporation © 2010 Rocket Software
  • 8. Information Management Use Cases for Cloning IMS Systems or Refreshing IMS Databases  Offload business reporting to an IMS clone – Improve production performance – Run pointer checker utility from the clone  Create or refresh test, development, or quality assurance environments – Development refresh – To stage data-warehouse loads  Aid in problem determination – Troubleshoot a production problem – Test new releases of IMS – Apply maintenance and verify integrity before applying to production  Refresh databases into a previously cloned IMS skeleton 8 © 2010 IBM Corporation © 2010 Rocket Software
  • 9. Information Management Why is Test Data Management Important?  Improving business critical applications – Improve application reliability – Increase functionality – Shorten time to market – Reduce development and testing costs  Sharing resources and skills for development and testing – Setting up a test environment usually involves: • Systems programmer • DBA • Application developer • Tester – Test databases are typically shared  Meeting technical requirements – Application testing done in a production replica – Technical changes tested before production implementation  Meeting regulatory requirements – Protection of personal or sensitivity data 9 © 2010 IBM Corporation © 2010 Rocket Software
  • 10. Information Management IMS System Cloning Today Production LPAR Test LPAR IMS001 IMS001 User User Catalogs Catalogs IMS002 IMS002 IMS System IMS System Data Sets Data Sets IMS003 IMS003 Application Application Libraries Libraries IMS004 IMS004 RECON RECON Data Sets Data Sets IMS005 IMS005 Data Base Data Base Data Sets Data Sets 10 © 2010 IBM Corporation © 2010 Rocket Software
  • 11. Information Management IMS Database Cloning Today z/OS Production IMS Test IMS IMS System IMS System Data Sets Data Sets Application ??? Application Libraries Libraries RECON Unload Load RECON Data Sets Data Sets Data Base Data Base Data Sets Data Sets Image Recovery Copy Index Rebuild 11 © 2010 IBM Corporation © 2010 Rocket Software
  • 12. Information Management What is Fast Replication?  An instant copy of a volume/data set at a specific point in time  Data Movement (CPU and I/O) offloaded to storage processor – Frees up resources on host processor  A relationship is established between a source and a target -> Pair – Geometrically similar devices  Consistency Groups – Group of volumes copied at exactly the same point in time while maintaining the order of dependent writes 12 © 2010 IBM Corporation © 2010 Rocket Software
  • 13. Information Management Variations of Fast-Replication  NOCOPY – Data read from source – COPY and WRITE occurs on source updates – Useful for situations where data only needed for temporary amount of time  BACKGROUND – COPY from source may occur on reads to target  PERSISTENT – Relationship between source and target maintained  INCREMENTAL – Only changed tracks copied – Requires PERSISTENT relationship to be maintained  Space Efficient or Virtual Devices (available 2007) – IBM Flashcopy SE – EMC Symmetrix Virtual Devices – Reduces disk space for target devices – Mappings or pointers to source data maintained in storage processors 13 © 2010 IBM Corporation © 2010 Rocket Software
  • 14. Information Management Space Efficient Usage Economics Enable Many Clone Copies Write to Original Track VDEV Track Source Virtual Save Device Device  The Snap target is accessible when the copy session is activated  The first time a track on the source device is written to: – Original data on the source device is copied to a save device (pool) – Pointer on the VDEV device is changed to point to the save device – The host write is then written onto the track of the source device  The track on the source device is then updated  Unchanged data stays in place on the source device 14 © 2010 IBM Corporation © 2010 Rocket Software
  • 15. Information Management Fast Replication Data Copy Options Fast copy processes offloaded to the storage processor No host CPU or I/O resources  Volume based fast replication options for IMS Fast Replication system cloning Commands from – FlashCopy (IBM,EMC,HDS) z/OS – SnapShot (IBM,STK) – TimeFinder/Clone Volume Snap (EMC) – TimeFinder/Snap (EMC) – Mirror processes •PPRC (IBM,EMC,HDS) •TimeFinder/Mirror, SRDF (EMC) •ShadowImage HUR (HDS)  Data set based fast replication options for IMS database refresh – Data Set FlashCopy (IBM,EMC,HDS) – Data set SnapShot (IBM,STK) – TimeFinder/Clone Data set Snap (EMC) 15 © 2010 IBM Corporation © 2010 Rocket Software
  • 16. Information Management Host Based Data Copy Options Data copy processes use host based CPU and I/O facilities Slower than storage-based fast replication  Volume copy options for IMS system cloning Host-based – TDMF (IBM) Copy Process – FDRPAS (Innovation Data Processing) – DFSMSdss (IBM) – FDR (Innovation Data Processing)  Data set copy options for IMS database refresh – Any traditional data set copy processes 16 © 2010 IBM Corporation © 2010 Rocket Software
  • 17. Information Management Challenges to Data Access On the Same or Shared LPAR  IMS system cloning is best done using volume replication – Large amounts of data – Volume replication much faster than data set replication • No allocation • Assumes volumes somewhat utilized  Volume data is replicated fast and easy but… – How do you access the cloned data?  Inherent Problems: – VOLSERs may have the same volume names as the source – Volume VTOC, VTOCIX and VVDS may be the same or different – Data set names are the same on source and target volumes – If you want to access the data from a common LPAR, how do you access the data? 17 © 2010 IBM Corporation © 2010 Rocket Software
  • 18. Information Management Challenges to Data Access On the Same or Shared LPAR - Volume ID Issues Source Target PIMS01 PIMS01 (offline) VTOC VTOC SYS1.VTOCIX.PIMS01 SYS1.VTOCIX.PIMS01 SYS1.VVDS.VPIMS01 SYS1.VVDS.VPIMS01 Result: 1. Source PIMS01 keeps its original VOLSER on the target volume 2. VTOC, VTOCIX, and VVDS reflect PIMS01 3. The target volume is offline because it has the same VOLSER as the source. It could be brought online on another non-sharing system to use the clone 4. Target volume can‟t be used on the same or shared disk LPAR without re-labeling the volume. 18 © 2010 IBM Corporation © 2010 Rocket Software
  • 19. Information Management Challenges to Data Access On the Same or Shared LPAR - Volume ID Issues (2) Source Target PIMS01 TIMS01 (online) VTOC VTOC SYS1.VTOCIX.PIMS01 SYS1.VTOCIX.PIMS01 SYS1.VVDS.VPIMS01 SYS1.VVDS.VPIMS01 Result: 1. Source PIMS01 becomes new VOLSER TIMS01 on the target volume 2. VTOC, VTOCIX, and VVDS reflect PIMS01 19 © 2010 IBM Corporation © 2010 Rocket Software
  • 20. Information Management Challenges to Data Access Data Set Name and Cataloging Issues Source Target A1.CAT PIMS01 TIMS01 ICF User VTOC VVDS VTOC VVDS Catalog VTOCIX VTOCIX A.DSN3 A.DSN3 A.DSN2 A.DSN2 A.DSN1 A.DSN1 Result: 1. Data sets on the volume are copied, but keep their original name 2. Only the source data sets are cataloged; even if the catalog is on the cloned volumes, it isn‟t connected to the system‟s master catalog 20 © 2010 IBM Corporation © 2010 Rocket Software
  • 21. Information Management IMS Cloning Tool Provides Data Access On the Same or Shared LPAR  Solution – IMS Cloning Tool — the Key to Data Access – Provides access to data sets on target volumes • Renames the VTOC, VTOCIX, and VVDS to match the target volume • Renames and catalogs all data sets to a new HLQ • Adjusts target IMS system to accommodate and accept the cloned data – Solution to enable the use of the data sets on the target volumes on the same image • No more complicated cloning • No more requirement for a separate image 21 © 2010 IBM Corporation © 2010 Rocket Software
  • 22. Information Management Clone IMS Systems Using IMS Cloning Tool IMS System Cloning Automation Production Database  Performs IMS system cloning automation – Takes an existing IMS system (complete installation and system generation process completed) and creates a new, IMS or cloned, IMS system from it without having to repeat the IMS Clone entire installation and system generation processes – Simplifies IMS cloning processes Cloning Tool – Reduces cloning time and administration costs Cloning  Leverages fast replication facilities to clone data Automation – Data can be cloned while online or offline  Performs rapid volume reconditioning and data set renaming on cloned volumes to solve the data access challenges – Target volumes retain their target volume label – Renames the VTOC, VTOCIX, and VVDS to match the Source Clone Database Database target volume Volumes Volumes – Renames and catalogs all data sets to a new HLQ  Adjusts target IMS to accommodate and accept the cloned data – IMS RECONs, PROCLIB, JOBS, JCL, MDA members 22 © 2010 IBM Corporation © 2010 Rocket Software
  • 23. Information Management Pre-IMS System Cloning Steps One Time Set Up - Target IMS System  User creation of target subsystem – Define target IMS to zOS – Determine what components and configuration options you want – Build dependent region PROCs • Create target PROCLIB members in source PROCLIB – Build IMS.JOBS members • Create target IMS.JOBS members in source IMS.JOBS data set  By default, because the clone is an exact replica of the source, the clone would have the same components as the source 23 © 2010 IBM Corporation © 2010 Rocket Software
  • 24. Information Management IMS System Cloning Steps Production IMS Target IMS „Source‟ IMS IMS Clone 1 IMS volume 5 Rename selection 2 A. Suspend IMS 6 Update cloned IMS systems data B. Consistency Group 7 Start cloned IMS 3 Volume copy 4 Resume IMS if 2A Clone Source Database Database Volumes Volumes 24 © 2010 IBM Corporation © 2010 Rocket Software
  • 25. Information Management Status of In-Flight Transactions  Status of In-Flight Transactions – An online cloning solution often results in transactions in flight – These in-flight transactions, cloned to the target system result in the same target system action that would happen on the source system if it were to have died at that same time and then restarted – The target restart is essentially an emergency restart of a failed system 25 © 2010 IBM Corporation © 2010 Rocket Software
  • 26. Information Management Cloned IMS System Updates  RECONs data sets – Data set names, IMS subsystem IDs, and VOLSERs are updated in the following RECON records: header record, database data set records, online log records, and back-out records  Optionally, the following RECON records are updated if they were on volumes that were cloned: – Image copy records, change accumulation records – System log data set (SLDS) records – Recovery log data set (RLDS) records 26 © 2010 IBM Corporation © 2010 Rocket Software
  • 27. Information Management Cloned IMS System Updates  IMS PROCLIB and JOBS and user JCL libraries – New values for IMSID, VOLSERS, and data set names in the JCL members within these libraries  MDA (MVS Dynamic Allocation) members for databases or system data sets, are updated to reflect the new data set names. Updates include: – The RECON data sets – Online Log data sets (OLDS) – Write-ahead data sets (WADS)  If IMS data sharing is involved, each additional IMS data sharing member is updated 27 © 2010 IBM Corporation © 2010 Rocket Software
  • 28. Information Management IMS Environments  IMS Active – IMS offline – IMS online • Hardware Consistency • Software Suspend • ERE for transaction consistency  Data Sharing – Many to less members – Data sharing to non-data sharing – Data sharing members may need to be restarted prior to being removed 28 © 2010 IBM Corporation © 2010 Rocket Software
  • 29. Information Management Refresh IMS Databases Using IMS Cloning Tool Database Refresh Automation Production Database  Performs automated IMS database refresh operations IMS IMS – Fast refresh of IMS databases Source Target – IMS DB support (FF, HALDB, DEDB) IMS Cloning Tool  Verifies source and target database compatibility Database Refresh  IMS data copied using storage-based data set Automation fast-replication – Target takes up the same amount of space as the source  Performs target system meta-data management – Updates DBRC information for target databases IMS IMS System Clone Volumes Volumes 29 © 2010 IBM Corporation © 2010 Rocket Software
  • 30. Information Management Refresh IMS Databases Using IMS Cloning Tool Production IMS Target IMS „Source‟ 1 Create target IMS databases IMS IMS if they do not exist Target Source Job Target Job 2 7 Update DBRC Database Selection for Target Databases 3 Compatibility Check 8 Start Target Databases Stop Source and 4 Target IMS databases 5 Data Set Copy Process Start Source 6 IMS databases 30 © 2010 IBM Corporation © 2010 Rocket Software
  • 31. Information Management IMS Database Refresh to a Cloned IMS Database Skeleton Production IMS Target IMS „Source‟ 3 Create Cloned IMS System Without IMS IMS Cloned Database Clone Data IMS System and Refresh IMS Databases Database Volumes IMS System IMS IMS 4 Refresh IMS 1 Database A Volume System Skeleton Selection Data Refresh IMS Sets System 5 2 Perform IMS System Database B Cloning Automation 6 Run Offload Using System Data A A Application on Set Volumes Only B Refreshed IMS B Databases 31 © 2010 IBM Corporation © 2010 Rocket Software
  • 32. Information Management Database Selection  Specify database names – Finds the IMS subsystem name and whether it is active – Finds the source and target databases and indexes – Determines data set names for each database and index – Verifies their existence IMSDBREFRESH - IMS-SSID(ssids,ssidt) - DBD(dbdname1,dbdname2) - INDEXES(Y) - DATA-MOVER(PGM(ADRDSSU),FASTREP(PREF)) – AUTO-START-SOURCE-DB(Y) - AUTO-STOP-TARGET-DB(Y) - AUTO-START-TARGET-DB(Y) - GLOBAL NOFEOV 32 © 2010 IBM Corporation © 2010 Rocket Software
  • 33. Information Management Compatibility  Verify compatibility between the source and target IMS – Gets attributes of IMS databases and indexes from source and target IMS RECON, ACBLIB, and MDALIB data sets Object type Attributes that must be identical for source and target object Database Type, Access Method, Number of segments, data set groups, Blocksize, Randomizer Parameters Partitioned Database Number of Partitions, Partition Selection Routine, Keystrng Segment Length, Key start and length, Uniqueness, Pointers, Edit Routine Index CIsize, record length, DDATA, source and target segment codes, Sparse routine Partitioned Index Number of Partitions, Keystrng 33 © 2010 IBM Corporation © 2010 Rocket Software
  • 34. Information Management Multi-threaded Data Set Copy  ADRDSSU PARALLEL Option  DSS_COPY_COMMANDS – Specifies number of COPY tasks  DSNS_PER_COPY – Specifies number of data sets to copy per task ADRDSSU PARALLEL mode 34 © 2010 IBM Corporation © 2010 Rocket Software
  • 35. Information Management Test Data Management  Optim reduces and scrubs data from IMS, DB2, VSAM, and sequential files  Cloning Tool creates target environments, refreshes databases and tablespaces IMS/DB2 Target A Reduced IMS data data IMS/DB2 Target A Cloning Optim Tool DB2 data IMS/DB2 Target A IMS/DB2 Target A VSAM and sequential 35 © 2010 IBM Corporation © 2010 Rocket Software
  • 36. Information Management Run Pointer Checker Utility on the Clone  DBA’s have to balance the value IMS pointer checker utility analysis provides with the adverse affect it can have on running production applications  The questions that database administrators must ask are: – What IMS pointer checker utility functions need to run? – How often do they need to be run? – How much impact will the utility functions have to production data? – How can the impact to production data be reduced?  Running pointer checker on the clone preserves the availability and performance of running IMS applications 36 © 2010 IBM Corporation © 2010 Rocket Software
  • 37. Information Management Copying HALDB Databases  Copying HALDB databases can be complex and can affect database availability – IMS Cloning Tool makes copying HALDB databases fast and easy. • Uses storage-based fast-replication to copy the data instantly • Updates the target IMS RECON to reflect the same partition and reorg numbers as the source RECON • Primary and secondary indexes and ILDS can be copied to eliminate index and ILDS rebuild time • Manages partition number and gap sequencing 37 © 2010 IBM Corporation © 2010 Rocket Software
  • 38. Information Management IMS Cloning Tool Session Summarization  IMS Cloning Tool simplifies and automates database administration tasks • Integrates and coordinates database and storage activities  Promote high availability solutions • IMS Cloning Tool clones IMS systems fast and effectively • IMS Cloning Tool provides fast and effective refresh operations  Integrates storage-based fast replication • Provides fast and non-intrusive cloning operations  Offloads copy to the storage processor – Reduces host CPU and I/O resource utilization 38 © 2010 IBM Corporation © 2010 Rocket Software
  • 39. Information Management Sources and Additional References  Simply and Improve Database Administration by Leveraging Your Storage System – By Ron Haupert, Rocket Software – http://www.mainframezone.com/it-management/simplify-and-improve-database- administration-by-leveraging-your-storag  IBM System Storage DS8000 Series: IBM FlashCopy SE – http://www.redbooks.ibm.com/redpapers/pdfs/redp4368.pdf  EMC Timefinder/Snap: Multiple Snaphots with Oracle 11g Databases – http://www.emc.com/collateral/software/white-papers/h5635-timefinder-snap-multiple- snapshots-wp.pdf  Using Hitachi ShadowImage and Hitachi TrueCopy software with the IBM DB2 Universal Database Enterprise Server Edition Version 8.1 – http://www.hds.com/assets/pdf/wp_150_db2.pdf 39 © 2010 IBM Corporation © 2010 Rocket Software
  • 40. Information Management Q&A 40 © 2010 IBM Corporation © 2010 Rocket Software