Implementing Softek zDMF technology.


Published on

In the mainframe industry, there are many data migration products and many ways to migrate data. Some are controller based or appliance based, while others are host based. This paper focuses on host-based migration products, as they provide the most flexibility and the fewest operational restrictions. In most cases, it is helpful to use combinations of products or tools to accomplish a migration, because no one tool can do everything exactly the way the user may desire.
For example, the user may want to migrate from older to newer technology while, at the same time, moving to larger-capacity devices such as from an 3390-3 device to a 3390-9 system. One option in this scenario would be to have IBM’s System Managed Storage (SMS) do the migration with redirection, but this would not move files that have not been deleted and reallocated in a timely manner.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Implementing Softek zDMF technology.

  1. 1. IBM Global Technology ServicesJune 2008 Implementing Softek zDMF technology. Supporting simple, effective and nondisruptive data migrations
  2. 2. Implementing Softek zDMF technology.Page 2 Introduction Contents In the mainframe industry, there are many data migration products and many ways to migrate data. Some are controller based or appliance based, while 2 Introduction others are host based. This paper focuses on host-based migration products, 3 zDMF capabilities as they provide the most flexibility and the fewest operational restrictions. In 5 zDMF architecture most cases, it is helpful to use combinations of products or tools to accomplish 8 The zDMF migration process a migration, because no one tool can do everything exactly the way the user 19 A typical migration may desire. 25 Performance considerations 29 Platform support For example, the user may want to migrate from older to newer technology 29 Restrictions while, at the same time, moving to larger-capacity devices such as from an 30 Other Softek migration 3390-3 device to a 3390-9 system. One option in this scenario would be to solutions have IBM’s System Managed Storage (SMS) do the migration with redirection, 31 Summary but this would not move files that have not been deleted and reallocated in a timely manner. Another option would be to use Hierarchical Storage Management (HSM) to migrate files after they have been archived and recalled. But again, if the files are in use, HSM would not archive the files. And, in most cases, it would be necessary to take manual action to adjust HSM policies and then reset them after a migration. Other scenarios could include the use of a disk copy utility such as Data Facility Data Set Services (DFDSS) at the disk and/or file level. However, as with the above scenarios, there is the issue of files being in use, not to mention the manual effort of building the control cards and finding an extended window for conducting the migration.
  3. 3. Implementing Softek zDMF technology.Page 3 To address some of these issues, Softek developed z/OS® Dataset Highlights Mobility Facility (zDMF) technology, host-based software that can move data at the file/extent level between Direct Access Storage Device (DASD) volumes without interrupting the applications that read/write to those Softek zDMF technology can help volumes. zDMF software can work with virtually any vendor’s storage that may with several different types of local or may not be under SMS control in a shared or non-shared DASD environment data migrations. including sysplexes. zDMF software is ideal for the following types of local data migrations: • Technology refresh, especially in heterogeneous or high-availability environments • Migrations from smaller to larger volumes • Implementation of tiered storage • Consolidation of storage • Dynamic data movement to fix performance problems (e.g., moving files from storage hot spots) The sections that follow provide a general overview of zDMF software’s capabilities, architecture and migration process. They also sketch a sample migration scenario and discuss performance factors. zDMF capabilities Simplicity of design zDMF technology’s distinct value is its simplicity. Built from the ground up as a fully integrated product, it features components that are designed to execute flawlessly together. zDMF software is designed to be simple to install, simple to use and simple to maintain.
  4. 4. Implementing Softek zDMF technology.Page 4 Nondisruptive switch Highlights zDMF software’s key differentiator is its ability to nondisruptively switch the source and target files, which moves the applications dynamically onto new Softek zDMF software can dynami- storage. This switchover feature, which occurs under user control, results in cally move applications onto new redirection of an application’s input/output (I/O) function (e.g., from the origi- storage by nondisruptively switch- nal source to the target). This occurs without any disruption to the application. ing the source and target files. Until the redirection, zDMF software continues to synchronously write to the source and target volumes, allowing fallback (the ability to fall back to the original source volume) at any time. Volumes that are no longer in applications’ I/O path can be taken offline without disruption. For the current release of zDMF technology, a small application bounce may be required after the migration in order to free up the original disk space. The bounce would be needed only for applications that have a persistent file allocation; it can be scheduled to occur when a window is available. Migration tuning parameters The software can control the migra- In order to adjust the performance impact on applications during a migration, tion rate using several parameters. zDMF software provides the ability to control migration rate using the follow- ing parameters: • MAXIO (determines the maximum overall I/O concurrency) • MAX_CHANNEL_ IO (determines the maximum I/O concurrency per channel path) • MAX_DEVICE_ IO (determines the maximum I/O concurrency per individual device) • MAXTRK (specifies the size of the I/O operation in tracks that zDMF software will use)
  5. 5. Implementing Softek zDMF technology.Page 5 Migration groups Highlights zDMF software supports the definition and concurrent migration of thousands of files. Because migration projects may involve several hundred files sup- zDMF software can migrate thou- porting a range of application types, zDMF technology provides the ability to sands of files concurrently and logically group files for efficient operational control. Each group’s migration enables monitoring of the process. parameters are independently configured and controlled to best suit the business applications supported. Monitoring features zDMF provides the ability to monitor the migration process from start to finish. Statistics include details such as elapsed time, copy rate and percent- age complete. Shared DASD zDMF works in a shared DASD environment. The DASD can be shared by individual LPARs running on multiple physical CPUs, shared within a sysplex or shared across a multiple sysplex environment. New Extended Address Volume functionality Softek zDMF software’s first two zDMF can streamline large-scale migrations via new Extended Address components are the zDMF TSO Volume functionality, which exceeds the previous industry-standard storage Monitor and zDMF server. capacity of 65 cylinders. zDMF architecture zDMF technology consists of four major components: • zDMF TSO Monitor: This is a user interface to establish a migration, issue commands and control migrations. zDMF software also offers a command line interface (CLI). • zDMF server: The server is a primary component of the zDMF product framework. The zDMF server is a z/OS Started Task that must run on each system that has access to the data that will be migrated.
  6. 6. Implementing Softek zDMF technology.Page 6 • zDMF I/O monitor: A subcomponent of the zDMF server. The zDMF Highlights I/O monitor is responsible for monitoring all I/O activity to source data sets (extents). The second two components • zDMF database: This file is used to store and share information about are the zDMF I/O monitor and zDMF data migration activity. The zDMF database is used by the zDMF zDMF database. server to store and communicate migration information across all 10708881 zDMF Components Figure 3 participating systems. TSO Monitor zDMF/Server MVS/W zDMF/Server MVS/X zDMF/Server MVS/A zDMF/Server MVS/Y zDMF/Server zDMF/Server MVS/V MVS/Z ESCON/FICON Director zDMF Database Figure 1 The zDMF components
  7. 7. Implementing Softek zDMF technology.Page 7 zDMF migration elements Highlights To facilitate data migration at the logical data level, zDMF software must understand the z/OS metadata structures that describe the logical data being Four items make up the metadata migrated. It must also be able to manipulate and update the metadata dynami- including the volume table of cally while applications and the system continue to access and modify the data. contents, VTOCIX, VSAM volume data set, catalogs and the The following items comprise the metadata in a z/OS system. coupling facility. Volume table of Contains information describing volume contents, including contents (VTOC) information about specific data sets (number of extents, size of extents, starting locations, etc.). VTOCIX Used in conjunction with the VTOC for identifying free space on a volume that is SMS managed. VSAM volume data Introduced with ICF catalogs, this contains information about a set (VVDS) data set similar to what is contained within the VTOC. Additionally, information such as details on VSAM data sets such as relative block addresses (RBA) also is kept here. Catalogs There can be, and usually are, multiple catalogs in a z/OS environ- ment. A data set is most typically catalogued, and this tells the system where the data set is located (volume or volumes on which Metadata structures must be it resides). Catalog information not only is kept on a DASD volume, understood by the zDMF software. but also is kept in memory in the catalog address space (CAS). Coupling facility If enhanced catalog sharing (ECS) is in use, then entries from the VVDS may be cached in the coupling facility.
  8. 8. Implementing Softek zDMF technology.Page 8 The zDMF migration process Highlights After the installation, an administrator can use the zDMF TSO Monitor to establish a migration that performs all the steps of a migration process. The distinct sequence of steps involved in a migration using zDMF software is as described below. Migration steps Description The first four of the eight migration 1. Group definition The data set(s) to be migrated are defined steps are group definition, activa- in a migration group using the TSO Monitor. tion of a migration group, copy 2. Activating a migration group Activating a migration group initiates the phase and synchronization phase. data migration process for the defined migration group. 3. Copy phase Data is asynchronously copied from the source data sets to the target data sets that are defined in a migration group. 4. Synchronization phase All final differences between source and target data sets in a migration group are synchronized and the migration group is prepared for mirroring. 5. Mirror phase The migration group is put into a state of The last four steps are mirror synchronous mirroring. phase, diversion phase, During the mirror phase, updates to source completion phase and post- and target data sets in the migration group completion phase. are applied simultaneously. 6. Diversion phase In this phase, the actual logical relocation of data sets occurs. Source and target data sets, along with the metadata, are modified and all I/O activity is redirected to the new location. At this point, the files have been migrated, and all references and updates to the files will occur at the target location. 7. Completion phase Although the metadata has been modi- fied, applications that were active before diversion will have their I/O redirected to the target location until they de-allocate the data set. For applications that continue to have the original source file allocation, a scheduled bounce will be required in order to free the original space. 8. Post-completion phase Once the migration has completed, migra- tion groups original source data sets and the storage resources they reside upon can be cleaned up.
  9. 9. Implementing Softek zDMF technology.Page 9 Group definition Highlights Before a data migration can begin, a group definition must first be created. A group definition consists of the following: First the migration administrator • Migration group name defines groups and gives them • Migration options basic information. • Source data set(s) • Exclude data set(s) — optional • Target data set(s) zDMF software has the ability to create groups for a migration. The grouping is decided upon by the migration administrator. This makes it easier to control the migration with a single command that is applied to the entire group. It is recommended that the grouping follow some convention, such as by application or by array that is being migrated. The software can group volumes To further refine grouping, zDMF software has the ability to group volumes together to further refine grouping. together. The source volume list name references a user-defined population of source volumes, refer to “Define zDMF Group – Panel 2” and “Define zDMF Group – Panel 3a” below. Define zDMF Group - Panel 1 Define zDMF Group Com mand ===> Scroll ===> CSR Primary Com mands : EXit NExt Group Name . . . . Source Options . . . N Replace Existing Data Sets (Y/N) Y Tolerate Allocation Failure (Y/N) Figure 2 Define zDMF Group – Panel 1
  10. 10. Implementing Softek zDMF technology.Page 10 The zDMF selection criteria for the source data sets can include wild cards. Highlights Care should be taken to make sure that the selection closely reflects the actual data sets that the user intends to migrate. Consider using IEHLIST or ISPF 3.4 Wild cards can be selected for to validate the selection. migration as well as exclusion lists. In further refining the selection criteria, it is possible to build a Data Set Exclude list. An example of this may be to exclude zDMFTEST.** data sets from being migrated after selecting all data sets on a particular volume. This would exclude any data set that starts with a high-level qualifier of zDMF TEST from being migrated. Define zDMF Group - Panel 2 Define zDMF Group Panels 2 and 3a from Softek zDMF Com mand ===> Scroll ===> CSR are shown here. Primary Com mands : EXit NExt Group Name . . . . . . . COCOCO Source Data Set Options . . N Trace (Y/N) N AllocSeq (D/S/N) . Y Sphere (Y/N) N Rename UnConditional (Y/N) . N Build Data Set Exclude List (Y/N) Source Data Set Na me/Mask Source Volume List Na me . . Storage Class . . . . . . Source Volume(s) . . Figure 3 Define zDMF Group – Panel 2 Depending on parameter settings, zDMF software may display one of the fol- lowing panels. Define zDMF Group - Panel 3a Define zDMF Group Com mand ===> Scroll ===> CSR Primary Com mands : EXit NExt Group Na me . . . . . . . Source Volu me List Na me . . . . SPMS13 Source Volu me List Mask SPMS13 Figure 4 Define zDMF Group – Panel 3a
  11. 11. Implementing Softek zDMF technology.Page 11 Highlights Define zDMF Group - Panel 3b Define zDMF Group Com mand ===> Scroll ===> CSR Primary Com mands : EXit IMport NExt Group Name . . . . . . . GROUP002 Data Set Excludes Mask Figure 5 Group definition panels 3b and 3c Define zDMF Group – Panel 3b are shown here. Define zDMF Group - Panel 3c Define zDMF Group Com mand ===> Scroll ===> CSR Primary Com mands : EXit NExt Group Na me . . . . . . . . . Group 002 Rena me Unconditional Prefix (Optional) Rena me Unconditional Mask Pairs Figure 6 Define zDMF Group – Panel 3c
  12. 12. Implementing Softek zDMF technology.Page 12 . . . . . . . After specifying the source data set information, specify the target data set Highlights mask, target volume storage class and/or target volume(s) parameters, as described in “Define zDMF Group – Panel 4.” Define zDMF Group - Panel 4 Panels 4 and 5, shown here, Define zDMF Group Com mand ===> Scroll ===> CSR require information regarding group definition. Primary Com mands : EXit BUild Group Name . . . . . . . COCOCO Target Data Set Na me/Mask . . . . . Target Volume Storage Class. Target Volumes(s). . . . . . Figure 7 Define zDMF Group – Panel 4 Once all the relevant group/pair parameters for the new group definition have been entered, type the BUILD command (or BU), and then press Enter. The new group definition’s configuration information is displayed in “Define zDMF Group – Panel 5.” Define zDMF Group - Panel 5 Define zDMF Group Row 1 to 19 of 21 Com mand ===> Scroll ===> CSR Primary Com mands : EXit MOre EDit SAve VErify PRomote Group (COCOCO) - Mode (LMIGR()) - TOLORATE_ALLOCATION_FAILURE(YES) MAXRC(8) - REPLACE(NO) SOURCE_VOLUME_LIST SPMS13 (- SPMS13 - ] SET - ALLOCSEQ(NONE) - TRACE(NO) - SPHERE(YES) SOURCE ( - DSN (SXM90.zDMF.TEST.*) - SOURCE_VOLUME_LIST SPMS13 - ) TARGET (- DSN (SXM90.zDMF.TEST.*)- VOLUME (- IBM408 - Figure 8 Define zDMF Group – Panel 5
  13. 13. Implementing Softek zDMF technology.Page 13 At this point, the primary commands can be used to add more source data sets to Highlights the group, or the user can edit, save, verify and promote the newly created group. Activating a migration group During the activation phase, the During the activation phase, zDMF software allocates data sets on the target software allocates data sets. volumes. If any errors occur during allocation, the migration group will either terminate the entire group or just the data set(s) that are in error, depending on the Tolerate Allocation Failure option selected when defining the group. Copy phase tasks During the copy phase, zDMF software initiates and performs the following tasks: The copying phase occurs after a • The zDMF server asynchronously copies the source data sets to the target group is activated. data sets. • The zDMF I/O monitor routines monitor all activity to all source data sets and track all modifications to the source data set. • Once the initial copy of source data sets is completed, the zDMF server refreshes the changed data repeatedly until it reaches a point where it can quickly synchronize the source and target data sets with minimal disruption to applications or the system. Once this point is achieved, zDMF software automatically moves to the synchronization phase for this migration group.
  14. 14. Implementing Softek zDMF technology.Page 14 In some instances, a migration group may take some time to complete the Highlights copy phase across all data sets in the migration group. The copy phase hap- pens after the group is activated. Managing performance during the copy phase The zDMF software can dynami- During the copy phase, depending on the size of a migration group, a good cally modify parameters during deal of I/O could possibly be driven by zDMF software. To control the pacing the copy phase. of I/O during the zDMF copy phase, the user can modify parameters that the zDMF server uses. The parameters that can be dynamically modified include the following: • MAX_CHANNEL_ IO • MAX_DEVICE_ IO The optional MAXTRK parameter specifies the size of the I/O operation in tracks that zDMF software should use to transfer data while in copy phase. The MAXTRK value is used to reduce the impact of zDMF software on the application response time immediately following activation. The MAXTRK parameter is defined in the zDMF server startup configuration file for the system that hosts the zDMF owning server.
  15. 15. Implementing Softek zDMF technology.Page 15 Synchronization phase Highlights A migration group enters the synchronization phase when zDMF software determines that it can quickly synchronize all data between source and target The synchronization phase occurs data sets in the migration group without causing disruption to the systems and when the software determines that applications using this data. it can synchronize data without causing system and application During the synchronization phase, zDMF software initiates and performs the disruptions. following tasks: • The zDMF I/O monitor routines dynamically hold all I/O to the source data set so final synchronization can be achieved. • zDMF software copies all remaining differences from the source data sets to the target data sets. At this point, there is very little difference between data sets, and this operation occurs very quickly. • zDMF software allows normal I/O operations to continue once synchroniza- tion has been achieved. Once the mirror state has been indi- When the synchronization phase has completed, the migration group indi- cated and the synchronization phase cates that it is in mirror state. completed, the mirror phase begins. Mirror phase Once the source and target data sets within the migration group have suc- cessfully synchronized, they enter the mirror phase. During the mirror phase, updates to source and target data sets in the migration group are applied simultaneously. If an I/O error occurs at the target volume, the group is sus- pended. Once the mirror phase has been achieved, mirroring continues until one of the following occurs:
  16. 16. Implementing Softek zDMF technology.Page 16 • A Divert command is issued against the migration group. Highlights • A Suspend command is issued against the migration group. Diversion phase Updates to data sets in the The diversion phase executes in two subphases: migration group are applied simultaneously. • Subphase 1: processing of the divert command • Subphase 2: diversion of I/O from active applications to the target data set Subphase 1: processing of the divert command During this subphase, zDMF software modifies all metadata for the source and target data set pairs within the migration group, effectively swapping the identities of the source and target. To accomplish this, zDMF software: During subphase 1 of the diversion • Serializes access to the metadata for collections of data sets catalogued in a phase, zDMF software swaps the particular source/target catalog pair identities of the source and target. • Updates all metadata to accomplish the identity swapping of the source and target data sets: - Modifies all volume-based metadata in the VTOC, VTOCIX and VVDS - Modifies the catalog entries for the source and target data set pairs, and refreshes catalog data buffers for the catalogs involved across all z/OS images in a shared storage complex.
  17. 17. Implementing Softek zDMF technology.Page 17 Subphase 2: diversion of I/O from active applications to target data set Highlights With the source and target data set identities switched, I/O to any of the source data sets previously allocated by ongoing applications is diverted to the During subphase 2, the source target data set instead. It is at this point that the data set(s) has been migrated and target data set identities are from the source to the target volume. All references and I/Os to the data set will switched. All references and I/Os now be to the target volume. are set to the target volume. The original source data set is renamed, is no longer in use and can not be accessed for alteration including being deleted until the migration goes into the completion phase. Note: Once the divert command processing is completed, any new application allocations will automatically be directed to the target data set. Post Migration Completion phase Migration groups will remain in the diversion phase until all applications have relinquished their allocation of migrating data sets and zDMF software no Group completion occurs when all longer has to redirect I/O to the targets. Group completion is achieved by logi- LPARs reach the same state. cal partition (LPAR) in the shared storage complex, depending on the data set usage of the applications on each server. Each server periodically examines allocation information to see if any active address space still has any source data set allocated. Once all such allocations are freed through the normal action or completion of the system or application processing, then it is no longer necessary for zDMF software to divert any I/O. Locally, the zDMF processing is complete for the group. However, the group cannot be marked fully complete until all LPARs reach the same state.
  18. 18. Implementing Softek zDMF technology.Page 18 Identifying address spaces being diverted Highlights Identifying the z/OS address spaces that are being diverted is a simple process with zDMF software. By using the zDMF TSO Monitor Option 2 – Interact zDMF software enables easy With Promoted Groups, the user can display all address spaces that are cur- identification of the z/OS address rently diverted across all z/OS images in a shared storage complex. spaces that are being diverted. Post-completion phase Data migrations in many cases are undertaken to free up a storage resource. Before freed-up resources Post-migration actions with the freed storage resource may include removing can be used, a user must it from the system to return to a leaseholder, or reusing the storage for other initiate post-completion application data needs. Regardless of the reason, before the resources can be storage resource cleanup. reused, the user must take the following actions. Initiate post-completion storage resource cleanup 1. Ensure that the migration group has completed. 2. Delete the migration groups target data sets (renamed original source file on original source volume). 3. Delete the migration group from the zDMF database. When a migration group is deleted, zDMF software performs the following tasks: • The zDMF server ensures that the migration group is in a state that will allow it to be deleted. • The zDMF server removes the migration group from internal z/OS storage (memory). This results in valuable system memory being freed. • The zDMF server deletes the migration group from the zDMF database.
  19. 19. Implementing Softek zDMF technology.Page 19 A typical migration Highlights It is very important to be aware that the most critical phase of any migration is establishing a good migration plan. The more time spent in developing a Developing a thorough plan, the better the odds are of a successful migration. This section details the migration plan is key to a steps of a typical storage migration. As with any complex process, the basic successful migration. questions of why, what, when and how are key to understanding what kind of planning process will be needed for this migration. Example migration scenario: • Why — A technology upgrade is being performed because the current storage A technology upgrade is taking subsystem is coming off lease. place because the current storage • What — Data needed to be migrated from old to new storage. While perform- system is coming off lease. ing this conversion, it was also decided to install some new higher-capacity drives. The migration parameters would be as follows: - 800 MOD-3 to 800 MOD-3 2270GB - 120 MOD-3 to 40 MOD-9 340GB - 200 MOD-9 to 200 MOD-9 1702GB - 240 MOD-9 to 80 MOD-27 2043GB - With 1360 volumes to 1120 volumes, the total is 6355GB. • When — The new subsystem was to arrive and be installed by the first week of March. The old subsystem would come off lease on March 31. To avoid costly storage overlap, management wanted the migration to be completed by the end of March so the old subsystem could be released. • How — It was quickly decided that conventional disk-to-disk copy was not feasible due to the long application outage window that would be required. Also, the vendor’s hardware migration utility was not usable due to compat- The IT organization chose a com- ibility issues going from an Enterprise Systems Connection (ESCON) to Fiber bination of host-based utilities and Connectivity (FICON) on the new subsystem. Ultimately, a combination of products to perform the migration. host-based utilities and products was chosen to perform the migration.
  20. 20. Implementing Softek zDMF technology.Page 20 Because the environment consisted of 80 to 90 percent SMS, the operations Highlights staff believed that they could take advantage of SMS redirection, DFDSS to copy unallocated files (as explained in the section “Executing the migration” below, the operations staff eventually decided to use zDMF software in place of DFDSS), Softek Transparent Data Migration Facility (TDMF™) for volume- level migrations, and Softek zDMF for allocated files. Selection logic The Softek products were selected by the manager of infrastructure stor- age based on colleagues’ recommendations, as the manager has never used Softek products before. In addition, the fact that Softek products are designed to support vendor-neutral, nondisruptive migration and received a strong endorsement from the storage vendor made for a fairly easy decision. Executing the migration Using Softek software, the team It was decided to perform the most complex migration first, i.e., migrating the decided to perform the most com- smaller to larger volumes. There were a number of reasons for this, which are plex migration first. highlighted below. Step 1: ICKDSF Minimal Init Rather than initializing the new volumes with VOLID that corresponded to their naming convention and including them in the proper SMS storage pool, operations would simply do an ICKDSF Minimal Init with a VOLID of XXucb#. No SMS updates are done to include these volumes in the SMS pools. Step 2: Migrating smaller to larger volumes with TDMF Migrating the smaller volumes to a TDMF software was used to migrate the first smaller volume to a larger volume, larger volume first eliminated pro- i.e., one MOD-9 to a MOD-27. This had the advantage of completing one-third duction disruption. of the migration that required resizing of capacity without any disruption to production. It took less than one hour to migrate 40 MOD-3 to MOD-9 vol- umes and about four hours to migrate 80 MOD-9 to MOD-27 volumes.
  21. 21. Implementing Softek zDMF technology.Page 21 As the source VOLID was being copied to the target volume, no SMS changes Highlights needed to occur because the volume was already part of their desired SMS rules. Also, the VOLID naming standards followed to the new volume. TDMF Dynamic ICKDSF EXTVTOC Function The ICKDSF EXTVTOC option The ICKDSF EXTVTOC option is only invoked if migrating from allows the user to specify the one size device to a different size device (e.g., smaller to larger) or if number of tracks for the VTOC. a volume had been previously migrated and no REFVTOC had been performed. Only indexed VTOCs are extended. Nonindexed VTOCs, including volumes with damaged indexes, will be “REFVTOCed.” The user can specify a specific number of tracks for the VTOC, or TDMF software can use its own algorithm as follows: The minimum size of the new VTOC is the greater of the current VTOC size on the source and target. The VTOC may be extended further depending on the number of data sets on the volume: • If the volume is less than half full, the VTOC will be extended to contain the current numbers of data sets multiplied by the ratio of target to source device size, plus 25 percent. • If the volume is more than half full, the VTOC will be expanded to handle the situation where the target volume is full of data sets with the same average size. If the index needs to be extended, but the VTOC does not, Softek TDMF software will attempt to extend the VTOC by one track. If the VTOC cannot be extended because there is a data set adjacent to it, REFVTOC will be performed, unless there is insufficient space in the index for the additional VPSMS. Softek TDMF migration was set up to dynamically invoke ICKDSF to reformat and expand a volume’s VTOC. This function is performed when the source VTOC characteristics do not match those of the target device.
  22. 22. Implementing Softek zDMF technology.Page 22 Step 3: Migrating using SMS processes Highlights The remaining two-thirds of the source volumes that had been designated as part of the consolidation to larger-capacity volumes were placed in SMS “DISNEW” status. This would automatically migrate files that were deleted and reallocated during daily and weekly production runs to the new higher- capacity devices. Also this would prevent SMS from allocating any new files/extents on the old volumes. Next, the IT staff began migration Operations staff could now go on to their other migration tasks and let normal using SMS processes. SMS routines manage moving many of the files. At some point, they would need to revisit this task to determine which files SMS did not migrate. Step 4: Migrating volumes one for one using TDMF The fourth step involved migrating In this step, operations staff used TDMF software to migrate the remaining volumes using Softek TDMF software. disks that were one for one. This included 800 MOD-3 to MOD-3 and 200 MOD-9 to MOD-9 DASD. Upon completion of the TDMF migration, operations started the zDMF migration to finalize data migration requirements in step three above that SMS did not migrate. Note: The TDMF migration in step four took three days to accomplish, working only during regular business hours (8 a.m. to 5 p.m., Monday through Friday). Additional time with no migration activity was allowed so that SMS could reallocate many files to the higher-capacity devices as described in step three.
  23. 23. Implementing Softek zDMF technology.Page 23 Step 5: Migrating using zDMF Highlights The only migration that remained to be performed was of the files that had not been reallocated by SMS to higher-capacity disks during steps one through four. Next, the teams migrated files that This migration would be completed by zDMF software. were not reallocated. The files that remained were files that had not been through a processing cycle, such as weekly job run; files that had been created a long time ago and never got deleted; and files that were constantly in use such as the IBM DB2® and IBM CICS® files. Consideration was given to using DFDSS to migrate the remaining files not in allocation, but it was determined that by using zDMF software, operations staff could accomplish the same thing and also handle data sets that were in use at the same time. In addition, zDMF software also provided the flexibility of allowing data sets to go through allocation during this process, whereas DFDSS would have had periodic allocation issues as production continued to run. Step 6: Completion The completion of the migration All files except the ones with persistent allocations were migrated to comple- involved files that remained in tion. The files that remained in zDMF diversion mode were identified using diversion mode. the zDMF TSO Monitor; those applications were then scheduled to be bounced over the weekend. Once zDMF software completed all data set migrations, the storage migration project to the new technology was complete.
  24. 24. Implementing Softek zDMF technology.Page 24 Step 7: Verification Highlights One additional day was spent to verify that all data had migrated as planned and that post-migration tasks had been completed. The staff spent an additional day verifying that all data was migrated. Migration timeline The following was the timeline for the migration described above: This list and diagram summarize • Premigration: install new storage first week the migration timeline. • Migration step 1: ICKDSF INIT new storage • Migration step 2: TDMF small volumes to large volumes • Migration step 3: SMS “DISNEW” redirection • Migration step 4: TDMF equal size volumes • Migration step 5: zDMF small to large volume “files” (files not moved by SMS redirection) • Migration step 6: Scheduled application bounce (if required) • Migration step 7: Clean up • Post-migration: Two weeks to remove old storage (installation, migration and de-installation complete in one month) Figure 9 Sample migration timeline
  25. 25. Implementing Softek zDMF technology.Page 25 Performance considerations Highlights TDMF performance management options TDMF software has a number of options to manage performance during a Softek TDMF software manages migration. Some specifically set parameters as to how TDMF software will performance during a migration. execute the migration, while others are dynamic and self-adjust depending on resource utilization at a given point in time. Non-dynamic Dynamic FASTCOPY PACING (dynamic or fixed) FULLSPEED CONCURRENT volume migrations Note: Pacing is based on user I/O perfor- SYNCgoal mance and CPU memory utilization. zDMF performance management options The zDMF performance can be zDMF performance can be adjusted during a migration with the following adjusted during a migration. options. In its current release, the dynamic options must be manually altered during the course of a migration because they are not self-adjusting. Non-dynamic Dynamic MAXIO MAX_CHANNEL_IO MAXTRK MAX_DEVICE_IO Note: Parameters may be changed during the course of a migration by the use of z/OS console commands.
  26. 26. Implementing Softek zDMF technology.Page 26 Performance rule of thumb Highlights All migrations, regardless of method, will vary in performance depending on the particular environment in which they are carried out. Among the factors Several factors can affect the migra- that can impact performance are: tion performance, including the data change rate. • Logical-to-physical disk layout (a 72GB physical disk can have 25 MOD3 logical disks) • Type and number of channels • End user/application I/O activity • Data change rate • Amount of cache and cache hit ratios • Number of concurrent data migrations. In addition, host-based migration products can be affected by CPU utilization and by the amount and usage of processor memory. Both TDMF and zDMF software act as typical disk-to-disk utilities with very low CPU utilization, but they are capable of driving the I/O subsystem to its capacity. The following section gives The sections below provide estimates for typical TDMF and zDMF throughput estimates for Softek zDMF and during an average migration. (Note that for the purposes of these examples, TDMF throughput during an numbers have been rounded off.) It is also important to notice that the esti- average migration. mates for zDMF software are lower than they are for TDMF software (100GB vs. 180GB per hour). Keep in mind that compared to TDMF software, zDMF technology must monitor many more migration entities (thousands of files/ extents) and deal with more components (such as adhering to the customer’s SMS policies and modifying metadata).
  27. 27. Implementing Softek zDMF technology.Page 27 TDMF throughput estimates Highlights • 180GB (64 MOD-3 volumes) per hour based on 12 concurrent volumes over ESCON Softek TDMF throughput esti- Sample TDMF migration: mates and a sample migration are given here. • 400 MOD-3 volumes (1.1TB) to 400 MOD-3 volumes — (64 volumes/hr = 6 hours) • 400 MOD-9 volumes (3.4TB) to 400 MOD-9 volumes — (21 volumes/hr = 19 hours) • No application outage • CPU less than 3 percent for the master and less than 1 percent per agent (TDMF software is required to run on each LPAR that is sharing DASD; one LPAR will be designated as the TDMF master, and a TDMF agent will run on each of the other LPARs.) zDMF throughput estimates • 100GB per hour Sample zDMF migration: • 800 MOD-3 volumes (2.3TB) to 400 MOD-9 volumes — (100GB/hr = 23 hours) • Note: Minimal (scheduled) application outage may be required. Although the files have been migrated, some data sets with persistent allocations will remain in diversion mode until a scheduled application bounce can be scheduled. The time estimate is only for file replication.
  28. 28. Implementing Softek zDMF technology.Page 28 Tips for conducting large migrations Highlights When a migration includes only like volume-to-volume migrations, consider using a product that migrates at the volume level, such as Softek TDMF soft- When a migration includes large ware. When a migration includes different disk size capacities for the source volumes of data, businesses and target volumes, consider using a combination of Softek TDMF and zDMF should consider using Softek software. Use TDMF technology to move the first source volume to the target TDMF software in conjunction and then use zDMF technology to move the other source volumes to the with zDMF software. same target. In addition to making this migration easier, this method of performing migra- tion has the advantage of not requiring users to introduce a “new” VOLID into the installation that must follow some standard as well as avoiding concern over introducing a new VOLID into the SMS environment. To use a simple analogy, picture the data to be migrated as a haystack that has to be moved from point A to point B. The farmer could use a forklift (TDMF software) to quickly move the haystack in large chunks, but the farmer would probably wind up having to leave behind a lot of the hay that was too small for the forklift to pick up. Alternatively, the farmer could use a pitchfork (zDMF software) that allows moving small bundles of hay, but it would be harder and take longer to move the haystack. The best way to tackle the job, then, would be to use some combination of both the forklift and the pitchfork, TDMF and zDMF technology, to quickly and completely move the hay.
  29. 29. Implementing Softek zDMF technology.Page 29 Platform support Highlights • TDMF software: IBM OS/390® 2.10 and all levels of IBM z/OS • zDMF software: IBM z/OS 1.4 and higher Restrictions Platform support and restrictions TDMF software: are included here. • Volumes with Active Page or Swap data sets • Volume containing the active TDMF SYSCOM data set for the session zDMF software: • System type data sets (e.g., data sets in LinkList or APF authorized to a specific volume) • VSAM KSDS files with IMBED, KEYRANGE and REPLICATE defined • Catalogues • ISAM data sets • Individual members within a partitioned data set (PDS) • UNIX® System Services (USS) HFS or zFS data sets • Page data sets • Undefined data sets (DSORG=U or DSORG=PSU) • VSAM Volume data sets (VVDS) • VTOCs • Temporary (&&) data sets
  30. 30. Implementing Softek zDMF technology.Page 30 In addition to the above, the following restriction also applies: Highlights • Control units must have equal or higher functionality. For example, move- ment from a 3990 control unit to a 2105 control unit is allowed; reversal of this movement, however, is not allowed. The reason for this restriction is that some applications such as DB2 technology take advantage of expanded channel command word (CCW) command sets. If such a migration were to be allowed, a command reject would ensue, and the application could expe- rience unexpected, and possibly problematic, results. Other Softek migration solutions Softek zDMF software is designed and optimized specifically for local data migrations at the file extent level. Softek TDMF for z/OS technology is designed for volume-level migrations involving movement for local or remote distances (also known as “global migrations”). Data can be moved across a TCP/IP net- work (LAN or WAN) or channel extenders. Softek TDMF and zDMF software Softek zDMF and TDMF products can both be used in the same environment can be used in the same envi- to address different migration project requirements. zDMF and TDMF soft- ronment to address different ware is designed to provide a very fast, easy, optimized migration solution migration needs. for data migrations. Softek also offers TDMF software for open system platforms.
  31. 31. Implementing Softek zDMF technology.Page 31 Summary Highlights IT organizations looking to take advantage of large-capacity volumes on today’s high-performance storage subsystems are faced with complex and disrup- zDMF software is designed to tive data conversions that can have a negative impact on important business support the needs of today’s applications. zDMF software can give users the power to consolidate data onto 24x7 business environment. large-capacity volumes without interruption to the 24x7 business environment. zDMF software can help ensure that applications maintain maximum availabil- ity even as the underlying storage infrastructure is phased out and taken offline. zDMF software was designed specifically to help customers, storage vendors and integrators successfully and quickly move to new storage technology. For more information For more information about implementing Softek zDMF technology, visit:
  32. 32. © Copyright IBM Corporation 2008 IBM Global Services Route 100 Somers, NY 10589 U.S.A. Produced in the United States of America 06-08 All Rights Reserved IBM, the IBM logo, CICS, DB2, zDMF, Nonstop Data Mobility, OS/390, Softek, TDMF and z/OS are trade- marks or registered trademarks of International Business Corporation and other companies in the United States, other countries, or both. Microsoft is a trademark of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. Performance/capacity results or other technical statistics appearing in this document are provided by the author solely for the purposes of illustrat- ing specific technical concepts relating to the Softek products discussed herein. The perfor- mance/capacity results or other technical statistics published herein do not constitute or represent a warranty as to merchantability, operation, or fitness of any Softek product for any particular purpose. SDW03009-USEN-02