HALDB Toolkit - IMS UG March 2013 Phoenix


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

HALDB Toolkit - IMS UG March 2013 Phoenix

  1. 1. IMS Tools 2011 ® HALDB ToolkitDennis EichelbergerIT SpecialistIMS Advanced Technical Skillsdeichel@us.ibm.com 1 © 2013 IBM Corporation
  2. 2. Agenda  HALDB Toolkit Overview  Application Support  HALDB and DBRC  Applications processing partitions  Maintaining HALDB  Converting to HALDB  Summary 2 © 2013 IBM Corporation
  3. 3. HALDB Toolkit Overview3 © 2013 IBM Corporation
  4. 4. HALDB Toolkit Overview   HALDB Database Types are defined by using  a single DBD that creates the same structure for each partition. The partitions are defined for access in the DBRC Recons. Each HALDB Partition is treated as a single dataset and may be allocated and reorganized individually, in sequential groups or all together as a single database. HALDB allows for a maximum of 1001 partitions per database. A reorganization of the partitions does not required rebuilding of a secondary index or logical relations. These can be self healed during regular processing.  Partitioned HDAM (PHDAM)  Partitioned HIDAM (PHIDAM)  Partitioned Secondary Index (PSINDEX) 4 © 2013 IBM Corporation
  5. 5. HALDB Toolkit Overview  DBRC and HALDB  Cloning a DBRC definition of HALDB to a different Recon dataset  Copy of a HALDB and its DBRC information to a different IMS system  Bypassing DBRC requirements  Analysis of HALDB structures, partitions and datasets  HALDB Toolkit Utilities  Splitting unload files to separate partition files  Rebuilding the Indirect List Keys  Adding or removing partitions  Convert an existing database to HALDB  ISPF interface 5 © 2013 IBM Corporation
  6. 6. HALDB and DBRC6 © 2013 IBM Corporation
  7. 7. Testing HALDB Databases Non-HALDB test databases  Often, test databases are not registered in DBRC RECONs  Each application programmer may have one or more versions of a database or set of databases All HALDB databases must be registered in the DBRC RECON datasets. This leads to considerations regarding testing environments.  DBRC does not allow multiple databases with the same name to be in the same Recon.  When using multiple versions of the same database the database must be defined in different RECONs  Multiple test versions of a database require multiple RECONs 7 © 2013 IBM Corporation
  8. 8. Defining Test Environments Using non-HALDB databases for testing can mean using the same DBD as a production database. Placing the DBD in a different DBDLIB or ACBLIB is sufficient for data separation between IMS environments and allowing testing to not effect a production IMS database. Replicating a HALDB into a test environment requires establishing the same partition definitions as defined in the Recons of the originating system.  Define the partitions in a test environment using the same names as the source database in a different Recon. or  Use the Partition Definition Utility EXPORT and IMPORT functions to move the definitions to the test environment Recon datasets. 8 © 2013 IBM Corporation
  9. 9. HALDB and DBRC Transporting a HALDB to a different IMS can be performed several ways.  By using an Unload/Reload using IMS utilities and a different Recon dataset with the DBD and partitions definitions.  Using an application load program into a new set of partitions and the DBD and partition definitions in a different Recon dataset.  By Using Image Copy Recovery process using the HP Image Copy product from one IMS system to another with a new set of Recon definitions.  DFDSS system services for dataset copying. 9 © 2013 IBM Corporation
  10. 10. HALDB and DBRC Transport a DB to a different IMS  Using the IMS Unload and Load tools  Redefine the partition definitions in target RECON  Reset the Independent List Keys as required HALDB Toolkit steps • Can create a copy of the DBD and the partition definitions in the target Recon based on the definitions in the source RECON. • Invoke the IMS utilities to unload the source database datasets and load the newly defined datasets. • The ILK reset utility during unload can be used to assure the ILK is current and correct. IHCHALDB IHCHALDB IHCHALDB COPYDBRC DBD(ddddd) RESETILK DBD(ddddd) INCLIND(Y) INCLIC(Y) TODD(OUTDBRC) 10 © 2013 IBM Corporation
  11. 11. Copy via Image Copy The Partition definitions must be identical in the target Recon as the source partitions.  The partition registration must be added to the target RECON.  The Image Copy record must be copied from the source RECON and added to the target Recon.  Change the dataset naming to be unique  Allocate the new target datasets. Recover the database using the target RECON. HALDB TOOLKIT  Will create the partition definition for the target RECON based on the source database definition.  Copy the Image Copy entry from the source RECON to the target RECON.  Create the IDCAMS statements to allocate the new partition datasets. 11 © 2013 IBM Corporation
  12. 12. Copy a HALDB using DFDSS Via DFDSS  This process is similar to the Image Copy process.  The target RECON information must reflect the same partition structure as defined in the source RECON.  The Database must be /DBRed. HALDB Toolkit • Create the partition definition control statements for the target RECON from the originals in the source RECON.  Create the DFDSS commands to copy the database datasets. 12 © 2013 IBM Corporation
  13. 13. HALDB Toolkit Bypassing DBRC13 © 2013 IBM Corporation
  14. 14. //ALLO EXEC PGM=IDCAMS//SYSPRINT DD SYSOUT=*//SYSIN DD *DELETE my.test.RECON1 HALDB and DBRCDELETE my.test.RECON2DELETE my.test.RECON3SET MAXCC=0 Test DatabasesDEFINE CLUSTER(NAME(my.test.RECON1) …DEFINE CLUSTER(NAME(my.test.RECON2) …. When multiple test databases areDEFINE CLUSTER(NAME(my.test.RECON3) ….//CRE EXEC PGM=IHCHALDB, REGION=60M needed a separate RECON dataset is//STEPLIB// DD DISP=SHR,DSN=IHC320.SIHCLOAD DD DISP=SHR,DSN=IMS910.SDFSRESL required for everybody needing a//DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL//IMS DD DISP=SHR,DSN=my.DBDLIB separate test database. This means//RECON1//RECON2 DD DISP=SHR,DSN=SOURCE.RECON1 DD DISP=SHR,DSN=SOURCE.RECON2 that DBRC knowledge is needed to//RECON3 DD DISP=SHR,DSN=SOURCE.RECON3//MSGPRINT DD SYSOUT=* create and maintain the definitions.//SYSUDUMP DD SYSOUT=*//SYSOUT DD SYSOUT=*//IHCSYSIN DD *COPYDBRC DBD(nnnnn) –DSNPREF(new.hlq) – TOOLKIT AssistedDSNDBD(DBD) - INCLIND(YES) –TODD(DBRCOUT)/* The cloning feature of the HALDB//DBRCOUT DD DISP=(,PASS),SPACE=(TRK,(1,1)),UNIT=SYSALLDA Toolkit will copy the partition definition//DBRC EXEC PGM=DSPURX00,REGION=50M,COND=(4,LE)//STEPLIB DD DISP=SHR,DSN=IMS910.SDFSRESL from the original Recon to a newly//RECON1 DD DISP=SHR,DSN=my.test.RECON1//RECON2 DD DISP=SHR,DSN=my.test.RECON2 created Recon. The procedure on the//RECON3 DD DISP=SHR,DSN=my.test.RECON3//DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL left shows a new recon is created with//IMS DD DISP=SHR,DSN=your.DBDLIB//SYSPRINT DD SYSOUT=* the partitions for a single HALDB.//SYSIN DD *INIT.RECON NOFORCER CATDS TAPEUNIT(3480)// DD DISP=(OLD,DELETE),DSN=*.CRE.DBRCOUT 14 © 2013 IBM Corporation
  15. 15. HALDB and DBRC – Bypassing DBRC There may be a need to have several HALDBs of the same name for testing or quality assurance purposes. In order to reduce the DBRC maintenance HALDB Toolkit provides a method to simulate a DBRC=N condition in order to have multiple HALDBs of the same name without multiple Recon datasets and their associated maintenance overhead.  This uses a single Recon that holds the HALDB master DBD and partition definitions. The partition names, numbers and key ranges, etc.  A DSNPREFX is substituted as part of the dataset name to avoid naming conflicts. The DBD and partition structure is used from the Recon definitions  DBRC is accessed for the definitions, however, all other DBRC authorization functions are bypassed. As an example, an Image Copy needed flag in the Recon will not stop an application job from accessing the database. 15 © 2013 IBM Corporation
  16. 16. HALDB and DBRC Allocate the HALDB repository dataset. This is a KSDS and a sample allocation job is in member the IHCXDBRI member of the SIHCSAMP library defined during installation. Specify the bypass parameter values and compile the IHCXMAC macro into the HALDB Toolkit loadlib. Link a DSPCRTR0 module into the HALDB Toolkit loadlib. This module will be used to access the RECON for the partition structure and will not return any DBRC authorization information. Execute the database job. The DD statements of the database datasets being bypassed must be included in the execution JCL. Bypassing DBRC completely for a job use the card //$$DBRCNO DD DUMMY 16 © 2013 IBM Corporation
  17. 17. Backup of DBRC definitions of HALDB Backup of the HALDB definitions. The Physical description of a HALDB is kept in the DBRC RECON datasets. This consists of the DBD and the partition definitions.  A DELETE.DB of the HALDB effectively makes the database unusable. The database no longer exists. There is no clue of the physical file definitions.  Even if Image Copies are still available they cannot be used without the partition definitions in a RECON.  HALDB Toolkit allows the DBRC definitions to be copied to a flat file as DBRC statements that may be re-input to the Recon at a later time if needed. //IHCSYSIN DD * BACKUPDBRC DBD(PHDO2) INCLIND(Y) TODD(outfile) /* 17 © 2013 IBM Corporation
  18. 18. Applications & HALDB18 © 2013 IBM Corporation
  19. 19. Parallel Loads of HALDB Partitions  The HALDB Toolkit allows an unload file to be split into single unload files per partition. This allows a load of the HALDB partitions to be run in parallel reducing elapsed time significantly.  The unload files are in HD format and contain only the records for their particular partition. Each file has its own header and trailer record. 19 © 2013 IBM Corporation
  20. 20. Parallel Loads 20 © 2013 IBM Corporation
  22. 22. Partition Selection API  Partition Selection API is a callable interface that may be used to split input on a partition boundary. An application program does not know what the key boundaries are in the partitions.  The Partition Selection API allows a database root segment key to be input and returns the partition name and number that the key belongs in.Selection call:CALL IHCUAPI using(″SEL″,HANDLE,RETCODE,RSNCODE,yourkey,PARTNAME,PARTNUM)The caller provides yourkey.The API returns PARTNAME and PARTNUM. 22 © 2013 IBM Corporation
  23. 23. Applications & HALDB Partitioning Selective Partition Processing Batch application programs may be set to process individual partitions or a set of partitions. The partitions to process are identified by specifying a DFSHALDB DD statement.DFSHALDB DD *HALDB PCB=(4,POHIDKA)HALDB PCB=(PCBNUM2,POHIDJA)DFSHALDB DD *HALDB PCB=(3,PVHDJ5A,NUM=4)HALDB PCB=(PCBNUM7,PVHDJ5B,NUM=3) 23 © 2013 IBM Corporation
  24. 24. Application Support //S2 EXEC PGM=IHCHALDB,TIME=(1,20), DFSHALDB statement // REGION=80M //STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD generator can be run a // DD DISP=SHR,DSN=IMS910.SDFSRESL step prior to the application //RECON1 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON1 job. //RECON2 //RECON3 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON2 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON3 //IMS DD DISP=SHR,DSN=IMSA.DBDLIB  Provide starting key and //$$DYNTR DD * //TRACE DD SYSOUT=* ending key //SYSPRINT DD SYSOUT=* //SYSABEND DD SYSOUT=* //MSGPRINT DD SYSOUT=* //IHCSYSIN DD * low key RUN PGM(IHCUDFSH) DBD(PHDO2) PCB(2) //KEYS DD * 08001000 98008000 high key //DFSHALDB DD UNIT=SYDSA,DISP=(,PASS), // DSN=&&HAL,SPACE=(TRK,1)  In subsequent application step, you specify The DFSHALDB DD is generated based on the key //DFSHALDB DD DSN=&&HAL,DISP=(OLD,DELETE) range input and passed to the next step. 24 © 2013 IBM Corporation
  25. 25. Maintaining HALDB25 © 2013 IBM Corporation
  26. 26. Production Maintenance Deleting a single partition Merging two DBDs into one DBD Changing partition boundaries Healing Index Pointers Reorganizing a PSINDEX Maintaining databases online 26 © 2013 IBM Corporation
  27. 27. Deleting a partition 21 22 23 nn    //IHCSYSIN DD * REORGIND DBD(PHDO2) DROP(PHDO22) /* 21 23 nn    27 © 2013 IBM Corporation
  28. 28. Splitting a partition 21 23 nn    //IHCSYSIN DD * RUN PGM(IHCSPLT) DBD(PHDO2) /* 21 23 nn 22    28 © 2013 IBM Corporation
  29. 29. Splitting a partition 21 23 nn 22  //IHCSYSIN DD * MAINTAIN DBD(PHDO2) PARTITION(name) PARTSIZE(4096)/* 21 23 nn 22    29 © 2013 IBM Corporation
  30. 30. Splitting a partition 21 23 nn 22  //IHCSYSIN DD * MAINTAIN DBD(PHDO2) PARTITION(name) KEYS(keysdd)/* 21 23 nn 22    30 © 2013 IBM Corporation
  31. 31. HALDB - ILDS There is an Indirect List Data Set (ILDS) is associated with each partition of a HALDB. The ILDS is a VSAM KSDS. The ILDS contains an RBA pointer to each secondary index or logical relationship target in the HALDB. An entry in the ILDS consists of an Independent List Entry (ILE), Independent List Key (ILK), segment code, partition ID, RBA and reorg number. ILDS entries may become “out of date” during HALDB maintenance. The “out of date” condition is noted when the reorganization number in logical relationship or secondary index pointer does not match the reorganization number of the partition. 31 © 2013 IBM Corporation
  32. 32. HALDB Overview – The EPS and ILEIf Partition ID in EPS is correct andreorg # in EPS matches reorg # in ILEPartition DBDS, we use EPS RBApointer, else we use the ILK to find ILDS ILKILE and use ILEs RBA Segment Code ILE Partition ID Current reorg #: 1 Segment A Current RBA EPS KEY ... Partition DBDS EPS Current reorg #:1 Current reorg #:1 Partition ID Segment B Reorg #:1 ILK RBA ILK 32 © 2013 IBM Corporation …
  33. 33. HALDB Overview – The EPS and ILEIf Partition ID in EPS is correct andreorg # in EPS matches reorg # in ILEPartition DBDS, we use EPS RBApointer, else we use the ILK to find ILDS ILKILE and use ILEs RBA Segment Code ILE Partition ID Current reorg #: 2 Segment A Current RBA EPS KEY ... Partition DBDS EPS Current reorg #:2 Current reorg #:2 Partition ID Segment B Reorg #:1 ILK RBA ILK 33 © 2013 IBM Corporation …
  34. 34. Database Pointer Healing Heal a secondary index pointer  EPS pointer of the entry in the ILDS is not “healed” after REORG. An example may be that a single partition was reorganized. The entries for that partition are now “out of date” in the sense that they point to where the target segments used to be before the partition is reorganized.  HALDB has a pointer healing process that updates the ILDS pointer with the correct RBA information during database access. Pointer healing during regular database access may not be considered directly related to the access and might be referred to as extra overhead.  HALDB Toolkit provides batch pointer healing.  The primary database must not be open for update  The secondary indexes must be offline. //IHCSYSIN DD * PTRHEAL DBD(PHDO2) /* 34 34 © 2013 IBM Corporation
  35. 35. HALDB Analyzer A HALDB is like any other database. It requires monitoring and maintenance. The HALDB analyzer can assist by verifying HALDB Structures. The analyzer can validate ILKs, root key location, EPS pointers against the ILE and performance information. The analyzer information regarding the perfomance related information is set by thresholds in the parameter of the resource being checked. The parameters are specified to give a low and high value. These are the thresholds that the analyzer uses to provide feedback. • PTRHEAL(50,80)  The first value is the Low threshold: a warning about an upcoming problem  The second value is the High threshold: problem is now more urgent and may need attention 35 35 © 2013 IBM Corporation
  36. 36. HALDB Analyzer 36 36 © 2013 IBM Corporation
  37. 37. HALDB Analyzer 37 37 © 2013 IBM Corporation
  38. 38. Health Checker  Runs very fast  Up to 8 datasets at the same time  Verification of ILK and ILE can be turned off  Identifies potential performance impacts  ILK/ILE missing or invalid  Partition Selection exit errors  Randomizer errors  Provides decision support on partition maintenance  Combining/splitting/rearranging segments  Adjusting randomizing parameters for partitions  User set warning thresholds  Normal and Critical  parm(norm,crit) 38 38 © 2013 IBM Corporation
  39. 39. Converting to HALDB39 © 2013 IBM Corporation
  40. 40. Migration to HALDB HDAM DatabasePHDAM Database    40 © 2013 IBM Corporation
  41. 41. Converting to HALDB The HALDB Toolkit provides the functions to convert an IMS full function database to HALDB:  Find partition boundaries  Convert the non-HALDB DBD to a HALDB DBD  Create the control card information for DBRC  Update the Recon dataset  Calculate appropriate size and allocate the new files for the database datasets of each partition  Invoke the appropriate utilities to unload, allocate, load and update the Recon information needed to convert the database. Can convert SHISAM, HISAM, DEDBs and HDAM databases. The database can remain open for update during the conversion process when IMS Online Reorg Facility is available and used for the conversion. This option can help maximize the availability of the database being converted. 41 41 © 2013 IBM Corporation
  42. 42. CONVERT//ICHSYSIN DD * CONVERT DBD(dbdname) Primary DBD name DBDPATT(*****…) to create a partition name ONLINE(N) to create a batch process DSNPREF(your.hlq) appended with DBD name PARTSIZE(2048)|PARTNUM(nn)|KEYS(ddname) Takeover concept  No renames, all dataset names are new  Activated only if successful  Use of temporary RECONs  RECON notification delayed  Activate changed DBD 42 © 2013 IBM Corporation 42
  43. 43. HALDB ISPF Interface43 © 2013 IBM Corporation
  44. 44. HALDB Toolkit - ISPF Interface• All functions previous addressed can be invoked through an ISPF interface for JCL creation and tracking projects that are in progress. – Environment set up – Project Tracker for Database conversion management – DBD Conversions – HALDB Maintenance Functions – DBRC for HALDB functions – Database Utilities 44 © 2013 IBM Corporation
  45. 45. HALDB Toolkit - ISPF Interface 45 © 2013 IBM Corporation
  46. 46. HALDB Toolkit - ISPF Interface• Environment setup  A VSAM cluster - IHCXKSDS is used as a repository for the toolkit information. → IMS utilities used for maintenance and conversion steps → Identification of databases in maintenance → Identification of database in conversion  This allows projects to be more easily tracked and controlled over a period of time. 46 © 2013 IBM Corporation
  47. 47. IMS HALDB Toolkit - Benefits HALDB maintenance assistance  Splitting and combining partitions  Single step pointer healing for indexes Full Function to HALDB single step conversion Application testing enhancements  DBRC information replication  DBRC authorization bypass 47 © 2013 IBM Corporation
  48. 48. IMS HALDB Toolkit - Benefits Invokes Standard IMS HD Reorganization utilities. Invokes High Performance IMS Tools. Interfaces with IMS Online Reorganization Facility. May use the High Performance I/O capabilities of the operating system. Provides a single point of control and management for on-going HALDB conversion and maintenance projects. 48 © 2013 IBM Corporation