HALDB Toolkit present - IMS UG May 2013 Helsinki


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

HALDB Toolkit present - IMS UG May 2013 Helsinki

  1. 1. ®IMS Tools 2013© 2013 IBM Corporation1HALDB ToolkitNordics IMS Technical UpdateMay 2013Krister PetterssonClient Technical Professionalkristerp@se.ibm.com
  2. 2. © 2013 IBM Corporation2AgendaHALDB Toolkit OverviewApplication SupportHALDB and DBRCApplications processing partitionsMaintaining HALDBConverting to HALDBSummary
  3. 3. © 2013 IBM Corporation3HALDB ToolkitOverview
  4. 4. © 2013 IBM Corporation4HALDB Toolkit OverviewHALDB Database Types are defined by usinga single DBD that creates the same structure for each partition.The partitions are defined for access in the DBRC Recons.Each HALDB Partition is treated as a single dataset and maybe allocated and reorganized individually, in sequential groupsor all together as a single database. HALDB allows for amaximum of 1001 partitions per database.A reorganization of the partitions does not required rebuildingof a secondary index or logical relations. These can be selfhealed during regular processing.Partitioned HDAM (PHDAM)Partitioned HIDAM (PHIDAM)Partitioned Secondary Index (PSINDEX)
  5. 5. © 2013 IBM Corporation5HALDB Toolkit OverviewDBRC and HALDBCloning a DBRC definition of HALDB to a differentRecon datasetCopy of a HALDB and its DBRC information to a differentIMS systemBypassing DBRC requirementsAnalysis of HALDB structures, partitions and datasetsHALDB Toolkit UtilitiesSplitting unload files to separate partition filesRebuilding the Indirect List KeysAdding or removing partitionsConvert an existing database to HALDBISPF interface
  6. 6. © 2013 IBM Corporation6HALDB and DBRC
  7. 7. © 2013 IBM Corporation7Testing HALDB DatabasesNon-HALDB test databasesOften, test databases are not registered in DBRC RECONsEach application programmer may have one or more versionsof a database or set of databasesAll HALDB databases must be registered in the DBRCRECON datasets. This leads to considerations regardingtesting environments.DBRC does not allow multiple databases with the same nameto be in the same Recon.When using multiple versions of the same database thedatabase must be defined in different RECONsMultiple test versions of a database require multiple RECONs
  8. 8. © 2013 IBM Corporation8Defining Test EnvironmentsUsing non-HALDB databases for testing can mean using the sameDBD as a production database. Placing the DBD in a differentDBDLIB or ACBLIB is sufficient for data separation between IMSenvironments and allowing testing to not effect a production IMSdatabase.Replicating a HALDB into a test environment requires establishingthe same partition definitions as defined in the Recons of theoriginating system.Define the partitions in a test environment using the same names asthe source database in a different Recon.orUse the Partition Definition Utility EXPORT and IMPORT functions tomove the definitions to the test environment Recon datasets.
  9. 9. © 2013 IBM Corporation9HALDB and DBRCTransporting a HALDB to a different IMS can be performedseveral ways.By using an Unload/Reload using IMS utilities and a differentRecon dataset with the DBD and partitions definitions.Using an application load program into a new set of partitions andthe DBD and partition definitions in a different Recon dataset.By Using Image Copy Recovery process using the HP ImageCopy product from one IMS system to another with a new set ofRecon definitions.DFDSS system services for dataset copying.
  10. 10. © 2013 IBM Corporation10HALDB and DBRCTransport a DB to a different IMSUsing the IMS Unload and Load toolsRedefine the partition definitions in target RECONReset the Independent List Keys as requiredHALDB Toolkit steps• Can create a copy of the DBD and the partition definitions in thetarget Recon based on the definitions in the source RECON.• Invoke the IMS utilities to unload the source databasedatasets and load the newly defined datasets.• The ILK reset utility during unload can be used to assure the ILKis current and correct.COPYDBRC DBD(ddddd)INCLIND(Y)INCLIC(Y)TODD(OUTDBRC)IHCHALDBRESETILK DBD(ddddd)IHCHALDBIHCHALDB
  11. 11. © 2013 IBM Corporation11Copy via Image CopyThe Partition definitions must be identical in the target Recon as thesource partitions.The partition registration must be added to the target RECON.The Image Copy record must be copied from the source RECONand added to the target Recon.Change the dataset naming to be uniqueAllocate the new target datasets.Recover the database using the target RECON.HALDB TOOLKITWill create the partition definition for the target RECON based onthe source database definition.Copy the Image Copy entry from the source RECON to the targetRECON.Create the IDCAMS statements to allocate the new partitiondatasets.
  12. 12. © 2013 IBM Corporation12Copy a HALDB using DFDSSVia DFDSSThis process is similar to the Image Copy process.The target RECON information must reflect the samepartition structure as defined in the source RECON.The Database must be /DBRed.HALDB Toolkit• Create the partition definition control statements for thetarget RECON from the originals in the source RECON.Create the DFDSS commands to copy the databasedatasets.
  13. 13. © 2013 IBM Corporation13HALDB ToolkitBypassing DBRC
  14. 14. © 2013 IBM Corporation14HALDB and DBRCTest DatabasesWhen multiple test databases areneeded a separate RECON dataset isrequired for everybody needing aseparate test database. This meansthat DBRC knowledge is needed tocreate and maintain the definitions.TOOLKIT AssistedThe cloning feature of the HALDBToolkit will copy the partition definitionfrom the original Recon to a newlycreated Recon. The procedure on theleft shows a new recon is created withthe partitions for a single HALDB.//ALLO EXEC PGM=IDCAMS//SYSPRINT DD SYSOUT=*//SYSIN DD *DELETE my.test.RECON1DELETE my.test.RECON2DELETE my.test.RECON3SET MAXCC=0DEFINE CLUSTER(NAME(my.test.RECON1) …DEFINE CLUSTER(NAME(my.test.RECON2) ….DEFINE CLUSTER(NAME(my.test.RECON3) ….//CRE EXEC PGM=IHCHALDB, REGION=60M//STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD// DD DISP=SHR,DSN=IMS910.SDFSRESL//DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL//IMS DD DISP=SHR,DSN=my.DBDLIB//RECON1 DD DISP=SHR,DSN=SOURCE.RECON1//RECON2 DD DISP=SHR,DSN=SOURCE.RECON2//RECON3 DD DISP=SHR,DSN=SOURCE.RECON3//MSGPRINT DD SYSOUT=*//SYSUDUMP DD SYSOUT=*//SYSOUT DD SYSOUT=*//IHCSYSIN DD *COPYDBRC DBD(nnnnn) –DSNPREF(new.hlq) –DSNDBD(DBD) - INCLIND(YES) –TODD(DBRCOUT)/*//DBRCOUT DD DISP=(,PASS),SPACE=(TRK,(1,1)),UNIT=SYSALLDA//DBRC EXEC PGM=DSPURX00,REGION=50M,COND=(4,LE)//STEPLIB DD DISP=SHR,DSN=IMS910.SDFSRESL//RECON1 DD DISP=SHR,DSN=my.test.RECON1//RECON2 DD DISP=SHR,DSN=my.test.RECON2//RECON3 DD DISP=SHR,DSN=my.test.RECON3//DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL//IMS DD DISP=SHR,DSN=your.DBDLIB//SYSPRINT DD SYSOUT=*//SYSIN DD *INIT.RECON NOFORCER CATDS TAPEUNIT(3480)// DD DISP=(OLD,DELETE),DSN=*.CRE.DBRCOUT
  15. 15. © 2013 IBM Corporation15HALDB and DBRC – Bypassing DBRCThere may be a need to have several HALDBs of the samename for testing or quality assurance purposes. In order toreduce the DBRC maintenance HALDB Toolkit provides amethod to simulate a DBRC=N condition in order to havemultiple HALDBs of the same name without multiple Recondatasets and their associated maintenance overhead.This uses a single Recon that holds the HALDB master DBD andpartition definitions. The partition names, numbers and key ranges,etc.A DSNPREFX is substituted as part of the dataset name to avoidnaming conflicts. The DBD and partition structure is used from theRecon definitionsDBRC is accessed for the definitions, however, all other DBRCauthorization functions are bypassed. As an example, an ImageCopy needed flag in the Recon will not stop an application job fromaccessing the database.
  16. 16. © 2013 IBM Corporation16HALDB and DBRCAllocate the HALDB repository dataset. This is a KSDS and asample allocation job is in member the IHCXDBRI member ofthe SIHCSAMP library defined during installation.Specify the bypass parameter values and compile theIHCXMAC macro into the HALDB Toolkit loadlib.Link a DSPCRTR0 module into the HALDB Toolkit loadlib.This module will be used to access the RECON for thepartition structure and will not return any DBRC authorizationinformation.Execute the database job. The DD statements of the databasedatasets being bypassed must be included in the executionJCL.Bypassing DBRC completely for a job use the card//$$DBRCNO DD DUMMY
  17. 17. © 2013 IBM Corporation17Backup of DBRC definitions of HALDBBackup of the HALDB definitions. The Physical description of aHALDB is kept in the DBRC RECON datasets. This consists of theDBD and the partition definitions.A DELETE.DB of the HALDB effectively makes the databaseunusable. The database no longer exists. There is no clue ofthe physical file definitions.Even if Image Copies are still available they cannot be usedwithout the partition definitions in a RECON.HALDB Toolkit allows the DBRC definitions to be copied to aflat file as DBRC statements that may be re-input to the Reconat a later time if needed.//IHCSYSIN DD *BACKUPDBRC DBD(PHDO2)INCLIND(Y) TODD(outfile)/*
  18. 18. © 2013 IBM Corporation18Applications & HALDB
  19. 19. © 2013 IBM Corporation19Parallel Loads of HALDB PartitionsThe HALDB Toolkit allows an unload file to be split intosingle unload files per partition. This allows a load of theHALDB partitions to be run in parallel reducing elapsedtime significantly.The unload files are in HD format and contain only therecords for their particular partition. Each file has its ownheader and trailer record.
  20. 20. © 2013 IBM Corporation20Parallel Loads
  22. 22. © 2013 IBM Corporation22Partition Selection APIPartition Selection API is a callable interface that may be usedto split input on a partition boundary. An application programdoes not know what the key boundaries are in the partitions.The Partition Selection API allows a database root segmentkey to be input and returns the partition name and number thatthe key belongs in.Selection call:CALL IHCUAPI using(″SEL″,HANDLE,RETCODE,RSNCODE,yourkey,PARTNAME,PARTNUM)The caller provides yourkey.The API returns PARTNAME and PARTNUM.
  23. 23. © 2013 IBM Corporation23Applications & HALDB PartitioningSelective Partition ProcessingBatch application programs may be set to process individualpartitions or a set of partitions. The partitions to process areidentified by specifying a DFSHALDB DD statement.DFSHALDB DD *HALDB PCB=(4,POHIDKA)HALDB PCB=(PCBNUM2,POHIDJA)DFSHALDB DD *HALDB PCB=(3,PVHDJ5A,NUM=4)HALDB PCB=(PCBNUM7,PVHDJ5B,NUM=3)
  24. 24. © 2013 IBM Corporation24Application SupportDFSHALDBstatement generatorcan be run a stepprior to theapplication job.Provide starting key andending keyIn subsequent applicationstep, you specifyThe DFSHALDB DD isgenerated based on thekey range input andpassed to the next step.//S2 EXEC PGM=IHCHALDB,TIME=(1,20),// REGION=80M//STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD// DD DISP=SHR,DSN=IMS910.SDFSRESL//RECON1 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON1//RECON2 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON2//RECON3 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON3//IMS DD DISP=SHR,DSN=IMSA.DBDLIB//$$DYNTR DD *//TRACE DD SYSOUT=*//SYSPRINT DD SYSOUT=*//SYSABEND DD SYSOUT=*//MSGPRINT DD SYSOUT=*//IHCSYSIN DD *RUN PGM(IHCUDFSH) DBD(PHDO2) PCB(2)//KEYS DD *0800100098008000//DFSHALDB DD UNIT=SYDSA,DISP=(,PASS),// DSN=&&HAL,SPACE=(TRK,1)low keyhigh key//DFSHALDB DD DSN=&&HAL,DISP=(OLD,DELETE)
  25. 25. © 2013 IBM Corporation25Maintaining HALDB
  26. 26. © 2013 IBM Corporation26Production MaintenanceDeleting a single partitionMerging two DBDs into one DBDChanging partition boundariesHealing Index PointersReorganizing a PSINDEXMaintaining databases online
  27. 27. © 2013 IBM Corporation27Deleting a partition//IHCSYSIN DD *REORGIND DBD(PHDO2) DROP(PHDO22)/*21 22 23 nn21 23 nn
  28. 28. © 2013 IBM Corporation28Splitting a partition//IHCSYSIN DD *RUN PGM(IHCSPLT) DBD(PHDO2)/*212223 nn21 23 nn
  29. 29. © 2013 IBM Corporation29Splitting a partition//IHCSYSIN DD *MAINTAIN DBD(PHDO2) PARTITION(name) PARTSIZE(4096)/*212223 nn21 23 nn22
  30. 30. © 2013 IBM Corporation30Splitting a partition//IHCSYSIN DD *MAINTAIN DBD(PHDO2) PARTITION(name) KEYS(keysdd)/*212223 nn21 23 nn22
  31. 31. © 2013 IBM Corporation31HALDB - ILDSThere is an Indirect List Data Set (ILDS) is associated with eachpartition of a HALDB. The ILDS is a VSAM KSDS.The ILDS contains an RBA pointer to each secondary index or logicalrelationship target in the HALDB. An entry in the ILDS consists of anIndependent List Entry (ILE), Independent List Key (ILK), segmentcode, partition ID, RBA and reorg number.ILDS entries may become “out of date” during HALDB maintenance.The “out of date” condition is noted when the reorganization numberin logical relationship or secondary index pointer does not match thereorganization number of the partition.
  32. 32. © 2013 IBM Corporation32HALDB Overview – The EPS and ILEEPSPartition IDReorg #:1RBAEPS KEYSegment AIf Partition ID in EPS is correct andreorg # in EPS matches reorg # inPartition DBDS, we use EPS RBApointer, else we use the ILK to findILE and use ILEs RBAILDSILEPartition DBDSCurrent reorg #:1ILKSegment BILEILKSegment CodePartition IDCurrent reorg #: 1Current RBA. . .Current reorg #:1
  33. 33. © 2013 IBM Corporation33HALDB Overview – The EPS and ILEEPSPartition IDReorg #:1RBAEPS KEYSegment AIf Partition ID in EPS is correct andreorg # in EPS matches reorg # inPartition DBDS, we use EPS RBApointer, else we use the ILK to findILE and use ILEs RBAILDSILEPartition DBDSCurrent reorg #:2ILKSegment BILEILKSegment CodePartition IDCurrent reorg #: 2Current RBA. . .Current reorg #:2
  34. 34. © 2013 IBM Corporation34Database Pointer HealingHeal a secondary index pointerEPS pointer of the entry in the ILDS is not “healed” after REORG.An example may be that a single partition was reorganized. Theentries for that partition are now “out of date” in the sense that theypoint to where the target segments used to be before the partitionis reorganized.HALDB has a pointer healing process that updates the ILDSpointer with the correct RBA information during database access.Pointer healing during regular database access may not beconsidered directly related to the access and might be referred toas extra overhead.HALDB Toolkit provides batch pointer healing.The primary database must not be open for updateThe secondary indexes must be offline.34//IHCSYSIN DD *PTRHEAL DBD(PHDO2)/*
  35. 35. © 2013 IBM Corporation35HALDB AnalyzerA HALDB is like any other database. It requires monitoring andmaintenance. The HALDB analyzer can assist by verifyingHALDB Structures. The analyzer can validate ILKs, root keylocation, EPS pointers against the ILE and performanceinformation.The analyzer information regarding the perfomance relatedinformation is set by thresholds in the parameter of the resourcebeing checked.The parameters are specified to give a low and high value.These are the thresholds that the analyzer uses to providefeedback.• PTRHEAL(50,80)The first value is the Low threshold: a warning about anupcoming problemThe second value is the High threshold: problem is nowmore urgent and may need attention35
  36. 36. © 2013 IBM Corporation36HALDB Analyzer36
  37. 37. © 2013 IBM Corporation37HALDB Analyzer37
  38. 38. © 2013 IBM Corporation38Health CheckerRuns very fastUp to 8 datasets at the same timeVerification of ILK and ILE can be turned offIdentifies potential performance impactsILK/ILE missing or invalidPartition Selection exit errorsRandomizer errorsProvides decision support on partition maintenanceCombining/splitting/rearranging segmentsAdjusting randomizing parameters for partitionsUser set warning thresholdsNormal and Criticalparm(norm,crit)38
  39. 39. © 2013 IBM Corporation39Converting to HALDB
  40. 40. © 2013 IBM Corporation40Migration to HALDBHDAM DatabasePHDAM Database
  41. 41. © 2013 IBM Corporation41Converting to HALDBThe HALDB Toolkit provides the functions to convert an IMS fullfunction database to HALDB:Find partition boundariesConvert the non-HALDB DBD to a HALDB DBDCreate the control card information for DBRCUpdate the Recon datasetCalculate appropriate size and allocate the new files for thedatabase datasets of each partitionInvoke the appropriate utilities to unload, allocate, load and updatethe Recon information needed to convert the database.Can convert SHISAM, HISAM, DEDBs and HDAM databases.The database can remain open for update during the conversionprocess when IMS Online Reorg Facility is available and used forthe conversion. This option can help maximize the availability ofthe database being converted.41
  42. 42. © 2013 IBM Corporation4242CONVERT//ICHSYSIN DD *CONVERT DBD(dbdname) Primary DBD nameDBDPATT(*****…) to create a partition nameONLINE(N) to create a batch processDSNPREF(your.hlq) appended with DBD namePARTSIZE(2048)|PARTNUM(nn)|KEYS(ddname)Takeover conceptNo renames, all dataset names are newActivated only if successfulUse of temporary RECONsRECON notification delayedActivate changed DBD
  43. 43. © 2013 IBM Corporation43HALDBISPF Interface
  44. 44. © 2013 IBM Corporation44HALDB Toolkit - ISPF Interface• All functions previous addressed can be invoked through anISPF interface for JCL creation and tracking projects that are inprogress.– Environment set up– Project Tracker for Database conversion management– DBD Conversions– HALDB Maintenance Functions– DBRC for HALDB functions– Database Utilities
  45. 45. © 2013 IBM Corporation45HALDB Toolkit - ISPF Interface
  46. 46. © 2013 IBM Corporation46HALDB Toolkit - ISPF Interface• Environment setupA VSAM cluster - IHCXKSDS is used as a repository for thetoolkit information.→IMS utilities used for maintenance and conversion steps→Identification of databases in maintenance→Identification of database in conversionThis allows projects to be more easily tracked andcontrolled over a period of time.
  47. 47. © 2013 IBM Corporation47IMS HALDB Toolkit - BenefitsHALDB maintenance assistanceSplitting and combining partitionsSingle step pointer healing for indexesFull Function to HALDB single step conversionApplication testing enhancementsDBRC information replicationDBRC authorization bypass
  48. 48. © 2013 IBM Corporation48IMS HALDB Toolkit - BenefitsInvokes Standard IMS HD Reorganization utilities.Invokes High Performance IMS Tools.Interfaces with IMS Online Reorganization Facility.May use the High Performance I/O capabilities of the operatingsystem.Provides a single point of control and management for on-goingHALDB conversion and maintenance projects.