Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Online Reorganization Facility - IMS UG March 2013 Phoenix


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Online Reorganization Facility - IMS UG March 2013 Phoenix

  1. 1. IMS Online Reorganization FacilityDennis EichelbergerIT SpecialistIMS Advanced Technical 1 © 2013 IBM Corporation
  2. 2. IMS Online Reorganization Facility Agenda• IMS ORF Overview• Database Types supported• Other required IMS Tools• Phases 2 2013 IBM Corporation
  3. 3. IMS Online Reorganization Facility Overview• A one-step reorganization of IMS databases with minimal impact to its availability• Reduces database downtime from hours to seconds• Eliminates the need for manual intervention before, during and after a reorganization• Supports: • Internal logical relationships • Secondary indexes • Full Function Databases • BMP Pause and Resume 3 2013 IBM Corporation
  4. 4. IMS Online Reorganization Facility Why Reorganize a Database?■ Performance – Fragmentation – Overflow – CI/CA Splits■ Space – Nearing Limitations – Current Definitions – Reclaim – IMS Limitations■ DBD Changes 4 2013 IBM Corporation
  5. 5. IMS Online Reorganization Facility All Full Function Databases■ HDAM■ HIDAM■ SHISAM■ HALDB - PHDAM/PHIDAM – Entire database – all partitions – Single partition■ Partitioned secondary indexes IBM IMS Online Reorganization – Entire index Facility – Single partition■ Secondary indexes – When primary DB reorganized – Index only 5 2013 IBM Corporation
  6. 6. IMS Online Reorganization Facility Other Required IMS Tools• Unload – IMS Basic Unload – IMS High Speed Sequential Retrieval (HSSR/HPUNLOAD)• Load – IMS Basic Reload – IMS High Performance Load (HPLOAD)• Prefix Resolution/Update – IMS Basic Prefix Resolution & IMS Basic Prefix Update – IMS High Performance Prefix Resolution/Update• Image Copy – IMS Basic Image Copy – IMS High Performance Image CopyNote: Secondary Index tools are not used; indexes are built by ORF 6 2013 IBM Corporation
  7. 7. The Offline Reorg Process Build Sec Image Copy Unload Indexes (w/Ptr Check) Load Prefix Res/Upd Data Unavailable /STA DB/DBR DB Page 7
  8. 8. The Online Reorg Process – with shadow data sets Allocate Shadow HP Unload Build Sec Image Copy Update data sets Indexes (w/Ptr Check) DBRC & Rename Data Copy Primary HP Load Prefix Res/Upd Apply Changes Sets DB /DBR DB /STA DB Start Change Finish Change Capture Capture Page 8
  9. 9. IMS Online Reorganization Facility Phases• ORF phases occur in the following sequence: – Verification – Copy – Reorganization – Apply – Takeover  The only phase allowing restart processing – Completion 9 2013 IBM Corporation
  10. 10. IMS Online Reorganization Facility Data Flow 10 2013 IBM Corporation
  11. 11. IMS Online Reorganization Facility Reorganization Process• ORF can be run while: – IMS system(s) are up – IMS system(s) are down – IMS databases are online – IMS databases are offline• No JCL changes or manual intervention needed for any of the processes above 11 2013 IBM Corporation
  12. 12. IMS Online Reorganization Facility VERIFICATION PhaseIf Normal Execution (not Restart): • IMS Release and component requirements are verified • Operating system requirements are verified • DBRC registration/database state requirements are verified • Database DBD requirements are verified • Database data set requirements are met (cannot exceed 42 characters in length and are cataloged) • ORF level for STEPLIB in batch job/IMS control regions is verified • JCL requirements are met • Control statement syntax is verified • Shadow data sets (.S data sets) are constructed/verified/allocated • Temporary (.T data sets) data sets deleted and deletions verified • Other miscellaneous checking is performed 12 2013 IBM Corporation
  13. 13. IMS Online Reorganization Facility COPY Phase• BMPs are paused (if usermod is installed)• Databases are closed/reopened using TOSI to flush buffers across all IMS subsystems and to update the catalog/VTOC• BMPs are resumed (if usermod is installed)• DBRC status remains unchanged - allocated and authorized• Waits for database syncpoint, then online component of ORF is notified to:  Begins capturing all changes made to the primary database data sets – type 50 records• Copies the original primary database only to shadow copy• Near the end of the Copy phase, another database close/reopen is done and:  Captured changes are sent to ORF batch utility & applied to shadow primary database datasets.  The shadow primary index (for HIDAM or PHIDAM) is created from the shadow primary database  The ILDS (for HALDB) is created for the shadow ILDS 13 2013 IBM Corporation
  14. 14. IMS Online Reorganization Facility REORGANIZATION Phase• Shadow database is Unloaded• Shadow database is Reloaded• Shadow Secondary indexes are rebuilt except for HALDB PSINDEX databases• Shadow internal logical relationships are rebuilt by Prefix Resolution/Update (if any)• Shadow databases are Image Copied (and optionally Pointer Checked if rquested)• Online component of ORF continues to capture update calls to the original databases and are written to a temporary data set for use in the subsequent Apply phase. 14 2013 IBM Corporation
  15. 15. IMS Online Reorganization Facility APPLY Phase– Captured changes are applied to the shadows– ORF pauses any BMPs and issues a /DBR FEOV command to stop original databases– DBRC flag is set to PROHIBIT AUTH– Apply phase ends after all remaining captured-updates have been applied to shadows If heavy activity, the apply Is done in iterations (continues to capture while applying)– Original database and its indexes are unavailable until after the Takeover phase (swapping the shadow database to the original database) 15 2013 IBM Corporation
  16. 16. IMS Online Reorganization Facility TAKEOVER Phase• TAKEOVER(Y)• Verifies the database and its indexes are unavailable • Notifies DBRC of the REORG with a timestamp after the /DBR command • Notifies DBRC of the image copy with a timestamp that is greater than the REORG timestamp • Notifies DBRC of a DB ALLOC that is needed for log data sets • Notifies DBRC of logs that were created during the Apply phase • Swaps the reorganized shadow data sets with the original database data sets • Performs the ACBGEN and ACBLIB or DMB replacements (if DBD changes have occurred) and copies the NEWDBD to the current DBDLIB if DBDCOPY(Y) is specified • Authorizes the databases in DBRC (turns off REORG INTENT and PROHIBIT AUTH flags) • Starts the databases by using a /START command • Continues to recover the log marked in error • Notifies DBRC of the log with the new timestamp 16 2013 IBM Corporation
  17. 17. IMS Online Reorganization Facility TAKEOVER (with DBD changes)– NEWDBD and ONLINECHANGE keywords are ignored for HALDB– If NEWDBD(ddname) was specified with ONLINECHANGE(Y) and DBDCOPY(Y), ORF automates the replacement of DBDs and ACBs/DMBs to eliminate manual intervention and increase database availability– Automatically generates an ACB– Copies changed DMB to ACBLIBA and ACBLIBB– Activates new DMB– Performs a DMB replacement in the staging and active ACBLIBs– Starts database(s) and resets DBRC flags– If NEWDBD(Y) was specified with ONLINECHANGE(N), ORF does not automate the replacement of DBDs and ACBs/DMBs; manual intervention is required– ORF does not automatically generate an ACB– Database(s) remain offline and DBRC flags are not reset– Manual intervention is required 17 2013 IBM Corporation
  18. 18. IMS Online Reorganization Facility TAKEOVER RestartIf ORF detects that there is a restart pending by reading the data that was saved in the restart data set:– Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords– Databases remain in PROHIBIT AUTH and REORG INTENT status until restart is complete– Databases remain stopped until restart is complete– ORF Takeover Restart utility is invoked– Automatically restarts from where it stopped in previous Takeover phase− DBRC flags are reset when TAKEOVER restart has successfully completed− Databases are started 18 2013 IBM Corporation
  19. 19. IMS Online Reorganization Facility TAKEOVER (scheduled) Scheduled Takeover phase  Allows the reorganization Takeover process to be scheduled within a window that you specify  Allows you to manually perform steps after the reorganization process and before the databases are brought online, if needed.  Use the TAKEOVER.WINDOW(time) keyword  Must specify TAKEOVER(Y) on REORG command  The database is left in DB recovery needed state with PROHIBIT AUTH and REORG status set in DBRC.  Restart information is saved in a restart data set.  Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords specified Format of Scheduled Time:  (begHH:MM[,endHH:MM[,endaction]])  Format 00:00 to 23:59. Times are in 24–hour time 19 2013 IBM Corporation
  20. 20. IMS Online Reorganization Facility COMPLETION PhaseFinal reports and messages written Shadow data sets deleted, if DELETE(Y) Data sets are deallocated Miscellaneous cleanup tasks performed ORF job completes 20 2013 IBM Corporation
  21. 21. IMS Online Reorganization Facility TAKEOVER (delayed) Allows you to delay the Takeover phase before the original database datasets are affected Allows you to manually perform steps after the reorganization process and before the databases are brought online, if needed Original job must have TAKEOVER(DELAY) Databases are left in DB recovery needed state with PROHIBIT AUTH and REORGI status set in DBRC Restart information is saved in a restart data set Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords 21 2013 IBM Corporation
  22. 22. IMS Online Reorganization Facility BMP Management• The Pause/Resume: – ORF sets a flag in a control block allocated in the CTL address space – ORF verify on a CHKP call if the flag has been set – The BMP checks during the suspend state if the flag has cleared – ORF uses logger exit for communication (DFSFLGX0) – It cleans up if a failure occurs (the XCF connection fails) – Failure is posted from XCF that the ORF job went away – ORF then cleans itself up – Once the BMP determine the suspend flag has been cleared, the BMP will be resumed from the last checkpoint     Note: DLIBATCH is not Supported by ORF: • If there is a DLI Batch running, IMS ORF will fail – Unable to obtain authorization from DBRC. • Vice Versa – If ORF is active, DLI batch will fail to initiate as the database is marked REORGI is active 22 2013 IBM Corporation
  23. 23. IMS Online Reorganization Facility Online ProcessingIMS• New transaction arrives when the database is DBR’d – Transaction placed on suspend queue • Exit/Automation to process suspend queue and reissue transaction • /STA DB will requeue messageCICS• SCHEDULE PSB request when the database is DBR’d – ORF detects that it has DBR’d the database • The Thread is put into temporary wait – Typically short lived 23 2013 IBM Corporation
  24. 24. IMS Online Reorganization Facility Online ProcessingODBA• APSB request when DB is DBR’d – ORF detects that it has DBR’d the DB • Application TCB is put into temporary wait – Typically short lived 24 2013 IBM Corporation
  25. 25. IMS Online Reorganization Facility Installation Considerations• IBM IMS Generic Exits is required and part of the IMS Tools Base – Partner Exit (DFSPPUE0) – Logger Exit (DFSFLGX0)• IBM IMS Tools Online System Interface (TOSI) is required• IBM IMS BMP Pauser Interface – DFSRRC99 – No JCL Changes • By aliasing DFSRRC99 for DFSRRC00 in the running reslib – With JCL Changes • By Steplib with DFSRRC99 aliased as DFSRRC00 and invoking HRFRRC00 as the region controller in the BMP• IMS Control Regions (STEPLIB) modification and recycle needed 25 2013 IBM Corporation
  26. 26. IMS Online Reorganization Facility Restrictions• IMS Online Reorganization Facility does not support: – Batch jobs running as DLI, DBB, or ULU types during reorganization – HSAM databases – HIDAM databases with compressed root keys – IMS Utility Control Facility (UCF) – Databases and data sets that are not registered to DBRC – External logical relationships – Fast Path databases – Reorganization of multiple HALDB partitions in one step (you can do this type of reorganization in parallel with separate IMS Online Reorganization jobs) – HALDB DBD changes – Reorganization of HALDB Multi-Volume data sets – Uncataloged or corrupted databases 26 2013 IBM Corporation
  27. 27. IMS Online Reorganization Facility DBD Changes not allowed• IMS Online Reorganization Facility does not allow the following DBD changes: – DBD name or type – Access method – Randomizer name – Segment changes • type • variable to fixed or fixed to variable lengths • length decrease – Conversion from Full Function to HALDB – Heirarchical structure change – Logical relationships added, modified or deleted – VSAM blocksize – Secondary index • delete • conversion from or to shared secondary index 27 2013 IBM Corporation
  28. 28. IMS Online Reorganization Facility Common Questions/Answers (continued)1) How many times do the databases get stopped and a FEOVis done?  They are temporarily paused twice (close and reopen the database (to get the control blocks updated in the catalog/VTOC) and stopped (/DBR’d) once with one force-end-of-volume unless FEOV(N) is specified.2) What does ORF use to communicate between the ORFbatch job and the IMS online control region(s)?  ORF uses XCF to communicate between shared IMS systems. Two XCFGROUP names need to be setup during ORF installation (one for TOSI (if none exists) and one for ORF online capturing). 28 2013 IBM Corporation
  29. 29. IMS Online Reorganization Facility Common Questions/Answers (continued)3) How does ORF handle CICS/ODBA applications when ORFneeds to pause/stop the databases?  When a CICS or ODBA APSB request detects that ORF needs to momentarily stop or issue a /DBR on a database, the thread in which the APSB request is made is put into a temporary wait state until ORF restarts the database.  No special implementation is needed.  This feature can be disabled, if desired.4) Do I need to recycle IMS to apply ORF maintenance?  Not always, depends on APAR. specified 29 2013 IBM Corporation
  30. 30. IMS Online Reorganization Facility Common Questions/Answers (continued)5) How does the BMP Pauser Interface work during an ORF jobwith concurrent BMP(s)?  If an ORF job needs to do a /DBR for a HALDB database, the PSB is automatically paused (no abend) after the next checkpoint and waits for ORF to restart the database and the BMP wait is released.  If an ORF job needs to do a /DBR for a non-HALDB database, the PSB is automatically stopped (U3303 abend) after the next checkpoint. After the database(s) are restarted, ORF restarts the BMP at the last checkpoint.  If a BMP attempts to start in the middle of an ORF job, a check is performed to see if a /DBR needs to be done or has not yet completed. If so, the job step TCB waits until ORF restarts the database.6) Do I need to preallocate the shadow datasets?  You must preallocate the shadows if SHADOWS(E) for Existing was specified  You do not need to preallocate the shadows if SHADOW(A) was specified. ORF will use the same attributes of the production database datasets. 30 2013 IBM Corporation
  31. 31. IMS Online Reorganization Facility Common Questions/Answers (continued)7. Can I run an ORF job even if no online systems are up?  Yes, ORF relies on DBRC subsystem records and if none exist, it can run without active IMS system(s).8. Can I restart a failed ORF job?  Yes, if the original job which failed had RESTART(Y) and it failed in the TAKEOVER Phase. Failures in any other phases are not restartable; however, those phases do not affect the production database so no special action is needed.9. How does ORF prevent any other utilities (reorg/recovery, etc.) from running during an ORF reorganization?  ORF sets the REORG INTENT flag in DBRC which prevents any other utilities from running against the databases being reorganized by ORF. 31 2013 IBM Corporation
  32. 32. IMS Online Reorganization Facility Benefits Reduced database downtime One step database reorganization Eliminates manual intervention during reorganization Automated replacement of DBDs and ACBs Improved DBA productivity Flexible reorganization schedule Integration with IBM IMS High Performance Tools 32 2013 IBM Corporation
  33. 33. IMS Online Reorganization FacilityQUESTIONS AND COMMENTS 33 2013 IBM Corporation