IMS Online Reorganization FacilityDennis EichelbergerIMS Toolsdeichel@us.ibm.com  1                                  © 201...
IMS Online Reorganization Facility               Agenda• IMS ORF Overview• Database Types supported• Other required IMS To...
IMS Online Reorganization Facility                Overview• A one-step reorganization of IMS databases with  minimal impac...
IMS Online Reorganization Facility Why           Reorganize a Database?■   Performance     – Fragmentation     – Overflow ...
IMS Online Reorganization Facility                All Full Function Databases■   HDAM■   HIDAM■   SHISAM■   HALDB - PHDAM/...
IMS Online Reorganization Facility Other               Required IMS Tools• Unload   – IMS Basic Unload   – IMS High Speed ...
The Offline Reorg Process                             Build Sec                             Image Copy          Unload    ...
The Online Reorg Process – with shadow data sets  Allocate  Shadow              HP Unload             Build Sec           ...
IMS Online Reorganization Facility                  Phases• ORF phases occur in the following sequence:  –Verification  –C...
IMS Online Reorganization Facility            Data Flow                                                 10                ...
IMS Online Reorganization Facility            Reorganization Process• ORF can be run while:  – IMS system(s) are up  – IMS...
IMS Online Reorganization Facility                VERIFICATION PhaseIf Normal Execution (not Restart):   •   IMS Release a...
IMS Online Reorganization Facility                 COPY Phase• BMPs are paused (if usermod is installed)• Databases are cl...
IMS Online Reorganization Facility        REORGANIZATION Phase• Shadow database is Unloaded• Shadow database is Reloaded• ...
IMS Online Reorganization Facility               APPLY Phase– Captured changes are applied to the shadows– ORF pauses any ...
IMS Online Reorganization Facility                TAKEOVER Phase• TAKEOVER(Y)• Verifies the database and its indexes are u...
IMS Online Reorganization Facility         TAKEOVER (with DBD changes)– NEWDBD and ONLINECHANGE keywords are ignored for H...
IMS Online Reorganization Facility            TAKEOVER RestartIf ORF detects that there is a restart pending by reading th...
IMS Online Reorganization Facility           TAKEOVER (scheduled)Scheduled Takeover phase   Allows the reorganization Take...
IMS Online Reorganization Facility             COMPLETION PhaseFinal reports and messages written  Shadow data sets delete...
IMS Online Reorganization Facility           TAKEOVER (delayed)Allows you to delay the Takeover phase before the originald...
IMS Online Reorganization Facility               BMP Management• The Pause/Resume:  – ORF sets a flag in a control block a...
IMS Online Reorganization Facility              Online ProcessingIMS• New transaction arrives when the database is DBR’d  ...
IMS Online Reorganization Facility               Online ProcessingODBA• APSB request when DB is DBR’d   – ORF detects that...
IMS Online Reorganization Facility             Installation Considerations• IBM IMS Generic Exits is required and part of ...
IMS Online Reorganization Facility                  Restrictions• IMS Online Reorganization Facility does not support the ...
IMS Online Reorganization Facility            DBD Changes not allowed• IMS Online Reorganization Facility does not allow t...
IMS Online Reorganization Facility              Common Questions/Answers1.   How many times do the databases get stopped a...
IMS Online Reorganization Facility          Common Questions/Answers (continued)5.       Do I need to preallocate the shad...
IMS Online Reorganization Facility        Common Questions/Answers (continued)7.   Can I run an ORF job even if no online ...
IMS Online Reorganization Facility              BenefitsReduced database downtimeOne step database reorganizationEliminate...
IMS Online Reorganization FacilityQUESTIONS AND COMMENTS                                                32                ...
Upcoming SlideShare
Loading in...5
×

Online reorganization facility

937

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
937
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Online reorganization facility

  1. 1. IMS Online Reorganization FacilityDennis EichelbergerIMS Toolsdeichel@us.ibm.com 1 © 2011 IBM Corporation
  2. 2. IMS Online Reorganization Facility Agenda• IMS ORF Overview• Database Types supported• Other required IMS Tools• Phases 2 2011 IBM Corporation
  3. 3. IMS Online Reorganization Facility Overview• A one-step reorganization of IMS databases with minimal impact to its availability• Reduces database downtime from hours to seconds• Eliminates the need for manual intervention before, during and after a reorganization• Supports: • Internal logical relationships • Secondary indexes • Full Function Databases • BMP Pause and Resume 3 2011 IBM Corporation
  4. 4. IMS Online Reorganization Facility Why Reorganize a Database?■ Performance – Fragmentation – Overflow – CI/CA Splits■ Space – Nearing Limitations – Current Definitions – Reclaim – IMS Limitations■ DBD Changes 4 2011 IBM Corporation
  5. 5. IMS Online Reorganization Facility All Full Function Databases■ HDAM■ HIDAM■ SHISAM■ HALDB - PHDAM/PHIDAM – Entire database – all partitions – Single partition IBM IMS Online■ Partitioned secondary indexes Reorganization Facility – Entire index – Single partition■ Secondary indexes – When primary DB reorganized – Index only 5 2011 IBM Corporation
  6. 6. IMS Online Reorganization Facility Other Required IMS Tools• Unload – IMS Basic Unload – IMS High Speed Sequential Retrieval (HSSR/HPUNLOAD)• Load – IMS Basic Reload – IMS High Performance Load (HPLOAD)• Prefix Resolution/Update – IMS Basic Prefix Resolution & IMS Basic Prefix Update – IMS High Performance Prefix Resolution/Update• Image Copy – IMS Basic Image Copy – IMS High Performance Image CopyNote: Secondary Index tools are not used; indexes are built by ORF 6 2011 IBM Corporation
  7. 7. The Offline Reorg Process Build Sec Image Copy Unload Indexes (w/Ptr Check) Load Prefix Res/Upd Data Unavailable/DBR DB /STA DB Page 7
  8. 8. The Online Reorg Process – with shadow data sets Allocate Shadow HP Unload Build Sec Image Copy Update DBRC data sets Indexes (w/Ptr Check) & Rename Data Sets Copy Primary HP Load Prefix Res/Upd Apply Changes DB /DBR DB /STA DB Start Change Finish Change Capture Capture Page 8
  9. 9. IMS Online Reorganization Facility Phases• ORF phases occur in the following sequence: –Verification –Copy –Reorganization –Apply –Takeover The only phase allowing restart processing –Completion 9 2011 IBM Corporation
  10. 10. IMS Online Reorganization Facility Data Flow 10 2011 IBM Corporation
  11. 11. IMS Online Reorganization Facility Reorganization Process• ORF can be run while: – IMS system(s) are up – IMS system(s) are down – IMS databases are online – IMS databases are offline• No JCL changes or manual intervention needed for any of the processes above 11 2011 IBM Corporation
  12. 12. IMS Online Reorganization Facility VERIFICATION PhaseIf Normal Execution (not Restart): • IMS Release and component requirements are verified • Operating system requirements are verified • DBRC registration/database state requirements are verified • Database DBD requirements are verified • Database data set requirements are met (cannot exceed 42 characters in length and are cataloged) • ORF level for STEPLIB in batch job/IMS control regions is verified • JCL requirements are met • Control statement syntax is verified • Shadow data sets (.S data sets) are constructed/verified/allocated • Temporary (.T data sets) data sets deleted and deletions verified • Other miscellaneous checking is performed 12 2011 IBM Corporation
  13. 13. IMS Online Reorganization Facility COPY Phase• BMPs are paused (if usermod is installed)• Databases are closed/reopened using TOSI to flush buffers across all IMS subsystems and to update the catalog/VTOC• BMPs are resumed (if usermod is installed)• DBRC status remains unchanged - allocated and authorized• Waits for database syncpoint, then online component of ORF is notified to: Begins capturing all changes made to the primary database data sets – type 50 records• Copies the original primary database only to shadow copy• Near the end of the Copy phase, another close/reopen is done and: Captured changes are sent to ORF batch utility & applied to shadow primary database datasets. The shadow primary index (for HIDAM or PHIDAM) is created from the shadow primary database The ILDS (for HALDB) is created for the shadow ILDS 13 2011 IBM Corporation
  14. 14. IMS Online Reorganization Facility REORGANIZATION Phase• Shadow database is Unloaded• Shadow database is Reloaded• Shadow Secondary indexes are rebuilt except for HALDB PSINDEX databases• Shadow internal logical relationships are rebuilt by Prefix Resolution/Update (if any)• Shadow databases are Image Copied (and optionally Pointer Checked)• Online component of ORF continues to capture update calls to the original databases and are written to a temporary data set for use in the subsequent Apply phase. 14 2011 IBM Corporation
  15. 15. IMS Online Reorganization Facility APPLY Phase– Captured changes are applied to the shadows– ORF pauses any BMPs and issues a /DBR FEOV command to stop original databases– DBRC flag is set to PROHIBIT AUTH– Apply phase ends after all remaining captured-updates have been applied to shadows » If heavy activity, the apply Is done in iterations (continues to capture while applying)– Original database and its indexes are unavailable until after the Takeover phase (swapping the shadow database to the original database) 15 2011 IBM Corporation
  16. 16. IMS Online Reorganization Facility TAKEOVER Phase• TAKEOVER(Y)• Verifies the database and its indexes are unavailable • Notifies DBRC of the REORG with a timestamp after the /DBR command • Notifies DBRC of the image copy with a timestamp that is greater than the REORG timestamp • Notifies DBRC of a DB ALLOC that is needed for log data sets • Notifies DBRC of logs that were created during the Apply phase • Swaps the reorganized shadow data sets with the original database data sets • Performs the ACBGEN and ACBLIB or DMB replacements (if DBD changes have occurred) and copies the NEWDBD to the current DBDLIB if DBDCOPY(Y) is specified • Authorizes the databases in DBRC (turns off REORG INTENT and PROHIBIT AUTH flags) • Starts the databases by using a /START command • Continues to recover the log marked in error • Notifies DBRC of the log with the new timestamp 16 2011 IBM Corporation
  17. 17. IMS Online Reorganization Facility TAKEOVER (with DBD changes)– NEWDBD and ONLINECHANGE keywords are ignored for HALDB– If NEWDBD(ddname) was specified with ONLINECHANGE(Y) and DBDCOPY(Y), ORF automates the replacement of DBDs and ACBs/DMBs to eliminate manual intervention and increase database availability » Automatically generates an ACB » Copies changed DMB to ACBLIBA and ACBLIBB » Activates new DMB » Performs a DMB replacement in the staging and active ACBLIBs » Starts database(s) and resets DBRC flags– If NEWDBD(Y) was specified with ONLINECHANGE(N), ORF does not automate the replacement of DBDs and ACBs/DMBs; manual intervention is required » ORF does not automatically generate an ACB » Database(s) remain offline and DBRC flags are not reset » Manual intervention is required 17 2011 IBM Corporation
  18. 18. IMS Online Reorganization Facility TAKEOVER RestartIf ORF detects that there is a restart pending by reading the data that was saved in the restart data set:– Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords– Databases remain in PROHIBIT AUTH and REORG INTENT status until restart is complete– Databases remain stopped until restart is complete– ORF Takeover Restart utility is invoked– Automatically restarts from where it stopped in previous Takeover phase− DBRC flags are reset when TAKEOVER restart has successfully completed− Databases are started 18 2011 IBM Corporation
  19. 19. IMS Online Reorganization Facility TAKEOVER (scheduled)Scheduled Takeover phase Allows the reorganization Takeover process to be scheduled within a window that you specify Allows you to manually perform steps after the reorganization process and before the databases are brought online, if needed. Use the TAKEOVER.WINDOW(time) keyword Must specify TAKEOVER(Y) on REORG command The database is left in DB recovery needed state with PROHIBIT AUTH and REORG status set in DBRC. Restart information is saved in a restart data set. Takeover can be completed by resubmitting your job with TAKEOVER(YES) and RESTART(AUTO) keywords specifiedFormat of Scheduled Time: (begHH:MM[,endHH:MM[,endaction]]) Format 00:00 to 23:59. Times are in 24–hour time 19 2011 IBM Corporation
  20. 20. IMS Online Reorganization Facility COMPLETION PhaseFinal reports and messages written Shadow data sets deleted, if DELETE(Y) Data sets are deallocated Miscellaneous cleanup tasks performed ORF job completes 20 2011 IBM Corporation
  21. 21. IMS Online Reorganization Facility TAKEOVER (delayed)Allows you to delay the Takeover phase before the originaldatabase datasets are affectedAllows you to manually perform steps after the reorganizationprocess and before the databases are brought online, if neededOriginal job must have TAKEOVER(DELAY)Databases are left in DB recovery needed state with PROHIBITAUTH and REORGI status set in DBRCRestart information is saved in a restart data setTakeover can be completed by resubmitting your job withTAKEOVER(YES) and RESTART(AUTO) keywords 21 2011 IBM Corporation
  22. 22. IMS Online Reorganization Facility BMP Management• The Pause/Resume: – ORF sets a flag in a control block allocated in the CTL address space – ORF verify on a CHKP call if the flag has been set – The BMP checks during the suspend state if the flag has cleared – ORF uses logger exit to communicate – It cleans up if a failure occurs (the XCF connection fails) – Failure is posted from XCF that the ORF job went away – ORF then clean up – Once the BMP determine the suspend flag has been cleared, The BMP resumes Note: DLIBATCH is not Supported by ORF: • If there is a DLI Batch running, IMS ORF will fail – Unable to obtain authorization from DBRC. • Vice Versa – If ORF is active, DLI batch will fail to initiate as the database is marked REORGI is active 22 2011 IBM Corporation
  23. 23. IMS Online Reorganization Facility Online ProcessingIMS• New transaction arrives when the database is DBR’d – Transaction placed on suspend queue • Exit/Automation to process suspend queue and reissue transaction • /STA DB will requeue messageCICS• SCHEDULE PSB request when the database is DBR’d – ORF detects that it has DBR’d the database • The Thread is put into temporary wait – Typically short lived 23 2011 IBM Corporation
  24. 24. IMS Online Reorganization Facility Online ProcessingODBA• APSB request when DB is DBR’d – ORF detects that it has DBR’d the DB • Application TCB is put into temporary wait – Typically short lived 24 2011 IBM Corporation
  25. 25. IMS Online Reorganization Facility Installation Considerations• IBM IMS Generic Exits is required and part of the IMS Tools Base – Partner Exit (DFSPPUE0) – Logger Exit (DFSFLGX0)• IBM IMS Tools Online System Interface (TOSI) is required• IBM IMS BMP Pauser Interface – DFSRRC99 – No JCL Changes • By aliasing DFSRRC99 for DFSRRC00 in the running reslib – With JCL Changes • By Steplib with DFSRRC99 aliased as DFSRRC00 and invoking HRFRRC00 as the region controller in the BMP• IMS Control Regions (STEPLIB) modification and recycle needed 25 2011 IBM Corporation
  26. 26. IMS Online Reorganization Facility Restrictions• IMS Online Reorganization Facility does not support the following: – Batch jobs running as DLI, DBB, or ULU types during reorganization – HSAM databases – HIDAM databases with compressed root keys – IMS Utility Control Facility (UCF) – Databases and data sets that are not registered to DBRC – External logical relationships – Fast Path databases – Reorganization of multiple HALDB partitions in one step (you can do this type of reorganization in parallel with separate IMS Online Reorganization jobs) – HALDB DBD changes – Reorganization of HALDB Multi-Volume data sets – Uncataloged or corrupted databases 26 2011 IBM Corporation
  27. 27. IMS Online Reorganization Facility DBD Changes not allowed• IMS Online Reorganization Facility does not allow the following DBD changes: – DBD name or type – Access method – Randomizer name – Segment changes • type • variable to fixed or fixed to variable lengths • length decrease – Conversion from Full Function to HALDB – Heirarchical structure change – Logical relationships added, modified or deleted – VSAM blocksize – Secondary index • delete • conversion from or to shared secondary index 27 2011 IBM Corporation
  28. 28. IMS Online Reorganization Facility Common Questions/Answers1. How many times do the databases get stopped and a FEOV is done? They are temporarily paused twice (close and reopen the database (to get the control blocks updated in the catalog/VTOC) and stopped (/DBR’d) once with one force-end-of-volume unless FEOV(N) is specified.1. What does ORF use to communicate between the ORF batch job and the IMS online control region(s)? ORF uses XCF to communicate between shared IMS systems. Two XCFGROUP names need to be setup during ORF installation (one for TOSI (if none exists) and one for ORF online capturing).1. How does ORF handle CICS/ODBA applications when ORF needs to pause/stop the databases? When a CICS or ODBA APSB request detects that ORF needs to momentarily stop or issue a /DBR on a database, the thread in which the APSB request is made is put into a temporary wait state until ORF restarts the database. No special implementation is needed. This feature can be disabled, if desired.2. Do I need to recycle IMS to apply ORF maintenance? Not always, depends on APAR. 28 2011 IBM Corporation
  29. 29. IMS Online Reorganization Facility Common Questions/Answers (continued)5. Do I need to preallocate the shadow datasets? Depends on the keyword specified: – You must preallocate the shadows if SHADOWS(E) for Existing was specified – You do not need to preallocate the shadows if SHADOW(A) was specified. ORF will use the same attributes of the production database datasets.6. How does the BMP Pauser Interface work during an ORF job with concurrent BMP(s)? – If an ORF job needs to do a /DBR for a HALDB database, the PSB is automatically paused (no abend) after the next checkpoint and waits for ORF to restart the database and the BMP wait is released. – If an ORF job needs to do a /DBR for a non-HALDB database, the PSB is automatically stopped (U3303 abend) after the next checkpoint. After the database(s) are restarted, ORF restarts the BMP at the last checkpoint. – If a BMP attempts to start in the middle of an ORF job, a check is performed to see if a /DBR needs to be done or has not yet completed. If so, the job step TCB waits until ORF restarts the database. 29 2011 IBM Corporation
  30. 30. IMS Online Reorganization Facility Common Questions/Answers (continued)7. Can I run an ORF job even if no online systems are up? Yes, ORF relies on DBRC subsystem records and if none exist, it can run without active IMS system(s).8. Can I restart a failed ORF job? Yes, if the original job which failed had RESTART(Y) and it failed in the TAKEOVER Phase. Failures in any other phases are not restartable; however, those phases do not affect the production database so no special action is needed.9. How does ORF prevent any other utilities (reorg/recovery, etc.) from running during an ORF reorganization? ORF sets the REORG INTENT flag in DBRC which prevents any other utilities from running against the databases being reorganized by ORF. 30 2011 IBM Corporation
  31. 31. IMS Online Reorganization Facility BenefitsReduced database downtimeOne step database reorganizationEliminates manual intervention during reorganizationAutomated replacement of DBDs and ACBsImproved DBA productivityFlexible reorganization scheduleIntegration with IBM IMS High Performance Tools 31 2011 IBM Corporation
  32. 32. IMS Online Reorganization FacilityQUESTIONS AND COMMENTS 32 2011 IBM Corporation
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×