IMS Batch Message Processing - IMS UG Singapore 8-2013

  • 380 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
380
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
12
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. IMS Batch Message Processing © 2013 IBM Corporation
  • 2. Agenda  BMP Overview  Typical daily processing workload  Recent IMS function to help BMPs  Converting a batch program to a BMP  BMP Programming facilities © 2013 IBM Corporation
  • 3. Batch Message Program (BMP) z/OS CICS Transaction Processing APPL PGM IMSCTL Control Region TELECOMMUNICATIONS DBRC Recovery Control DLISAS DL/I Subordinate Address-space BMP Batch Message Processing LOG CTL DL/I ACTION MODULES APPL PGM MESSAGE SCHEDULING LOGGING/ RESTART ONLINE LOG DATA SET RECON CORPORATE DATABASE  BMP is not started by the IMS control region, but by submitting a batch job.  The batch job then connects to an IMS control region  Many BMPs can run concurrently – same or different programs  BMPs do not normally process online transactions, but are designed for larger bulk processing of data. DB/DC SUBSYSTEM © 2013 IBM Corporation
  • 4. Typical Banking Batch (BMP) Processing over 24 hours Overnight BMPs BMPs during the day Start overnight BMP Processing Online Transactions in CICS or IMS  Large Australian Bank BMP workload  4176 steps run over 24 hour period  Of these, 2227 (53%) ran between 08:00 and 18:00 © 2013 IBM Corporation
  • 5. Comments from Large Australian Bank  “all the BMPs must checkpoint if they are locking data, and checkpoint frequently enough to not cause any undue impact on the online systems.”  “There has to be enough capacity to ensure the BMP runs and doesn’t get delayed because of CPU/paging..etc while they hold locks.”  “The database pools have to sized correctly”  ...etc © 2013 IBM Corporation
  • 6. Overnight Batch Processing Benchmark  IMS Version 2.2 Overnight Processing Redbook – Strict timeframe for processing Batch (BMP) workloads – For a Major Bank, with their Retail Banking application – Goal: Prove the overnight batch processing could be finished within the timeframe  Conclusions – IMS is able to process the Batch Applications in the Overnight Timeframe. – Store data into multiple key ranges, across separate IMS databases • Similar to HALDB function which is available today in IMS – Parallel application processing, • One stream for each key range • Merge the data results at the end of the parallel processing © 2013 IBM Corporation
  • 7. IMS V12 Dynamic Full Function Database Buffer Pools  IMS 12 adds dynamic buffer pool support for full function databases – Users can dynamically manage full function buffer pools – Buffer pool definitions can change without taking IMS down – Application activity is internally quiesced • Allows new buffer pools to be created • Allows existing buffer pools to be changed or deleted  Benefits – Improved buffer pool management • Provides type-2 commands for better usability • Eliminate system down time for modifications to buffer pool definitions • Improve application performance with improved buffer pool specifications 8.7 © 2013 IBM Corporation
  • 8. Overview of Dynamic Full Function Dynamic Buffer Pools IMS Proclib System Initialization DFSVSMxx IMS CONTROL REGION 2048,10000 4096,10000 POOLID=VCCC 2048,10000 4096,10000 1 Define OSAM and VSAM Changes in DFSDFxxx IMS Proclib 2 DFSDFxxx <SECTION=VSAM111> POOLID=VCCC 2048,30000 4096,30000 3 8.8 IMS CONTROL REGION 2048,30000 4096,30000 UPDATE POOL TYPE(DBAS) SECTION(VSAM111) © 2013 IBM Corporation
  • 9. Fast Path 64 Bit Buffer Manager  Fast Path buffers above the 2GB bar in control region address space – Optional – Multiple subpools with different buffer sizes • Subpools expand and compress dynamically  Implementation – DFSDFxxx PROCLIB member <SECTION=FASTPATH> FPBP64=Y • FPBP64=N is the default • Changing FPBP64 value requires a cold start of IMS • User does not specify the number of buffers. FP handles this DBBF, BSIZ, and DBFX are ignored when FPBP64=Y is specified 9 IMS 11 | May 2010 © Copyright IBM Corporation 2010 © 2013 IBM Corporation
  • 10. Fast Path 64 Bit Buffer Manager  64 bit buffer pool – Initial allocation of buffers in each subpool is determined by the number of areas using the CI size – Subpools expand and compress depending on buffer use by applications • Maximum storage used above the bar is 2GB  ECSA pool – Every subpool has a small number of buffers in ECSA • Used for: MSDBs Buffer headers (DMHRs) and some control blocks System buffers including those used for SDEP inserts – Initial allocation of ECSA storage is 64K (for tracking of 2GB above the bar) • ECSA storage may grow as needed. 10 IMS 11 | May 2010 © Copyright IBM Corporation 2010 © 2013 IBM Corporation
  • 11. Converting a Batch Program to BMP  Requirements for converting a batch program to a BMP are: – A BMP must have an I/O PCB • You can obtain an I/O PCB in batch by specifying the compatibility option in the Program Specification Block (PSB) for the program. – BMPs should issue Checkpoint calls more frequently than batch programs • Batch programs in a data-sharing environment must also issue Checkpoint calls frequently. © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 12. Additional Programming Features  "DUMMY" I/O-PCB  System Service Calls – Checkpoint – Restart  GSAM: Generalized Sequential Access Method – Simple, non-hierarchic data set – No update capability – Compatible with non-IMS data sets © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 13. I/O PCB  If an application does not have an I/O-PCB, it prevents: – INQY,SETS – CHKP, XRST – System Service Calls  'AD' status code if non I/O-PCB is used for these calls © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 14. "Dummy" I/O-PCB  Input/Output PCB is normally used only for "terminal" I/O PROGRAM LOGIC ENTRY .. CALL 'XRST' .. CALL 'LOG' .. CALL 'GU' .. CALL 'ISRT' .. CALL 'GN' .. CALL 'CHKP' PCB-AREA I/O-PCB  Some DB calls require the I/O-PCB – CHECKPOINT – RESTART – LOG  Built by CMPAT=Y parameter during PSBGEN DB-PCB-1  Also used for: – Testing "online" programs – Conversion to "online" DB-PCB-2 GSAM-PCB © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 15. Checkpoint/Restart Program Flow DATA AREAS FUNCTION CODES I/O-AREAS WORK AREA LIST OF USER-DEFINED AREAS PROGRAM LOGIC ENTRY CALL 'XRST' .. PROCESS .. CALL 'CHKP' TERMINATION © Copyright IBM Corporation 2008 RESTART LOGIC CHKPT-ID DATE/TIME // EXEC .... PARM='... CHKPT-ID IN WORK AREA YES NO NO CHKPT-ID IN PARM FIELD YES RESTART PROCESSING CONTINUE NORMAL PROCESSING TREAT XRST AS NOP --------------------SET FLAG FOR CHKP © 2013 IBM Corporation
  • 16. Dynamic Backout QPOOL STOP! MSG/BMP TRAN A MASTER INPUT MSG GU IOPCB 1 ISRT DBPCB STOP! TRAN PGM LTERM ABEND CODE SEG1 OF INPUT PGM X ISRT IOPCB OUTPUT MSG 2 ABEND!! 4 DATA BASE 3 Delete message In process Dynamic backout! (of all updates since last sync point) © Copyright IBM Corporation 2008 5 Cancel all msgs ISRT'd (except express) since last sync point © 2013 IBM Corporation
  • 17. Checkpoint Programming Call 'CBLTDLI' count, function, pcb-name, ioalen, CALL langTDLI PLITDLI ioarea, arealen, area,... PARAMETER COUNT FUNCTION CODE ('CHKP') FIRST PCB ADDR (I/O-PCB) I/O AREA LENGTH (NOT USED) 30 BYTE I/O AREA (CONTAINS CHECKPOINT ID) CHECKPOINT AREA LENGTH CHECKPOINT AREA  The "checkpoint area length, checkpoint area" pair may be specified up to 7 times. © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 18. Restart Programming Call 'CBLTDLI' count, function, pcb-name, ioalen, ioarea, arealen, area,... PLITDLI CALL langTDLI PARAMETER COUNT FUNCTION CODE ('XRST') FIRST PCB (I/O-PCB) I/O AREA LENGTH (NOT USED) 30 BYTE I/O AREA CHECKPOINT AREA LENGTH CHECKPOINT AREA  The "checkpoint area length, checkpoint area" pair may be specified up to 7 times, and must (of course) be the same as specified on the CHKP call. © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 19. Exception Handling  ROLL Calls – The Roll Back (ROLB) call will dynamically back out database changes and return control to your program – The Roll (ROLL) call will abnormally terminate your program and dynamically back out database changes. – The Roll Back to SETS (ROLS) call will back out to a processing point set by a prior SETS or SETU call.  SET Calls – The Set a Backout Point (SETS) call will set an intermediate backout point or cancel all existing backout points. – The SET Unconditional (SETU) call operates like the SETS call, except that the SETU call is accepted even if unsupported PCBs exist or an external subsystem is used. © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 20. ROLL (Rollback and ABEND) Call 'ROLL' Call 'CBLTDLI' CALL langTDLI count, function, pcb-name PLITDLI PARAMETER COUNT FUNCTION CODE ('ROLL') I/O PCB  Requests that the current application be ABENDed: – Current application's database updates are backed-out – U0778 ABEND issued  The ROLL call is not a substitute for good programming. © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 21. ROLB (Rollback, no ABEND) Call 'ROLB' Call 'CBLTDLI' CALL langTDLI count, function, pcb-name PLITDLI PARAMETER COUNT FUNCTION CODE ('ROLB') I/O PCB  'ROLB' like the 'ROLL' call, except the application retains control  Current application's database updates are backed-out – Log dataset must be on DASD – BKO=Y parameter must be coded in JCL PARM  Control returns to the application program © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 22. SETS Call 'SETS' Processing: 'CBLTDLI' CALL langTDLI count, function, pcb-name, ioarea, token PLITDLI  Establishes (sets) up to 9 intermediate backout points  I/O-AREA (before call): LL ZZ DATA TO BE RETURNED  Data to be returned on corresponding 'ROLS‘ XXXX  TOKEN (after call): 4-byte identifier to be used on later 'ROLS‘ – Used by later ROLS call to control "partial" rollback © Copyright IBM Corporation 2008 © 2013 IBM Corporation
  • 23. ROLS Call 'ROLS' Call IO-PCB 'CBLTDLI' CALL langTDLI count, function, or pcb-n , ioarea, token PLITDLI DBPCB  Backs out database changes to 'SETS' "token" – Program retains control LL ZZ RETURNED DATA  I/O-AREA (after call): – Same format as area supplied on the SETS call  TOKEN (before call): XXXX – 4-byte identifier set on prior 'SETS' call  Requires prior SETS call to establish token  If DB-PCB is supplied, it causes U3303 ABEND – Is an appropriate response to 'BA' status code after INIT © 2013 IBM Corporation
  • 24. Generalized Sequential Access Method  Non-Hierarchic USING  Sequential Database VSAM(ESDS) BSAM • Fixed • Variable • Undefined OR RECFM • Fixed • Variable © 2013 IBM Corporation
  • 25. GSAM Programming Calls 'CBLTDLI' PLITDLI CALL langTDLI parm-count, call-func, pcb-name, ioarea, rsa-addr PARAMETER COUNT ADDRESS FUNCTION CODE ADDRESS (GU,GN,ISRT,OPEN,CLSE) GSAM PCB ADDRESS I/O AREA ADDRESS RECORD SEARCH ARGUMENT ADDRESS RSA FORMAT: APPLICATION PGM REQUIRED MOVE RSA CALL 'GU' RSA ADDR OPTIONAL RSA ADDR GSAM 2 FULL WORDS: • BSAM TAPE - RELATIVE BLOCK ADDRESS • BSAM DASD - ACTUAL TRACK ADDRESS • VSAM - RELATIVE BYTE ADDRESS CALL 'GN' or 'ISRT' © 2013 IBM Corporation
  • 26. Summary  BMP Overview  Typical daily processing workload  Recent IMS function to help BMPs  Converting a batch program to a BMP  BMP Programming facilities © 2013 IBM Corporation