• Save
LE z/VSE CEETRACE - New features, Tips And Tricks
Upcoming SlideShare
Loading in...5
×
 

LE z/VSE CEETRACE - New features, Tips And Tricks

on

  • 1,024 views

Do you miss the “good 'ol days“ of the DOS/VS COBOL “ready-trace” functionality? Would you like to be able to recover your COBOL/VSE source code from an available load-module? Would you like ...

Do you miss the “good 'ol days“ of the DOS/VS COBOL “ready-trace” functionality? Would you like to be able to recover your COBOL/VSE source code from an available load-module? Would you like to be able to archive and back-up your COBOL/VSE source code with your load-modules? Do heap storage corruption abends keep your application programmers busy resulting in delays to development of new innovative
application functionality?

Statistics

Views

Total Views
1,024
Views on SlideShare
1,024
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

LE z/VSE CEETRACE - New features, Tips And Tricks LE z/VSE CEETRACE - New features, Tips And Tricks Document Transcript

  • IBM Language Environment for z/VSE CEETRACE New Application Debugging Features, Tips and Tricks _______________________________________________________________________________________ CEETRACE Tips and Tricks - Technical Document 1.0 Page 1 of 18
  • Preface Do you miss the “good 'ol days“ of the DOS/VS COBOL “ready-trace” functionality? Would you like to be able to recover your COBOL/VSE source code from an available load-module? Would you like to be able to archive and back-up your COBOL/VSE source code with your load-modules? Do heap storage corruption abends keep your application programmers busy resulting in delays to development of new innovative application functionality? Then CEETRACE is the tool for you! CEETRACE will help reduce your application debugging time, freeing-up your programmers time, provide a “secondary” source-code repository back-up option and help identify quickly and efficiently application code responsible for heap storage or other environment corruption abends. Table of Contents Preface ................................................................................................................................................................2 Introduction.....................................................................................................................................................3 Implementation...............................................................................................................................................4 Covered Subjects........................................................................................................................................4 Download and Documentation Information .............................................................................................4 Review new CEETRACE attention routine commands.............................................................................5 1. The “auto_rprt_epn” Command .......................................................................................................5 2. The “auto_rprt” Command................................................................................................................6 3. High Level Language Statement Exit Option...................................................................................7 New CEETRACE options and Analysis Data............................................................................................8 1. The new “mini_dump” option...........................................................................................................8 2. Condition Information Block............................................................................................................8 3. Machine State Information Block.....................................................................................................9 4. mini_dump Option Control...............................................................................................................9 Environment Validation Enhancements...................................................................................................10 1. Environment Validation Activation.................................................................................................10 2. Sample Heap storage corruption CEETRACE statement execution history report........................11 3. Environment Validation Performance Considerations.....................................................................11 COBOL/VSE Source-code Archiving and Recovery Capabilities..........................................................12 1. Pre-requisite PTFs...........................................................................................................................12 2. Compile processes/JCL modifications for SEPARATE option.......................................................12 3. COBOL/VSE Source Code Archive and Recovery Concept..........................................................12 Planning for CEETRACE .......................................................................................................................14 Resource Requirements.......................................................................................................................14 CEETRACE resource settings and configuration...............................................................................15 1. Recommendations for Production-based Environments.............................................................15 2. Recommendations for Development/Testing-based Environments............................................16 Conclusion................................................................................................................................................17 Trademarks ...................................................................................................................................................17 Comments and Questions.........................................................................................................................17 Author..................................................................................................................................................17 About the Author......................................................................................................................................18 CEETRACE Tips and Tricks - Technical Document 1.0 Page 2 of 18
  • Introduction Initially available with z/VSE 4.2 is the new download-able CEETRACE Feature for LE z/VSE that provides innovative new tools. These tools are designed to improve application development and problem diagnosis for all supported Language Environment for z/VSE (LE z/VSE) enabled high level languages (HLL). Provided in APAR PQ74143 and the supporting language APAR (English : PQ74144) for the COBOL/VSE compiler is the ability to store as a separate VSE librarian member the COBOL module symbol table that is normally generated when the TEST(xxx,SYM) compiler option is used. This APAR(s) PTF provides a new sub-option for the TEST compiler option – SEPARATE – which can be abbreviated to SEP. When this sub-option is used the compiler will store the complete symbol table in the named librarian member which is referred to as the “SYSDEBUG” file. Along with the symbol table will also be stored a heavily compressed copy of the COBOL/VSE programs source code. The main advantages of using the SEPARATE sub-option are : • Separation of the symbol table from the load module into a librarian member. • Reduced load-module size due to the separation of the symbol table. • Saving of source code used to produce the symbol table and compiled object module. The benefits associated with reducing an applications load module size are obvious and especially beneficial in the CICS environment when that application may need to be resident below the 16MB line. This document will concentrate on new application problem analysis improvements provided with CEETRACE V1.2.0 (supported by z/VSE 5.1). Specifically with regards to the use of new CEETRACE Attention Routine (AR) tracing commands available to all LE z/VSE HLLs when enabled to use CEETRACE. Some of the information provided will also be applicable to CEETRACE V1.1.2 (z/VSE 4.3) users. Next this document will review recently available CEETRACE options that have been designed to assist application development debugging by producing further detailed application failure information in addition to the LE z/VSE dump details. With the difficultly associated analysing HEAP storage abends using just the information provided by the LE z/VSE dump output and the high possibility that the corruption responsible for the abend occurred long before the abend and dump was taken, the CEETRACE environment validation function has been enhanced to help identify the culprit program-name and statement number. We will look at how to use this enhancement and what information is provided by CEETRACE to assist with identifying the application and code responsible. We will also review just one potential new concept now available with regards to utilising the COBOL/VSE source code storage capability available via the SYSDEBUG file and the associated utility supplied with CEETRACE to re- construct this source code. Then in conclusion a summary of recommended CEETRACE and LE z/VSE option settings that are designed to optimise the use of CEETRACE in certain system configurations will be discussed. CEETRACE Tips and Tricks - Technical Document 1.0 Page 3 of 18
  • Implementation Covered Subjects • Review new CEETRACE attention routine commands. ◦ Statement execution history “Auto-Report” Feature ◦ New CEETRACE High Level Language (HLL) User exit. • New CEETRACE options. ◦ The new “mini_dump” option and output. • Fix-packs V1.1.2b and V1.2.0a Environment Validation Enhancements ◦ An over-view of the changes introduced with the fix-packs to improve HEAP storage overlay abends analysis. • COBOL/VSE Source-code Archiving and Recovery Capabilities. ◦ Pre-requisite (eg COBOL/VSE compiler) PTFs. ◦ Compile processes/JCL modifications for SEPARATE option. ◦ COBOL/VSE Source Code Archive and Recovery Concept • Planning for CEETRACE ◦ Resource requirements. ◦ CEETRACE resource settings and configuration. Download and Documentation Information • Download the latest CEETRACE documentation and available fixpacks from : ◦ http://www-03.ibm.com/systems/z/os/zvse/downloads/tools.html#ceetrace CEETRACE Tips and Tricks - Technical Document 1.0 Page 4 of 18
  • Review new CEETRACE attention routine commands With CEETRACE version 1.2.0 (z/VSE 5.1) and above there are a number of new options and operator attention routine commands available which will allow the user to set specific trace activation and termination points within CEETRACE-enabled HLL applications. See the CEETRACE Feature Installation and User Guide documentation chapter “Preparing an application for use with the CEETRACE feature” for details on how to enable an application for use with the CEETRACE feature. The first enhancement we will look at is the Auto-Report feature which has options to allow the specification of a entry-point name and statement number at which time a CEETRACE statement execution history report will automatically be produced and sent to the defined output destination. The auto-report statement selection option allows either a specific statement number within the specified entry-point name to be the report activation point or for the report to be produced after a set number of program statements (between 1 and 65535) have been executed within the CEETRACE-enabled application. The Auto-Report feature attention routine commands consist of two separate commands. 1. The “auto_rprt_epn” Command When using an entry-point name as a CEETRACE monitoring selection criteria then this AR command must be specified first. For example, with a PL/1 VSE subroutine entry point that has the name “plisub1”, you would issue the following operator console AR command : s cee,ceetrace=(auto_rprt_epn=plisub1) This command establishes that any automatic execution trace reports produced are to be limited to execution within the “plisub1” sub-routine. The load-module name is not considered so if your application consists of subroutines that contain the same name within different load-modules then these other subroutines will also be considered for automatic execution trace report production. The command will accept an entry-point name up to 12 characters long. For COBOL applications the majority will have the entry-point name the same as the program-name. The possible exception is if the COBOL “ENTRY” verb has been specified. Check your COBOL applications link-edit map to verify if an alternate entry-point name has been used. Special Requirements - • If you wish to have an automatic report produced at a specific statement number then the “auto_rprt_epn” command must be set to the appropriate entry point name that contains the statement number that when executed is to initiate an automatic execution statement history report. • If you simply want an automatic execution report to be produced after a certain number of statements within the entire application have been executed, then you must set the “auto_rprt_epn” option to “off”. CEETRACE Tips and Tricks - Technical Document 1.0 Page 5 of 18
  • 2. The “auto_rprt” Command This command/option actually consists of two separate concepts. Which one will be applicable to your particular situation will depend firstly upon what you set the “auto_rprt_epn” option to as discussed earlier. The auto_rprt option defines the monitoring of either a complete CEETRACE-enabled application execution as a total number or the execution of a specific statement number within an already defined entry-point name. So if you have set auto_rprt_epn to “off” your only option is to set the auto_rprt option to a statement number execution count at which point, when this count has been reached, an execution statement history report will be produced and sent to the defined destination. For example. If you want to have an execution statement report produced after a total of 25,000 language statements have been executed within any CEETRACE-enabled routines within the application, then you would issue the following commands on the console : s cee,ceetrace=(auto_rprt_epn=off) s cee,ceetrace=(auto_rprt=r25000) When the next CEETRACE-enabled application is executed a statement execution history report will be automatically produced after 25,000 statements have been executed. Notes : 1. The maximum number you can specify for a statement number count is 65,535. 2. CEETRACE-enabled applications must be executed either within an “included” partition or not within an “excluded” partition. See the CEETRACE.INI file for details on partition inclusion or exclusion. Alternatively, if your situation has meant you were able to specify an entry-point name for auto_rprt_epn, then for the auto_rprt option you need to simply define the statement number within this entry-point name that is to be monitored. Then when this statement number is executed a current execution statement history report will automatically be produced and sent to your defined destination. For example. If you wanted a statement execution history report produced if/when statement number 435 is executed within PL/1 subroutine “plisub1” then in conjunction with the earlier auto_rprt_epn option setting (see point 1 on page 5) you would subsequently issue the following console command : s cee,ceetrace=(auto_rprt=s435) Then whenever statement number 435 within entry point name “plisub1” is executed, a statement execution history report will be produced and sent to your defined destination. CEETRACE Tips and Tricks - Technical Document 1.0 Page 6 of 18
  • 3. High Level Language Statement Exit Option CEETRACE now includes a user exit point to allow an external user-exit program to get control at each statement executed by any CEETRACE-enabled high-level language (HLL) application. At this exit point CEETRACE will provide specific information on the executing application that can be used by the exit for its own processing. Not only can the name of this exit module can be defined in the default CEETRACE.INI file, this exit module name can be dynamically changed by the operator using a CEETRACE AR command. For example : If you had defined the default HLL exit to your CEETRACE.INI file and then decided that you want to try out your own version called EMPHLLX.PHASE you would enter the following operator attention routine command to activate your exit routine : s cee,ceetrace=(exit_mod=emphllx) When the next CEETRACE-enabled application is executed in a CEETRACE-enabled (or not excluded) partition CEETRACE will attempt to load the exit module EMPHLLX.PHASE. If this load is not successful then the HLL exit point will be disabled for that application. If the load is successful then the exit routine will be called with the documented parameters ( CELHLLDS.A ) to allow site-specific relevant processing to be performed. An example HLL exit is provided in your LE z/VSE installation library as member CELHLLXT.Z and a copy of the Application Programming Interface (API) DSECT parameters in member CELHLLDS.A. The sample shows a simple program that counts the number of statements executed by any CEETRACE- enabled application and when it has reached 15 executed statements a console message is issued including the name of the application and the number of statements executed. The exit then disables itself so as to not impact on any application performance unnecessarily. This sample should be used as a basis on how to develop and implement any site specific processing that may include such things as the number of times a certain statement (perhaps to an external database API) is executed, a specific program is called/executed, its language and even its elapsed CPU time. The exit can also be used to extend the capabilities of CEETRACE by allowing the user exit to gain control at each HLL statement executed and then perform other processing as required before continuing with application execution. The CEETRACE HLL statement exit can only be developed in LE-enabled Assembly language. Other HLLs are not supported as a user exit routine. See the “CEETRACE HLL Statement Exit ” section in the CEETRACE Installation and Users Guide documentation for details on the parameters provided and more with available CEETRACE HLL exit. CEETRACE Tips and Tricks - Technical Document 1.0 Page 7 of 18
  • New CEETRACE options and Analysis Data 1. The new “mini_dump” option With CEETRACE version 1.2.0 (z/VSE 5.1) and above is the new “mini_dump” CEETRACE option. The purpose of this option is to provide some further abend analysis information on the details of a failure as part of a program execution statement history report when produced due to an application failure. The “mini_dump” information should be thought of as complimenting the available LE z/VSE Trace-Back report and associated dump information. This, along with the program execution statement history, is designed to accelerate any application failure diagnosis process so that a correction can be made quickly to minimise any delays due to application failure. It should be noted that the LSTQ destination support provided by both LE z/VSE and CEETRACE is not supported by the mini_dump option. If “mini_dump” is activated when the LSTQ destination is set the “mini_dump” output will be suppressed. The mini_dump option, when activated, is not limited only to HLL applications. If an LE-enabled assembler program experiences a failure then the mini_dump output will still be produced. 2. Condition Information Block The mini_dump will appear after the LE z/VSE condition information in the CEETRACE execution history report unless the LSTQ destination is used. In which case the mini_dump output will be suppressed. The first part of the mini_dump output will contain a detailed expansion of the LE z/VSE Condition Information Block (CIB). In the following example a COBOL/VSE program has forced a data-exception (OC7 or CEE3207S) and the following is displayed for the mini_dump report: CEE3207S The system detected a data exception. CEETRACE requested MINI_DUMP begins. CIB for : 004FD4E0 +000000 CIB_Eye.. CIB CIB_Back. 00000000 CIB_Frwd. 00000000 CIB_Size. 010C CIB_Ver.. 0008 +000010 CIB_Plat. 00000005 Reserved. 00000000 CIB_Cond. 00030C87 59C3C5C5 00000000 CIB_Mach. 004FD5EC +000028 CIB_OLDc. 00030C87 59C3C5C5 00000000 CIB_Flg1. 00 CIB_Flg2. 00 CIB_Flg3. 00 +000037 CIB_Flg4. 00 CIB_HDsf. 00000000 CIB_HDen. 804A6A00 CIB_HDrs. 00000000 CIB_RMsf. 00522100 +000048 CIB_RMpt. 004209E0 CIB_RSmh. 00444948 Reserved. 00000005 00000000 00000000 00000000 00000000 +000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 +000088 00000000 00000000 CIB_Vsr.. 00000000 00000000 CIB_Vsto. 00000000 +00009C CIB_Vpsa. 00000000 CIB_Mcb.. 00000000 CIB_Mrn.. 00000000 00000000 CIB_Mflg. 00 +0000AD Reserved. 000000 CIB_flg5. 40 CIB_flg6. 27 CIB_flg7. 40 CIB_flg8. 00 +0000B4 CIB_ABcd. 00000020 CIB_ABrc. 00000007 CIB_ABnm. CIB_Pl... 0042008C CIB_SV2.. 00522100 +0000CC CIB_SV1.. 00522100 CIB_Int.. 004209DA CIB_Qdat. 00000000 CIB_FdBk. 00000000 CIB_Fun.. 00000001 +0000E0 CIB_Toke. 00522100 CIB_Mid.. 00000005 CIB_Stat. 0000000A CIB_Rtcc. 00000014 CIB_Ppav. 00000001 +0000F4 CIB_ABte. CEL4RPRT CIB_Sdwa. 00520028 As can be seen, the data-exception is reported (CEE3207S) and then the “Condition Information Block” (CIB) details are displayed. This provides us with information such as the active Dynamic Save-Area (DSA) at the time of the failure (CIB_SV1), the abend code (CIB_ABcd) and reason code (CIB_ABrc) that confirms our program check (X'20') and Data-exception (X'07'). We are also given the interrupt address (CIB_int) of where the Data- exception occurred. We can also see that the CEETRACE program execution statement history reporter (CEL4RPRT) was active as the “Abnormal Termination Exit” (CIB_ABte) as would be expected. CEETRACE Tips and Tricks - Technical Document 1.0 Page 8 of 18
  • Further detailed information on debugging an application using the LE z/VSE Condition Information Block (CIB) can be found in the LE z/VSE Debugging and Run-Time Messages Guide and in the supplied macro CEECIB.A in the LE z/VSE Installation Library. 3. Machine State Information Block Following the Condition Information Block (CIB) information will be a dump of the Machine State information extracted from LE z/VSE as it trapped the application failure. Machine State: 004FD5F4 +000000 MCH_reg0. B76317D0 MCH_reg1. 004490D4 MCH_reg2. 004C77EC MCH_reg3. 004209A6 MCH_reg4. 004200B0 +000014 MCH_reg5. 00520268 MCH_reg6. 0052641C MCH_reg7. 00000000 MCH_reg8. 00000000 MCH_reg9. 00526858 +000028 MCH_regA. 0042018C MCH_regB. 00420838 MCH_regC. 0042016C MCH_regD. 00522100 MCH_regE. 804209DA +00003C MCH_regF. 00000000 MCH_psw.. 07DD3000 804209E0 MCH_ilc.. 0006 MCH_intc. 0007 +00004C MCH_Rsvd. 00000000 MCH_Fltp. 42310000 00000000 4BC5704C 46EAD000 40404040 40404040 40404040 +00006C 40404040 MCH_Rsvd. 00000000 00000000 00000000 00000000 00000000 00000000 00000000 +00008C 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 +0000B0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 MCH_Bear. 004B8DDA CEETRACE requested MINI_DUMP complete. This report gives us all the general purpose register (MCH_reg0 – MCH_regf) contents as at the time of the failure along with the PSW interrupt address (MCH_psw) and interrupt information such as the instruction length (MCH_ilc) and interrupt code (MCH_intc). The last saved breaking- event address register information (MCH_Bear) is also included. The MCH_Bear contents will be most useful in situations where an OC1 (Operation exception) has occurred due to a wild branch as this field reports the last successfully executed branch source address. More information on how to debug an application using the machine information block can be found in the LE z/VSE Debugging and Run-Time Messages guide in the “Using the Machine State Information Block” section. 4. mini_dump Option Control The new “mini_dump” option can be specified in the CEETRACE default initialization file (CEETRACE.INI) and also using the CEETRACE attention routine command. For example, if you wanted to activate the mini_dump function temporarily for all your CEETRACE-enabled partitions and applications, you would enter the following AR command on the operators console : s cee,ceetrace=(mini_dump=yes) Then, when you no-longer want the mini_dump information produced you would enter the following AR command again on the operators console to de-activate the option : s cee,ceetrace=(mini_dump=no) CEETRACE Tips and Tricks - Technical Document 1.0 Page 9 of 18
  • Environment Validation Enhancements Provided with fix-pack V1.1.2b (z/VSE 4.3) and V1.2.0a (z/VSE 5.1) are enhancements to assist with Language Environment corruption abends. Such as when a HEAP overlay abend (CEE0802C) occurs. Attempting to identify the culprit from the produced Language Environment abend information, even when using the HEAPCHK run-time option, can be difficult at best. Especially when trying to identify what statement in the application is actually responsible for the overlay when that corruption could have occurred long before the abend was triggered. This is complicated even further by LE z/VSE only checking HEAP storage areas, such as the free storage chain, when an actual HEAP storage request is made. Meaning if a lot of processing occurs between HEAP storage acquisitions the corruption could have been performed long before it was detected. Which usually results in the dump produced by the abend being taken too late. Meaning even comprehensive analysis of the dump will not provide the desired information for identifying the culprit. With the earlier mentioned fix-packs installed, CEETRACE is now able to assist with the identification of the program-name and statement number that was responsible for the environment corruption. CEETRACE provides environment validation at run-time for core Language Environment control blocks (MIN), HEAP storage corruption (HEAP) and the combination of both validation checks simultaneously (FULL). 1. Environment Validation Activation Activation of this capability is as simple as either setting the option in your CEETRACE.INI file and then reloading or by issuing an operator console command to activate the option with your desired setting. For example : S CEE,CEETRACE=(env_validation=heap) Will activate the heap storage validation processing for any subsequently executed CEETRACE-enabled applications executed in an “included” partition or not executed in an “excluded” partition. If CEETRACE detects corruption of the defined environment three actions will be performed. 1. Message CELT060E will be issued on the operator console. 2. The current environment validation request will be deactivated. 3. A statement execution history report will be sent to the defined destination indicating the program- name and statement number that was responsible for the detected corruption. Eg X1 0049 CELT060E Environment checking trapped corruption. X1 0049 CELR021W Language Environment for z/VSE CEETRACE report complete. CEETRACE Tips and Tricks - Technical Document 1.0 Page 10 of 18
  • 2. Sample Heap storage corruption CEETRACE statement execution history report. As can be seen from the sample output above, the last entry in the statement execution history report indicates that the previous trace entry identifies the responsible program-name and statement number that was last executed successfully before the specified environment validation processing failed. To de-activate the environment validation checking you can again either modify your CEETRACE.INI file and then perform a “reload” or issue the operator console command : S CEE,CEETRACE=(env_validation=off) When the HEAP environment validation is activated (or the FULL option is used) CEETRACE will also turn on automatically the LE z/VSE HEAPCHK run-time option processing. This is to enable as much information as possible to be produced when the corruption is detected. As the application may continue processing even after CEETRACE has reported on the corruption culprit there may be another CEETRACE statement execution history report produced. This can be in response to the abend that may be issued when LE z/VSE detects the same corruption. As CEETRACE will trap this abend a statement execution history report will therefore be produced. This is to be expected and is not an error. The COBOL program source code used in the above example can be found in the CEETRACE V1.2.0a fix- pack zip file. 3. Environment Validation Performance Considerations Use of the environment validation option should be restricted as much as possible to development and testing environments. There will be a noticeable CPU over-head associated with activating this option which should be considered when being used in any CPU constrained Production environment. CEETRACE Tips and Tricks - Technical Document 1.0 Page 11 of 18
  • COBOL/VSE Source-code Archiving and Recovery Capabilities 1. Pre-requisite PTFs With the “SYSDEBUG” capability added to the COBOL/VSE compiler via APAR PQ74143 and CEETRACE able to utilise this new feature through the LE/COBOL run-time support, there now exists new opportunities for performing source-code archiving and recovery within z/VSE. 2. Compile processes/JCL modifications for SEPARATE option What APAR PQ74143 does is allow the COBOL/VSE compiler (when the correct compile and JCL PARM card options are set) to create a separate z/VSE librarian member (file-type SYSDEBUG) which will contain not only the COBOL/VSE programs symbolic table information (used by the LE z/VSE dump formatting routines) but also a heavily compressed copy of the COBOL/VSE source-code along with compile-time option flag indicators. CEETRACE uses this compressed copy of the COBOL/VSE programs source-code to find related source- code statement information based upon the executed statement number when the appropriate CEETRACE options are set. This allows an improved “first-glance” statement history report review for any programmer when an application failure occurs. The application programmer might consider changing any COBOL/VSE compile processes or JCL skeletons to set the “SD” JCL PARM option and also the COBOL/VSE compiler TEST option to use at least STMT with both SYM and SEP options for your application development and any pre-Production testing environments. These options can be used in a “Production” environment also but for production environments that are highly CPU constrained it is recommended that CEETRACE only be activated (using the available AR commands) when attempting to diagnose production application issues. This is because there will be a minimum increase of at least approximately 10% CPU utilisation over-head for every CEETRACE-enabled application executing in a CEETRACE-included (or not excluded) partition along with the documented extra storage requirements to support CEETRACE processing. 3. COBOL/VSE Source Code Archive and Recovery Concept A potentially new source-code management benefit of using the SYSDEBUG COBOL/VSE compiler feature is that your applications source-code is now archived as a compressed z/VSE librarian member which can then be included in any normal rotational system backup processes. Thereby archiving source-code along with any regular system backups. Providing the source-code recovery ability is the CEETRACE feature utility that can uncompress and extract the source-code from any “SYSDEBUG” librarian member where there is a matching COBOL/VSE load- module (PHASE). Also included in this source code recovery feature is a report of the compile-time options used. CEETRACE Tips and Tricks - Technical Document 1.0 Page 12 of 18
  • Currently only COBOL/VSE programs that have an associated load-module are supported by the extraction utility. COBOL/VSE programs that exist only as object code which are statically linked into other load- modules are not directly supported. However, so long as a matching SYSDEBUG member exists, the object code can be link-edited into a load-module and then the extraction utility will be able to extract the compressed source-code. This utility now becomes an option as a source-code recovery tool should the “master” copy of a COBOL/VSE program source code become lost or unusable. The utility will extract the compressed source- code and then punch out to a VSE/POWER spooled punch device the complete source-code used to produce the matching object deck. The compile-time options used to compile the source-code will be sent to the extraction job output member used to execute the CEETRACE utility. All copy statements will be included and all copy-books used expanded. From z/VSE 4.3 (CEETRACE V1.1.2) onwards the CEETRACE utility can be controlled to comment out the COBOL “COPY” statements produced from the SYSDEBUG member if desired. This will allow easier re- compilation of the source-code sent to the punch device. To activate this function the JCL used to execute the utility needs to specify // UPSI 00000100 . For more information on this utility including sample JCL and required parameters see the CEETRACE Installation and Users Guide, section titled “COBOL/VSE Source Code Extraction Utility ”. CEETRACE Tips and Tricks - Technical Document 1.0 Page 13 of 18
  • Planning for CEETRACE Resource Requirements Before implementing CEETRACE, especially in a Production Environment, the resources required to use CEETRACE effectively should be reviewed so as to minimise any negative impacts on production application performance. How you implement CEETRACE will depend upon how your production, development and/or test systems are configured. With the large scalability of z/VSE systems these days it would not be unusual to use a single z/VSE system for both production and “QA” (quality assurance) testing. And then have another z/VSE system devoted to performing development and testing functions. CEETRACE was designed with the intention of having as small as possible impact on an applications processing so as to allow the easy debugging of a production problem even in a production environment - within the system's available capacity. Much of the work CEETRACE does is performed only after an application failure has actually occurred thereby ensuring that resource intensive processing is only performed after the application has failed. For example – the loading and use of the LE z/VSE SYSDEBUG file is only performed when an actual application failure occurs along with the associated trace table output formatting and report production. It should be remembered that ultimately you can control what applications CEETRACE will trace and which will be exempted simply by compiling the applications with TEST (to use CEETRACE) or NOTEST (exempted from CEETRACE). CEETRACE configuration options are only used if the application has been compiled with the required TEST compile-time option. Then there is the partition inclusion/exclusion options. Running an application in an excluded partition will eliminate any possible CEETRACE overhead and also failure reporting. Just as executing an application in a partition that is not specified in the inclusion list will also eliminate CEETRACE from the application. Finally, only applications that actually use the LE z/VSE run-time, will be able to utilise CEETRACE. So, for example, specifying partitions that are used for interactive DITTO/VSE sessions in the exclusion list is not required as DITTO/VSE does not use the LE z/VSE run-time. It should be noted also that for COBOL/VSE applications which use the SYSDEBUG compile-time option the SYSDEBUG file will be loaded into ANYHEAP storage but only if/when required. Thus there is not really any point in making an environment-wide change to accommodate this behaviour. There is no reliable way of knowing when this storage will be required or how much will be needed. Also the need for this ANYHEAP storage will be completely dependant upon application failures and not successful executions. But you should be aware of this behaviour when reviewing any LE z/VSE storage usage reports (eg RPTSTG(ON) reports) which involve the use of COBOL/VSE applications that have an associated SYSDEBUG file. CEETRACE Tips and Tricks - Technical Document 1.0 Page 14 of 18
  • CEETRACE resource settings and configuration. 1. Recommendations for Production-based Environments • CICS system requirements : ◦ For use of CEETRACE at “critical” times in the on-line CICS systems, ensure you have at least 64K of EUDSA storage available for the CEETRACE support run-time load modules. ◦ Ensure you are using the RUWAPOOL SIT option in all of the production CICS systems. This is recommended even without CEETRACE being used. ◦ For CPU constrained production systems, consider starting CEETRACE with ceetrace=off and then using the attention routine commands to activate CEETRACE when an application problem occurs. • Where CEETRACE is active by default - LE z/VSE CICS requirements : ◦ To support the CEETRACE tracing table and assorted control blocks, increase your CICS ANYHEAP run-time option initial storage value by a minimum of 2040 bytes. ◦ For example, assuming the supplied default CICS run-time options where in use : ▪ ANYHEAP=((6120,4080,ANYWHERE,FREE),OVR). ◦ If you have a sizeable number of AMODE24 applications that will also be used with CEETRACE then also increase your BELOWHEAP CICS run-time options to something similar to the following: ▪ BELOWHEAP=((6120,4080,FREE),OVR), • If the majority of CICS applications running will not be using CEETRACE (not compiled appropriately) or if only irregular (ie on-demand via AR command) use of CEETRACE will be performed then no adjustment of the ANYHEAP/BELOWHEAP run-time options should be required for such intermittent CEETRACE use. • Where CEETRACE is active by default - LE z/VSE BATCH requirements: ◦ Generally the default BATCH ANYHEAP (or BELOWHEAP) values will be sufficient for normal CEETRACE execution. ◦ If you have specific options tailored for special applications (eg CEEUOPT or JCL PARM overrides) these should be reviewed and any BELOWHEAP (for AMODE24 applications) or ANYHEAP values adjusted accordingly. • CEETRACE options : ◦ For an environment that is heavily utilised with little to no available capacity, it is recommended that CEETRACE initially be deactivated (ceetrace=off in the CEETRACE.INI file). Then when a problem is encountered where CEETRACE can be used, the AR (attention routine) activation command be issued and the production application re-executed. This applies equally to both the batch and CICS environments. ◦ For an environment with at least ~25% or more CPU capacity available only the smallest CEETRACE involvement should be activated. Eg ▪ env_validation=off (unless targeting a particular application corruption problem) ▪ timer=off ▪ warn_msgs=off (unless you are having problems with CEETRACE itself) ◦ In the batch environment, when response-time critical applications are being executed and CEETRACE is active, either execute these in an excluded partition or not in a partition that has been included. If elapsed times or CPU consumption remains high issue the “s cee,ceetrace=off” AR command to temporarily deactivate CEETRACE. It should be noted that this will deactivate CEETRACE system-wide. CEETRACE can then be re-activated again (using s cee,ceetrace=on) CEETRACE Tips and Tricks - Technical Document 1.0 Page 15 of 18
  • when an application problem arises that CEETRACE can be used with or when more system resources become available. 2. Recommendations for Development/Testing-based Environments • CICS system requirements : ◦ Ensure you have at least 64K of EUDSA storage available for the CEETRACE support run-time load modules. ◦ Ensure you are using the RUWAPOOL SIT option in all of the CICS systems. This is a recommended setting even without CEETRACE being used. • LE z/VSE CICS requirements : ◦ To support the CEETRACE tracing table and assorted control blocks, increase your CICS ANYHEAP run-time option initial storage value by a minimum of 2040 bytes. ◦ For example, assuming the supplied default CICS run-time options where in use : ▪ ANYHEAP=((6120,4080,ANYWHERE,FREE),OVR). ◦ If you have a sizeable number of AMODE24 applications that will also be used with CEETRACE then also increase your BELOWHEAP CICS run-time options to something similar to the following: ▪ BELOWHEAP=((6120,4080,FREE),OVR), ◦ If the majority of applications running will not be using CEETRACE then no adjustment of the ANYHEAP/BELOWHEAP run-time options should be performed for CEETRACE use. • BATCH requirements : ◦ Generally the default BATCH ANYHEAP (or BELOWHEAP) values will be sufficient for normal CEETRACE execution. ◦ If you have specific options tailored for special applications (eg CEEUOPT or JCL PARM overrides) these should be reviewed and any BELOWHEAP (for AMODE24) or ANYHEAP values adjusted accordingly. • CEETRACE options : ◦ Use of CEETRACE environment checking is recommended. Eg ▪ env_validation=min (“heap” or “full” would be preferred) ▪ timer=on ▪ warn_msgs=on ( until use of CEETRACE is comfortable ) ◦ When performing “production-like” testing within this environment execute these applications in an excluded partition, not in an included partition or compile with the NOTEST option before testing. All of the above recommendations should be only be considered a guide as you may need to further refine or adjust the values or options mentioned to accommodate your application or systems specific behaviour. There is also the possibility that certain applications (most commonly COBOL/VSE) will invoke specific language verbs, run-time functions or experience highly repetitive iterative loops that will result in the normally expected overhead from CEETRACE being greatly exaggerated. Should this occur it is recommended that these applications be run either in a partition that is not specified in the inclusion list or in a partition that is specified in the exclusion list. CEETRACE Tips and Tricks - Technical Document 1.0 Page 16 of 18
  • Conclusion New features available with CEETRACE provide a more flexible and improved application development and debugging experience for all LE z/VSE supported high level languages. The COBOL/VSE compiler SYSDEBUG support along with LE z/VSE COBOL run-time support for the SYSDEBUG file and the ability of CEETRACE to exploit the compressed source-code stored within this member can now provide COBOL/VSE with a new and improved method for the archiving and recovery of source code. With the enhanced environment corruption functionality available with CEETRACE application abends due to heap or LE z/VSE environment corruption can now be quickly and easily identified. Trademarks The following terms are trademarks of International Business Machines Corporation in the United States, or other countries, or both: CICS, IBM, Language Environment, VSE/ESA, z/VSE Other company, product, or service names, may be the trademarks or service marks of others. © Copyright International Business Machines Corporation 2013. All rights reserved. Comments and Questions All comments or questions on this documentation are welcome. Please send your comments to: vsesupportLE@de.ibm.com Or  Post a question on the LE z/VSE IBM DeveloperWorks Blog. LEzVSE Blog : https://www.ibm.com/developerworks/community/blogs/lezvse/?lang=en Author Mr Garry Hasler Language Environment for z/VSE Development and Service IBM Australia Development Laboratory (ADL), Perth, Western Australia, Australia LinkedIn : http://au.linkedin.com/pub/garry­hasler/68/a9b/557/ Revision Date : 9 July 2013 Time: 08:43:18 AM CEETRACE Tips and Tricks - Technical Document 1.0 Page 17 of 18
  • About the Author Garry has been working with VSE since the days of VSE/SP 3.x. Initially as a computer operator and then eventually working towards and graduating to a senior and then consultant Systems Programmer role while working for a number of companies in his native home-land of New Zealand. After immigrating to Australia in the late 1990s to work for IBM at the ADL Garry has been involved with LE/VSE since the original LE/VSE 1.1.0 version was originally ported from OS/390 and made available with VSE/ESA 1.4. After servicing the later release of LE/VSE 1.4.0 Garry became the lead development architect for LE z/VSE and maintenance of the same product. This involves detailed customer problem analysis, problem resolution code development and implementation through to the shipped PTF and any associated documentation updates. He is also responsible for the design, implementation, testing and documentation of all LE z/VSE specific enhancements (including CEETRACE) and also for any ported LE z/OS functionality. The IBM Australia Development Laboratory (ADL), which is located in Perth Western Australia, is home to many highly experienced System/z developers and infrastructure support personal. ADL develop and service a wide variety of products over many System/z areas including zVSE, zVM and zOS. Products such as but not limited to the zVM C/C++ XL Compiler and Binder, EGL for VSE plugin and the VSE Build Server for Rational Business Developer (RBD), Rational Team Concert for System/z, SCLM and ISPF for z/OS. CEETRACE Tips and Tricks - Technical Document 1.0 Page 18 of 18