Your SlideShare is downloading. ×
Jon Sayles - Using RAAi together with RDz for Traditional z ...
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Jon Sayles - Using RAAi together with RDz for Traditional z ...

1,907
views

Published on


0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,907
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • The problems of software maintenance are well known: a labor-intensive activity conducted by less than well-prepared people using inadequate documentation. This paper presents a constructive approach to software maintenance that centers on a highly operational software development technique. We argue that maintenance practitioners are served well by an operational view of a system, in which the sequential behavior is emphasized, as opposed to the usual hierarchy of functions viewpoint.
  • From Frieda: Split a program - PL/I Undocumented programs Programs
  • To debug this application, use the following steps: 1. Locate the error message for the current condition in the Condition Information section of the dump, shown in Figure 94 on page 250. The message is CEE3209S The system detected a fixed-point divide exception (System Completion Code=0C9). For additional information about this message, see z/OS Language Environment Run-Time Messages . 2. Note the sequence of calls in the call chain. COBOLZ1 called IGZCFCC, which is a COBOL library subroutine used for dynamic calls; IGZCFCC called COBOLZ2; COBOLZ2 then called IGZCFCC; and IGZCFCC called ASSEMZ3. The exception occurred at this point, resulting in a call to CEEHDSP, a Language Environment condition handling routine. The call to ASSEMZ3 occurred at statement 11 of COBOLZ2. The exception occurred at offset +64 in ASSEMZ3. 4. Check offset +64 in the listing for the assembler routine ASSEMZ3, shown in Figure 96 on page 252. This shows an instruction to divide the contents of register 4 by the variable pointed to by register 6. You can see the two instructions preceding the divide instruction load register 6 from the first word pointed to by register 1 and prepare register 6 for the divide. Because of linkage conventions, you can infer that register 1 contains a pointer to a parameter list that passed to ASSEMZ3. Register 6 points to a 0 value because that was the value passed to ASSEMZ3 when it was called by a higher level routine. Note: To translate assembler instructions, see z/Architecture Principles of Operation, SA22-7832. 5. Check local variables for COBOLZ2 in the Local Variables section of the dump shown in Figure 97. From the dump and listings, you know that COBOLZ2 called ASSEMZ3 and passed a parameter in the variable DV-VAL. The two variables DV-VAL and D-VAL have 0 values. 6. In the COBOLZ2 subroutine, the variable D-VAL is moved to DV-VAL, the parameter passed to the assembler routine. D-VAL appears in the Linkage section of the COBOLZ2 listing, shown in Figure 98 on page 253, indicating that the value did pass from COBOLZ1 to COBOLZ2. 7. In the Local Variables section of the dump for program COBOLZ1, shown in Figure 99, D-VAL has a 0 value. This indicates that the error causing a fixed-point divide exception in ASSEMZ3 was actually caused by the value of D-VAL in COBOLZ1.
  • Why understand the COPYBOOKS? Understanding copybooks Rapid system understanding is a mixture between looking at global system information like the call graph and looking in more detail at a specific program in order to obtain a feeling of its particularities. One of the aims during rapid system understanding is to reduce the number of programs that need to be studied in detail. Having the copybook relation explicitly available will help to avoid looking at copybooks that are in fact never included. Do the systems contain reusable code? Based on the call graph several programs and sections were identified which, after inspection, turned out to contain well-isolated reusable functionality. Which data fields represent business entities? The tools help to identify those data fields that are written to file and used by most programs: the indexed representation helps to browse these fields and filter out certain fields that are perceived as “non-business”. Which statements describe business rules? An inventory of the data fields used is made per section: those dealing with several fields are likely to describe business-oriented procedures. What fraction of code is platform-specific? Of the 340 sections of Share 177 (approximately50%), refer to at least one data field. Thus, an initial estimate is that the other 50% is likely to contain platform-specific code. For Mortgage, 510 of the 2841 sections (only 18%) refer to the data items stored on file. Thus, 82% appears to be platform-oriented rather than business oriented. Inspection of the functionality shows that this is the case: a large part of Mortgage deals with CICS-specific details (implementing a layer on top of CICS). Rapid system understanding, in which fast comprehension is more important than highly accurate or detailed understanding, plays an important role in the planning, feasibility assessment and cost estimating phases of system renovation projects. Comprehension models Searching through code using lexical analysis can be viewed as browsing in order to answer questions and verify hypotheses. Recent studies in system and program understanding have identified code cognition models emphasizing this hypothesis verification aspect [3, 12, 13]. From our experience with the two COBOL cases we observed that many of our actions were aimed at reducing the search space . Thus, rather than verifying hypothesis immediately, we started by organizing the set of programs such that the chance of looking at less relevant programs was minimized. It seems interesting to study how this search space reduction fits in some of the existing code cognition models.
  • Thanks to John DelMonaco for assisting with the research in the above.
  • Transcript

    • 1. RAAi and Maintaining z/OS COBOL Applications z/OS Traditional Development – Upgraded to the 21 st Century
    • 2. Terms & Definitions- What is Software Maintenance?
      • Software maintenance is that collection of activities that relates to correcting, adapting, or perfecting software in production use .
        • "in production use" means the software has been formally accepted and put into use by its users
        • Once software is "in production" it rapidly evolves into "Legacy"
      • So, Legacy Applications are: " Applications that work "
    • 3. Maintenance Project Problems
      • 1. Programmer Effectiveness
        • Almost all maintenance is done by programmers who did NOT write the code they're maintaining:
          • Gaining a complete and thorough understanding of the application semantics of a single legacy program, much less an entire system is incredibly challenging
        • Much maintenance is done by programmers who do not have the depth of experience necessary to understand:
          • z/OS-specific technical issues and problems
          • Complexities introduced by large-scale application scope
      • 2. Software Quality
        • Making matters worse, z/OS "Legacy Applications" – which is the stuff of z/OS Maintenance Projects – have become over the decades:
          • "Brittle" – and easily broken because of short-term "patches" applied over the years
          • Complex – due to the number of different coding styles that have revised the logic
          • With useless documentation and embedded program comments
            • Op Ed – luckily, z/OS Legacy Applications were written primarily in COBOL which is self-documenting.
      • 3. Machine requirements
        • Processing costs (development MIPS costs) and LPAR availability
    • 4. The Need for Automated Maintenance Solutions
      • "[The] current practice of documentation and coding does not encourage efficient understanding as it compartmentalizes knowledge by type of document and rarely provides the cross references that are needed to support programmers’ cognitive needs."
        • A. von Mayrhauser, Senior Member, IEEE, and A.M. Vans, Colorado State U.
      •  Translation:
        • Application understanding, or "gaining intellectual control" (Larry England/IBM) over the code you're about to change is the crux of software maintenance
          • And almost all maintenance is done by individuals who did NOT write the original code
        • Programmers use a multilevel approach to application understanding (building cognitive procedural models of the semantics):
          • Top down
          • Bottom up
          • And frequently switch between program/situation (bottom-up), and domain (top-down) models.
        • Without automated solutions you have two alternatives for
          • ISPF
          • Studying compile listings
    • 5. Why Automated Maintenance Solutions?
      • Same basic functionality since the  late 1970's
      With RDz/RAAi  Work much more productively Lower defect rate Save MIPS
    • 6. Studying Listings
      • What the bosses think it's like
      • Manual process dates back to Grace Hopper and the first generation of business programmers modifying production source code
      • What it's really like
    • 7. Optimizing Maintenance
      • Research, over the last 20 or so years in the field of z/OS maintenance has proven the following:
        • The needs of maintenance – for process and tools differ from development
        • There are specific best practices, tools and tool features that improve maintenance productivity and effectiveness. These improvements include (but are not limited to):
          • General tools and features:
            • The ability to view the application landscape, to see/analyze as much code at-a-glance as possible
            • Program navigation
            • Functionality that keeps up with developer's thought processes
          • Specific maintenance tools/features assistance
            • Finding semantic dependencies – in key elements within the application
            • Rendering code in a semantically meaningful way
            • Enabling optimized analysis – throughout the lifecycle
      RDz RAA
    • 8. RAA and z/OS Traditional Maintenance RAA reveals the procedural semantics buried within an application's source code. It catalogs all elements and all of the RELATIONSHIPS among the application elements. RAA presents these elements and their relationships in simple, visual, hyper-linked and easily consumed reports and graphics It is the shortest path to grasping an application – gaining top-down and bottom-up "intellectual control"
    • 9. Rational Asset Analyzer – Application Understanding Quickly understand code with little or no documentation, and relationships across the enterprise Batch job diagram WAR file graph
      • Group artifacts into user-defined groups called Applications to limit scope to area of interest
      • Use various types of diagrams for understanding how the application “hangs together”
      • Use annotations to capture knowledge from SMEs e.g. Business function, description, etc.
      • Create user-defined relationships for situations where relationships cannot be determined through static analysis
      • Perform enterprise-level keyword searches
    • 10. RAAi – RAA/RDz integration
      • Installed on top of RDz
      • RAA functionality in Eclipse/RDz based interface
      • Seamless transition from the RDz user interface to remote RAA / WSAA for enterprise information access
      • Local/remote functions are independent
        • Local: RDz + RAAI + RAA
        • Remote: RDz + RAAI + remote connection to:
          • RAA
          • WSAA
    • 11. RAAi – RAA/RDz Functionality Summary
      • Scanning of files in local workspace projects into a local RAA repository.
      • Search for Programs and Data Elements in local and remote RAA repositories.
      • View local and remote programs
            • A tree view showing the components that make up the program broken down by component type.
            • A diagram showing the control flow of the program.
            • A diagram showing the structure of the diagram.
            • A table showing information about all data elements that are in a program.
            • View Dead code in Cobol program
      • Perform impact analysis on files, programs, and data elements, and view their results.
      • Integration between RAA and WSAA functions and source in LPEX RDz editor window
    • 12. Demo
      • RAAi with RDz
        • Integration
        • Views
        • Functionality
        • Reports
        • Access to RAA
        • Batch Jobs
    • 13. Net
      • You can continue to maintain z/OS applications the way developers did in the '70s
    • 14. Or you can use RDz/RAAi
      • Which – do you reckon:
      • Makes you more productive?
      • Nets you better quality code (fewer software defects)?
      • Saves MIPS, Time and Money?
    • 15. Software Maintenance Lifecycle – and IBM/Rational Tools
      • IBM/Rational solutions which automate the stages of the Software Maintenance Lifecycle
      Team Collaboration IDE Workbench Start Here… Note: RTCz/RDz enable all lifecycle stages Rational Software Architect Debug Tool Fault Analyzer Debug Tool Fault Analyzer Debug Tool Fault Analyzer
    • 16. Standard ISPF – versus RDz – Application Development and Maintenance Functionality COBOL, PL/I, Assembler, Java, CICS, SQL, C Not available Language-specific editors and tools (COBOL, PL/I, Assembler, Java, EXEC SQL/CICS, etc.) Can offload significant operational sub-systems on to: AIX, System P Cannot reduce MIP consumption with mainframe run-time Reduce MIP consumption for mainframe run-time environment Significant MIPS reduction possible (between 40% and 70%) Cannot reduce MIP consumption with mainframe tools Reduce MIP consumption for mainframe development
      • RDz + Debug Tool
      • Local
      • Host-based
      • Intuitive, integrated, easy to use
      Only available with ISPF + CompuWare products + extremely TSO-cycle intensive Local Debug not available Not available
      • Edit/Compile/Debug/IDE Integration:
        • Line-by-line step
        • Monitor and Watch Lists
        • Modify variable and even register values dynamically
        • Local Debug and/or Remote Debug
        • End-to-end debug between mainframe and WebSphere Java/J2EE applications
      RDz + Fault Analyzer Perspective. More seamlessly integrated tools No software integration. Only available with PD Tools Integrate the IDE and source editor with ABEND resolution software (click on the ABENDing line, and go directly to the COBOL source line where the instruction error occurred) One language – one IDE - one toolset – used by either COBOL/PL/I, CICS/IMS, Java or Web 2.0 developers Y Cannot be done with ISPF Cannot be done with ISPF Cannot be done with ISPF
      • Use ONE common development environment, and IDE for:
        • Traditional mainframe COBOL/PL1/CICS/IMS/DB2 development
        • Java/J2EE modeling, development and maintenance
        • Web 2.0 (Rich Internet Application) Development
      One language – one IDE - one toolset – used by either COBOL or Web 2.0 developers Requires Java Developers with separate tools working with COBOL developers Create Web 2.0 (Rich Internet Application) interfaces to existing COBOL/CICS/IMS applications One language – one IDE - one toolset – used by either COBOL or Java developers Requires Java Developers with separate tools working with COBOL developers Create Web Interfaces to existing COBOL/CICS/IMS applications Automated, Wizard-driven process Not available UML  COBOL model transformation Y – both N - Only system print Local or system print capabilities Y Not available "Paint" using Drag & Drop CICS and IMS Screens. Generate BMS/MFS + JCL for assembling RDz + Mainframe Debug. More seamlessly integrated tools Available only through batch job PD Tools Integrated Source level Debug and RUN Y – built in to product features Can be done with additional custom REXX and TSO Edit macros (and file tailoring) Generate JCL for compiling, linking and map editing Y Y Submit a batch z/OS Job. Monitor the job and view the output in JES Y Y Launch a z/OS session – Example CICS, IMS TM, or logon to TSO Y Not available Generate complete CRUD/DB2 Application – call-able through Web Services Y Y z/OS Batch Jobs (Submit, Monitor and View) Automated, Wizard-driven process – available from Workstation Manual costly and time-consuming ISPF coding and testing process Generate using wizards, and debug DB2/COBOL/PL1 Stored Procedures Automated, Wizard-driven process Manual costly and time-consuming ISPF coding and testing process
      • Create “top-down”, “bottom-up”, and “meet-in-the-middle” Web Services from existing :
        • CICS/COBOL/PL1
        • IMS/COBOL/PL1
      RDz TSO/ISPF Use Case Scenarios
    • 17. ISPF – versus RDz Basic File, Database and Data Set Manipulation Functionality Y Y Y Y N N
      • Work with datasets, PDS/PDSE and PDS members
        • Allocate, compress, copy, move, migrate, compress with backup datasets
        • Copy files between local (PC) and remote (mainframe) systems
        • Copy files and datasets between LPARs
      Y Integrated RDz + File Manager Partial Only available with CompuWare File-Aid
      • Work with VSAM (indexed sequential or relative record) datasets
        • Direct file editing format through a COPYBOOK
        • Specify selection for VSAM file sub-setting
      Y Integrated RDz + File Manager Partial Only available with CompuWare File-Aid
      • Work with large, complex QSAM (sequential) datasets
        • Direct file editing format through a COPYBOOK
        • Specify selection for VSAM file sub-setting
      Y Y Y N N N
      • Work with IMS
        • DL/I View and hierarchy map generation
        • Navigate SYSGEN resource definitions
        • MFS Map Editing
      Y Y Y Y Y Y N N N N N Partial – using SDF II
      • Work with CICS
        • View and navigate resource definitions
        • Define Resources
          • From scratch
          • From existing production resource tables
        • Run locally (offload mainframe test cycles)
        • BMS Map Editing and assembling
      Y Y Y Y Y Y Y Y Y N N N N N N N
      • Work with DB2 tables, views, stored procedures, DDL, Extract and Load
        • Run coded SQL Statements interactively
        • Save SQL statement run-results for unit-test compare purposes
        • Generate SQL Statements with wizard
        • Generate DB2 Stored Procedures using a wizard
        • From a single login, work with multiple databases or work with databases on different subsystems or LPARs
        • Edit a table or view directly (like an Excel spreadsheet) modify/add/delete rows, etc.
        • Load a table from a comma-delimited file
        • Unload a table into a comma-delimited file
      RDz TSO/ISPF Use Case Scenarios
    • 18. Standard ISPF Facilities – versus RDz for Data Source Editing Y N Find/Change across multiple file types Y – Local or Remote N – Must submit Batch z/OS Job Interactive JCL Scan (Syntax Check) for JCL run-stream and Procs Y N Open declaration on Selection Partial – Most but not all functionality Y Issue TSO commands and other TSO and custom REXX Editing Macros and CLISTS Y N Copybook expansion Y N Refactor (modify) Paragraph and variable names Y N Syntax Check Dynamically as you type (syntax validation without compilation) Y – all keys Partial - limited to PF-Key customization Customize Key Mapping – change to editor best suited to the task Y N Syntax/Compiler Error Automatic Statement Selection Y N Block Source Editing (copy rectangular block of code) Y N Commenting/un-commenting executable lines in one operation Y Y Find/Change – Global/Discretionary – repeat Find/Change, and w/in columns Y Y Hex EDIT Y N Filtering data sets into logical folders (for find/change) Y Partial – can use SuperC Source File Electronic Compare Y N COBOL Perform Hierarchy View (shows nesting of Perform chains) Y – available with F1 over keyword N COBOL Keyword and Language syntax HELP Y N Local History for Edit Compare, or for undo of saved changes by timestamp Y – Local or Remote N – Must submit Batch z/OS Job Interactive Compile (Syntax Check) for COBOL Y Partial – Cannot Save Bookmarks Bookmarks (create and save between edit sessions) and Bookmark views Y N Show lines that have been changed during Edit before Save Y N Templatized COBOL and PL/I program and JCL file create process and code Snippets Y N Content Assist – build COBOL statements dynamically from data elements and paragraphs (provides serious productivity improvements and fewer compile errors) Y N Outline (hierarchical) view of COBOL program- Paragraphs and Copybook records Y N Collapse/Expand source view - show only 01, Paragraphs, SQL, CICS, etc. or detail code 46  76 lines of source (editing fonts) 17 lines (Default ) Comprehensive/Expansive source view (how many lines code can you see at once?) Y – unlimited use of editable views Limited # of vertical-only views – Editing in one Split Screen – vertical or horizontal. View and work in (edit in) > 2 splits at once Y – ISPF mode or GUI Edit Mode Y Block (multi-line) editing (Insert/Delete/Copy/Move, Exclude, Copy with Overlay, Shift) Y – ISPF mode or GUI Edit Mode Y Single line edit commands (Insert/Delete/Copy/Move, Exclude, Copy with Overlay, Shift) RDz TSO/ISPF Functionality
    • 19. TSO/ISPF – versus RDz + COBOL Analytics Tools – For Development Standard (in-the-box) RAA reports During the development of this app, the users realized they forgot about a key piece of information that will be added to a DB2 table. We need to know what COBOL/PL1 programs and Java entities (Servlets, JSPs and EJBs) reference this table – as they will more than likely need process the information in the new table/column
      • All integrated tooling – and allows you to answer the critical questions: “Where, What, Why and How” did this problem happen:
        • RDz
        • PD Tools
        • RAA
      • Manual, iterative, costly, error-prone and time-consuming ISPF – and can only answer the question: “What Happened”
        • Search
        • Edit/Compile/Retest
        • May require additional tools:
          • ABEND-AID
          • File-AID
      • Debug an error that occurred during application development unit-test – including but not limited to:
        • Data exception
        • Operation exception
        • Arithmetic exception
        • Divide By Zero
        • Job Cancelled/Timeout (infinite loop)
        • Module Not Found
        • IMS Database Error or VSAM File Error Condition (invalid status code)
        • DB2 / SQL Error Condition
        • Incorrect Output (No ABEND Code – just incorrect results)
      Standard (in-the-box) RAA reports
      • Manual, iterative, costly, error-prone and time-consuming ISPF:
        • Search
        • Edit – document with comments
      • We’re planning on transforming several key VSAM and QSAM files into DB2 tables. In order to cost and scope this effort, we need to know:
        • What JCL will have to be changed
        • How many temporary (&&Temp and SortWork) files are affected (for QSAM)
        • All areas of the COBOL programs that access the VSAM file that will have to be re-written
        • FDs
        • Copybooks and all other related WORKING STORAGE structures and variables
        • Linkage Section
        • PROCEDURE DIVISION paragraphs
        • All calls to sub-routines passing values
      RAA Reports
      • Manual, iterative, costly, error-prone and time-consuming ISPF:
        • Search
        • Edit – document with comments
      • Note graphical documentation is (for all practical purposes) not supported
      • We would like to document the following for all new applications:
        • JCL run-stream jobs, Procs and Parmlib data
        • Data Flow Diagram – all QSAM/VSAM files, DL/I databases and DB2 Tables/Views
        • A process flow diagram – all of the programs in each job
        • A list of all programs in the application. For each program:
          • A graphical/hierarchical structure chart of the flow of control
          • A list of all files accessed
          • A list of all copybooks
        • If online program:
          • Show the transaction flow – front-to-back
          • List associated system resources (CICS/IMS table gens)
          • List online screens – and which programs send and receive them
        • Java / JavaScript web pages that access the application’s:
          • Process (COMMAREA or LINKAGE)
          • Data
        • A set of analysis metrics – that calculate overall code complexity
      Debug Tool Virtually impossible to achieve with ISPF search techniques We need to ensure that the unit and integration test data beds have sufficient Code Coverage Analysis before moving the application into production RDz + RAA TSO/ISPF Use Case Scenarios
    • 20. TSO/ISPF versus RDz + COBOL Analytics Tools For Maintenance – 1 of 2 Included in the product functionality Very costly and difficult to achieve – error-prone We would like to statically analyze the program’s execution path (both forwards and backwards) in order to determine how a particular routine ended up being invoked (we’re looking for fall-through logic) Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches Lately a certain batch job is taking too long to finish. We need to find out data sets and tables are being accessed in order to determine if there are contention issues Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches Find COPYBOOKS that are no longer used Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches Find INSERTS into a VSAM file Built-in code metrics reports Subjective opinion Should we keep investing in maintenance or redesign this application (how complex is it)? Built-in code metrics reports Subjective opinion What is the risk that this application will have performance issues in the near future? Built-in code metrics reports Subjective opinion Is this application a good candidate for outsourcing? Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches – by expert shop developers
      • A key algorithm (that’s hard-coded in several modules in a large application) for computing tax and revenue has changed, and we want to:
        • Create a reusable sub-routine for this algorithm
        • Find all of the modules that contain the code
        • Replace the existing hard-coded logic with a call to the sub-routine
        • Understand what applications paths must be tested
      Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches We just bought a new SORT package and want to change all of the JCL in our shop to reference the new product Included in the product functionality Manual iterative, costly and time-consuming ISPF searches – more likely combined with green-bar listing/printout study
      • Bill and Julia are retiring next month (they’ve each worked here for 35+ years – and are mainly responsible for the maintenance and support of 7 mission critical systems. We have hired and trained new COBOL developers, but how can they grasp the:
        • Programs in an online transaction
        • Key data flows throughout the system (both in batch and online transactions)
        • Flow of control (both the batch and online transaction parts of these applications)
        • Job sequence and the JCL run-stream/Proc datasets
        • Inter and Intra-program dependencies and data relationships
        • Programs’ structure and relationships
      Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches We need to enlarge a DB2 table column, list the DB2 affected artifacts, COBOL copybooks, WORKING-STORAGE and LINKAGE SECTION fields, program literals, Files – including JCL DD card (LRECL/BLKSIZE) parameters (for the file itself and affected files (GDGs, &&TEMP and SORTWRK), print files, online screen macros, online system table and including: XML files, WSDL definitions, EJBs, Property Files, Java Servlets & JSP and any HTML pages that have to be changed Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches List the programs that use a data element of a particular name Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches List the programs called by a particular program & List the programs that call a particular program Included in the product functionality Very costly and difficult to achieve – error-prone Find dead (un-executed) code in any program in an application Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches We need to change a file’s LRECL. What other jobs and COBOL programs will be affected? Included in the product functionality Virtually impossible to achieve with ISPF search techniques We need to ensure that the unit and integration test data beds have sufficient Code Coverage Analysis before moving the application into production RDz + RAA TSO/ISPF Use Case Scenarios
    • 21. TSO/ISPF versus RDz + COBOL Analytics Tools For Maintenance – 2 of 2 We want to migrate a COBOL program from z/OS to AIX COBOL - What percentage of the code is platform-specific Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches We want to change the over-night batch cycle, but are worried about contention issues. List all of the common datasets between two applications (not Procs, not JCL entire applications) Available queries against the RAA catalog Manual, iterative, costly and time-consuming ISPF searches We are trying to reengineer a large COBOL application. In the initial data analysis phase we need to remove homonyms from the code. List all copybooks with the same name as other copybooks. Built-in code metrics reports Subjective opinion A production proc needs to be changed. List all of the datasets referenced. Available queries against the RAA catalog Subjective opinion We are trying to recover disk space. List all QSAM and VSAM datasets that are not used (referenced) in either the Batch or Online application systems Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches Make a list of all unused Copybooks (see above, reclaiming disk space) Included in the product functionality Manual, iterative, costly and time-consuming ISPF searches We plan to reengineer a QSAM file to DB2. List all of the CRUD operations against it, in order to begin the analysis and cost estimates. Included in the product functionality Very costly and difficult to achieve – error-prone Create a sorted list of all COBOL literals used in the PROCEDURE DIVISION. Available queries against the RAA catalog Manual, iterative, costly and time-consuming ISPF searches The users have decided to change a screen. Provide an initial estimate (scoping) of the amount of effort required by listing all transactions and programs that will be impacted by this. Available queries against the RAA catalog Virtually impossible to achieve with ISPF search techniques We need to change a date parm for batch streams, from USA format to ISO. Identify all of the batch streams that will be affected. RDz + RAA TSO/ISPF Use Case Scenarios
    • 22. TSO/ISPF – versus RDz + COBOL Analytics Tools – For ABEND Resolution and Production Support Depending on the type of error, use either the file/program dependency reports, or impact analysis to track backwards from the error to the source Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage. VSAM File Error Condition Use RAA reports to verify the logic sequencing, that caused an R.I. problem during INSERT. Or, if some other SQL error use Impact Analysis tracing to find out the cause of the problem. Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • DB2 / SQL Error Condition
        • Invalid or unexpected SQLCodes (Referential integrity problem, -811, Duplicate Primary key, etc.)
      Use RAA reports to verify the logic sequencing, that caused the invalid DL/I Return Code. Or, if some other DL/I error use Impact Analysis tracing to find out the cause of the problem. Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • IMS Database Error Condition
        • Invalid or unexpected PCB status codes
      Use RAA code understanding schematics to grasp the logic leading to (typically) an infinite loop. Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Job Cancelled / Timeout condition
        • 122/222/322
      Use RAA automatic impact analysis functionality – which spans variables, modules, jobs data files/databases and transactions Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Module Not Found
        • 806
      Use RAA automatic impact analysis functionality – which spans variables, modules, jobs data files/databases and transactions Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Debug an Operation Exception
        • 0C1
        • 0C4
      Use RAA automatic impact analysis functionality – which spans variables, modules, jobs data files/databases and transactions Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Arithmetic Exception
        • 0C8
        • 0CA
        • 0CB (Divide By Zero)
      Use RAA automatic impact analysis functionality, on incorrect values to reverse trace variables, modules, jobs data files/databases etc. in the transaction or unit of work Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Incorrect Output
        • No ABEND Condition
      Use RAA automatic system tracing functionality – which provides Data Flow analysis, and spans variables, modules, jobs data files/databases and transactions. May also need Impact Analysis – also built-in to the product Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Debug a Data Set type of exception
        • 001
        • 002
        • 013
        • 813 (file open error)
        • B37 (out of space condition)
      By using RAA the developer will be able to understand the context of the problem (What, Where, How, When and most-importantly Why) some of these problems happen Manual, iterative costly, error-prone and time-consuming ISPF searches – working backward in the source code, often across multiple jobs, attempting to follow the instruction set that wrote invalid (non-numeric) data into storage.
      • Debug a data exception
        • S0C7 – Essentially it’s necessary to perform a usage (intelligent-reference) analysis of what data has overlaid the contents of
      RDz + RAA TSO/ISPF Use Case Scenarios
    • 23. TSO/ISPF – versus RDz + COBOL Analytics Tools – For SOA Custom Queries from the RAA Catalog Manual iterative, costly and time-consuming ISPF – and probably PC-file searches We need to change a web service. Find all programs, applications and Java Classes that use a particular WSDL " Manual iterative, costly and time-consuming ISPF searches Which statements in the program describe business rules " Manual iterative, costly and time-consuming ISPF searches Which variables in the programs we want to SOA-enable represent business entities? “ Manual iterative, costly and time-consuming ISPF searches Should we keep investing in maintenance or redesign this application (how complex is it)? “ Manual iterative, costly and time-consuming ISPF searches Identifying data validation assumptions from screen interactions, and ensuring that the data validation is done somewhere, potentially as part of the contract with the calling service “ Manual iterative, costly and time-consuming ISPF searches Removing screen calls down in a call chain and replacing this with error return codes. “ Manual iterative, costly and time-consuming ISPF searches Separating the business logic that you want to reuse from what is not relevant. “ Manual iterative, costly and time-consuming ISPF searches List programs that are doing persistence or presentation functionality RAA standard reports and functionality included in the tools Manual, iterative, costly, error-prone and time-consuming ISPF searches List most frequently called (hub) programs (identify as good candidate for SOA) RDz + RAA TSO/ISPF Functionality
    • 24. TSO/ISPF – versus RDz + COBOL Analytics Tools – For Testing " Manual iterative, costly and time-consuming ISPF searches We are making a change that impacts a very large # of jobs across the enterprise, identify all of the input files (and not temporary inter-job files/output files) by using the job-step sequence in the JCL procs “ Manual iterative, costly and time-consuming ISPF searches Validate the code-coverage of our existing test-plan – down to the field level of affected variables due to a modification of a file copybook “ Manual iterative, costly and time-consuming ISPF searches Removing screen calls down in a call chain and replacing this with error return codes. “ Manual iterative, costly and time-consuming ISPF searches Separating the business logic that you want to reuse from what is not relevant. “ Manual iterative, costly and time-consuming ISPF searches Identify the screen and data resources that are needed to do testing: files, databases, etc. RAA standard reports and functionality included in the tools + custom SQL queries against meta-data stored in the catalog Manual, iterative, costly, error-prone and time-consuming ISPF searches I've changed a particular program, sub-routine, field or copybook – identify all the places it is used – across the enterprise - so I can build quality test cases RDz + RAA TSO/ISPF Functionality
    • 25. TSO/ISPF – versus RDz + COBOL Analytics Tools – For Data Analysis and Database Work RAA Standard product Impact analysis tracing process Manual, iterative costly, error-prone and time-consuming ISPF searches
      • Need to rename or modify the datatype of a column in a table – Show all impacts:
        • Indexes
        • Views
        • Application program copybooks – and all associated WORKING-STORAGE fields that are associated with the changed copybook field(s)
        • SQL Statements
        • Java/J2EE web pages
      RAA Custom reports and meta-dictionary lookups Manual, iterative costly, error-prone and time-consuming ISPF searches We need to further normalize a table design. What programs and DCLGENs are affected RAA Custom reports and meta-dictionary lookups Manual, iterative costly, error-prone and time-consuming ISPF searches We would like to explore the use in our shop of DB2 “outer joins”, as they are potential performance problems – where can we get a list of them? “ Manual, iterative costly, error-prone and time-consuming ISPF searches We would like to offer 24/7 batch update support. What table and index combinations are candidates for locking and contention issues? “ Manual, iterative costly, error-prone and time-consuming ISPF searches What columns are good candidates for indexes (based on source reference/usage model) “ Manual, iterative costly, error-prone and time-consuming ISPF searches What tables are accessed online? Batch? Both? Also list the report by Job Stream and by Transaction “ Manual, iterative costly, error-prone and time-consuming ISPF searches List any SQL statements that join “large” tables without indexes on their foreign keys “ Manual, iterative costly, error-prone and time-consuming ISPF searches Our current Data Center manager believes heavily in Stored Procedures. What are the best candidates for DB2 Stored Procedures among our existing static plans? RAA Custom reports and meta-dictionary lookups Manual, iterative costly, error-prone and time-consuming ISPF searches We would like to build a CRUD matrix against our DB2 tables, Views and VSAM files RAA Standard reports Manual, iterative costly, error-prone and time-consuming ISPF searches We have just upgraded our DASD and need to re-visit the blocking factor (BLKSIZE) for the QSAM files on a given LPAR. “ Manual, iterative costly, error-prone and time-consuming ISPF searches We are designing new tables, and need to understand the application data access model (Select …vs… Update, Delete, Insert) in order to find the best performance-oriented and clustering index combinations “ Manual, iterative costly, error-prone and time-consuming ISPF searches We would like to partition our largest and most heavily used tables to support CPU and I/O access parallelism. What are the best candidates? And what’s a good clustering key for partitioning? “ Manual, iterative costly, error-prone and time-consuming ISPF searches Lately a certain batch job is taking too long to finish. What data resources – including QSAM, VSAM files and DB2 tables does it access? “ Manual, iterative costly, error-prone and time-consuming ISPF searches What tables are frequently (or almost always) used together? We may want to de-normalize the design. RAA Custom reports Manual, iterative costly, error-prone and time-consuming ISPF searches Added an index – Packages/Plans to re-bind. List programs to re-analyze RDz/Explain Tool – and SQL Best Practices queries. Manual, iterative costly, error-prone and time-consuming ISPF searches Find inefficient SQL Queries RDz + RAA TSO/ISPF Functionality
    • 26. TSO/ISPF – versus RDz + COBOL Analytics Tools, For Capacity Planning/System Tuning Note – the above static system documentation would be used to complement standard SMF data and Tivoli/OMEGAMON output analysis RAA cumulative reports in the meta-model on system resources The business side is predicting a 15% up-tick in growth and numbers of concurrent users. What effect will the associated increase in CICS/DB2 traffic be on our network, production and testing sub-systems?
      • RAA standard reports on new application resources:
      • Program modules + online screens
      • I/O statements & SQL activity
      • If I add these 4 new batch jobs, what will the affect be on:
        • CPU Utilization
        • I/O Storage
      RAA cumulative reports of batch and online system resources Limited to traditional manual and ISPF multi-search methods Do I have enough resource (CPU, I/O, Storage,..) to do the job today? RAA standard reports on resources How do I size a new application? Custom catalog reports on JCL dataset extents over time captured in the database How do I establish my growth? RAA standard reports What is the combined VSAM file allocation currently required by a given application? “ How can I more accurately model my current workload? Standard reports with SQL output Limited to traditional manual and ISPF multi-search methods DB2 Buffer Pool Allocation Custom catalog queries on SQL statements that incur DB2 sort Limited to traditional manual and ISPF multi-search methods DB2 sort pool sizing RDz + RAA TSO/ISPF Functionality
    • 27. TSO/ISPF – versus RDz + COBOL Analytics Tools – For Project Management and Outsourcing Governance RAA Standard reports and metrics N - Limited to traditional manual and verbal methods Lower the cost and length of the bidding process RAA Standard reports and code visualization N - Limited to traditional manual and verbal methods Reduce dependency on a client's time and expertise by automating application intelligence RAA Standard reports and metrics – and code visualization Y – but limited to PF-Keys Bring new consultants up to speed with client code far faster RAA coding best practices reports Limited to traditional manual and verbal methods Improve client audit compliance through management information and status reporting RAA Standard reports and metrics No generation capabilities, only expensive and time-consuming manual methods Generate core systems documentation in minutes, providing factual backup to strategic decisions and aiding application understanding for future projects RAA coding best practices reports Limited to traditional manual and verbal methods Support consistent delivery of high-quality projects through fully automated, easily repeated assessments across the entire core system landscape RAA Standard reports and metrics Limited to traditional manual and verbal methods Improve year-on-year ROI for both supplier and client RAA Standard reports and metrics – showing application size, scale, dependencies, data flow, and complexity metrics
      • Discuss with application Subject Matter Experts
      • Manual iterative costly and time-consuming ISPF searches
      • Read through existing documentation
      Lower the cost and length of the bidding process RDz + RAA TSO/ISPF Functionality
    • 28. TSO/ISPF – versus RDz + COBOL Analytics Tools – Enlarge a DB2 Column (Just Application Work – not DBA portion) Run RAA Standard impact analysis reports.
      • 1. Use a 3rd Party tool to determine what DB2 PLANS reference the table. Or using SPUFI/QMF, write and execute a number of DB2/DBA-level System Catalog queries – referenced catalog tables include (but are not limited to): SYSPLANS, SYSPLANDEP, SYSPACKAGES, SYSDBRMS, SYSTMT & Stored Procedure tables (if they are used)
      • 2. Save the DB2/SQL Statement results to a TSO file – or printout. On an average large-scale z/OS production system, this could be well over 100 entries.
      • 3. For each SQL Statement (for each of the (average) over 100 results listed in step 2)
        • Open and browse in the COBOL source
        • Find the statement
        • Find the host variable for the column to be enlarged
        • Note the DCLGEN record name (will have to be re-created)
        • (The hard part) Do iterative, recursive, discrete and explicit TSO text searches, to list all COBOL variables that this field interacts with (essentially, all COBOL "data manipulation" and "data comparison" statements) that will need to be modified. Here is a partial list of explicit keywords, each if found factors into the recursive search:
          • MOVE – and all variations of MOVE such as MOVE CORRESPONDING – which means you now must trace GROUP moves, not just the individual field itself
          • (if a numeric field) ADD, SUBTRACT, DIVIDE, MULTIPLY, COMPUTE, SET
          • IF, EVALUATE, INSPECT ( WHEN clause)
          • Literals that will need to be changed – esp. if part of comparisons
          • STRING, UNSTRING and Reference Modification statements
          • CALL USING/ENTRY USING
          • WRITE, READ
          • A host of LE calls and COBOL intrinsic functions
          • PERFORM UNTIL loops with (if a variable is part of PERFORM USING)
          • … more
        • Disregard all hits (false-positives) in * comment lines
        • Repeat the above for each field affected (all fields related to any changed field become part of a linked list or, " association chain ").
        • Be careful not miss any fields – anywhere in the association chain – this gets very difficult, when fields are externalized – see steps, 4  9 below)
        • List all of the Records affected in the FILE SECTION, or DL/I segments, or other DB2 DCLGENs
          • Look for QSAM – VSAM file - DL/I database and DB2 table write operations
        • Create a list or printout of all affected fields and records
      • 4. For all File Records found in step three affected Look for QSAM – VSAM file - DL/I database and DB2 table write operations
        • If DB2 Table SQL statement – return to Step 1
        • If any records are QSAM/VSAM files – including SORT/MERGE, &&TEMP files, SYSOUT, potentially log files:
          • Find all JCL associated with the files
          • The LRECL and BLKSIZE Parameters – that will need modification. SPACE parameter might need to be recalculated and extents
          • If they are VSAM files or IMS databases built on VSAM files:
            • you will need to modify the IDCAMS utility JCL used to DELETE/DEFINE the datasets
          • If there are DL/I databases, you will have to change the DBD and modify the DL/I database – deleting and reloading (let's hope not)
      • 5. If the field is passed from one program to another in CALL … USING/ENTRY USING
        • Repeat steps 3,4 and this step 5 – for all affected programs
      • 6. If the field – or any associated field anywhere in the chain participates in printed reports:
        • Recalculate print columns to line up the fields according to the required report output
      • 7. If the field – or any associated field anywhere in the chain is used in a CICS or IMS TM application
        • Recreate the screen UI – modify the BMS or MFS macros
        • Modify all COBOL variables affected by changes to the screen I/O
        • For CICS, you will have to factor in changes to the TS QUEUEs and Transaction Work Areas
        • For older IMS conversational programs, you will have to factor in changes for the IMS SPA definitions in the SYSGEN
      • 8. (And here's the killer) For every file that any field in the chain is written to, you will need to trace each file through every JCL job stream and:
        • Repeat steps 3, 4, 5, 6, 7 and 8 for every program/file/field combination - and every new association chain produced
      RAA TSO/ISPF
    • 29. Collateral list
      • Rational Asset Analyzer
        • http://www-01.ibm.com/software/awdtools/raa/
        • http://www.ibm.com/developerworks/rational/products/raa/
      • z/OS Programmers – Book Excerpt from Capers Jones
        • http://www-949.ibm.com/software/rational/cafe/thread/2274?tstart=15
      • z/OS Terms and Concepts:
        • http://www-949.ibm.com/software/rational/cafe/docs/DOC-3129
      • COBOL's future:
        • http://answers.google.com/answers/threadview/id/328090.html