Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IBM IMS Tools Smarter IMS - IMS UG Phoenix 12-2013


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

IBM IMS Tools Smarter IMS - IMS UG Phoenix 12-2013

  1. 1. On the road to a Smarter IMS Janet LeBlanc Silicon Valley Lab Janet LeBlanc IMS Tools Strategist © 2010 IBM Corporation
  2. 2. IMS Tools Solution Packs IMS Database Solution Pack IMS Fast Path Solution Pack IMS Recovery Solution Pack IMS Performance Solution Pack Autonomics IBM Administration Console IMS Administration Console IMS Administration Console Online Reorg Facility DB Reorg Expert HP Unload HP Load HP Prefix Resolution Index Builder HP Image Copy HP Pointer Checker IMS DB Repair Facility Library Integrity Utilities HALDB Toolkit 2 Autonomics IMS Administration Console HP FP Utilities FP Advanced Utilities FP Online Utilities IMS DB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities DB Recovery Facility HP Change Accumulation HP Image Copy DRF Extended Functions IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator © 2013 IBM Corporation
  3. 3. Our Vision – Putting information to work for you User-defined policies and thresholds Gather database statistics at regular intervals Policies Sensors Automation Modernization Recommending and taking corrective actions 3 Rich, web-based user interface © 2013 IBM Corporation
  4. 4. Database space monitoring Its common purpose is to prevent these situations • Free space shortage o Caused by running-out of data set extents or free space on volumes and resulting in service outage • Inefficient use of database space o Caused by free space fragmentation, which can eventually result in free space shortage • Performance degradation o Caused by free space fragmentation, long synonym chains, a large percentage of overflow segments, etc. and sometimes can result in “batch job overrun” 4 © 2013 IBM Corporation
  5. 5. Database space monitoring What activities are needed? • Collect space statistics data Notice o % of max data set size, … Decide • Analyze the data o Is a threshold reached? … Analyze Prepare • Notice exceptional states o Fragmentation, space shortage, … Collect Act • Decide actions o Free-space reorg, DBD/randomizer change, (re-)partitioning… • Prepare for action(s) o Schedule a free space reorg, plan tuning for actions, … • Take action(s) o Do the structure change, (re-)partitioning, … 5 © 2013 IBM Corporation
  6. 6. Challenges in the current IT environment Especially in IMS database space management • Decrease in o Skilled IMS DBAs o Maintenance window time • Increase in o Number of databases o Data volume Notice Decide Notice Decide Notice Decide Notice Decide Notice Decide Analyze Prepare Analyze Prepare Analyze Prepare Analyze Prepare Analyze Prepare Collect Act Collect Act Collect Act Collect Act Collect Act • Increasing business pressures o “Do more with less.” • Flat or less IT cost • Less people 6 © 2013 IBM Corporation
  7. 7. We recommend simplification and automation for the efficiency of the process • Simplify each step o Data collection o Data analysis o Exception notification o Action determination o Action preparation Notice Decide Notice Decide Notice Decide Notice Decide Notice Decide Analyze Prepare Analyze Prepare Analyze Prepare Analyze Prepare Analyze Prepare Collect Act Collect Act Collect Act Collect Act Collect Act o Action taking • Automate repetitive tasks o Starting from data collection through action decision 7 © 2013 IBM Corporation
  8. 8. IBM IMS Tools … can simplify and automate the process • IMS Database Solution Pack database IMS Database Solution Pack sensor feature can simplify the ‘Collect’ step + • Autonomics Director (IBM Tools Base) and Policy Services can simplify and automate the ‘Analyze’, ‘Notice’, and ‘Decide’ steps • Administration Console (IBM Tools Base) can simplify access to space monitoring information needed for the ‘Prepare’ step • IMS Database Solution Pack also provides reorganization tools that can make execution of space maintenance ‘Actions’ simpler and accelerate the performance 8 IBM Tools Base Notice Decide Analyze Collect Prepare Act © 2013 IBM Corporation
  9. 9. You can do more with less people and time … by using the IBM IMS Tools Solution • Space statistics can be collected as byproducts of your Pointer Checker and/or Image Copy jobs • Data analysis, exception notification, and action recommendation can be made automatically based on exception detection/notification policies IMS Database Solution Pack + IBM Tools Base Notice Decide Analyze • Space statistics, detected exceptions, and recommended actions can be accessed through a comprehensive graphical web interface Collect Prepare Act • Space maintenance actions can be simplified and accelerated 9 © 2013 IBM Corporation
  10. 10. Automating monitoring process 10 © 2013 IBM Corporation
  11. 11. Autonomics Director and Policy Services Infrastructure for automation • Autonomics Director server o Periodically evaluates space statistics data collected by IMS DB Solution Pack utilities, and detect and notifies exceptions if any Automation Infrastructure Policy Services Policy Evaluation & Exception Notification Autonomics Director Server • AD server configuration: o Database monitor list o Data evaluation schedule o Data collection schedule IMS DB Solution Pack IMS DB Solution Pack IMS DB Solution Pack utilities can collect and utilities can collect and utilities can collect and stores space statistics stores space statistics stores space statistics IMS Tools Knowledge Base Server o Exception detection rules o Exception notification targets and policies 11 Repositories © 2013 IBM Corporation
  12. 12. Data Collection Database Sensor capability in utilities • You can collect database space status as “sensor data” • Sensor data can be collected during o Pointer Checker run Notice Decide Analyze Collect Prepare Act o Image Copy run • Sensor data are stored In ITKB Sensor Data Repository • Stand-alone DB Sensor can collect sensor data of one or more online full-function databases and OLR-cursor-inactive HALDB partitions 12 IMS Database Solution Pack High Performance Pointer Checker High Performance Image Copy Stand-alone Database Sensor IBM Tools Base IMS Tools Knowledge Base (ITKB) Policy Services Sensor Data Store © 2013 IBM Corporation
  13. 13. Data Analysis You don’t need to look at / parse a large number of reports • AD invokes “policy evaluation” o New sensor data are notified to Autonomics Director o Then, AD invokes data analysis process based on the evaluation schedule and the policy preconfigured for each database that is registered in the Monitor List • Policy evaluation o A process of applying thresholddetection rules and exceptionnotification policy to the collected sensor data • Each rule detects an exceptional state of a database or a partition 13 Notice Decide Analyze Collect Prepare Act IBM Tools Base Autonomics Director (AD) Policy Services Policy Evaluation IMS Tools Knowledge Base (ITKB) o Policy can be selected or defined for each database type, for an individual database, or for a group of databases. © 2013 IBM Corporation
  14. 14. Exception Notification You’ll be notified of exceptions • Action Recommendation o A policy associates exceptions with an action recommendation (e.g., REORG) • Notification List o A policy can have a list of destinations of exception notification • Exception Notification o The action recommendation is sent to the destinations listed in the Notification List in the form of WTO, an e-mail, a short message, or a TSO message 14 Notice Decide Analyze Collect Prepare Act IBM Tools Base Autonomics Director (AD) Policy Services Exception Notification IMS Tools Knowledge Base (ITKB) o An exception consists of: •An exception name •A severity •A short message © 2013 IBM Corporation
  15. 15. Simplifying access to monitoring information 15 © 2013 IBM Corporation
  16. 16. Action Determination Typical questions that need to be answered • Which databases need attention now? • Why they need attention? Notice Decide Analyze Prepare • What actions are recommended? • How those databases have gotten to the current state? • How are free spaces and DASD volumes used? o What did HP Pointer Checker report? • What the average database record length and is the current block/CI size optimal for it? Collect Act Information needed • Summary of latest status • Trend of space status changes • Database utilities reports for “drill-down” research o What did HP Unload report? 16 © 2013 IBM Corporation
  17. 17. Administration Console Provides a holistic view of data and information IBM OneUI Theme Detected exceptions and action recommendation • Admin. Console also provides a table of the number of exceptions and highest severities for each user-defined group of databases and partitions. Search and group databases • A typical use case scenario and various screen shots are available on Information Center. Charts of various space statistics data items 17 History of utility reports Online help for space monitoring © 2013 IBM Corporation
  18. 18. DBD Visualization – “DBD Map Viewer” New in IMS Database Solution Pack, Version 2.1 • An Admin. Console add-on o Can be installed by using IBM Installation Manager • It provides six views for a DBD: o DBD map o DBD source o DBD XML o Segment properties o Data sets o Outline 18 © 2013 IBM Corporation
  19. 19. DBD Visualization – “DBD Map Viewer” DBD Source View DBD Map View Data Sets View DBD XML View Segment Properties View 19 Outline View © 2013 IBM Corporation
  20. 20. Simplifying space management tasks 20 © 2013 IBM Corporation
  21. 21. Space management actions Three main action types • Free space reorganization o Offline reorganization o Online reorganization Notice Analyze Decide Prepare • Definition change reorganization o Space allocation change Collect Act Notice Decide o DBD/randomizer change o Partitioning/re-partitioning • Tuning for definition changes o Planned to prepare for an immediate or future definition/structure change 21 Analyze Collect Prepare Act © 2013 IBM Corporation
  22. 22. Offline free space reorganization solution High Performance reorganization utilities • IMS Database Solution Pack provides following utilities o IMS High Performance Unload o IMS High Performance Load o IMS Index Builder o IMS High Performance Prefix Resolution o Smart Reorg utility • An integrated reorganization utility that runs utility functions above and post-reorganization Image Copy and Pointer Checker in a single job step //REORG EXEC PGM=HPSGMAIN,PARM='DBD=dbdname,DBRC=Y' //STEPLIB DD DISP=SHR,DSN=IMSTOOLS.LOADLIB // DD DISP=SHR,DSN=IMS.SDFSRESL //IMS DD DISP=SHR,DSN=IMS.DBDLIB //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //HPSIN DD * (REORG) DBSHARE=YES IMSCMD=YES,TOSI TOSIXCFGRP=REORG SPACEALLOC=YES INDEXBLD=YES IC=YES NAMESWAP=YES DISPOLDDS=NEWSHADOW STARTDB=YES (UNLOAD) ... (RELOAD) ... (INDEXBLD) ... /* //ICEIN DD * GLOBAL VIC=Y, VICDSN=&DDN..UC1, ICHLQ=TESTDS, DSN=&ICHLQ..&DDN..IC1, UNIT=SYSALLDA, SPACE=(CYL,10,10) /* An example JCL statements for Smart Reorg (for a database that has no logical relationship) 22 © 2013 IBM Corporation
  23. 23. Online free space reorganization solution IMS Online Reorganization Facility (ORF) for z/OS • IMS Database Solution Pack ORF provides online reorg. solution for full-function databases and HALDBs ORF COPY Processing Phase UNLOAD IMAGE COPY RELOAD T APPLY K O Update and re-register logs DB Update Capture Database Status Log UPDATE Database Update Capture UPDATE UPDATE UPDATE Time Database Status UPDATE Read-Only or Offline UPDATE Name Swap (NS) Legend (for both ORF and Smart Reorg) Processing Phase Unload, Reload, Index Build, Image Copy, Pointer Check in a single job step NS DB Data Set before Reorg Shadow Data Set Reorganized DB Data Set Post-Reorg Image Copy Smart Reorg 23 © 2013 IBM Corporation
  24. 24. Tuning by partitioning/re-partitioning IMS HALDB Toolkit helps you DBD1 DBD2 DBD3 DBD4 • The Toolkit provides HALDB maintenance, modeling, and analysis functions, including DBDS1 DBDS2 DBDS3 DBDS4 100-199 200-299 300-399 Key Ranges: 001-099 RECON o Splitting a partition o Consolidating partitions o Adding an empty partition to the end of a database o Merging HALDBs DBDH1 DBDH3 DBDH4 PART11 PART21 PART31 PART41 Key Ranges: 001-099 100-199 200-299 300-X’FFFFFF’ RECON Merge a set of singlepartition HALDBs to a single HALDB with multiple partitions. DBDH Partition Definition Key Ranges:001-099 24 DBDH2 Partition Defiinition PART1 • The Toolkit can help reduce the time and minimize the skills that are required to perform HALDB maintenance tasks Convert userpartitioned full-function databases to singlepartition HALDBs. PART2 PART3 100-199 200-299 RECON DBDH PART4 300-X’FFFFFF’ Maintain partitions (partition split, etc.) Partition Definition PART1 Key Ranges:001-099 PART2 PART3 PART31 100-199 200-259 260-299 PART4 300-X’FFFFF’ © 2013 IBM Corporation
  25. 25. User-defined policies and thresholds Gather database statistics at regular intervals Policies Automation Recommending and taking corrective actions Sensors Modernization Rich, webbased user interface New in Autonomics: – Active Autonomics – New User GROUP management – Group Reorganization Management 25 Notice Analyze Collect Decide Prepare Act © 2013 IBM Corporation
  26. 26. Active Autonomics Automatic Reorganization Determined by Sensor and Policy Evaluation Supported by IMS DB Reorg Expert DB Maintenance Windows User controlled DB maintenance windows Submission of Reorganization Jobstreams Manages job distribution across sysplex Controls for parallelism Tracks jobstream thru completion User Notifications Email, Text Job plan and results viewable via AD Plan restart capability Determines what ran successfully 26 Only reruns/runs what failed or is yet to run © 2013 IBM Corporation
  27. 27. User Defined Groups Easy to create – Select from any DBs/partitions/areas in system DBDLIBs – Select from any DBRC or other user group – Name masking support: PAY* Contains – List of DB’s in group – Consolidated group attributes and options for various tools and functions – Reorganization, Back-up and Recovery, etc. Automatic membership checking/warning for logical relationships Useful for – Adding DBs to the monitor list – Viewing recommendation list – Scheduling on demand Basis for Group Managed Reorganization 27 © 2013 IBM Corporation
  28. 28. Group Managed Reorganization Pre-GMR • /DBR and DBRC authorization changes • Allocate common/work files • Pre-reorganization utility • Runs IMS Reorg Expert for DBs that need reorg • Optional DB scan for logically related DBs that don’t need reorg •Consistent feed for prefix resolution/prefix update • One job per DB/partition/area Reorganization • DBs reorganized in parallel or serially or combination Post-Reorg Post GMR 28 • Invoked after reorg phase • One or more jobs run serially or parallel as identified by user • Prefix Resolution, Prefix Update • Invoked after post-reorg phase • One or more jobs run serially or parallel as identified by user • Dataset cleanup, Image Copies © 2013 IBM Corporation
  29. 29. IMS Database Solution Pack for z/OS, 2.1 IMS Database Solution Pack combines the following tools into a single offering: • IMS Database Reorganization Expert for z/OS • IMS Online Reorganization Facility for z/OS* • IMS High Availability Large Database (HALDB) Toolkit for z/OS* • IMS High Performance Unload for z/OS • IMS High Performance Load for z/OS • IMS High Performance Prefix Resolution for z/OS • IMS Index Builder for z/OS • IMS High Performance Image Copy for z/OS • IMS High Performance Pointer Checker for z/OS • IMS Library Integrity Utilities for z/OS** • Database Repair Facility * Added in Version 2.1 • Database Sensor ** Provides DBD Map Viewer server capability 29 © 2013 IBM Corporation
  30. 30. DEDB space management • DEDB space management activity is composed of the following tasks: o Monitoring space utilization o Tuning areas and randomizer o Reorganizing free space or restructuring areas 30 © 2013 IBM Corporation
  31. 31. Background and challenge of space monitoring • DEDB space utilization must be monitoring on a regular basis for the following reasons: o To avoid application failures that might be caused by out-ofspace conditions o To determine whether reorganization is required o To determine the optimal timing for database expansion • To maintain hundreds and thousands of DEDB areas, DBA needs deep IMS skill and experiences for database space analysis , and it takes a lot of time to check a large number of reports that IMS tools generates 31 © 2013 IBM Corporation
  32. 32. Space monitoring solution • DEDB area analysis and pointer validation o o o o o Collect raw space statistics (called sensor data) Report detailed space usage statistics Ensure that your database pointers are error free Help you detect and correct database errors quickly Provide both online and offline utilities • IMS Tools Knowledge Base o The reports that are generated by the FPA functions, FPO tools, and IMS HPIC can be stored in a repository o You can search historical copies of those reports by database name, date and time, or other search criteria o Threshold criteria for detecting database exceptions and recommendations can also be defined and stored in a repository • Autonomics Director and Administration Console o Simplify DEDB space monitoring tasks 32 © 2013 IBM Corporation
  33. 33. Activating DB Sensor function in FPSP Utilities • DB Sensor function is supported in following utilities o o o FP Advance Tool (ANALYZE, UNLOAD, RELOAD, CHANGE, and REORG) FP Online Tool (ANALYZE (Online PC), REORG (Online Expert Reorg)) HP Image Copy • DB Sensor is activated just by adding three control statement options o o o SENSOR=YES (required) ITKBSRVR=your_itkb_xcf_group_name (required) ADXCFGRP=autonomic_director_xcf_group_name (optional) A JCL Example for FPA ANALYZE //FPA EXEC PGM=HFPMAIN0 //STEPLIB DD DISP=SHR,DSN=HPFP.SHFPLMD0 // DD DISP=SHR,DSN=ITB.SHKTLOAD // DD DISP=SHR,DSN=IMSVS.SDFSRESL // DD DISP=SHR,DSN=IMSVS.PGMLIB //IMSACB DD DISP=SHR,DSN=IMSVS.ACBLIB //IMSDALIB DD DISP=SHR,DSN=IMSVS.MDALIB //HFPSRPT DD SYSOUT=* //HFPSYSIN DD * GLOBAL DBRC=YES, ITKBSRVR=FPQSRV01, ADXCFGRP=IAVADM00 ANALYZE DBD=DEDBJN22,IAREA=(DB22AR0,DB22AR1), PTRCHKLVL=FULL,SDEPCHKLVL=FULL, SENSOR=YES REPORT /* 33 A JCL Example for OPC //OPC EXEC FPUTIL,DBD=DEDBJN22,IMSID=IMS1 //STEPLIB DD DISP=SHR,DSN=HPFP.SHFPLMD0 // DD DISP=SHR,DSN=ITB.SHKTLOAD // DD DISP=SHR,DSN=IMSVS.SDFSRESL // DD DISP=SHR,DSN=IMSVS.PGMLIB //FPXIN DD * GLOBAL DBRC=YES, ITKBSRVR=FPQSRV01, ADXCFGRP=IAVADM00 ANALYZE AREA=(DB22AR0,DB22AR1), FULLSTEP=YES, TYPRUN=RPT, SENSOR=YES /* For details and more JCL samples, see “DEDB monitoring and tuning guide” in the User’s Guide. © 2013 IBM Corporation
  34. 34. Online free space monitoring acceleration • Support light-weight monitoring besides full monitoring Light-weight monitoring of free spaces (new) OPC o OPC with TYPRUN=FS now can store both the sensor data and reports on free space statistics in a single job step • Elapsed time of OPC with FULLSTEP=YES is significantly reduced by o Processing multiple areas concurrently o Eliminating I/Os for intermediate data sets 34 area IMS A subset of sensor data No SORT • TYPRUN=FS • SENSOR=YES • FULLSTEP=YES Free space report * Note: Consumption of IMS online DB buffer resource is same as that for TYPRUN=RPT. Full monitoring of space utilization OPC area IMS Full set of sensor data SORTs SORTs SORTs • TYPRUN=RPT • SENSOR=YES Full analysis • FULLSTEP=YES report © 2013 IBM Corporation
  35. 35. Stand-alone DB Sensor • • • • • Process one DEDB in a job step Be used to collect and store sensor data for multiple areas Process both offline and online areas Report sensor data values in a report Does NOT produce Area Analysis Reports A JCL Example for Stand-alone DB Sensor //HFP EXEC PGM=HFPSENSR //STEPLIB DD DISP=SHR,DSN=FPSP.SHFPLMD0 // DD DISP=SHR,DSN=IMS.SDFSRESL // DD DISP=SHR,DSN=ITB.SHKTLOAD //IMSACB DD DISP=SHR,DSN=IMS.ACBLIB //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB1 //HFPSPRT DD SYSOUT=* //HFPPRINT DD SYSOUT=* //HFPSYSIN DD * GLOBAL DBRC=YES, ITKBSRVR=FPQSRV01, ADXCFGRP=ADSRV01 DATABASE DBD=DEDBJN23, IAREA=*ALL /* • FPA, FPO, and HPIC with SENSOR=YES can also produce this report. 35 Sensor Data Statistics Report IMS HPFP UTILITIES - DB SENSOR 56555655-W14 V1R3 DBD NAME: DEDBJN24, AREA NAME: DB24AR0 , AREA NUMBER: 1 --------------------------------------------------------============================================== AREA DEFINITION ============================================== Data Element Name Value ------------------------ -------------------DB_AREADEF_CISIZE 1,024 DB_AREADEF_UOW1 20 DB_AREADEF_UOW2 4 DB_AREADEF_ROOT1 6,800 DB_AREADEF_ROOT2 425 DB_AREADEF_NUM_SDEP_CIS 12,478 ============================================== FREE SPACE PERCENTAGE STATISTICS ============================================== Data Element Name Value ------------------------ -------------------DB_PCT_BYTES_FS_RAA 23% DB_PCT_BYTES_FS_DOVF 75% DB_PCT_BYTES_FS_IOVF 97% DB_PCT_BYTES_FS_SDEP 100% ============================================== OVERFLOW USAGE STATISTICS ============================================== Data Element Name Value ------------------------ -------------------DB_PCT_NUM_UOW_USE_DOVF 98% DB_AVG_NUM_DOVFCI_BY_UOW 1.50 DB_MAX_NUM_DOVFCI_BY_UOW 4 DB_PCT_NUM_UOW_USE_IOVF 2% DB_NUM_UOW_USE_IOVF 110 DB_AVG_NUM_IOVFCI_BY_UOW 2.20 DB_MAX_NUM_IOVFCI_BY_UOW 11 DB_MIN_NUM_IOVFCI_BY_UOW 1 DB_PCT_NUM_IOVFCI_USED 3% DB_PCT_NUM_RAPCI_OVFL 24% ... © 2013 IBM Corporation
  36. 36. Automated exception detection for sensor data • Autonomics Director provides: o Automatic and on-demand analysis of the state of your databases against user-defined threshold criteria o Generation of user-recommended actions for subject databases and database groups • Exception detection rules are almost same as those supported by thresholds for DEDB Pointer Checker and FPA ANALYZE function and OPC o o o o o 36 Free space availability Overflow status Randomizing Synonyms DB Record Length (Avg. & Max) Physical I/O © 2013 IBM Corporation
  37. 37. 1 Administration Console work flow for DEDB area space monitoring Summary Table for a DEDB Check the summary of latest DEDB area space diagnosis done by Autonomics Director. 2 Exceptions Widget Check the itemized diagnosis result for the resource. 3 Help Widget Read the help for each of the detected exceptions. 4 Reports Widget Look at FP Solution Pack utility reports to see detailed analysis of the space usage of the subject area. 5 Sensor Widget Look at the trend of specific sensor data values 37 Main dashboard for a DEDB area © 2013 IBM Corporation
  38. 38. Background of database tuning • When an exceptional state of an area is detected by space monitoring o Tuning these areas to obtain the optimal database definition parameters, then reorganize or restructure them based on the tuning results, might resolve the exception • Tuning a database often involves significant and timeconsuming database administration tasks o DBDLIB and ACBLIB need to be generated for each iterative if DBD structure change is required o Unloading and reloading the area to determine if the changes are effective is time-consuming 38 © 2013 IBM Corporation
  39. 39. Database tuning solution • Adjust the DBD parameters or/and the randomizing routine to suit the current characteristics of the database records in the area • Goal of tuning is to achieve the following results: o Shorter synonym chains with fewer unused RAPs o Higher percentage of segments in RAA BASE o Fewer physical I/Os to access a root segment and a complete database record • The following database specifications can be changed and evaluated: o o o o o 39 Randomizing module UOW parameter values ROOT parameter values Number of database areas CI sizes © 2013 IBM Corporation
  40. 40. FPA TUNE function • Simulate changes of the DBD parameters and the randomizing routine without loading the segments to allow user to select the physical database attributes that satisfy the performance and space utilization requirements o Provide JCL ease of use o Allow user to specify new DBD parameters and new randomizing routine with keywords, instead of new ACBLIB o Tune multiple areas in parallel o Provide a set of complete reports to verify effects of changes 40 © 2013 IBM Corporation
  41. 41. Database tuning simplification • Database tuning function facilitates multiple iterations of the database tuning process, thereby allowing you to select physical database attributes that meet your requirements o Even if the tuning attribute is determined once (without iteration), the steps are reduced significantly o It is unnecessary to build DBDLIB and ACBLIB for each iteration o Complete reports are provided to verify effects of changes FPSP V1.2 DBAs generate DBDLIB for the new DBD definition if DBD structure change is required. DBAs generate ACBLIB for the new DBD definition if DBD structure change is required. DBAs run FPB DEDB Tuning Aid (DEDBTA) using the above new ACBLIB and/or the new randomizer and threshold in 6 job steps. FPSP V1.3 Not necessary Not necessary DBAs run FPA Tune function with no AREA scan mode specifying the candidate of the new DBD definition and threshold to the keywords and/or the new randomizer in 1 job step. DBAs need to compare some reports of before/after the DBAs can confirm the effectiveness of the tuning based tuning to confirm the effectiveness of the tuning. on the threshold values in the area tuning summary report. 41 © 2013 IBM Corporation
  42. 42. Command set of FPA TUNE function • TUNE command o Specify general information for tuning, such as DBD name, area name, dataset mask, and so on • SIMULATE subcommand o Provide resources for tuning, such as new randomizing routine name, and so on • AREADEF subcommand o Define the new area parameters for tuning, such as CISIZE, UOW, ROOT, and so on //HFPTUNE //HFPTUNE EXEC PGM=HFPMAIN0 DISP=SHR,DSN=HPF HPF.SHFPLMD0 //STEPLIB DD DISP=SHR,DSN=HPF.SHFPLMD0 // DD DISP=SHR,DSN=IMS.SDFSRESL //IMSACB DD DISP=SHR,DSN=IMS.ACBLIB DISP=SHR,DSN=HFP HFP.DB30AR0.ADS1 //DB30AR0 DD DISP=SHR,DSN=HFP.DB30AR0.ADS1 //HFPRPTS DD SYSOUT=* //HFPPRINT DD SYSOUT=* //HFPSYSIN DD * GLOBAL SCAN=NO, DBRC=NO TUNE DBD=DEDBJN30, IAREA=DB30AR0, IAREA=DB30AR0, AREASCAN=YES SIMULATE OAREA=DB30AR0 AREADEF OAREA=DB30AR0, UOW=(28,8) THRESHOLD DOVFFS_PCT=30, IOVFFS_PCT=50 /* Sample JCL for mode 2 iteration area FPA TUNE AREASCAN=YES & TAR files specified TAR file FPA TUNE iteration tuning reports AREASCAN=NO & TAR files specified Mode 1: Create TAR files at first, then iterate tuning process using the existing TAR files 42 area FPA TUNE tuning reports AREASCAN=YES & No TAR files specified Mode 2: Iterate tuning process using the intermediate TAR files that are deleted at the end of each iteration © 2013 IBM Corporation
  44. 44. © 2013 IBM Corporation ----------009,901 009,901 ----------SDROCER BD FO REBMUN -------- ---------------- --------)XEH(ABR #KCOLB -----------------TRATS PEDS --------- --------- --------- --------388,181 388,31 000,84 000,021 009,551 009,11 000,42 000,021 --------- --------- --------- --------LATOT FVOI FVOD ESAB --------------------------------------S'IC LATOT ------------)005,0056( )005,0056( ------------=TOOR 44 List the database definitions and the number of database records in the current areas and the simulated areas ------------*)8,82( )4,42( ------------- ------- ----- -------840,2 1 0RA03BD 840,2 1 0RA03BD ------- ----- -------.ON EMAN =WOU EZIS-IC AERA AERA ecnereffid eht swohs * saera detalumis fo sdrocer esabatad fo rebmun dna noitinifed eht swohs aera hcae rof wor rewoL saera tnerruc fo sdrocer esabatad fo rebmun dna noitinifed eht swohs aera hcae rof wor reppU :noitubirtsiD droceR esabataD dna noitinifeD esabataD 15+ %89 *%74 %05 TCP_SFFVOI 13+ %75 *%62 %03 TCP_SFFVOD ------------------ ---------------- ---------------- ---------------- -----------------------ecnereffiD eulaV detalumiS eulaV tnerruC eulaV dlohserhT drowyeK -----------------------------------------------------------------------------------------------0RA03BD :EMAN AERA List the threshold value for each factor that is evaluated (supplied through the THRESHOLD subcommand keywords) .noitpecxe dlohserht fo tniop eht swohS :* :scitsitatS aerA ------------------------------------------------------------------------------------------------------------------*** detceted saw noitpecxe oN *** ------------------------------------------------------------------------------------------------------------------saera detalumis morf snoitpecxE ------------------------------------------------------------------------------------------------------------------.0RA03BD aera ni dlohserht a woleb llef FVOI ni ecaps eerf fo egatnecrep ehT .0RA03BD aera ni dlohserht a woleb llef FVOD ni ecaps eerf fo egatnecrep ehT ------------------------------------------------------------------------------------------------------------------saera tnerruc morf snoitpecxE List the threshold exceptions that are detected in the current areas and the simulated areas 84:22:61 31-90-3102 1 :EGAP :snoitpecxE dlohserhT 44CDHFBD : REZIMDNAR BILBCA.SMI :EMANSD BILBCA :SECRUOSER DETALUMIS 44CDHFBD : REZIMDNAR BCASMI :EMANDD BILBCA :SECRUOSER TNERRUC )deificeps era sdnammocbus FEDAERA( ERUTCURTSER :EDOM NOITALUMIS 03NJBDED :EMAN DBD 3R1V 41W-5565 ENUT APF - SEITILITU PFPH SMI "tropeR yrammuS gninuT aerA" Area Tuning Summary Report
  45. 45. Other reports generated by FPA TUNE function • Provide two sets of the comprehensive analysis reports to evaluate the effects of tuning o The reports beginning with TUNED in the title contain information that obtained from the simulated areas o The reports without beginning with TUNED in the title information that obtained from the current areas o The following reports are generated: • • • • • • • 45 (TUNED) DEDB Area Analysis report (TUNED) Process Summary of Analysis report (TUNED) Segment Length Distribution report (TUNED) UOW report (TUNED) UOW Exception report (TUNED) Largest Database Records report (TUNED) Consolidated Largest Database Records report © 2013 IBM Corporation
  46. 46. More information in area analysis report • Solution o Add new report header lines that indicate: • Input data set name • Input data set type (and the timestamp for image copy) o Add SDEP information in the following reports: • SDEP LB/LE information in Freespace Analysis report • Number of occurrences in Segment Placement Analysis report o Optionally add segment prefix length information in the DB Record Profile Analysis report o Optionally add Segment Length Distribution report • Customer benefit o o o o 46 Sure identification of the source of statistics data Comprehensive reporting of SDEP space usage information Better estimation of space use by each segment type Quick identification of segment length © 2013 IBM Corporation
  47. 47. More information in area analysis report (cont.) Sample 1: New report header lines Input: Area data set Input: A batch image copy data set Input: A concurrent image copy data set 47 47 © 2013 IBM Corporation
  48. 48. More information in area analysis report (cont.) Sample 2: SDEP information SDEP LB/LE information Number of SDEP segment occurrences 48 48 © 2013 IBM Corporation
  49. 49. More information in area analysis report (cont.) Sample 3: Segment prefix length information These information are reported when SEGLSTAT=DETAILED is specified. (The default is SEGLSTAT=COMPACT.) 49 49 © 2013 IBM Corporation
  50. 50. More information in area analysis report (cont.) Sample 4: Segment Length Distribution report This report is generated when SEGLDIST=YES is specified. (The default is SEGLDIST=NO.) 50 50 © 2013 IBM Corporation
  51. 51. Database-level Largest DB Records Report • Consolidate and sort the statistics data for large database records and create a single report across entire database o Provide a second parameter for the LARGEREC_COUNT keyword to allow user to specify the number of reporting entries o Consolidated Largest Database Records Report • Be helpful for quick identification of large database records in a database that has a large number of areas 51 Largest Database Records Report for each area area 1 area 2 area 4 area 5 area 3 … arean Merge and sort raw information internally within FPA/FPO “Consolidated Largest Database Records Report” © 2013 IBM Corporation
  52. 52. Consolidated Largest Database Records report A Sample Report IMS HPFP UTILITIES 56555655-W14 V1R2 DBDNAME: DEDB0003 TOTAL AREA#: "CONSOLIDATED LARGEST DATABASE RECORDS REPORT" KEY LENGTH: 10 RECORD COUNT: PAGE: 1 2012-0917:07:41 2012-09-14 17:07:41 30 19 DB230 230014 AREA: DB23AR1 DB23AR2 DB23AR3 DB23AR4 DB23AR5 DB230006 DB230007 DB230008 DB230009 DB230010 DB230011 DB230012 DB230013 DB230014 DB230015 DB230016 DB230017 DB230018 DB230019 RECORD SIZE ----------4,012 4,012 4,012 4,012 4,012 4,012 3,760 3,760 3,760 3,760 3,760 3,760 3,510 3,510 3,510 . . . # OF SEG'S ---------24 13 24 13 24 13 14 11 14 11 14 11 19 15 19 . . . AREANAME -------DB23AR1 DB23AR1 DB23AR4 DB23AR4 DB23AR5 DB23AR5 DB23AR1 DB23AR1 DB23AR4 DB23AR4 DB23AR5 DB23AR5 DB23AR1 DB23AR1 DB23AR4 . . . AREA# ----1 1 4 4 5 5 1 1 4 4 5 5 1 1 4 . . . ROOT RBA -------00070808 00015408 00008744 0002FBA6 00070808 00015408 00023AC4 000276C4 000532C4 0005CAC4 00023AC4 000276C4 0000A008 0000DC08 00015BA6 . . . RAP# ----9 48 9 48 9 48 84 93 84 93 84 93 21 30 21 . . . RAP RBA -------00004400 00014800 00008000 0002F800 00004400 00014800 00023800 00027400 00053000 0005C800 00023800 00027400 00009400 0000D000 00015800 . . . ROOT SEGMENT SEQUENCE FIELD (HEX) --------------------------------F1F0F0F0 F0F0F0F1 F0F1 F1F0F0F0 F0F0F0F4 F9F1 F4F0F0F0 F0F0F0F1 F0F1 F4F0F0F0 F0F0F0F4 F9F1 F5F0F0F0 F0F0F0F1 F0F1 F5F0F0F0 F0F0F0F4 F9F1 F1F0F0F0 F0F0F0F8 F5F1 F1F0F0F0 F0F0F0F9 F4F1 F4F0F0F0 F0F0F0F8 F5F1 F4F0F0F0 F0F0F0F9 F4F1 F5F0F0F0 F0F0F0F8 F5F1 F5F0F0F0 F0F0F0F9 F4F1 F1F0F0F0 F0F0F0F2 F2F1 F1F0F0F0 F0F0F0F3 F1F1 F4F0F0F0 F0F0F0F2 F2F1 . . . (CHARACTER) *----------------* ----------------* --*1000000101 * *1000000491 * *4000000101 * *4000000491 * *5000000101 * *5000000491 * *1000000851 * *1000000941 * *4000000851 * *4000000941 * *5000000851 * *5000000941 * *1000000221 * *1000000311 * *4000000221 * . . . 52 52 © 2013 IBM Corporation
  53. 53. Make space management simpler and easier 53 © 2013 IBM Corporation
  54. 54. Background and challenge of space management • When an exceptional state of an area is detected by space monitoring, reorganize free space or extend the size of areas based on the tuning results o If the database record profile and the number of database records in the area are relatively unchanged, reorganizing free space might resolve the exception o If the available free space fall below the thresholds, extending the size of the DEDB areas might resolve the exception • In operating environments that cannot be stopped, unload/reload cannot be used to increase free space in areas, and ineffective reorganization causes unnecessary impact upon online performance 54 © 2013 IBM Corporation
  55. 55. Space management solution • DEDB area expansion o Dynamically extend the IOVF portion and the SDEP portion of DEDB area without take it offline o Support volume management in SMS environment • DEDB area reorganization o Extend the capability of High-Speed DEDB Direct Reorganization utility (DBFUHDR0) o Evaluate UOWs in DEDB areas, rank the UOWs, and reorganize only highly ranked UOWs to reduce the volume of database update logs o Collect statistics and generate area analysis reports of the reorganized area o Take image copies of the reorganized area 55 © 2013 IBM Corporation
  56. 56. Online area expansion • Increase IOVF and/or SDEP parts size while allowing online access to the area that is being extended o Use DEDB create utility (DBFUMRI0) o Process a single area in one step • Two expansion types Shadow expansion Original expansion IOVF ADS1 avail IOVF OAE IMS ADS1 1 avail IOVF IOVF OAE ADS1 ADS2 IMS avail avail IOVF IOVF IOVF copy back ADS2 ADS2 Rename (TOSI) 56 unavail ADS1 unavail © 2013 IBM Corporation
  57. 57. Simpler parameter specification • The extended size for SDEP portion can be specified with a specific number or by a percentage of CIs within the total number of CIs defined in the SDEP portion o SDEP_CI=xx or xx% • The extended size for IOVF portion can be specified with a specific number or by a percentage of UOWs within the total number of UOWs defined in the IOVF portion o IOVF_UOW=xx or xx% • These specifications allow OAE to be executed multiple times without JCL modification 57 © 2013 IBM Corporation
  58. 58. Volume management support • Provide a keyword VOLCNT to allow user to specify the maximum number of volumes for the extended area data set o OAE uses volume where space is available o The VOLCNT keyword is supported only for SMS environment • Make OAE execution a success although some volumes are out of space 58 © 2013 IBM Corporation
  59. 59. Online free space reorganization • Save online system resource and eliminates unnecessary reorganizations by identifying only those databases that need to be reorganized based on user-defined criteria ROOT ROOT Base Base ROOT DDEP DDEP ROOT DDEP DDEP ROOT DDEP DDEP ROOT ROOT DDEP OER ROOT DDEP DDEP IMS DDEP DOVF DDEP DOVF DDEP UOW 59 IOVF UOW IOVF © 2013 IBM Corporation
  60. 60. Free space reorganization acceleration • Minimize the impact on other critical applications accessing the area, and maximize the effect of reorganization o Reorganize only the subset of the most fragmented UOWs o Provide a keyword REORGLIM to allow user to specify an allowable maximum number of UOWs to be reorganized • Internally invoke OPC after reorganization to produce reports to verify the effects of reorganization, to take image copy of the reorganized areas 60 © 2013 IBM Corporation
  61. 61. Big picture of Fast Path Solution Pack and Tools Base • Tune database IMS FP Solution Pack V1.3 Tune functions Extend function • Extend database 4 Reorganize functions • Reorganize free space 5 TSO Analyze & DB Sensor functions 1 • Analyze and collect sensor data • Change database attributes Restructure functions 2 • Notify exceptions 3 • Notify sensor data • Store sensor data (and reports) z/OS Env. Autonomics Director IBM Tools Base V1.4 Open Env. Administration Console (+ Policy Services) Autonomics Director Server • Policy-based DEDB Area Diagnosis 61 IMS Tools Knowledge Base Repository Server Distributed Access Infrastructure Repository © 2013 IBM Corporation
  62. 62. DEDB utility solutions Solution Reorganizing Function FPA (offline) FPB (offline) Unload UNLOAD DEDB Unload Reload RELOAD DEDB Reload Change CHANGE Reorganize REORG Online Data Extract (ODE) Online Expert Reorganization (OER) Extend Online Area Extender (OAE) Build FP Secondary Index INDEXBLD Integrity verification and analysis ANALYZE -DEDB AREAs -Secondary Index DBs Print DMAC information DMACPRT Extracting Extract segments EXTRACT Unload/Reload user exit routines Tuning Tuning Aid TUNE DEDB Tuning Aid Other Initialize DEDB areas INITIALIZE Analyzing 62 FPO (online) DEDB Pointer Checker - DEDB AREAs Online Pointer Checker (OPC) -DEDB AREAs Online DMAC Print (ODM) Online Data Extract (ODE) © 2013 IBM Corporation
  63. 63. Summary IMS Tools from IBM provide, integrated, easy-to-use solutions that fit your company’s needs IBM is dedicated to the continued success and support of IMS and the mainframe. We’re invested for the long term, right beside you. We are continuing to invest in our IMS Tools technology and have a vision for our IMS Tools that centers around autonomic computing 63 63 © 2013 IBM Corporation