IMS Tools - Smarter Reorgs

1,024 views

Published on

IMS Tools – Smarter Reorgs
Fix the Right Problem and Reorg it and You’re Done

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,024
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
36
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

IMS Tools - Smarter Reorgs

  1. 1. IBM Software Group | Information ManagementIMS Tools – Smarter ReorgsFix the Right Problem and Reorg it and You’re DoneIMS Regional User’s Group Meeting Janet LeBlanc leblancj@ca.ibm.com Silicon Valley Lab, San Jose, CA © 2011 IBM Corporation
  2. 2. IBM Software GroupAgenda IMS™ database reorganization today IMS Database Reorganization Expert overview Reorganization Expert features Implementation Considerations Summary2 © 2011 IBM Corporation
  3. 3. IBM Software Group Disclaimer © Copyright IBM Corporation 2011. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com, IMS, and z/OS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml3 © 2011 IBM Corporation
  4. 4. IBM Software Group IMS Database Reorganization Today Background: motivation for developing this tool Background: motivation for developing this tool4 © 2011 IBM Corporation
  5. 5. IBM Software GroupIMS Database Reorganization Today Typical reorganization processes and their planning Free space defrag – Unload and reload without DBD change – Regularly scheduled: weekly, biweekly, monthly, etc. Database structure change – Performed for better space utilization and performance – Needs periodic collection and analysis of database statistics – Needs proactive planning Block Size Change Free Space Defragging (Compaction) Various Types of DB Definition Changes Repartitioning and/or Multiple DSGs5 © 2011 IBM Corporation
  6. 6. IBM Software GroupIMS Database Reorganization Today Workload for data management increases as your business grows Increasing volume of repetitive tasks such as free space defrag – Processing efficiency is becoming important More frequent needs for database logical/physical structure changes – Deep IMS DBA skills are needed for these complex tasks Each may be simple, Each may be simple, IMS DBAbut the total work volumebut the total work volume is getting larger is getting larger Diagnose Databases Diagnose Databases Plan/Do Actions Plan/Do Actions Check Key Diagnosis Result Diagnosis Result Simple Repetitive Actions Processing efficiency Processing efficiency Stats Items Exception 1 (e.g. free space defrag) is becoming important is becoming importantPointer .Checker . Exception 2 . Statistics ValuesReports Deep IMS DBA Deep IMS DBA % of segment splits Exception 3 Complex Actions skills are needed skills are needed Avg nbr of FSEs (e.g. Structure Change) in planning actions Exception 4 in planning actions % of max. dataset size . . Number of extents . When can When can . it be done? How to do it How to do it . . it be done? this time? this time?6 © 2011 IBM Corporation
  7. 7. IBM Software GroupObjective of IMS Database Reorganization Expert Help customers perform these tasks more efficiently with decreasing number of skilled DBAs. Each may be simple, Each may be simple, IMS DBAbut the total work volumebut the total work volume is getting larger is getting larger Diagnose Databases Diagnose Databases Plan/Do Actions Plan/Do Actions Check Key Diagnosis Result Diagnosis Result Simple Repetitive Actions Processing efficiency Processing efficiency Stats Items Exception 1 (e.g. free space defrag) is becoming important is becoming importantPointer .Checker . Exception 2 . Statistics ValuesReports Deep IMS DBA Deep IMS DBA % of segment splits Exception 3 Complex Actions skills are needed skills are needed Avg nbr of FSEs (e.g. Structure Change) in planning actions Exception 4 in planning actions % of max. dataset size . . Number of extents . When can When can . it be done? How to do it How to do it . . it be done? this time? this time?7 © 2011 IBM Corporation
  8. 8. IBM Software Group IMS Database Reorganization Expert Helps customers: 1. Document DBA knowledge for regular repetitive tasks 2. Automate those tasks using the documented knowledge 3. Keep the knowledge and results of operations in one place8 © 2011 IBM Corporation
  9. 9. IBM Software GroupIMS Database Reorganization Expert– 3 objectives 1. Policy-based database space management Document DBA knowledge Document DBA knowledge Exceptional states of database space usage are named Rules for exception detection are documented in a policy Target/method of exception notification are documented in the policy 2. Policy-based automated operation Automate repetitive tasks Automate repetitive tasks The policy is used to detect reorganization need Free space reorg. can be done conditionally based on the policy Effectiveness of each free space reorg. can be evaluated 3. Central management of information and data Keep info/data in one place Keep info/data in one place Policies are kept and managed in a repository Statistics data needed for exception detection are kept in a repository History of exceptions and reorganizations are kept also in a repository9 © 2011 IBM Corporation
  10. 10. IBM Software GroupIMS Database Reorganization Expert– Follow-on product of IMS Parallel Reorganization New Smart Reorg Utility extends Parallel Reorg Driver capability to provide smarter way of reorganization. Smart Reorg 5 Major Features A Smart Reorg Utility Job 1. Policy definition Evaluate 3 z/OS® Data with 5 Notify Exceptions consoles Policy 4 (and/or 2. Statistics data collection TSO users) Store 2 5 Store 3. Exception detection Stats Report Define & 1 Store XCF Policies 4. Conditional reorganization XCF IMS Tools ISPF 5. Exception reporting Knowledge Base XCF Search & Server Read Reports ISPF 1 3 2 5 Policy Sensor Data Report Repository Repository Repository All information and data are managed by IMS Tools Knowledge Base10 © 2011 IBM Corporation
  11. 11. IBM Software GroupIMS Tools Knowledge Base– Central management of information and data Repository Management User Interface IMS Tools Knowledge Base ISPF Infrastructure for centrally managed repositories for IBM IMS Tools Provided at no charge as part of XCF IMS Tools Base A prerequisite of IMS DB Reorg IMS Tools Enables Sysplex- wide access to Expert Knowledge Base repositories from Server ISPF UI and Tools. Managed information and data It is important in smarter database management to keep and manage relevant information and data in Managed Various Repositories one place11 © 2011 IBM Corporation
  12. 12. IBM Software GroupIMS Tools Knowledge Base– Information and data for smarter reorganization Policy UI Report UI Smart Reorg Smart Reorg Utility Batch Uses and/or generates the following ISPF ISPF Job information and data Policy API Report API Policy API Sensor API Report API Repository Client API Repository Client API Repository Client API – Reorganization Policies – Database Sensor Data (database space statistics) Retention of – Database Diagnosis Reports IMS Tools sensor data Knowledge Base and reports are ISPF Policy User Interface managed by Server IMS Tools KB Provided by IMS Tools Base – Policy Services Used to define and manage policies Input Sensor Data Output ISPF Report User Interface Repository Repository Repository Used to search and view Reorg Sensor Diagnosis reports of various IMS Tools Policies Data Reports12 © 2011 IBM Corporation
  13. 13. IBM Software Group Policy-based Database Space Management Decision criteria and recommended actions are documented in a policy, and policies are kept in a centrally managed repository.13 © 2011 IBM Corporation
  14. 14. IBM Software GroupPolicy-based Database Space Management– A policy describes decision criteria A Smart Reorg Utility Job Reorganization policy Describes Select and use a reorganization policy – Criteria for exception detection – Action for detected exceptions Define & – Destinations and methods of Store exception notification Policies IMS Tools Can be defined Knowledge Base – For a database type Server Policy UI – For an individual database (ISPF) – For a group of databases Can be associated – With one or more Smart Reorg Policy jobs through utility control Repository statements14 © 2011 IBM Corporation
  15. 15. IBM Software GroupPolicy-based Database Space Management– Structure of a policy description Policy Major components of a policy Rule Set Rule Rule Rules that detect exceptions Rule Rule An Action List Entry Exception-based action list Rule Rule Action List Exception Action Action Class+Level Action A Rule Action Has two elements: Rule – Condition Condition Exception – Exception Action List Condition Exception Defines an exception-to-action Condition Description Exception Class mapping Threshold set Severity Level Message Text15 © 2011 IBM Corporation
  16. 16. IBM Software GroupPolicy-based Database Space Management– A rule describes an exception detection criteria A rule condition is defined as a pair of: An evaluation condition Evaluation Condition – Describes what data are evaluated and how Name of Threshold Set A set of threshold values – Customizable (( DB_PCT_OF_MAX_DS_SIZE DB_PCT_OF_MAX_DS_SIZE &1 )) AND (( DB_PCT_BYTES_FREE_SPACE &1 AND DB_PCT_BYTES_FREE_SPACE &2 )) &2 in one of data sets that compose the database in one of data sets that compose the database Sensor Data to Evaluate DB_PCT_OF_MAX_DS_SIZE The percentage of allocated bytes Threshold Values (bytes for High Allocated RBA) compared to the maximum size (4 GB or 8 GB). DB_PCT_BYTES_FREE_SPACE The percentage of bytes of total free spaces to the total used bytes for the data set.16 © 2011 IBM Corporation
  17. 17. IBM Software GroupPolicy-based Database Space Management– Describing an exception Exception Class An Example of Exception Class Exception Class: Exception Class: A specific category of database FRAGMENTED_FREE_SPACES FRAGMENTED_FREE_SPACES states being monitored * Name of the rule that detects this exception: * Name of the rule that detects this exception: IBM.FRAGMENTATION.10 IBM.FRAGMENTATION.10 Defined by IBM Exception Severity Level An Exception Levels The severity of an exception Class CRITICAL CRITICAL detected by the Rule Condition Severity Level SEVERE SEVERE Can be chosen from three levels Message Text WARNING WARNING Exception Message Text The text describing the exception An Example of Exception Message Can be modified “The fragmentation of free space in “The fragmentation of free space in %RESOURCE% has increased” %RESOURCE% has increased” * The symbol %RESOURCE% is replaced by a DBD * The symbol %RESOURCE% is replaced by a DBD name or a partition name. name or a partition name.17 © 2011 IBM Corporation
  18. 18. IBM Software GroupPolicy-based Database Space Management– Describing an action Exception-to-Action mapping Reorganization (free space defrag through unload and reload without Each rule is associated with DBD change) is recommended an exception class The threshold set selected for the rule is associated with a An Example severity level of the exception Threshold Exception Class Action Set Name + Severity Level An action is associated with a “HIGH” CRITICAL REORG pair of an exception class and “MED” SEVERE MESSAGE its severity level “LOW” WARNING Policy Rule Set Rule Rule Rule Rule Rule Condition Exception Rule Rule An Action List Entry Action List Exception Alerting the user of the exception Formula Exception Class Action Class +Level Action for preparation for user actions, or Severity Level ActionThreshold Set Action for immediate user actions Message Text18 © 2011 IBM Corporation
  19. 19. IBM Software GroupPolicy-based Database Space Management– IBM-provided templates IBM provides policies and rules A PDS A PDS Provided as a PDS member Exported Exported package package Installed into ITKB Input Repository as a set of policy/rule “templates” Export Import Customization Install Policy Services UI The installed policy/rule templates Installation package Customize can be copied and customized Policy Services Import and export Template Library (PDS) IMS Tools KB The copied/customized templates Server can be exported Policy/Rule – For importing them into Templates another environment (installed or customized) – For backup IMS Tools KB Input Repository19 © 2011 IBM Corporation
  20. 20. IBM Software Group Policy-based Automated Operation Smart Reorg Utility provides capability of policy-based automated exception detection and conditional reorganization.20 © 2011 IBM Corporation
  21. 21. IBM Software GroupPolicy-based Automated Operation– Using Smart Reorg Utility Four features provided by Smart Reorg Utility 1. Statistics data collection 2. Exception detection 3. Conditional reorganization 4. Exception reporting Three modes of execution of Smart Reorg Utility 1. Diagnosis Mode 2. Conditional Reorganization Mode 3. Unconditional Reorganization Mode21 © 2011 IBM Corporation
  22. 22. IBM Software GroupUsing Smart Reorg Utility– Statistics Data Collection Sensor data for an IMS full-function database A set of values of database statistics data elements at a specific time Used to detect exceptions by evaluating them with a reorg. policy Data elements supported in the first release Total of about 60 data elements: – Database Record Statistics Per database or HALDB partition – Randomizer Statistics Per HDAM or PHDAM partition – Volume/Extents Statistics – Data Set Space Usage Statistics Per data set – IMS Space Utilization Statistics – HISAM/SHISAM Statistics Complete list of data elements Can be found in Policy Services User’s Guide (SC19-2718)22 © 2011 IBM Corporation
  23. 23. IBM Software GroupUsing Smart Reorg Utility– Statistics Data Collection… DB Sensor Is a sensor data collector for IMS full- A Smart Reorg Utility Job function databases DB Sensor DB Sensor Is integrated in Smart Reorg Utility Can collect sensor data from a database or HALDB partition while it is online – One non-HALDB database or IMS Tools Knowledge Base HALDB partition at a time Server Stores collected data in IMS Tools KB Sensor data retention is configurable, and is Sensor Data Repository managed by IMS Tools KB. Is much faster than HASH Check of High Sensor Data Performance Pointer Checker Repository – DB Sensor bypasses pointer DB Sensor Data for FF Databases integrity check23 © 2011 IBM Corporation
  24. 24. IBM Software GroupUsing Smart Reorg Utility– 3 modes of execution Diagnosis Mode Sensor data is collected and evaluated by a policy to detect exceptions A diagnosis report can be stored in IMS Tools KB Output Repository Existence of an exception can be notified in various ways Conditional Reorganization Mode Same as Diagnosis Mode But, performs reorganization when it is recommended by the diagnosis – The reorganized database is diagnosed again to see effectiveness of the reorganization and remaining exceptions Unconditional Reorganization Mode Always performs reorganization regardless of the database status24 © 2011 IBM Corporation
  25. 25. IBM Software GroupUsing Smart Reorg Utility– Unconditional Reorganization Mode Single Step Parallel Execution of Reorganization Utility Tasks Same as Parallel Reorg Driver of IMS Parallel Reorganization 3.2 Smart Reorg Driver (IPR Driver) Parallel Reorganization Service (used only when reorg needed) CHANGE.DB AUTH READOFF Unload Task DISPOLDDS Processing DBDS Name Swapping Making DB Offline NOTIFY.REORG NOTIFY.IC/UIC Restarting DB Reload Task Image Copy Task (can include Pointer Check) Index Builder Task Index Builder’s SORT Address Space Index Builder’s SORT Address Space /DBR DB Index Builder’s /STA DB Sub Address Spaces IC Renamed to Shadow DBDS Renamed toto Renamed Shadow DBDS Renamed to original name original name original name original name Original DBDS Original DBDS Original DBDS Original DBDS IC Renamed to Original Index IC Renamed toto Renamed Original Index Original Index Original Index Original Index Index Index Shadow original toto Renamed Renamedname Renamedname original to Original Index original name original name Index original name original name Online DB is Off-lined Online25 © 2011 IBM Corporation
  26. 26. IBM Software GroupUsing Smart Reorg Utility– Conditional Reorganization Mode Basic Scenario of Conditional Reorganization Mode Step 1: Sensor data are collected and stored in the Sensor Data Repository Step 2: The sensor data are evaluated with the specified policy Step 3: Reorganization is performed if it was recommended in Step 2 – Sensor data of the reorganized database are stored – New sensor data is evaluated with the same policy again Step 4: A Diagnosis Report of the evaluation(s) is stored and the existence of remaining exceptions, if any, can be notified Step 5: The Diagnosis Report is reviewed by DBA and necessity of further action (e.g., planning for database tuning) is determined26 © 2011 IBM Corporation
  27. 27. IBM Software GroupUsing Smart Reorg Utility– Benefits of Conditional Reorganization A database can be reorganized only when it is really needed The Diagnosis Report provides an evidence of the necessity of reorganization and effectiveness of the reorganizationReorganization needReorganization need is checked is checked Smart Reorg Driver (IPR Driver) Parallel Reorganization Service (used only when reorg needed) Diagnosis of Online DB CHANGE.DB AUTH READOFF Conditional Reorg Unload Task Sensor data are Post Processing Sensor data are DISPOLDDS Processing DBDS Name Swapping collected during collected during Making DB Offline NOTIFY.REORG NOTIFY.IC/UIC Reorg Reload Restarting DB Reload Task Reorg Reload Image Copy Task (can include Pointer Check) Index Builder Task Index Builder’s SORT Address Space /DBR DB Index Builder’s SORT Address Space Index Builder’s /STA DB Detected or remaining Sub Address Spaces exceptions are notified (based on the policy) IC Renamed to Shadow DBDS Renamed toto Renamed Shadow DBDS Renamed to original name original name original name original name Original DBDS Original DBDS z/OS consoles Original DBDS Original DBDS (and/or TSO users) IC Renamed to Original Index IC Renamed toto Renamed Original Index Original Index Original Index Original Index Index Index Shadow original toto Renamed Renamedname Renamedname original to Original Index original name original name Index original name original name Online DB is Off-lined Online27 © 2011 IBM Corporation
  28. 28. IBM Software GroupUsing Smart Reorg Utility– Exception detection in Diagnosis Mode Basic Scenario of Diagnosis Mode Step 1 & 2: Same as those in Conditional Reorganization Mode Step 3: A Diagnosis Report of the evaluation is stored in the Output Repository and existence of an exception can be notified Step 4: The Diagnosis Report is reviewed by DBA and actions are determined. Or, the notification is used to control succeeding batch jobs or jobsteps. A Smart Reorg Utility Jobstep Evaluate 2 Statistics Data Collection Data with 3 Notify Exceptions z/OS consoles Policy (and/or TSO users) Exception Detection 1 3 Store Store Report Stats Exception Reporting XCF DBA reads Diagnosis Reports IMS Tools XCF and determines actions Knowledge Base Server Input Sensor Data Output Repository Repository Repository28 © 2011 IBM Corporation
  29. 29. IBM Software GroupUsing Smart Reorg Utility– Various ways of notifying exceptions Method 1: Notification by a WTO message You can add z/OS consoles as the notification targets – Including E-MCS consoles This could be used for message-based automation Method 2: Notification by Smart Reorg jobstep return code You can change the return code when a critical exception is detected This could be used to control the execution of succeeding job steps Method 3: Notification to TSO users You can add TSO user IDs as the notification targets Notification messages will be sent by using the TSO SEND command29 © 2011 IBM Corporation
  30. 30. IBM Software GroupNotification Lists Exception notification is made from A Smart Reorg Job WTO CONSOLE1 each Smart Reorg job An Policy Evaluation SEND Notification is based on the Summary Message TSOUSER1 SEND Notification Lists specified in the Selected TSOUSER2 SEND policy used TSOUSER3 A Notification List can be defind for Policy Rule Set a group of consoles and/or TSO Rule Rule user IDs Rule Rule Rule Rule Notification Lists are stored in the Action List ITKB Input Repository and can be Action referred to from policies by their Action Action LISTA names CONSOLE1 Notification Lists TSOUSER1 Notification Lists can be created, LISTA TSOUSER2 exported, and imported by using the LISTB Policy Services UI LISTB TSOUSER2 TSOUSER330 © 2011 IBM Corporation
  31. 31. IBM Software GroupUsing Smart Reorg Utility– Exception reporting by Diagnosis Report Information provided by Diagnosis Report Summary of Policy Evaluation – Policy that was applied to the database – Reorganization need – Result of policy evaluation – Result of policy evaluation after reorganization • if reorganization was performed – An evaluation summary message Sensor data values before and after reorganization – And their differences31 © 2011 IBM Corporation
  32. 32. IBM Software Group IMS DB Reorg Expert - V4R1 Database IMS DB Reorg Expert - V4R1 Database Diagnosis Report Diagnosis Report Page: Page: 2 2 5655-S35 5655-S35 Date: 09/14/2010 Date: 09/14/2010 Time: 12.05.14 Time: 12.05.14 Summary of Policy Evaluation (DBD: BKDB Summary of Policy Evaluation (DBD: BKDB ) ) ----------------------------------------------------------------- ----------------------------------------------------------------- Name of Policy Applied............ SYS.DBDTYPE.HIDAM Name of Policy Applied............ SYS.DBDTYPE.HIDAM Policy used for this job Policy Locale..................... Global Policy Locale..................... Global Reorganization Need............... Yes Reorganization Need............... Yes Reorganization was Exceptions before Reorganization Exceptions before Reorganization recommended -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- The number of available extents for a data set of BKDB is small The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 IBM.DBDS_EXTENTS.10 Rule: G:IBM.DBDS_EXTENTS.10 IBM.DBDS_EXTENTS.10 Threshold Set: HIGH Threshold Set: HIGH The size of a data set in BKDB, which still has a certain amount of free space, The size of a data set in BKDB, which still has a certain amount of free space, has increased has increased Class: GROWING_DBDS_WITH_FREE_SPACES Class: GROWING_DBDS_WITH_FREE_SPACES Level: CRITICAL Level: CRITICAL -> REORG -> REORG Rule: G:IBM.DBDS_GROWTH.20 IBM.DBDS_GROWTH.20 Rule: G:IBM.DBDS_GROWTH.20 IBM.DBDS_GROWTH.20 Threshold Set: HIGH Threshold Set: HIGH The fragmentation of free space in BKDB has increased The fragmentation of free space in BKDB has increased Class: FRAGMENTED_FREE_SPACES Class: FRAGMENTED_FREE_SPACES Level: CRITICAL Level: CRITICAL -> REORG -> REORG Rule: G:IBM.FRAGMENTATION.10 IBM.FRAGMENTATION.10 Rule: G:IBM.FRAGMENTATION.10 IBM.FRAGMENTATION.10 Threshold Set: HIGH Threshold Set: HIGH A data set of BKDB has many pointers that point to other bocks or CIs A data set of BKDB has many pointers that point to other bocks or CIs or or Class: EXCESSIVE_SEGMENT_SCATTERING Class: EXCESSIVE_SEGMENT_SCATTERING Level: SEVERE Level: SEVERE Rule: G:IBM.SEGM_SPREAD.10 IBM.SEGM_SPREAD.10 Rule: G:IBM.SEGM_SPREAD.10 IBM.SEGM_SPREAD.10 Threshold Set: MED Threshold Set: MED -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Exceptions after Reorganization Exceptions after Reorganization -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- BSN2800I GENERAL STATUS: RESOURCE=BKDB ACTION_NAME=REORG BSN2800I GENERAL STATUS: RESOURCE=BKDB ACTION_NAME=REORG EXECUTION_STATUS=SUCCESSFUL EXECUTION_STATUS=SUCCESSFUL The number of available extents for a data set of BKDB is small The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Class: DATA_SET_EXTENTS_AVAILABILITY Level: WARNING Level: WARNING Rule: G:IBM.DBDS_EXTENTS.10 IBM.DBDS_EXTENTS.10 Rule: G:IBM.DBDS_EXTENTS.10 IBM.DBDS_EXTENTS.10 Threshold Set: LOW Threshold Set: LOW -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Summary of Policy Evaluation and Action: Summary of Policy Evaluation and Action: Policy Evaluation -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- BBE2901I BKDB BBE2901I BKDB IN RECONID=RECON207 HAS BEEN REORGANIZED, BUT SOME WARNING IN RECONID=RECON207 HAS BEEN REORGANIZED, BUT SOME WARNING Summary Message EXCEPTIONS REMAIN. EXCEPTIONS REMAIN. -------------------------------------------------------------------------------- --------------------------------------------------------------------------------32 © 2011 IBM Corporation
  33. 33. IBM Software Group IMS DB Reorg Expert - V4R1 IMS DB Reorg Expert - V4R1 Database Diagnosis Report Database Diagnosis Report Page: Page: 5 5 5655-S35 5655-S35 Date: 09/14/2010 Date: 09/14/2010 Time: 12.05.14 Time: 12.05.14 Data Set Statistics (DBD: BKDB Data Set Statistics (DBD: BKDB , DSG: 01) , DSG: 01) -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- =================================== =================================== Data Set Space Usage Statistics Data Set Space Usage Statistics =================================== =================================== Note: The mark * in column P means that the data element is used in the policy. Note: The mark * in column P means that the data element is used in the policy. Data Element Name Data Element Name P P Before Reorg Before Reorg After Reorg After Reorg Difference Difference ------------------------ ------------------------ - - ---------------- ---------------- ------------------ ---------------- ---------------- ------------------ DB_BLOCK_SIZE DB_BLOCK_SIZE 4,096 4,096 4,096 4,096 0 0 DB_NUM_DBDS_BLOCKS DB_NUM_DBDS_BLOCKS 999,179 999,179 673,780 673,780 -325,399 -325,399 DB_MAX_DS_SIZE DB_MAX_DS_SIZE 4G 4G 4G 4G 0G 0G DB_PCT_OF_MAX_DS_SIZE DB_PCT_OF_MAX_DS_SIZE * * 96% 96% 65% 65% -31 -31 DB_RBA_HIGH_ALLOC DB_RBA_HIGH_ALLOC 4,128,768,000 4,128,768,000 2,801,664,000 2,801,664,000 -1,327,104,000 -1,327,104,000 DB_RBA_HIGH_USED DB_RBA_HIGH_USED 4,092,641,280 4,092,641,280 2,759,806,976 2,759,806,976 -1,332,834,304 -1,332,834,30433 © 2011 IBM Corporation
  34. 34. IBM Software Group IMS DB Reorg Expert - V4R1 IMS DB Reorg Expert - V4R1 Database Diagnosis Report Database Diagnosis Report Page: Page: 6 6 5655-S35 5655-S35 Date: 09/14/2010 Date: 09/14/2010 Time: 12.05.14 Time: 12.05.14 Data Set Statistics (DBD: BKDB Data Set Statistics (DBD: BKDB , DSG: 01) , DSG: 01) -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- ==================================== ==================================== IMS Space Utilization Statistics IMS Space Utilization Statistics ==================================== ==================================== Note: The mark * in column P means that the data element is used in the policy. Note: The mark * in column P means that the data element is used in the policy. Data Element Name Data Element Name P P Before Reorg Before Reorg After Reorg After Reorg Difference Difference ------------------------ ------------------------ - - ---------------- ---------------- ------------------ ---------------- ---------------- ------------------ DB_BYTES_SEG DB_BYTES_SEG 2,465,800,000 2,465,800,000 2,465,800,000 2,465,800,000 0 0 DB_BYTES_FREE_SPACE DB_BYTES_FREE_SPACE 1,610,723,680 1,610,723,680 283,136,678 283,136,678 -1,327,587,002 -1,327,587,002 DB_BYTES_UNIDENTIFIED DB_BYTES_UNIDENTIFIED * * 110 110 0 0 -110 -110 DB_PCT_BYTES_FREE_SPACE DB_PCT_BYTES_FREE_SPACE * * 39% 39% 10% 10% -29 -29 DB_PCT_BYTES_SEG DB_PCT_BYTES_SEG * * 60% 60% 89% 89% +29 +29 DB_PCT_UNUSED_BYTES DB_PCT_UNUSED_BYTES * * 1% 1% 1% 1% 0 0 DB_NUM_SEG DB_NUM_SEG 85,620,000 85,620,000 85,620,000 85,620,000 0 0 DB_NUM_VLSEG DB_NUM_VLSEG 8,560,000 8,560,000 8,560,000 8,560,000 0 0 DB_NUM_VLSEG_SPLIT DB_NUM_VLSEG_SPLIT * * 0 0 0 0 0 0 DB_PCT_NUM_VLSEG_SPLIT DB_PCT_NUM_VLSEG_SPLIT * * 0% 0% 0% 0% 0 0 DB_NUM_UNIDENTIFIED DB_NUM_UNIDENTIFIED * * 51 51 0 0 -51 -51 DB_AVG_NUM_UNIDENTIFIED DB_AVG_NUM_UNIDENTIFIED * * 0 0 0 0 0 0 DB_NUM_FSE DB_NUM_FSE * * 23,728,638 23,728,638 673,759 673,759 -23,054,879 -23,054,879 DB_AVG_NUM_FSE DB_AVG_NUM_FSE * * 23.75 23.75 1.00 1.00 -22.75 -22.75 DB_NUM_FSE_MIN DB_NUM_FSE_MIN * * 23,558,660 23,558,660 673,759 673,759 -22,884,901 -22,884,901 DB_NUM_FSE_MAX DB_NUM_FSE_MAX * * 718,662 718,662 673,759 673,759 -44,903 -44,903 DB_AVG_NUM_NOREUSE_FSE DB_AVG_NUM_NOREUSE_FSE * * .17 .17 0 0 -.17 -.17 DB_NUM_PTR DB_NUM_PTR 85,070,408 85,070,408 84,946,241 84,946,241 -124,167 -124,167 DB_NUM_PTR_DIFF_BLK DB_NUM_PTR_DIFF_BLK * * 32,424,879 32,424,879 663,666 663,666 -31,761,213 -31,761,213 DB_PCT_NUM_PTR_DIFF_BLK DB_PCT_NUM_PTR_DIFF_BLK * * 38% 38% 1% 1% -37 -3734 © 2011 IBM Corporation
  35. 35. IBM Software Group Using Smart Reorg Utility – A Simple Sample JCL //CREORG EXEC PGM=HPSGMAIN,PARM=DBD=SAMPLEDB,DBRC=Y //CREORG EXEC PGM=HPSGMAIN,PARM=DBD=SAMPLEDB,DBRC=Y //STEPLIB DD DISP=SHR,DSN=IMSTOOL.LOADLIB //STEPLIB DD DISP=SHR,DSN=IMSTOOL.LOADLIBThree types of policy selection // // DD DISP=SHR,DSN=IMS.SDFSRESL DD DISP=SHR,DSN=IMS.SDFSRESL // // DD DISP=SHR,DSN=USER.PGMLIB DD DISP=SHR,DSN=USER.PGMLIB By database type //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //IMS //IMS DD DISP=SHR,DSN=IMS.DBDLIB DD DISP=SHR,DSN=IMS.DBDLIB – This is the default. //HPSIN //HPSIN DD * DD * (REORG) Specifies Diagnosis Mode Specifies Conditional Reorg Mode (REORG) – IBM-provided CONDREORG=YES,DYAGONLY CONDREORG=YES SYS.DBDTYPE.type ITKBSRVR=FPQSVR00 ITKBSRVR=FPQSVR00 Specifies XCF Group Name SPACEALLOC=YES SPACEALLOC=YES for ITKB Server to connect policies is selected NAMESWAP=YES NAMESWAP=YES (CONDREORG) (CONDREORG) By database name POLICYBY=DBTYPE Specifies how a policy is to be selected POLICYBY=DBTYPE – This is intended to be /* /* used to specify a By database type Policy to be selected database specific policy POLICYBY=DBTYPE POLICYBY=DBTYPE SYS.DBDTYPE.type By policy name (type = HDAM, HIDAM, PHDAM, PHIDAM, HISAM, or SHISAM) – This is intended to be By database name used to specify a same POLICYBY=DBDNAME SYS.DBDNAME.dbdname POLICYBY=DBDNAME policy for a group of By policy name databases POLICYBY=NAME POLICYBY=NAME MY.POLICY.GROUP1 POLICYNM=MY.POLICY.GROUP1 POLICYNM=MY.POLICY.GROUP1 35 © 2011 IBM Corporation
  36. 36. IBM Software Group Policies Policies deliver the intelligence to an intelligent reorg36 © 2011 IBM Corporation
  37. 37. IBM Software GroupIMS Database Reorganization Expert– 3 objectives 1. Policy-based database space management Document DBA knowledge Document DBA knowledge Exceptional states of database space usage are named Rules for exception detection are documented in a policy Target/method of exception notification are documented in the policy 2. Policy-based automated operation Automate repetitive tasks Automate repetitive tasks The policy is used to detect reorganization need Free space reorg. can be done conditionally based on the policy Effectiveness of each free space reorg. can be evaluated 3. Central management of information and data Keep info/data in one place Keep info/data in one place Policies are kept and managed in a repository Statistics data needed for exception detection are kept in a repository History of exceptions and reorganizations are kept also in a repository37 © 2011 IBM Corporation
  38. 38. IBM Software GroupSummary of information and data for Smart ReorgITKB Repository Input Sensor Data OutputInformation / Data Reorg Policies DB Sensor Data Diagnosis ReportsPolicy Services UI Define, read, and store policiesSmart Reorg Utility Requests policy Collects, stores, and Stores policy evaluation with reads data for policy evaluation result as a Policy Services evaluation reportPolicy Services Searches and Evaluates the dataRun-time reads the selected with the policy and policy notify the resultReport UI Searches and reads reports to track exceptions38 © 2011 IBM Corporation
  39. 39. IBM Software GroupPolicy Services overview Policy Services provides an ISPF user interface for Policy Services ISPF User Interface defining and managing Smart Reorg job policies for IMS databases of Reorg Expert •Only full-function databases are supported in the first release Setup policies and store them in an Smart Reorg utility uses Policy ITKB repository Services –to evaluate the threshold sets Ask Policy Evaluation Use Policy specified in the policy definition for a state observed for a given database, and Read Sensor Data –to take action or actions that are specified in the policy Store Sensor Data definition39 © 2011 IBM Corporation
  40. 40. IBM Software GroupMajor components of a policy definition 1. One or more condition Policy expressions (rules) Rule Set – used to evaluate sensor data Rule Rule 2. A resulting exception when Rule Rule a condition threshold has Rule Rule been reached or exceeded Action List 3. List of actions Action Action – An action is selected for each Action exception (instance) detected Rule by evaluation of sensor data against a rule condition in the Condition Exception policy40 © 2011 IBM Corporation
  41. 41. IBM Software GroupComponents of a rule Policy Rule Set Rule Rule Rule condition Rule Rule – Conditional expression (formula) Rule Rule • The formula that compares stored data Action List Action element values with the limits specified Action for this data (threshold values) Action Rule – Threshold sets • A threshold set is a set of threshold Condition Exception values that specify the limits for the specific database states being governed by this rule Threshold exception Condition Exception – Exception class Expression Class – Exception (severity) level Threshold set Level – Exception message text Threshold set Message Text Threshold set41 © 2011 IBM Corporation
  42. 42. IBM Software GroupData elements and rule condition description Examples of data elements A Sample Rule DB_AVG_DBREC_LENGTH The average length of database records. DB_NUM_VLSEG_SPLIT The number of split segment occurrences in the data set. DB_PCT_NUM_VLSEG_SPLIT The percentage of the split variable- segment occurrences compared to the total number of variable-segment occurrences in the data set. DB_PCT_OF_MAX_DS_SIZE The percentage of allocated bytes (bytes for High Allocated RBA) compared to the maximum size (4 GB or 8 GB). DB_PCT_BYTES_FREE_SPACE The percentage of bytes of total free spaces to the total used bytes for the data set. A named set of threshold values for the threshold variables that are referred to in the evaluation formula description above is called a threshold set42 © 2011 IBM Corporation
  43. 43. IBM Software GroupAttributes of an exception An Example of Exception Class Exception class Exception Class: Exception Class: – Represents the specific database FRAGMENTED_FREE_SPACES FRAGMENTED_FREE_SPACES event category being monitored Name of the rule that detects the this exception: Name of the rule that detects the this exception: Exception severity level IBM.FRAGMENTATION.10 IBM.FRAGMENTATION.10 – Is a category representing the severity of the detected exception – There are fixed three levels: Exception • WARNING • SEVERE Class WARNING WARNING • CRITICAL Level SEVERE SEVERE Exception message CRITICAL CRITICAL – Is the text that can be used by the Message Text resulting policy action to describe the database event that crossed a rule threshold set An Example of Exception Message – Users can modify the message text “The fragmentation of free space in %RESOURCE% “The fragmentation of free space in %RESOURCE% has increased” has increased” * The symbol %RESOURCE% is replaced by a DBD * The symbol %RESOURCE% is replaced by a DBD name or a partition name. name or a partition name.43 © 2011 IBM Corporation
  44. 44. IBM Software GroupAction selection Policy Rule Set An action is the result of a Rule Rule rule condition being reached Rule Rule An Action List Entry or exceeded during a policy Rule Rule Exception Action List MESSAGE evaluation Action Class and or Action REORG Level A rule threshold set has Action been mapped to a severity level for the exception class An Example of threshold/exception/action mappings Threshold Exception Class Action associated with the rule Set + Severity Level MYHIGH In turn, the severity level is HIGH CRITICAL mapped to an action REORG MED SEVERE MESSAGE Note: In IBM-provided REORG policies, LOW WARNING severity-level-to-action mappings are MYLOW fixed for each exception class and are not customizable Note: Threshold values in each threshold set, system-provided or user-defined, can be customizable.44 © 2011 IBM Corporation
  45. 45. IBM Software GroupA sample IBM-provided policy The “locale” means where this policy is defined. BSNGLOBL means that it is defined globally independent of a specific RECON environment. This column indicates the locale of a rule that Part of threshold/exception/action mappings is referred to in the policy. G: Global R: RECON environment to which the policy belongs45 © 2011 IBM Corporation
  46. 46. IBM Software GroupSelecting a set of rules for a policy You can unselect pre-selected rules and select new rules to customize your version of this policy Rule locale Action command “V” opens the panel on the next page V The “O” status indicates the rules that were part of the original policy template.46 © 2011 IBM Corporation
  47. 47. IBM Software GroupView/Update Rule List of database types to which this rule can be applied Exception messageIBM-providedthresholdsets Shows the content of Update threshold values Add your own View the rule condition the selected threshold specified in the selected threshold set formula (description) for set threshold set this rule47 © 2011 IBM Corporation
  48. 48. IBM Software GroupUpdate a threshold set Some attributes of data elements that are used Valid range of the threshold as threshold variables (&1, &2, …) are displayed value Data type Short description of the data element (threshold variable)48 © 2011 IBM Corporation
  49. 49. IBM Software GroupChecking rule condition formula The threshold set “LOW” is selected this timeThreshold valuesspecified for thisthreshold set Description of rule condition49 © 2011 IBM Corporation
  50. 50. IBM Software GroupAssociating actions with rule threshold sets For each rule in the list on this panel, you can – select/unselect threshold sets you want to use for this policy, and – associate actions with those selected threshold sets Action command “S” opens the panel on the next page S50 © 2011 IBM Corporation
  51. 51. IBM Software GroupAssociating actions with rule threshold sets (cont.) This panel shows the mapping that follows: Threshold Exception Class Action Set + Severity Level HIGH CRITICAL REORG MED SEVERE MESSAGE LOW WARNING51 © 2011 IBM Corporation
  52. 52. IBM Software GroupIBM-provided policies (policy templates) in V4R1 Policy Name in Installation Policy Name for Run-Time Use Description Smart Reorg Remark Package Automated Policy Selection by DB type IBM.DBDTYPE.HDAM SYS.DBDTYPE.HDAM IBM basic policy for HDAM databases Yes IBM.DBDTYPE.HIDAM SYS.DBDTYPE.HIDAM IBM basic policy for HIDAM databases Yes IBM.DBDTYPE.PHDAM SYS.DBDTYPE.PHDAM IBM basic policy for PHDAM Yes databases IBM.DBDTYPE.PHIDAM SYS.DBDTYPE.PHIDAM IBM basic policy for PHIDAM Yes databases IBM.DBDTYPE.HISAM SYS.DBDTYPE.HISAM IBM basic policy for HISAM databases Yes IBM.DBDTYPE.SHISAM SYS.DBDTYPE.SHISAM IBM basic policy for SHISAM Yes databases IBM.DBDTYPE.HDDB SYS.DBDTYPE.HDDB IBM basic policy for HD databases No The policy name must be specified explicitly in the Smart Reorg job IBM.DBDTYPE.FFDB SYS.DBDTYPE.FFDB IBM basic policy for full-function No The policy name must be databases specified explicitly in the Smart Reorg job * Description of the content of each policy, including rules selected for the policy, can be found in the Policy Services User’s Guide. * Description of each IBM-provided rule can also be found in the User’s Guide.52 © 2011 IBM Corporation

×