Ibm tivoli monitoring for network performance v2.1 the mainframe network management solution sg246360
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,237
On Slideshare
1,237
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Front coverIBM Tivoli Monitoring forNetwork Performance V2.1The Mainframe Network Management SolutionManaging TCP/IP network performancefrom z/OSSample implementationscenariosOperational examplesand tips Budi Darmawan Venugopal Devarasetti Gary Kalatucka Garth Madella Tian Huat Peh Giancarlo Rodolfiibm.com/redbooks
  • 2. International Technical Support OrganizationIBM Tivoli Monitoring for Network Performance V2.1:The Mainframe Network Management SolutionOctober 2004 SG24-6360-00
  • 3. Note: Before using this information and the product it supports, read the information in “Notices” on page ix.First Edition (October 2004)This edition applies to Version 2, Release 1, Modification 0 of IBM Tivoli Monitoring for NetworkPerformance (product number 5698-FNP).© Copyright International Business Machines Corporation 2004. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp.
  • 4. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Chapter 1. Introduction to network performance monitoring . . . . . . . . . . . 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 The automation blueprint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 IBM Tivoli Monitoring for Network Performance . . . . . . . . . . . . . . . . . . . . . 4 1.4 Redbook environment and scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Document organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Chapter 2. Components and architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Components and functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Web application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2.1 Web application structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2.2 Web application user interface functions . . . . . . . . . . . . . . . . . . . . . 12 2.2.3 Role-based security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.4 Problem determination for Web application . . . . . . . . . . . . . . . . . . . 19 2.3 Monitor functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.1 Process structure of the monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3.2 Files used by the monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.3.3 Performance data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.3.4 Setting options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.3.5 Problem determination for the monitor . . . . . . . . . . . . . . . . . . . . . . . 31 2.4 Communication and security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.4.1 User authentication mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.4.2 Communication port usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.4.3 Certificates and authentication with SSL. . . . . . . . . . . . . . . . . . . . . . 38 2.4.4 Transport between DB2 and monitor . . . . . . . . . . . . . . . . . . . . . . . . 39 2.5 Database structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5.1 Configuration tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5.2 Measurement tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Chapter 3. Implementation scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.1 Implementation components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48© Copyright IBM Corp. 2004. All rights reserved. iii
  • 5. 3.2 Implementation scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2.1 Distributed servers environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.2.2 Pure z/OS environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3 Scenario comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3.1 Distributed consideration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3.2 z/OS only consideration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3.3 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.4 Scenario implementation roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.5 User operation and roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Chapter 4. AIX Web application implementation . . . . . . . . . . . . . . . . . . . . 57 4.1 Web application on AIX overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.2 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2.1 File system space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2.2 Setting up the Java Messaging Service . . . . . . . . . . . . . . . . . . . . . . 61 4.2.3 User authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.2.4 Enabling DB2 password encryption . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.2.5 WebSphere access to DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.3 Web application implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.3.1 Implementation steps overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.3.2 Web application installation details . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.3.3 Setting up LDAP in z/OS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.4 Configuring WebSphere Application Server for LDAP on z/OS . . . . 74 4.3.5 Verification of the distributed implementation . . . . . . . . . . . . . . . . . . 78 4.4 Some problems and their solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.4.1 Problem with console users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.4.2 LDAP user ID character constraint . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.4.3 Uninstallation from admin console . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.4.4 Set up the X-windows DISPLAY properties to enable graphics . . . . 86 4.4.5 LTPA key generation problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.4.6 Problem with SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.5 Start and stop procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 4.6 Backup and recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.6.1 File system backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.6.2 DB2 database backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.6.3 DB2 database restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.6.4 File system restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Chapter 5. Mainframe Web application implementation . . . . . . . . . . . . . 101 5.1 Scenario overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.2 Preparing for the implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 5.2.1 Preparing HFS files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 5.2.2 Preparing DB2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104iv IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 6. 5.2.3 Graphic package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.2.4 Preparing WebSphere Application Server . . . . . . . . . . . . . . . . . . . 1105.3 Web application implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 5.3.1 Installation procedure overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 5.3.2 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1155.4 Start and stop procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 5.4.1 Start up the Web application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 5.4.2 Shut down the Web application. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1195.5 Backup and recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 5.5.1 Backup and restore of file systems . . . . . . . . . . . . . . . . . . . . . . . . . 120 5.5.2 Backup and restore of DB2 database . . . . . . . . . . . . . . . . . . . . . . . 121Chapter 6. Monitor implementation and operation . . . . . . . . . . . . . . . . . 1256.1 Monitor installation process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.1.1 Before the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.1.2 Preparing the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.1.3 Parameters for itmnp.properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296.2 Some problems and their solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 6.2.1 Missing Tivoli common directory . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 6.2.2 Setting APF authorized attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . 1316.3 Start and stop procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.4 Sample monitor configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1346.5 Monitoring best practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.5.1 Monitored metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 6.5.2 Monitoring configuration design . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 6.5.3 Monitoring information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6.5.4 Monitoring storage usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 6.5.5 Monitoring network interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157Chapter 7. Discovery and alert interfaces. . . . . . . . . . . . . . . . . . . . . . . . . 1617.1 NetView Integrated TCP/IP services component . . . . . . . . . . . . . . . . . . 162 7.1.1 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 7.1.2 Configure NetView Integrated TCP/IP Services Component . . . . . 162 7.1.3 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1637.2 Event integration with IBM Tivoli Enterprise Console . . . . . . . . . . . . . . . 164 7.2.1 Customizing TEC rulebase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 7.2.2 Configuring event forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 7.2.3 Sample event automation program . . . . . . . . . . . . . . . . . . . . . . . . . 1737.3 Event integration with IBM Tivoli NetView for z/OS. . . . . . . . . . . . . . . . . 175 7.3.1 Setting up Event Automation Service . . . . . . . . . . . . . . . . . . . . . . . 176 7.3.2 Defining threshold and event generation . . . . . . . . . . . . . . . . . . . . 177 7.3.3 Automating NetView alert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181Chapter 8. Historical reporting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Contents v
  • 7. 8.1 Tivoli Data Warehouse overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 8.2 Tivoli Data Warehouse setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 8.2.1 Installation for distributed data source . . . . . . . . . . . . . . . . . . . . . . 186 8.2.2 Installation for mainframe data source . . . . . . . . . . . . . . . . . . . . . . 186 8.3 Installation of the Warehouse Enablement Pack. . . . . . . . . . . . . . . . . . . 187 8.3.1 Back up TWH databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 8.3.2 Warehouse Enablement Pack installation. . . . . . . . . . . . . . . . . . . . 188 8.4 ETL processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 8.4.1 ETL overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 8.4.2 Testing, scheduling, and promoting the ETLs . . . . . . . . . . . . . . . . . 199 8.5 Sample reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 8.5.1 Configuring Crystal Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 8.5.2 Available reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 8.5.3 Accessing the Crystal ePortfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Appendix A. ITMNP configuration files and test programs. . . . . . . . . . . 217 AIX itmnp.properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 z/OS itmnp.properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Sample server TSO REXX program. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Sample object REXX client program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 Appendix B. z/OS LDAP SDBM back end configuration files . . . . . . . . . 223 z/OS LDAP setup configuration file: ldap.profile . . . . . . . . . . . . . . . . . . . . . . 224 z/OS LDAP configuration file: SLAPDCNF. . . . . . . . . . . . . . . . . . . . . . . . . . . 225 z/OS LDAP started procedure: LDAPSRV . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Appendix C. Tips collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 WebSphere Application Server Version 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 J2EE based application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Startup and shutdown procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 DB2 Universal Database Version 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Relational database system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 DB2 on z/OS systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 DB2 on AIX or Window systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Differences between z/OS and distributed DB2 processing . . . . . . . . . . . 231 Important tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 z/OS Communication Server TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 IBM Tivoli NetView for z/OS Version 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Alerts structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Automation support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Event automation service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 IBM Tivoli Enterprise Console Version 3.9. . . . . . . . . . . . . . . . . . . . . . . . . . . 235 IBM Tivoli Enterprise Console structure . . . . . . . . . . . . . . . . . . . . . . . . . . 235vi IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 8. IBM Tivoli Data Warehouse Version 1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Data warehouse concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 Warehouse enablement pack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Appendix D. Underlying technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241Light-weight Directory Access Protocol (LDAP) . . . . . . . . . . . . . . . . . . . . . . . 242eXtensible Markup Language (XML) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Microsoft Excel for browsing XML files . . . . . . . . . . . . . . . . . . . . . . . . . . . 250Certificates and encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Secret Key Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Public Key Cryptography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Refinements on cryptographic techniques . . . . . . . . . . . . . . . . . . . . . . . . 256Secure Sockets Layer protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 The Record Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 The communication protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Contents vii
  • 9. viii IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 10. NoticesThis information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the usersresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send licenseinquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.The following paragraph does not apply to the United Kingdom or any other country where such provisionsare inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDESTHIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM maymake improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirmthe accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrates programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment to IBM for the purposes ofdeveloping, using, marketing, or distributing application programs conforming to IBMs applicationprogramming interfaces.© Copyright IBM Corp. 2004. All rights reserved. ix
  • 11. TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both: AIX® IMS™ RDN™ Cloudscape™ iSeries™ RMF™ CICS® Lotus® SecureWay® Database 2™ MQSeries® System/390® Domino® MVS™ Tivoli Enterprise™ DB2 Universal Database™ NetView® Tivoli Enterprise Console® DB2® OS/390® Tivoli® Eserver® pSeries® VTAM® Eserver® Redbooks™ WebSphere® IBM® Redbooks (logo) ™ z/OS® ibm.com® RACF® zSeries®The following terms are trademarks of other companies:Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,other countries, or both.Java and all Java-based trademarks and logos are trademarks or registered trademarks of SunMicrosystems, Inc. in the United States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Other company, product, and service names may be trademarks or service marks of others.x IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 12. Preface This IBM® Redbook explains the new IBM Tivoli Monitoring for Network Performance Version 2.1. This version of IBM Tivoli Monitoring for Network Performance provides a complete redesign of the z/OS® TCP/IP management tools that was started by the NetView® Performance Monitor for TCP/IP. IBM Tivoli Monitoring for Network Performance provides a comprehensive TCP/IP stack monitoring for z/OS. It collects performance metrics from the z/OS Communication Servers system management interface, measuring response time and Simple Network Management Protocol (SNMP) Management Information Base (MIB) variable collection. IBM Tivoli Monitoring for Network Performance uses strategic IBM software platforms, such as WebSphere® Application Server, as the Web application platform, and DB2® Universal Database™ as the central repository. This redbook starts with exploring the architecture of IBM Tivoli Monitoring for Network Performance and its components. We also discuss various implementation scenarios and evaluate the benefits and drawbacks of each scenario. Implementation planning and consideration is presented and operational consideration is explained.The team that wrote this redbook This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, Austin Center.© Copyright IBM Corp. 2004. All rights reserved. xi
  • 13. Figure 1 The redbook team Budi Darmawan is a project leader at the International Technical Support Organization, Austin Center. He writes extensively and teaches IBM classes worldwide on all areas of systems management on distributed and z/OS. Before joining the ITSO five years ago, Budi worked in IBM Indonesia as a lead IT Specialist performing implementation services and solution architecting. His current interest is advanced business service management solution. Venugopal Devarasetti is a Software Engineer at IBM Software Labs, Bangalore, India. He has been with IBM for four years, after receiving his Engineering degree from Kuvempu University, India. He has been involved with the IBM Java™ Development Kit on AIX® and z/OS. His area of expertise includes J2EE, Web Services, and JVM Internals. Gary Kalatucka is a Senior IT Consultant for Tivoli® Services Americas in the United States. He has 26 years of experience in the z/OS field, including four years of Tivoli software experience. His areas of expertise include z/OS, SNA, z/OS Automation products, and various Tivoli software products. Garth Madella is a Information Technology Specialist with IBM South Africa. He has 18 years of experience in the System/390® networking field. He has worked with IBM for eight years. His areas of expertise include VTAM®, SNA TCP/IP,xii IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 14. and sysplex. He has written extensively on TCP/IP and Enterprise Extender issues. Tian Huat Peh is a Advisory IT specialist with IBM Singapore. He has nine years of experience in the OS/390® system and TCP/IP field. His areas of expertise include z/OS USS, OSA-Express implementation, and z/OS TCPIP implementation. Giancarlo Rodolfi is a zSeries® Certified Consultant TSS in Brazil. He has 18 years of experience in zSeries field. His areas of expertise include zSeries and Linux. He has written extensively on z/OS Communication Server. Thanks to the following people for their contributions to this project: Wade Wallace International Technical Support Organization, Austin Center Bob Haimowitz and Richard M Conway International Technical Support Organization Laura Knapp and Douglas Gibbs IBM US, Tivoli SoftwareBecome a published author Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. Youll team with IBM technical professionals, Business Partners and/or customers. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, youll develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html Preface xiii
  • 15. Comments welcome Your comments are important to us! We want our Redbooks™ to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at: ibm.com/redbooks Send your comments in an Internet note to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. 0SJB Building 003 Internal Zip 2834 11400 Burnet Road Austin, Texas 78758-3493xiv IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 16. 1 Chapter 1. Introduction to network performance monitoring This chapter introduces you to the network performance monitoring, specifically TCP/IP network performance monitoring. It begins by putting performance management in context and develops the discussion of the usage of IBM Tivoli Monitoring for Network Performance in this perspective. The discussion in this chapter is divided into the following sections: 1.1, “Introduction” on page 2 1.2, “The automation blueprint” on page 2 1.3, “IBM Tivoli Monitoring for Network Performance” on page 4 1.4, “Redbook environment and scope” on page 5 1.5, “Document organization” on page 5© Copyright IBM Corp. 2004. All rights reserved. 1
  • 17. 1.1 Introduction Mainframes are playing a vital role in the current network computing environment and on-demand initiatives. They serve as the premier servers servicing hundreds and thousands of users. These mainframes are interconnected together and they interact with other machines and services through the network. Primary communication protocol shifted from the traditional Systems Network Architecture (SNA) to the Transmission Control Protocol/Internet Protocol (TCP/IP). This introduces a new challenge for managing the TCP/IP network communication on these mainframe servers. The IBM Tivoli Monitoring for Network Performance provides the facility to manage all aspects of TCP/IP communication from the z/OS’s IBM Communication Server TCP/IP. IBM Tivoli Monitoring for Network Performance Version 2.1 brings a new management paradigm on this area. It is a complete re-design of the product from previous versions and NPM/IP product. This redbook discusses the new IBM Tivoli Monitoring for Network Performance Version 2.1 implementation and operation scenarios. Since this is a critical component in the overall system management and automation environment, we will also discuss its role in the automation blueprint.1.2 The automation blueprint The IBM Tivoli solution is the base of providing automation on the overall system management for the on demand world. Automation is so critical for businesses to achieve resiliency, efficiency, responsiveness, and flexibility. The IBM automation platform shows the structure of the automation components in providing on demand automation capability. The IBM automation blueprint is shown in Figure 1-1 on page 3.2 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 18. Business Service Management Policy Based Orchestration Availability Assurance Optimization Provisioning Virtualization Software Resources System ResourcesFigure 1-1 IBM automation blueprintThe IBM automation blueprint is a game-changing plan for reducing thecomplexity of technology to allow you to focus on the business goals and theapplication of resources to business objectives rather than the management oftechnology. The blueprint enables enterprises to implement automation in anevolutionary fashion that acknowledges the heterogeneous nature of theinfrastructure.At the bottom of the blueprint is the foundation – the Software and SystemResources with native automation capabilities required for higher levelautomation functions. Many of these resources may be virtualized to the othercapabilities. Here, the key point is that in order to achieve the highest levels of ondemand automation, resources need to be virtualized so that they can bedynamically provisioned as business policies require.Above the resources are the key automation capabilities: Availability helps ensure that systems are available 24x7. Reliance or security keeps your systems protected from threats and provides the functions for a great user experience in accessing applications and data they need – while keeping out unwelcome users. Chapter 1. Introduction to network performance monitoring 3
  • 19. Optimization provides tools to make the most of the resources you have – so that they are running at peak performance and efficiency and providing you with the maximum return on your investment. Provisioning focuses on the self-configuring, dynamic allocation of individual elements of your IT infrastructure – so that Identities or Storage or Servers are provisioned as business needs dictate. The next layer, Policy-based Orchestration, helps customers automatically control all the capabilities of the four areas we just discussed so that the entire IT infrastructure is responding dynamically to changing conditions according to defined business policies. This orchestration builds on the best practices of the customer’s collective IT experience, and helps to ensure that complex deployments are achieved with speed and quality – on demand. Finally, Business-driven Service Management capabilities provide the tools you need to manage service levels, meter system usage and bill customers for that usage, as well as model, integrate, connect, monitor, and manage your business processes end-to-end for complete linkage of IT and business processes. Being able to view IT resources in context of business systems is a unique capability that we need. The IBM Tivoli Monitoring for Network Performance provides availability monitoring for mainframes TCP/IP network servers. It resides in the availability function and provides networking server performance monitoring.1.3 IBM Tivoli Monitoring for Network Performance IBM Tivoli Monitoring for Network Performance Version 2.1 provides a centralized monitoring of the TCP/IP networking protocol. IBM Tivoli Monitoring for Network Performance meets your daily tactical needs as well as your long-term strategic systems management goals, providing an effective way to gain control of mission-critical network resources, performance issues, and workload distributions. IBM Tivoli Monitoring for Network Performance provides timely analysis of performance related metrics, such as response time, traffic flow, system workload, and CPU utilization. Using the IBM Tivoli Monitoring for Network Performance Web application, operators can monitor the performance of the network in an effort to anticipate problems and resolve them before they occur. The performance data can be used to detect bottlenecks and other potential problems, which eliminates the need for network systems programmers to manually scan through extensive amounts of performance data. The network systems programmer can use this data to perform detailed problem determination for problems that cannot be4 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 20. resolved by the network operators. In addition, the network systems programmer can use this data to validate service level agreements and improve network performance through network tuning. After the data is summarized and aggregated into the Tivoli Data Warehouse, the capacity planner can use this data to do trend analysis and forecasting to improve performance and better plan for network growth.1.4 Redbook environment and scope This redbook discusses the implementation and operation scenarios of the IBM Tivoli Monitoring for Network Performance Version 2.1. The discussion is divided into two scenarios: the mainframe based scenario and the combination of mainframe and distributed scenario. The detailed environment is presented in 3.4, “Scenario implementation roadmap” on page 54.1.5 Document organization The discussion in this document is divided into the following chapters: Chapter 1, “Introduction to network performance monitoring” on page 1, this chapter, serves as the book introduction. Chapter 2, “Components and architecture” on page 7 explains, in detail, the IBM Tivoli Monitoring for Network Performance components and their interaction. Chapter 3, “Implementation scenarios” on page 47 discusses possible implementation scenarios and how we will discuss them in this redbook. Chapter 4, “AIX Web application implementation” on page 57 explains the implementation of the AIX-based Web application. Chapter 5, “Mainframe Web application implementation” on page 101 explains the implementation of the z/OS-based Web application. Chapter 6, “Monitor implementation and operation” on page 125 explains the monitoring components and some discussion in monitoring process. Chapter 7, “Discovery and alert interfaces” on page 161 describes integration with IBM Tivoli NetView for IP resource discovery and also alerts usage with IBM Tivoli Enterprise™ Console or IBM Tivoli NetView for z/OS. Chapter 8, “Historical reporting” on page 183 explains the data collection implementation for IBM Tivoli Monitoring for Network Performance data. Chapter 1. Introduction to network performance monitoring 5
  • 21. The appendices provide program listing and additional information for your reference in reading this redbook.6 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 22. 2 Chapter 2. Components and architecture This chapter discusses the product architecture of IBM Tivoli Monitoring for Network Performance. This chapter includes the following sections: 2.1, “Components and functions” on page 8 discusses the primary components of IBM Tivoli Monitoring for Network Performance and their functions. 2.2, “Web application” on page 10 describes the Web application structure. 2.3, “Monitor functions” on page 22 explains the monitoring subsystem. 2.4, “Communication and security” on page 34 shows the communication between the components of IBM Tivoli Monitoring for Network Performance. As it is a distributed application, security of the communication is of a concern. 2.5, “Database structure” on page 41 describes the underlying data structure for IBM Tivoli Monitoring for Network Performance.© Copyright IBM Corp. 2004. All rights reserved. 7
  • 23. 2.1 Components and functions IBM Tivoli Monitoring for Network Performance monitors the performance of the TCP/IP network in your enterprise from the mainframe perspective. Performance data from all monitored systems are stored in a central DB2 database. The data collected by IBM Tivoli Monitoring for Network Performance can be accessed using the Web application and also be used as source information for historical reports generation using Tivoli Data Warehouse. The basic function of the IBM Tivoli Monitoring for Network Performance is to collect performance data, which requires you to install the IBM Tivoli Monitoring for Network Performance monitor component on each of the z/OS systems that you want to monitor. In addition, you must install the IBM Tivoli Monitoring for Network Performance Web application on a system that has WebSphere Application Server and DB2 database run. The supported operating systems for the IBM Tivoli Monitoring for Network Performance Web application are z/OS and AIX. Figure 2-1 shows the overall IBM Tivoli Monitoring for Network Performance architecture. NetView Integrated TCP/IP WebSphere Application Server Services Component IBM Tivoli IBM Tivoli Monitoring Monitoring for Windows/AIX for Network JMS Network Performance Performance Web Application monitor DB2 z/OS Monitor Configuration Tivoli Data Warehouse Performance IBM Tivoli Server Central Data data Enterprise Console Warehouse and Crystal Data marts AIX or z/OS Enterprise IBM Tivoli NetView Server for z/OS WindowsFigure 2-1 IBM Tivoli Monitoring for Network Performance components As shown in Figure 2-1, the IBM Tivoli Monitoring for Network Performance architecture consists of several components. The required components are shown in the darker boxes, while optional components are shown in white boxes and connected by dotted lines.8 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 24. The required components are: WebSphere Application Server and DB2: The WebSphere Application Server and the DB2 are required for the IBM Tivoli Monitoring for Network Performance Web application. Both components must be run on the same server. WebSphere Application Server provides the platform for running the IBM Tivoli Monitoring for Network Performance Web application. DB2 acts as the central repository that contains a database that holds the configuration and performance data. See 2.5, “Database structure” on page 41 for more information. Web application: The IBM Tivoli Monitoring for Network Performance Web application runs as a WebSphere Application Server application. It is installed using Install Shield Multi-Platform (ISMP) on AIX or Windows® platform or using a script from the UNIX® Systems Services command line for z/OS. The product interface is a role-based Web application that is accessible from your Web browser. The Web Application provides the interface to view and configure the IBM Tivoli Monitoring for Network Performance environment. See 2.2, “Web application” on page 10 for more information. Monitors: The IBM Tivoli Monitoring for Network Performance Monitor is configured and runs on each z/OS operating system. The monitor performs three separate functions: collect performance data for the z/OS operating system, collect SNMP data from configured IP addressable, and collect availability and response time data from IP-addressable resources in the enterprise that the monitor is able to ping. In a normal configuration, there would be multiple instances of the z/OS monitors recording data to the DB2 database for the IBM Tivoli Monitoring for Network Performance Web application. See 2.3, “Monitor functions” on page 22 for more information.The optional components are: Event receivers: IBM Tivoli Monitoring for Network Performance can be configured to forward events in Event Integration Facility (EIF) format. This means that the events can be forwarded to IBM Tivoli Enterprise Console® (TEC) or IBM Tivoli NetView for z/OS. IBM Tivoli Monitoring for Network Performance provides the necessary TEC classes to receive events. Events forwarded to IBM Tivoli NetView for z/OS need to be received using the Event Automation Service. NetView Integrated TCP/IP Services Component (ITSC): It is highly recommended you install the NetView Integrated TCP/IP Services Component, which provides automatic discovery of IP-addressable resources in your enterprise. ITSC can discover and classify any IP-addressable resources that are running SNMP agent. The SNMP information can be queried for additional information, such as resource type, which can be used to group these resources into SmartSets. It can automatically detect z/OS systems and other TCP/IP stack information from z/OS. NetView Integrated Chapter 2. Components and architecture 9
  • 25. TCP/IP Services Component software is provided with IBM Tivoli Monitoring for Network Performance. Tivoli Data Warehouse: IBM Tivoli Monitoring for Network Performance captures performance data that can be collected into Tivoli Data Warehouse. Tivoli Data Warehouse is a strategic cross-application infrastructure for collecting historical data and generating reports. IBM Tivoli Monitoring for Network Performance provides a set of extract, transform, and load (ETL) programs utilities to summarize and migrate historical performance data to Tivoli Data Warehouse. Historical data stored in Tivoli Data Warehouse is used to generate historical performance reports using Crystal Enterprise, which is provided as part of Tivoli Data Warehouse.2.2 Web application The Web application is the main component of IBM Tivoli Monitoring for Network Performance. It serves as the central contact point for other components. It uses WebSphere Application Server and DB2 database. This section discusses the structure of the Web application. The discussion is divided into: 2.2.1, “Web application structure” on page 10 2.2.2, “Web application user interface functions” on page 12 2.2.3, “Role-based security” on page 17 2.2.4, “Problem determination for Web application” on page 192.2.1 Web application structure Figure 2-2 on page 11 shows the overview of the Web application components and their connections.10 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 26. itmnp21 Enterprise Application JMS messaging JDBC interface ITMNPDB ITMNP ITMNP JMX EJB itmnpItsc itmnpUI SERVLETs Web interface Netview ITSC monitors Web browser nvexportdFigure 2-2 The Web application component structureThe Web application consists of several modules. Some of these modules haveexternal interfaces to interact with other components. The following are themodules of the itmnp21 J2EE application: The itmnpJMX is a Java resource module that contains Java Management eXtension classes. These classes are subclasses that represent the JMX objects that reside in the monitors. This is an internal component of the Web application. This resource package for the JMX connector is stored in a resource archive (rar) file. The itmnpEJB module is an Enterprise Java Beans (EJBs) module for WebSphere Application Server. This module consists of EJBs for various objects that are managed by IBM Tivoli Monitoring for Network Performance, such as nodes, ITSC processes, commands, and so on. These EJBs provide the means to search and retrieve objects from a relational database through Java programs. Some of the objects here communicate using Java Messaging Service (JMS). The itmnpItsc module is the module that uses servlets and interacts with EJBs for IBM Tivoli Monitoring for Network Performance. It provides the interface to the monitors and NetView ITSC. The z/OS version of this module does not support SSL or HTTPS communication. This module utilizes the EJB and JMX modules for IBM Tivoli Monitoring for Network Performance. The itmnpUI module is the Web application user interface module that interacts with an operator using a Web browser. This module contains Java Chapter 2. Components and architecture 11
  • 27. Server Pages and static Web content for the operator Web console. This module can be accessed using either HTTP or HTTPS protocol on all platforms. Database access is performed though the JDBC interface to the DB2 database.2.2.2 Web application user interface functions The Web application serves the IBM Tivoli Monitoring for Network Performance user interface to a Web browser. This section describes the user interface functions of the Web application: A product interface for a user to access by browser The user interface is a role-based Web application that is accessible from a Web browser. When security is enabled, a user name and password are required to sign on to the Web application. The initial login window is shown in Figure 2-3. The Administrator Full Access check box allows configuration and maintenance functions to be available when the user is authorized. Figure 2-3 Initial login window Function based portfolio When the user is initially logged in, the left side of the window contains the menu portfolio that the user can access. Figure 2-4 on page 13 shows a user12 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 28. with administrator role menu. It has both the display and management menus. The management options are shown in the red box.Figure 2-4 Portfolio menu for an administrator Maintains and manages the configuration for the monitors The Web application provides three configuration wizards to create the monitor definition. The wizards are for: – z/OS monitor – SNMP monitor – Availability and response time monitor The following are the steps that you will go through with the wizard: – Defines one or more monitor locations The monitor location table contains the z/OS system name and IP address (or fully qualified host name) of one or more monitors in your enterprise. Chapter 2. Components and architecture 13
  • 29. The IP addresses or host names that you enter into this table are used by the IBM Tivoli Monitoring for Network Performance Web application when communicating with the monitor component. – Provides a list of resources to monitor The list of resources to monitor will differ dependent on the type of monitor definition you are creating. • z/OS monitor definition: You will specify the list of TCP/IP stacks on each z/OS system that you want to monitor. By default, you will monitor the TCP/IP stack you provided when you defined the monitor locations. If the z/OS systems you are monitoring has multiple TCP/IP stacks, you will provide the IP address or fully qualified host name of each additional stack that you want to monitor. The TCP/IP stacks must reside on the same z/OS system where the monitor resides. • SNMP monitor definition: You will specify the IP address or fully qualified host name of each resource you want to monitor. You must have network connectivity to each of these resources from the monitor and the SNMP agent must be running on each of these resources in order to retrieve data using an SNMP query. • Availability and Response Time monitor definition: You will specify the IP address or fully qualified host name of each resource you want to monitor for availability and response time. In order to collect availability and response time data, you must be able to ping each of these resources from the monitor. Availability and Response time monitor definitions should be created to monitor the most critical IP resources in your environment. – Specifies the type of data to collect The type of data to collect will differ depending on which type of monitor definition you are creating. • z/OS monitor definition: You will choose from a list of 10 different categories of data to collect, such as TCP, IP, UDP, FTP, and so on. • SNMP monitor definition: You will chose from a list of MIBS that contain performance data. • Availability and Response Time monitor definition: Availability and response time data will be collected for each of the resources you are monitoring. – Specifies threshold and rearm values Threshold values can be specified for each monitored metric. The threshold value is used to determine the point to which an alert is generated. Each piece of performance data that is collected by the monitor is compared to the threshold value to determine if the threshold14 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 30. has been crossed for all data that has been displayed on the user interface. If the threshold has been crossed, a red indicator will be placed next to the value of that metric when it is displayed. In addition, you can choose to generate an event when a threshold is crossed.– Define schedule: Collection time and interval A schedule entry defines the day of the week and time that you want to start collecting data and the day of the week and time that you want to stop collecting data. Additionally, for each schedule entry, you will define a collection interval and frequency. Note: Data collection for FTP and TN3270 sessions happens immediately following the completion of the session. This data is not stored based on the interval. Multiple monitor definitions that request the same data are combined to collect the data only once. When different collection definitions contain different schedules for similar data, IBM Tivoli Monitoring for Network Performance data collection engine must choose which intervals are used. The basic algorithm for collecting z/OS Communications Server data is as follows: i. It selects the shortest collection interval from all the collection instruction. ii. If more than one definition is currently active that have the same collection interval, it chooses one based on the order in which the definitions were delivered to the monitor component, meaning the order is an arbitrary choice by the monitor.Sets operation preferences for the environmentYou must set the operational values for your environment before creating anddeploying the monitor configurations. Doing so ensures that the monitorcollects performance data and generates events. The following are theoperation preferences:– Database purge preferences The Database Purge Preferences task is used to define the number of days to retain data in the database, the time of day the daily purge occurs, and whether the purge is dependent upon completion of the ETL process.– SNMP preferences The SNMP preferences task must be defined in order to collect performance data from SNMP resources in your enterprise. The SNMP agent must be running on each IP resource that you want to monitor. Chapter 2. Components and architecture 15
  • 31. – Define event receivers. An event is generated when specific thresholds are crossed. Events can be viewed using IBM Tivoli Enterprise Console or any applications that is capable of receiving and displaying events. The event receiver defines the fully qualified host name or IP address and port for the IBM Tivoli Enterprise Console Server or the IBM Tivoli NetView for z/OS. Data view in graphics and table format The IBM Tivoli Monitoring for Network Performance Web application shows performance data in table and graphical format view of the performance data collected in the DB2 database. Figure 2-5 shows the table view of the performance data. This is the default view mode. Figure 2-5 Table view of performance data Figure 2-6 on page 17 shows the graphic view of the performance data. For more information on getting the graph, refer to the IBM Tivoli Monitoring for Network Performance Operator Guide, SC31-6365.16 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 32. Figure 2-6 Graphical view Note: The graphic engine for a z/OS Web application server uses a shareware program called pja toolkit, while for an AIX based Web application server it uses X-Windows graphic classes; therefore, the DISPLAY environment variable must be set to refer to a server with an active X-server.2.2.3 Role-based security The user interface is secured with a set of roles. The Web application roles are configured using the WebSphere Application Server Web-based administration. Chapter 2. Components and architecture 17
  • 33. An authenticated user ID and password are required to sign in to the Web application. Depending on the platform and authentication mechanism, the creation of the user IDs are the responsibility of the security administrator. The user IDs are then associated with a role in the Web application. The IBM Tivoli Monitoring for Network Performance Web application uses three roles for user assignment: ITMNP Operators can view collected performance metrics for various configured monitors through the Web application. ITMNP Admin can configure and manage monitors to collect performance data, and set run-time preferences and global defaults for the product. ItscAllAuthority is used for connecting to the itmnpItsc module; this is for the monitor or NetView ITSC component when it connects to the Web application. The default is the ItscAllAuthority is assigned to everyone. Do not change this, as the monitors may fail. User authentication is handled by WebSphere Application Server. This can be a local operating system user ID or a remote LDAP directory server user ID. Most installations may want to use a z/OS based user ID to be consistent. The user ID needs to be assigned to a role in WebSphere Application Server. Role assignment is performed using WebSphere Application Server’s administrative console. Figure 2-7 on page 19 shows the role assignment dialog for IBM Tivoli Monitoring for Network Performance.18 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 34. Figure 2-7 Role assignment for the Web application2.2.4 Problem determination for Web application All components of the Web application run on a single Java Virtual Machine (JVM) started by the WebSphere Application Server. This JVM has the standard output and standard error redirected to files called SystemOut.log and SystemErr.log respectively. Those files reside under the path: <WebSphere directory>/AppServer/logs/<server name> where: WebSphere directory The path where WebSphere is installed; on AIX, it is typically /usr/WebSphere. server name The WebSphere server name; on AIX, it is typically called server1. Our z/OS server is called ws611sc61. Chapter 2. Components and architecture 19
  • 35. You can activate tracing for a WebSphere component to get more detail on the processing of the component from the WebSphere Application Server administrative console. From the administrative console, select Troubleshooting → Logs and Trace, as shown in Figure 2-8. Figure 2-8 Trace menu Select the server name that you want to modify and select the Diagnostic Trace property. The diagnostic trace setting dialog is shown in Figure 2-9 on page 21.20 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 36. Figure 2-9 WebSphere trace settingIn Figure 2-9, the trace is enabled for Java Messaging Services and it is writing toa file with a size of 20 MB. You may need to modify this size, as it may not beenough for a busy system. If you want to modify the components to be traced,click on the Modify button and you get the setting page, similar to Figure 2-10 onpage 22. Chapter 2. Components and architecture 21
  • 37. Figure 2-10 Setting component trace Click on the Apply, Close, OK, Save, and then Save again to save the setting. For more information, refer to IBM Tivoli Monitoring for Network Performance: Messages and Troubleshooting, SC31-6366.2.3 Monitor functions The IBM Tivoli Monitoring for Network Performance monitor component runs on z/OS systems to collect TCP/IP network performance data and send the collected data to a DB2 database. The monitor performs the following functions: Collects performance data from the z/OS system where the monitor resides. The z/OS performance data is stored in a central DB2 database and can be displayed using the IBM Tivoli Monitoring for Network Performance Web application.22 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 38. Collects SNMP performance data from IP-addressable resources in the network that are running the Simple Network Management Protocol (SNMP) agent. The SNMP data cannot be shown using the Tivoli Monitoring for Network Performance Web application, but it is stored in the DB2 database. Collects availability and response time data from IP-addressable resources in the network. It uses the ping command or ICMP protocol to collect this information. Availability and response time data is stored in the central database and can be displayed using the IBM Tivoli Monitoring for Network Performance Web application. Sends events when performance metrics indicate a possible problem in the network. The events are sent in Event Integration Facility (EIF) format. The monitor configuration defines which resources to monitor, which types of data to collect, when to start and stop collecting data, and how often to collect data. A monitor configuration can consist of one or more monitor definition. The monitor configuration is defined using the Web application.2.3.1 Process structure of the monitor The monitor process structure is shown in Figure 2-11. itm np M o n ito r.sh L a un che r M o n ito r_ cs3 9 0 Java V irtu al M ach in e IT M N P M o nitor (U S S ) z/O S C o m m un ica tion S N M P d ae m on S e rve r z/O S W e b A p plicatio n D B2 Figure 2-11 Monitor startup process Chapter 2. Components and architecture 23
  • 39. As shown in Figure 2-11 on page 23, the processes for the monitor are: A shell script, itmnpMonitor.sh, initializes environment variables before starting the launcher. The launcher starts the two main processes running in z/OS Unix System Services (USS), and a C-language program monitor_cs390 collects performance data from z/OS Communication Server and a Java Virtual Machine (JVM) that communicates with the Web application and DB2. – The JVM establishes the communication link with the Web application through the itmnpItsc servlets and with DB2 through Java Database Connectivity (JDBC). The JVM reads the monitor configuration and setup data collection. The JVM process also collect SNMP and ICMP monitors. – The monitor_cs390 collects data for the z/OS Communication server API. It retrieves data for TCP/IP stack, FTP, TN3270, TCP applications, TCP connections, TCP storages, UDP endpoints, Enterprise Extender (EE) and High Performance Routing (HPR), Common Storage Manager (CSM) storage, and so on.2.3.2 Files used by the monitor The files used by the IBM Tivoli Monitoring for Network Performance monitor process is shown in Figure 2-12. $tivoli_common_dir/FNP/ logs/fnp_config.xml $DBCacheDirectory/dbcache ITMNP Monitor $CONFIG_DIR/Itmnp.properties $tivoli_common_dir/FNP/logs/msg_fnp_monitor*.log $FNPlogPropertiesLocation $tivoli_common_dir/FNP/logs/trace_fnp_monitor*.log /etc/ibm/tivoli/common/cfg/log.properties Figure 2-12 The monitor file usage24 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 40. From Figure 2-12 on page 24, the monitor uses the following configurationinformation:itmnp.properties This is the main configuration file. The location of this file is under the $CONFIG_DIR path, as specified in the startup script for the monitor.log.properties This configuration file contains the path to the Tivoli common directory. It typically resides in the /etc/ibm/tivoli/common/cfg; however, if you set the environment variable $FNPlogPropertiesLocation in the startup script, you can specify the location of the file.fnp_config.xml This is the configuration of the monitoring process. This file is retrieved from the Web application when the monitor is started. This configuration is used to build the objects for metric collection. It resides in the common log directory.The monitor uses the following files for output:dbcache This is the location of information caching in a Cloudscape™ database before being written to DB2. The location of this directory is specified in the variable $DBCacheDirectory from the itmnp.propertiesmsg_fnp_monitor*.log and trace_fnp_monitor*.log These are the messages and traces log files, which reside under the tivoli common directory, as specified in the log.properties file.The configuration XML file, fnp_config.xml, is the latest configuration XML filethat the monitor has received from the Web application. It is stored in the samedirectory as the log files. The structure of the XML file is shown in Figure 2-13 onpage 26. Chapter 2. Components and architecture 25
  • 41. Poller Configuration Document Poller Configuration Collection Collection CS390 Collection SNMP Collection SNMP Expression CS390 Target MIBData CS390 Data UDPTable FTP TN3270 TCPDetail TCPStor TCPListen SNMP Expression MIBData Threshold Value CS390Attributes CS390 System Data CSMStor EEHPT Interval CS390 System Target Event Forwarding Interval Event Forwarding Figure 2-13 Structure of fnp_config.xml As shown in Figure 2-13, the configuration of the monitor is split into multiple collection instructions. Each collection represents the requirement for data and the target table monitor to collect. There are three types of collection: OS390 collection, which collects data from the z/OS communication server SNMP collection, which acquires MIB data from SNMP ICMP collection, which records availability and response time data Each collection is further qualified with the interval definition and event forwarding destination information.2.3.3 Performance data sources The monitor uses a set of network management interfaces provided by the z/OS Communications Server to collect most of the z/OS performance data. The remaining performance data is collected using SNMP queries for performance data that is stored in MIBs. SNMP data can be collected from any IP-addressable resource in the enterprise that is running the SNMP agent. Tivoli Monitoring for Network Performance has provided a specific set of performance metrics that you can choose from when creating an SNMP monitor definition. For a complete list of the SNMP performance metrics supported by IBM Tivoli Monitoring for Network Performance, refer to Appendix B, “SNMP Performance Data”, in the IBM Tivoli Monitoring for Network Performance Version 2 Release 1 Administrator Guide, SC31-6364.26 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 42. Note: Some of the z/OS performance data is collected using SNMP queries. To ensure that you collect the necessary performance data, you must have the OSNMPD agent running on each of the z/OS TCP/IP stacks that you want to monitor.Availability and response time data can be collected from any IP-addressableresource in the enterprise that the monitor is able to ping. The monitor uses theping command to determine the availability of the resource. It uses the pingresponse times to calculate an average response time for each resource beingmonitored.The data collection schedule is determined by the start and end times and theintervals specified in the monitor definitions. The interval determines how oftenthe monitor will collect data. For example, if the interval is set to 30 minutes, themonitor will collect data every 30 minutes. This is true for all types of data exceptfor FTP data and data that is displayed on the TN3270 Server Sessions screen.FTP and TN3270 Server Session data is provided by the z/OS CommunicationsServer as events occur. As a result, the data on these screens will not changeaccording to the specified interval. Table 2-1 shows where the performance datais collected from.Table 2-1 Performance data sources Performance data category Data source TCP stack SNMP V2 Management Information Base for Transmission Control Protocol using SMIv2 and z/OS Communications Server management interfaces. UDP stack SNMPv2 Management Information Base for the User Datagram Protocol using SMIv2. IP stack SNMPv2 Management Information Base for the Internet Protocol using SMIv2. TN3270 and FTP sessions z/OS Communication Servers management interfaces. HPR and EE information z/OS Communication Servers management interfaces. Interface Interface group MIB. Adapter interface IBM OSA-Express Direct SNMP Enterprise-Specific MIB. Chapter 2. Components and architecture 27
  • 43. Performance data category Data source Memory data z/OS Communications Server management interfaces. Response time Result of ping command. SNMP performance data Use SNMP queries to collect performance data that is stored in MIBs. More discussion on monitors and the data related to them can be found in 6.5.1, “Monitored metrics” on page 138.2.3.4 Setting options The itmnp.properties file contains configuration parameters for the monitor. This file is typically copied to /etc/itmnp/ directory. This directory is specified in the variable $CONFIG_DIR within itmnpMonitor.sh shell script. The itmnp.properties file consists of the configuration parameters for the monitor. The detailed information for each parameters are provided as comments in the member; see our examples in “AIX itmnp.properties” on page 218 and “z/OS itmnp.properties” on page 218. The parameters can be grouped into: Monitor configuration monitor_hostname The fully qualified domain name of the system where the monitor is located. This name is used to retrieve the monitor configuration from the Web application and must match the monitor name provided when defining the monitor configuration. bind_interface The interface or stack which the monitor binds on this host. CSAPIport The first of the two ports used by the monitor for internal communication. The monitor uses the port and the next numbered port (such as 1670 and 1671). This is the port monitor_cs390 process is listening on. Communication between the monitor and Web application; see also 2.4, “Communication and security” on page 34 local_httpport The port number that the monitor listens on, and the Web application used to communicate with the monitor. This port is used by the Web application to transmit the monitor configuration.28 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 44. WAS_hostname The host name of the WebSphere Application Server where the Web application runs. WAS_httpport The port number that the monitor uses to communicate with the Web application. socksServer, socksPort The fully qualified URL and port for the Socks server the monitor will use to communicate with the WebSphere Application Server. (Optional) Database properties DBName The name of the DB2 database where the performance data will be stored. DBUserName The database user who has the authority to store data in the DB2 database. DBPassword The password of the DB2 database user. DBHostName The fully qualified host name of the system where DB2 is located. This is the same system as the WebSphere Application Server host name. DBPort The port number that DB2 database server is listening on. Typical value for AIX is 50000, while in z/OS you can see it from the DSNL004I message in the DB2 startup. DBDriverType The type of driver used to establish connections with the database. Use the value of UNIVERSAL.Tip: Although this parameter defaults to DBDriverType=UNIVERSAL, wefound that we need to comment it out for a DB2 for z/OS database. Explicitlyspecifying DBDriverType=UNIVERSAL make the connection to a distributedDB2 database. DBCacheDirectory The directory to be used by the monitor to store the collected data in a local cache. This cache stores the collected data before it is sent to the DB2 database. This directory is only used when the DBCachRowLimit and DBCacheTimeout are > 1. DBCacheTimeout The maximum number of seconds that the monitor will store data in a local cache before sending the data to the DB2 database. Data will be transferred to the DB2 database when either the maximum number of seconds is reached or the maximum number of rows is reached. Chapter 2. Components and architecture 29
  • 45. DBCacheRowLimit The maximum number of rows of data that the monitor stores in a local disk cache before attempting to store the data in the DB2 database. enableCloudscape Use Cloudscape as a cache. This may improve the performance for high volumes systems. db2Security This is the setting for the security level between the monitor and DB2 database. See 2.4.4, “Transport between DB2 and monitor” on page 39. SSL properties; all the keyStore and trustStore settings are for HTTPS protocol with the Web application resides on AIX. WebSphereServletProtocol The protocol that the monitor uses to communicate with the Web application. For non-secure, it is HTTP, and for secure, it is HTTPS. trustStoreName The name of the file where the WebSphere Application Server and monitor certificates are stored. trustStorePassword The password used to access the trust store file. keyStoreName The name of the file where the WebSphere Application Server and monitor keys are stored. keyStorePassword The password to access the key store file. keyManagerPassword The password to access the key manager interface. This should be the same as keyStorePassword. Message and trace log properties; see 2.3.5, “Problem determination for the monitor” on page 31 for more details trace.maxFiles The maximum number of trace files that may exist at one time. When the monitor closes a full trace file, it will ensure that no more than the maximum number exists by deleting the oldest trace file before creating a new trace file. When the number is 1, the file size limit is ignored. You can specify individual type (Java, CLI, or C) separately. trace.maxFileSize The maximum size (in kilobytes) each trace file may contain before the monitor closes the old trace log and creates a new trace log. You can specify individual type (Java, CLI, or C) separately. message.maxFiles The maximum number of message files that may exist at one time. When the monitor closes a full message file, it will ensure that no more than the maximum30 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 46. number exists by deleting the oldest message file before creating a new message file. You can specify individual type (Java, CLI, or C) separately. message.maxFileSize The maximum size (in kilobytes) each message file may contain before the monitor closes the old trace log and creates a new message log. You can specify individual type (Java, CLI, or C) separately. monitor.trace.level The logging level for the trace logs. This controls how much trace data is collected. The possible values are DEBUG_MIN, DEBUG_MID, and DEBUG_MAX.2.3.5 Problem determination for the monitor For more information on problem determination, refer to IBM Tivoli Monitoring for Network Performance: Messages and Troubleshooting, SC31-6366. This section contains our experience on performing the troubleshooting. In debugging the monitors, the main information sources are the XML trace and log files. These files are located under the Tivoli common directory, typically /var/ibm/tivoli/common/FNP/logs. This directory contains the following files: fnp_config.xml: The current configuration file; if this file does not exist, the monitor initialization may fail or the monitor cannot communicate with the Web application. msg_fnp_monitor*.log: The message file for the JVM process shown in Figure 2-11 on page 23. The database connection and configuration file processing messages are here. The number of files and their size are governed by the parameters message.Java.maxFiles and message.Java.maxFileSize. trace_fnp_monitor*.log: Trace file for detailed information on the JVM process; the number of these files and their size are governed by the parameters trace.Java.maxFiles and trace.Java.maxFileSize. msg_fnp_monitorc*.log: The message file for the monitor_cs390 and the launcher process, as shown in Figure 2-11 on page 23. The calls to Communication Server API messages are shown here. The number of files and their size are governed by the parameters message.C.maxFiles and message.C.maxFileSize. trace_fnp_monitorc*.log: Trace file for detailed information on the monitor_cs390 process; the number of this files and their size are governed by the parameters trace.C.maxFiles and trace.C.maxFileSize. Chapter 2. Components and architecture 31
  • 47. Note: The parameters trace.maxFiles, trace.maxFileSize, message.maxFiles, and message.maxFileSize contain the default value for the derivative trace.*.maxFiles, trace.*.maxFileSize, message.*.maxFiles, and message.*.maxFileSize parameters. The tracing levels provided are DEBUG_MIN, DEBUG _MID, and DEBUG_MAX. We recommend running the processes with the DEBUG_MIN level, as these files have an XML format and are huge. In our environment, a two minute interval produces more than 50 MB of trace files with a DEBUG_MAX level. The serviceability consideration must be take into account when allocating the HFS dataset. The trace level can also be modified using a Command Line Interface. You start the CLI using the command itmnpMonitor.sh cli. This will give the menu shown in Figure 2-14. VBUDI @ SC66:/itmnp/V2R1M0/bin>./itmnpMonitor.sh cli Launching ITMNP Monitor CLI Main Menu Current state: Running Configuration last loaded: Tue Jun 15 12:59:00 EDT 2004 Build level: Thu Apr 29 09:00:00 2004 1) Change trace logging levels 2) Suspend collection of all measurements 3) Resume collection of all measurements 4) Exit Type your selection number: Figure 2-14 Monitor CLI menu The trace level is changed by selecting 1 from the menu in Figure 2-14. This gives the screen in Figure 2-15 on page 33.32 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 48. Change Trace Logger Levels Menu 1) All loggers 2) CS390 Data Layer DEBUG_MIN 3) CS390 Layer DEBUG_MIN 4) CS390 Socket Data Layer DEBUG_MIN 5) Command Line Interface DEBUG_MIN 6) Database Layer DEBUG_MIN 7) ICMP Layer DEBUG_MIN 8) JMX Layer DEBUG_MIN 9) Monitor Base Code DEBUG_MIN 10) Monitor CS390 - Base DEBUG_MIN 11) Monitor CS390 - CSM Storage DEBUG_MIN 12) Monitor CS390 - EE and HPR DEBUG_MIN 13) Monitor CS390 - FTP and TN3270 DEBUG_MIN 14) Monitor CS390 - Hash table DEBUG_MIN 15) Monitor CS390 - SNA Management DEBUG_MIN 16) Monitor CS390 - TCP Applications DEBUG_MIN 17) Monitor CS390 - TCP Connections DEBUG_MIN 18) Monitor CS390 - TCP Storage DEBUG_MIN 19) Monitor CS390 - UDP Endpoints DEBUG_MIN 20) Monitor Services Code DEBUG_MIN 21) Monitor utilities DEBUG_MIN 22) Monitor utilities - Socket DEBUG_MIN 23) SNMP Layer DEBUG_MIN 24) Return to previous menu. Type your selection number:Figure 2-15 Trace settingAs an example, if you want to change the CS390 Data Layer setting, select 2 andthen specify the trace level, as shown in Figure 2-16. Type your selection number: 2 Select New Logging Level For: CS390 Data Layer 1) DEBUG_MIN 2) DEBUG_MID 3) DEBUG_MAX 4) Return to previous menu. Type your selection number:Figure 2-16 Trace level Chapter 2. Components and architecture 33
  • 49. 2.4 Communication and security As discussed in 2.1, “Components and functions” on page 8, IBM Tivoli Monitoring for Network Performance is a distributed application. With such a structure, communication is a vital part. Components of IBM Tivoli Monitoring for Network Performance communicate using the TCP/IP protocol, which raises some concern about the security of the management infrastructure. The platform selection for the Web application, on AIX or a z/OS system, determines the available options for securing the communication. The discussion in this section is divided into: 2.4.1, “User authentication mechanism” on page 34 2.4.2, “Communication port usage” on page 35 2.4.3, “Certificates and authentication with SSL” on page 38 2.4.4, “Transport between DB2 and monitor” on page 392.4.1 User authentication mechanism The IBM Tivoli Monitoring for Network Performance Web application relies on WebSphere Application Server for the user authentication. It supports two security authentication mechanisms: For AIX, IBM Tivoli Monitoring for Network Performance supports authentication using Lightweight Directory Access Protocol (LDAP) or local operation system (LocalOS). For z/OS, IBM Tivoli Monitoring for Network Performance supports authentication using LocalOS through z/OS Security Server or Resource Access Control Facility (RACF®) or another System Authorization Facility (SAF)-compliant product. The WebSphere Application Server security authentication mechanism supports implementation for most major LDAP directory servers or LDAP servers that can be used as the repository for user and group information. These LDAP servers are called by the WebSphere Application Server for authenticating user and other security related entities. If you are running IBM Tivoli Monitoring for Network Performance in a z/OS environment, you can use RACF to perform native authentication using z/OS V1 R2 SecureWay® Lightweight Directory Access Protocol (LDAP) Server. This arrangement makes it easy for the various components of IBM Tivoli Monitoring for Network Performance to access the information without having to replicate the data in other repositories.34 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 50. The z/OS implementation of this protocol provides the ability to use user and group information and update passwords in the z/OS Security Server or RACF. There are two directory access mechanisms provided for the RACF LDAP server: Traditional Database Manager (TDBM) or Security Database Manager (SDBM). This means that when an application attempts to authenticate an entry in the LDAP server, the password verification is performed by the SDBM information in RACF. If the password that was sent to the LDAP server matches the password that is stored in the Security Server user ID associated with that entry, then the application is authenticated as the DN of the directory entry in TDBM. Figure 2-17 shows the communication between SDBM and TDBM requests for a LDAP client. Authentication Request Via LDAP Client Bind() LDAP Server Web application SDBM TDBM _passwd() _ACLs RACF DB2 Figure 2-17 A z/OS LDAP server using an SDBM/TDBM configuration The basic usage of z/OS native authentication of IBM Tivoli Monitoring for Network Performance Web application does not requires a TDBM database configuration.2.4.2 Communication port usage Transport communication using TCP/IP can be prone to eavesdropping and spoofing. In this section, we discuss various port usage that can be mapped to IBM Tivoli Monitoring for Network Performance components depending on the encryption mechanism. Understanding communication port assignment is important to putting a firewall between the components. By default, the transport between the monitor and WebSphere Application Server is non-secure, meaning that the communication transport between the monitors Chapter 2. Components and architecture 35
  • 51. and WebSphere Server is not encrypted and not authenticated. If WebSphere Application Server is running on AIX, you can secure the transport by encrypting communication to and from the Web application. Non-secure communication In a non-secure configuration, the Hyper Text Transfer Protocol (HTTP) is used. Figure 2-18 describes the communication flows and the ports used. 9080 IBM Tivoli Monitoring for 9081 IBM Tivoli Monitoring for Network J2C Network Performance Performance Monitor Web Application z/OS Netview Integrated WebSphere Application Server TCP/IP services component HTTP DB2 38030 for z/OS 50000 for AIX z/OS or AIXFigure 2-18 Non-secure communication ports configuration In Figure 2-18, the following communication configuration happens: The monitor contacts WebSphere Application Server for configuration information on port 9080 using non-secure HTTP protocol. Port 9080 is specified with the was_httpport parameter in the itmnp.properties configuration file. The Java Messaging Extension (J2C) inside WebSphere Application Server responds on non-secure port 9081 using the HTTP protocol. Port 9081 is specified with the local_httpport parameter in the itmnp.properties configuration file. The NetView Integrated TCP/IP Services Component (ITSC) sends information IP resources to the Web application on non-secure port 9080 using HTTP. Port 9080 is specified with the WebsphereServletPort parameter in the nvexportd.properties file. Users accessing the Web application uses non-secure (HTTP) port 9080.36 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 52. Secure communication In a secure communication configuration, the HTTP secure (HTTPS) is used. HTTPS is a protocol based on Secure Socket Layer (SSL). Figure 2-19 shows the default ports and communication flows for a secure configuration for AIX Web application. Note: For the z/OS Web application, the communication between the WebSphere Application Server and the monitor cannot be HTTPS. IBM Tivoli Monitoring for Network Performance does not support authentication and encryption of this communication transport when running WebSphere Application Server and the DB2 product on a z/OS system. The Web application console user will still have an HTTPS session through port 9443. IBM Tivoli Monitoring for IBM Tivoli Monitoring for 9455 J2C HTTPS Network Performance 9454 Network Performance Monitor HTTPS Web Application 9454 HTTPS Netview TCP/IP WebSphere Application Server z/OS services component 9453 HTTPS DB2 50000 AIXFigure 2-19 Secure communication ports configuration for AIX Web application In this configuration: The monitor contacts WebSphere Application Server for configuration information on port 9454 using HTTPS. Port 9454 is specified with the was_httpport parameter in the itmnp.properties configuration file. The Java Messaging Extension (J2C) inside WebSphere Application Server responds on port 9455 using HTTPS. Port 9455 is specified with the local_httpport parameter in the itmnp.properties configuration file. Additional information that must be provided in the itmnp.properties file includes the TrustStore file name and password and the KeyStore file name and password. Chapter 2. Components and architecture 37
  • 53. The NetView Integrated TCP/IP Services Component sends information IP resources to the Web application on secure port 9454. Port 9454 is specified with the WebsphereServletPort parameter in the nvexportd.properties file. Additional information that must be provided in the nvexportd.properties file includes the TrustStore name and password and the KeyStore name and password. Internal port used by monitor The monitor component requires two ports for internal communications. The monitor will use the port specified on the CSAPIport parameter in the monitor itmnp.properties file as the first port. The second port used for internal communication will be the next numbered port. The default value for CSAPIport is 1670, which means ports 1670 and 1671 will be used by the monitor for internal communications. These ports are used regardless of a non-secure or secure configuration.2.4.3 Certificates and authentication with SSL The IBM Tivoli Monitoring for Network Performance includes a set of self-signed client and server certificates. These certificates are provided in .jks format as well as .arm format. The files itmnpMonitorCert.arm and itmnpServerCert.arm contain exported self-signed certificates that the security administrator can import into existing certificate stores and key rings by using tools such as IBM Global Security Kit (GSKIT) or the keytool command supplied with the Java Development Kit (JDK). Prior understanding of Secure Sockets Layer (SSL) and certificates is assumed; for more information on SSL and certificates, refer to Appendix D, “Underlying technology” on page 241. Table 2-2 shows the SSL trust stores, key stores, and certificates required to create secure SSL connections. You can also use this table to replace the self-signed certificates with certificates from a trusted Certificate Authority (CA).Table 2-2 SSL related files and certificates for secure connection Repositories TrustStore Trusted Certificate KeyStore WebSphere itmnpServerTrustStore.jks itmnpServerCert.arm itmnpServerKeyStore.jks Application itmnpItsccert.arm Server Monitor itmnpMonitorTrustStore.jks itmnpMonitorCert.arm itmnpMonitorKeyStore.jks itmnpServerCert.arm38 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 54. Repositories TrustStore Trusted Certificate KeyStoreNetView itmnpItscTrustStore.jks itmnpServerCert.arm itmnpItscKeyStore.jksIntegratedTCP/IP ServicesComponent The server certificates are installed when you install the Web application on the WebSphere Application Server. The server key store file contains the public and private keys for the server. The server trust store file contains the self-signed certificates for the monitors and NetView Integrated TCP/IP Services Component. The client certificates are installed when you install the monitor component. The client key store file contains the public and private key of the client. The client trust store file contains the monitor’s and server’s self-signed certificates. The secure ports used by the Web application and the monitor are described in 2.4.2, “Communication port usage” on page 35. To secure the transport between WebSphere Application Server and the monitors, you must specify values for the trustStoreName and keyStoreName parameters in the itmnp.properties file for each monitor. To secure the transport between the NetView Integrated TCP/IP Services Component and WebSphere Application Server, you must copy the client certificates from the IBM Tivoli Monitoring for Network Performance CD-ROM and store them on the server where the NetView Integrated TCP/IP Services Component is running. You must then specify values for the trustStoreName and keyStoreName statements in the nvexportd.properties file. If you choose to secure the transport, all of the monitors and systems where the NetView Integrated TCP/IP Services Component is running must be configured to secure the transport.2.4.4 Transport between DB2 and monitor The DB2 product uses one port to listen for communication from the monitors. The port number is specified using the DBPort parameter in the monitor itmnp.properties file. The default value for this port is 50000 for AIX. Note: The discussion in this section applies to a Web application running on AIX. Chapter 2. Components and architecture 39
  • 55. By default, IBM Tivoli Monitoring for Network Performance uses password-protected security when it transmits data to and from the DB2 database. A user ID and password are verified at the DB2 server, and the password is transmitted using CLEAR_TEXT_PASSWORD_SECURITY. Security may be increased by configuring for encrypted passwords. Security may be reduced by configuring for CLIENT verification. Communication between the monitors and the DB2 database can be secured using security mechanisms provided by the DB2 server. One of these mechanisms should be used to protect against unauthorized access to your databases. The DB2 product supports five security mechanisms, including two that require Kerberos. However, the IBM Tivoli Monitoring for Network Performance does not support Kerberos as an option. As a result, only three of the mechanisms are supported in this environment. The following DB2 security mechanisms are supported by IBM Tivoli Monitoring for Network Performance. The security parameter db2Security in itmnp.properties determines how and where authentication of a user takes place. The supported types are: CLIENT: Authentication takes place at the client. As a result, authentication is not performed at the server. This is the IBM-supplied default value for IBM Tivoli Monitoring for Network Performance. SERVER: User ID and password are sent from the client to the server so that authentication can take place on the server SERVER_ENCRYPT: User ID and password are sent from the client to the server so that authentication can take place on the server. This is the same as SERVER, except that any passwords sent over the network are encrypted. The IBM Tivoli Monitoring for Network Performance monitor component includes a DB2 universal driver that allows the monitors to communicate with the DB2 server from a remote system. The DB2 universal driver configuration options reside in the itmnp.properties file, known as DBDriverType. The names of these security mechanisms do not exactly match the names of the security mechanisms on the DB2 server. However, they are related. Table 2-3 on page 41 contains the supported DB2 server mechanisms and the universal driver security mechanisms that are compatible with each one.40 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 56. Table 2-3 DB2 universal driver configuration options DB2 Server Security Mechanism Values for db2Security statement in the itmnp.properties file CLIENT USER_ONLY_SECURITY SERVER CLEAR_TEXT_PASSWORD_SECURITY SERVER_ENCRYPT CLEAR_TEXT_PASSWORD_SECURITY ENCRYPT_PASSWORD_SECURITY ENCRYPT_USER_AND_PASSWORD_SECURITY2.5 Database structure IBM Tivoli Monitoring for Network Performance relies heavily on the data stored in the DB2 database central repository. We feel that it is helpful to understand some of the data structure in the database. This section serves as an overview of the database structure. Refer to the actual installed database for a detailed database structure. The database itself can be considered to consist of two distinct set of tables: The configuration tables, which consists of configuration information from the Web application The measurement tables, which contain the metrics and thresholds for network performance2.5.1 Configuration tables The tables in this area define most of the collection and node information. The relationship between tables can be seen in Figure 2-20 on page 42. Chapter 2. Components and architecture 41
  • 57. SMARTSET_OBJ NETWORK_OBJ SS_NODE_RELN SS_INT_RELN SS_OBJ_RELN T_SET_SS_RELN NODE_OBJ INTERFACE_OBJ NETVIEW_ID_MAP T_SET_NODE_RELN T_SET_IFACE_RELN TARGET_SET T_SET_IP_RANGE T_SET_IP_WILD T_SET_HN_WILD EVENT_DEST PRIM_POLL_PARM MONITOR_INST_CFG COL_EV_DEST_RELN COL_SET_RELN INTERVALGRP INTRVL_TBL SNM_POLL_PARMS MONITOR_COL_RELN COLL_TBL COL_INTERVAL_RELN COL_ATTR_RELN COL_REARM_RELN COL_THRESH_RELN COL_ATTR_GRP REARM_GRP THRESH_GRP SNMP_REARM_EXPR CS390_REARM_EXPR ICMP_COL_ATTR ICMP_THRESH_EXPR ICMP_REARM_EXPR SNMP_COL_ATTR CS390_COL_ATTR SNMP_THRESH_EXPR CS390_THRESH_EXPR BEST_PRACTICES MIB_OBJ MIB_EXPRFigure 2-20 Configuration tables database structure In Figure 2-20, the lines with names are relationships that were stored as tables, while the dotted lines are relationships by content of a key column. Some of the interesting tables are: NODE_OBJ All TCP/IP addressable nodes that are either entered manually or discovered using NetView ITSC. INTERFACE_OBJ Interface list of all discovered nodes. SMARTSET_OBJ Smart-set list of smart set objects in NetView ITSC.42 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 58. NETVIEW_ID_MAP List of NetView engines that discover the configuration. NETWORK_OBJ Network list that is discovered by the NetView ITSC engine. TARGET_SET A collection of nodes and interfaces that can be monitored in a single collection. COLL_TBL A collection list, which is the primary configuration table for collection. This relates to the collection tag in the fnp_config.xml (see Figure 2-13 on page 26). EVENT_DEST List of event destinations that are configured from the Web application run-time preferences. MONITOR_INST_CFG List of monitor instances and active collection in them. INTERVL_TBL List of schedules, that together with the INTERVALGRP table, build the monitoring schedule. This makes up the Interval tag in the fnp_config.xml file. COLL_ATTR_GRP A link to either SNMP_COL_ATTR, ICMP_COL_ATTR, or CS390_COL_ATTR, depending on the collection type. As shown in Figure 2-13 on page 26, a collection can be a CS390 collection, ICMP collection, or SNMP collection. REARM_GRP, THRESH_GRP A link to the threshold and rearm tables for CS390, ICMP, and SNMP collection. BEST_PRACTICES Link from SNMP collection to MIB objects to be collected. MIB_OBJ, MIB_EXPR Tables for MIB specification. There are additional tables that are not mentioned in the diagram; they are: LAST_ETL_RUN This is used by Tivoli Data Warehouse as a processing bookmark. ENUM_TYPES Enumeration value list.2.5.2 Measurement tables The measurement tables are a set of tables with the suffix of _MSMT. Most of these tables have matching tables with the suffix of _THSH, which contains information for measurements that cross the threshold. The following are in the measurement table list: CSM_SUMM_MSMT CSM storage summary CSM_MON_MSMT CSM storage monitoring Chapter 2. Components and architecture 43
  • 59. FTP_SESS_MSMT FTP sessions FTP_CTRANS_MSMTFTP Client transfer records FTP_STRANS_MSMTFTP Server transfer records HPR_AVL_MSMT HPR availability and response time HPR_PIPE_MSMT HPR pipe measurement HPR_TT_MSMT HPR throughput and traffic EE_AVL_MSMT EE availability and response time EE_TT_DET_MSMT EE throughput and traffic detailed measurement EE_TT_MSMT EE throughput and traffic ICMP_RTT_MSMT ICMP response time IF_STATUS_MSMT Interface status IF_UNICAST_MSMT Unicast performance metrics IF_MULTI_MSMT Multicast/broadcast performance metrics OSA_ETH_TT_MSMTOSA Ethernet throughput and traffic OSA_STATUS_MSMTOSA status OSA_TT_MSMT OSA throughput and traffic OSA_TT_UTIL_MSMTOSA utilization SNMP_EXPR_MSMT SNMP expression collection SNMP_INT_MSMT SNMP integer MIB collection SNMP_STRING_MSMTSNMP string MIB collection STK_IP_TT_MSMT IP Stack Throughput and Traffic STK_TCP_AVL_MSMTTCP stack availability and response time STK_TCP_TT_MSMT TCP stack throughput and traffic TCP_APP_AVL_MSMTApplication availability and response time TCP_CONN_AVL_MSMTConnection availability and response time TCP_CONN_TT_MSMTConnection throughput and traffic TCP_PRV_MEM_MSMTPrivate memory TN3270_AVL_MSMT TN3270 session availability TN3270_RESP_MSMTTN3270 sliding-window response time TN_RTT_BKT_MSMT TN3270 response time counts by time bucket TN_RTT_BND_MSMTTN3270 response time counts by bound session STK_UDP_TT_MSMT UDP stack throughput and traffic44 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 60. UDP_EP_TT_MSMT UDP endpoint throughput and trafficTRACERT_MSMT Trace route performanceMeasurement tables that do not have a corresponding threshold tables areTN_RTT_BND_MSMT and OSA_TT_UTIL_MSMT. Chapter 2. Components and architecture 45
  • 61. 46 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 62. 3 Chapter 3. Implementation scenarios This chapter discusses the implementation scenarios for IBM Tivoli Monitoring for Network Performance. We first evaluate the different pieces that made up IBM Tivoli Monitoring for Network Performance and come up with the scenarios and discuss each of them. The discussion is divided into: 3.1, “Implementation components” on page 48 3.2, “Implementation scenarios” on page 48 3.3, “Scenario comparison” on page 50© Copyright IBM Corp. 2004. All rights reserved. 47
  • 63. 3.1 Implementation components The implementation of IBM Tivoli Monitoring for Network Performance can be thought of as a series of implementations of its components. There are several components that are of interest, as they provide options on how they can be implemented. They are: Web application platform The Web application can run on a z/OS or AIX server. This has several implications, as there are differences in the architecture of the operating systems and the application itself. Implementation on AIX will introduce a new operating system on the management platform. Event receiver The event receiver can be IBM Tivoli Enterprise Console or IBM Tivoli NetView for z/OS, depending on which platform the existing automation resides; the decision must be made to optimize the existing infrastructure. Historical data Currently, IBM Tivoli Monitoring for Network Performance only supports Tivoli Data Warehouse to collect and report historical data. If your current historical reporting utilizes Tivoli Decision Support for OS/390, you need to create your own collection programs. Communication security The highly distributed nature of IBM Tivoli Monitoring for Network Performance makes the communication critical. Some components of IBM Tivoli Monitoring for Network Performance support encryption using SSL and HTTPS, while some do not.3.2 Implementation scenarios Based on the component implementation selection in 3.1, “Implementation components” on page 48, we choose two scenarios to implement in this project. The scenarios are: Distributed servers managing a z/OS environment z/OS based implementation The discussion here is divided into the following sections: 3.2.1, “Distributed servers environment” on page 49 3.2.2, “Pure z/OS environment” on page 5048 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 64. 3.2.1 Distributed servers environment In this environment, we bring in several non z/OS servers that will be used to manage our z/OS environment. The implementation scope is shown in Figure 3-1. Netview Integrated TCP/IP Services Component Websphere Application Server IBM Tivoli Monitoring Win2K - Laredo IBM Tivoli Monitoring for for Network WAS Network Performance JMX Performance Web Application Monitor DB2- ITMNPDB z/OS – SC62 Monitor Configuration Tivoli Data Tivoli Enterprise Warehouse Performance Console Server Server Central Data data Warehouse and Data marts AIX - Jakarta Win2K - Laredo Crystal Enterprise Server Win2K - PhoenixFigure 3-1 Components for IBM Tivoli Monitoring for Network Performance The configuration in Figure 3-1 consists of: IBM Tivoli Monitoring for Network Performance monitors running on z/OS machines. The IBM Tivoli Monitoring for Network Performance Web application component with DB2 and WebSphere Application Server are installed on an AIX machine to monitor z/OS machines. The IBM Tivoli Enterprise Console server receives events when specific thresholds are crossed in IBM Tivoli Monitoring for Network Performance. NetView Integrated TCP/IP Services Component (ITSC) provides support to automatically discover IP-addressable resources in the enterprise. The IP-addressable resources can be very helpful when configuring the IBM Tivoli Monitoring for Network Performance monitors in your enterprise. A Tivoli Data Warehouse control server with Crystal Enterprise running, which collects data captured by IBM Tivoli Monitoring for Network Performance. The data can be summarized and migrated into the Tivoli Data Warehouse. The historical data is distributed to the user using Crystal Enterprise through a Web browser. Chapter 3. Implementation scenarios 49
  • 65. Communication between these components supports either HTTP or HTTPS.3.2.2 Pure z/OS environment In this environment, we use a z/OS system to act as the management server. We run the Web application and IBM Tivoli NetView for z/OS there. We still need a non-z/OS platform for this solution for the following items: The historical reporting portion of IBM Tivoli Monitoring for Network Performance for Tivoli Data Warehouse Network object discovery using NetView ITSC Our configuration is shown in Figure 3-2. z/OS WTSC61 Tivoli Data Crystal Ware Enterprise house product DB2 DB2 ITMNPDB TWH_CDW Netview Integrated TCP/IP Services Component z/OS Itmnp21 itmnpMonitor WTSC52 Web MQ Appication (JMS) WMQX itmnpMonitor WebSphere EAS Netview Figure 3-2 z/OS configuration3.3 Scenario comparison The comparison in this section is divided into the following sections: 3.3.1, “Distributed consideration” on page 51 3.3.2, “z/OS only consideration” on page 5250 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 66. 3.3.3, “Summary” on page 533.3.1 Distributed consideration The IBM Tivoli Monitoring for Network Performance has the ability to run in a distributed way. In this scenario, we have the IBM Tivoli Monitoring for Network Performance monitor component running on zSeries, and the Web application component running on pSeries®. This distributed approach can be really useful: When you have pSeries machines being used in the enterprise and have enough capacity to install the Web Application component on it. Refer to IBM Tivoli Monitoring for Network Performance: Planning, Installation, and Configuration, SC31-6363 for the AIX system requirements. Having WebSphere and DB2 skills is not a necessity, but in some enterprises, those skills does not belong to the zSeries personnel. If the enterprise already has the WebSphere Application Server and DB2 running on a pSeries machine running AIX, you could install the Web Application component (WebSphere application Server and DB2) on it to save time. You can use the same WebSphere Application Server and DB2; do not forget that the WebSphere Application Server and DB2 must be in the same operating environment. You can save some zSeries CPU cycles if you deploy the Web Application component on a pSeries machine. The cost of software acquisition. WebSphere Application Server and DB2 for AIX are included with IBM Tivoli Monitoring for Network Performance. When you have requirements for a security configuration, with authentication and SSL communication between the monitor and the browser with the Web Application, you have to run the Web Application on a pSeries machine under AIX. On the other hand, running IBM Tivoli Monitoring for Network Performance in a distributed way can bring more complexity to the scenario. There are some considerations you have to look at: Managing a distributed environment will require more skills, perhaps more people, and more than one team. Do not forget that other components like Tivoli Data Warehouse can be used on the solution and this can include other platforms like Windows 2000. Most enterprises have different teams responsible for each one of the platforms. Troubleshooting in a distributed environment can be really difficult. As you have the components running on different operating systems and, sometimes, remote locations, you will have to deal with network problems Chapter 3. Implementation scenarios 51
  • 67. and different types of operating systems, each one with its own problems and peculiarities. If you are using a LDAP server to authenticate the users and this LDAP server is not available for any reason, you will not be able to log on to the Web Application to look at the data being collected. Losing one of the components could really cause some problems for the whole solution. There is no rule to define if a distributed scenario is the best way to implement a ITMNP solution. You have to consider a lot of enterprise factors to decide which implementation will better fit your enterprise.3.3.2 z/OS only consideration As shown in 3.2.2, “Pure z/OS environment” on page 50, all the major components of IBM Tivoli Monitoring for Network Performance can run on zSeries: the Monitor component and the Web Application component. Running a mainframe only scenario can have some advantages: You have only a single platform to manage. All the major components will be running on a single environment, which means you have to work with only one platform. This will require less people being involved with the solution. Of course, there are at least three major components or subsystems to be considered here: z/OS Communication Server, WebSphere Application Server, and DB2. The backup and recovery implementations on the mainframe platform tend to be more reliable, and most of the enterprises have them fully implemented, especially if they have DB2 and WebSphere Application Server already running. From the troubleshooting perspective, as everything is running on the same environment, it is easier to do problem determination. From the network perspective, if there are remote systems on the configuration, the network could be a concern. The new machines z890 and z990 have a new CP that can play an important role while running Java applications on a z/OS 1.6 using JDK 1.4 (the zAAP processor). If you have those prerequisites and are planning to implement Java applications on a zSeries machine, this can be really helpful, saving some CPU cycles from the CPs and not paying additional software costs for the additional CPU capacity The drawback of running a mainframe only scenario is: If the enterprise is not eligible to run the zAAP processor, you will have some additional CPU consumption to run this application, and maybe some52 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 68. additional software costs. The distributed scenario can be favorable in this manner.3.3.3 Summary In this section, we try to summarize the comparison between the benefits and drawbacks of each solution. The results are shown in Table 3-1.Table 3-1 Solution comparison Area z/OS only Distributed server Software licenses WebSphere Application Server and DB2 WebSphere Application Server and DB2 need to be acquired separately. for AIX are already included with the product. Administration The main components reside in a Needs to administer monitors skill single platform, although they run on running on UNIX System Services; UNIX System Services. Web application server running on Additional skill is required for Tivoli AIX; Event management on IBM Data Warehouse administration. Tivoli Enterprise Console and Tivoli Data Warehouse for historical The core function needs some z/OS reporting. system programmer skill for WebSphere, DB2, and network. The core function needs a team of administrators for these different functions: z/OS network administrator, Tivoli Framework administrator and AIX system administrator. Hardware Having the Web application running on Offloading the Web application function consideration z/OS machine requires a Java in AIX may save some memory and CPU environment. JVM in general requires a constraint on a zSeries partition, which large amount of virtual memory and the typically has a higher price compared to semi interpreted nature of Java requires a pSeries machine. more CPU cycles. This may result in a dedicated logical partition (LPAR) for the management system, unless the system has a zAAP processor that helps the Java based application run more efficiently. Communication Secured HTTPS communication only All communication to and from the Web requirement supported with the Web console user. application can be secured using the The communication with the monitor and HTTPS protocol. NetView ITSC is not secure (HTTP). Chapter 3. Implementation scenarios 53
  • 69. Area z/OS only Distributed server Authentication Native local OS authentication. Distributed authentication using LDAP to the z/OS security server or a local AIX authentication. Backup and Backup process can be coordinated Backup needs to be performed on the recovery purely from z/OS perspective. monitors (z/OS) and Web application with its database (distributed).3.4 Scenario implementation roadmap We decided to make the scenario similar to the overall configuration shown in Figure 3-3. Mainframe scenario Distributed scenario ITMNP ITMNP ITMNP ITMNP monitor monitor monitor monitor z/OS – SC52 z/OS – SC62 z/OS – SC66 ITMNP ITMNP Web application Web application IBM Tivoli Enterprise Console WebSphere WebSphere Application Application IBM Tivoli NetView Server Server ITSC + DB2 + DB2 JMS JMS Windows 2000 - laredo AIX - jakarta Tivoli Data Tivoli Data IBM Tivoli NetView Warehouse Warehouse for z/OS Crystal Enterprise Crystal Enterprise Windows 2000 - kingston z/OS – SC61 Windows 2000 - phoenixFigure 3-3 Overall implementation environment The implementation discussion is divided into the following: Distributed Web application implementation is discussed in Chapter 4, “AIX Web application implementation” on page 57. This includes the following: – Preparation for the Web application implementation54 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 70. – Web application installation tips – Backup procedure for the Web application Mainframe z/OS implementation is discussed in Chapter 5, “Mainframe Web application implementation” on page 101. This includes the following: – Preparation for the Web application implementation – Web application installation tips – Backup procedure for the Web application Monitor implementation is discussed in Chapter 6, “Monitor implementation and operation” on page 125. This includes the following: – Preparation for the monitor implementation – Monitor implementation tips – Monitor configuration discussion – Backup procedure for the monitor Discovery and alert interface implementation is discussed in Chapter 7, “Discovery and alert interfaces” on page 161. This includes the following: – IBM Tivoli NetView for discovery – IBM Tivoli Enterprise Console as the event receiver – IBM Tivoli NetView for z/OS as the event receiver Historical reporting with Tivoli Data Warehouse is discussed in Chapter 8, “Historical reporting” on page 183. This includes the following: – Installation process overview for distributed systems – Installation process overview for z/OS database – Warehouse Enablement Pack installation – Running the ETL program – Sample reports3.5 User operation and roles IBM Tivoli Monitoring for Network Performance can be used by various users and, depending on their roles, they may use different parts of the overall solution. Basically, the users of IBM Tivoli Monitoring for Network Performance can be categorized into: Network system programmer The network system programmer uses IBM Tivoli Monitoring for Network Performance as both an online Chapter 3. Implementation scenarios 55
  • 71. monitoring tool and historical analysis of performance and capacity information. They need access to the Web application for configuring the monitor, monitoring the result, and managing the historical data. Network administrator The distributed network administrator will be interested in getting information on how the network performs in regard to accessing the mainframe applications and resources. Network operator The network operator can monitor network availability through the graphical interface for IBM Tivoli Monitoring for Network Performance Web application and get information about potential performance problems. Capacity planner The capacity planner, looking at the collected historical data in Tivoli Data Warehouse, can predict the network communication usage and capacity for years to come. Service Level Coordinator The service level coordinator uses the information collected from IBM Tivoli Monitoring for Network Performance to ensure no service level violation can and will occur. Database administrator The database administrator needs to help the IBM Tivoli Monitoring for Network Performance maintainer to manage the database. Automation specialist The automation specialist evaluates events from IBM Tivoli Monitoring for Network Performance and programs specific responses for automating problem resolution. Web site administrator The Web master uses performance information collected by IBM Tivoli Monitoring for Network Performance to improve the Web pages that he/she administers. Security administrator The security administrator coordinates the security of the access to the IBM Tivoli Monitoring for Network Performance application, specifically on the z/OS platform.56 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 72. 4 Chapter 4. AIX Web application implementation This chapter documents our installation experiences on implementing IBM Tivoli Monitoring for Network Performance on AIX in a distributed environment scenario. The following topics are covered in this chapter: 4.1, “Web application on AIX overview” on page 58 4.2, “Preparing for the installation” on page 59 4.3, “Web application implementation” on page 66 4.4, “Some problems and their solutions” on page 82 4.5, “Start and stop procedures” on page 90 4.6, “Backup and recovery” on page 91© Copyright IBM Corp. 2004. All rights reserved. 57
  • 73. 4.1 Web application on AIX overview The distributed scenario discussed in 3.2.1, “Distributed servers environment” on page 49 uses an AIX Web application server. This chapter deals with the implementation of this AIX-based IBM Tivoli Monitoring for Network Performance Web application. The Web application in this scenario manages the monitors running on SC62 and SC66. The overall structure of this scenario and the host name associated with it are shown in Figure 4-1. The DB2 and WebSphere Application Server are installed on AIX machine jakarta. The implementation is using a secure transport between the monitors and the Web application. Netview Integrated TCP/IP Services Component Websphere Application Server IBM Tivoli Monitoring Win2K - Laredo IBM Tivoli Monitoring for for Network WAS Network Performance JMX Performance Web Application Monitor DB2- ITMNPDB z/OS – SC62 Monitor Configuration Tivoli Data Tivoli Enterprise Warehouse Performance Console Server Server Central Data data Warehouse and Data marts AIX - Jakarta Win2K - Laredo Crystal Enterprise Server Win2K - PhoenixFigure 4-1 Distributed environment for IBM Tivoli Monitoring for Network Performance As shown in Figure 4-1, there are some additional machines that we use. We have a Windows 2000 system, laredo, with NetView Integrated TCPIP Services Components (ITSC) and IBM Tivoli Enterprise Console (TEC) installed on it. Tivoli Enterprise Console Server The Tivoli Monitoring for Network Performance monitors can be configured to generate events to notify operators when specific thresholds are crossed. Events can be received by the IBM Tivoli Enterprise Console server. To use TEC as a receiver, you must create an IBM Tivoli Enterprise Console RuleBase with IBM Tivoli Monitoring for Network Performance classes.58 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 74. NetView Integrated TCP/IP Services Component NetView Integrated TCP/IP Services Component provides support to automatically discover IP-addressable resources in the enterprise. The IP-addressable resources can be very helpful when configuring the IBM Tivoli Monitoring for Network Performance monitors in your enterprise. We have a Windows 2000 system, phoenix, with Crystal Enterprise Server and Tivoli Data Warehouse. The performance data captured by IBM Tivoli Monitoring for Network Performance can be summarized and migrated into the Tivoli Data Warehouse, the Tivoli strategic cross-application infrastructure for archiving data and generating reports. The IBM Tivoli Monitoring for Network Performance uses a set of extract, transform, and load (ETL) utilities to summarize and migrate historical performance data. The ETL files are SQL scripts that execute from the Tivoli Data Warehouse server. ETLs summarize and migrate data into the Tivoli Data Warehouse central data warehouse database and then onto the Tivoli Monitoring for Network Performance data mart database. Data from the data mart can be used to generate historical reports with the Crystal Enterprise product, which is provided as part of Tivoli Data Warehouse. The report files are read by the Crystal Enterprise Server to produce reports. Reports can be viewed from any Web browser through a connection to the Crystal Enterprise Server.4.2 Preparing for the installation Before we install the IBM Tivoli Monitoring for Network Performance Web application, we need to preinstall WebSphere Application Server V5 and DB2 Universal Database Version 8 on the AIX machine. We went through the software and hardware prerequisites in Appendix A, “Software and hardware prerequisites“, in IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363 and made sure that we met the prerequisites, especially the required APARs.4.2.1 File system space The installation of the Web application uses an Install-Shield Multi Platform (ISMP) wizard. The wizard can install the database and the Web application automatically. You need to understand where the files are created and prepare sufficient file system space in the AIX machine. Chapter 4. AIX Web application implementation 59
  • 75. Checking allocated file systems In AIX, the physical disk is partitioned in file systems, typically the Journaled File Systems (JFS). For z/OS users, this is much like the Hierarchical File Systems (HFS) or ZFS in UNIX System Services. Use the df commands to get the allocated and free sizes of the file systems, as shown in Example 4-1. The -k option gives the result in kilobytes instead of 512 byte segments. Example 4-1 Checking file systems in AIX # df -k Filesystem 1024-blocks Free %Used Iused %Iused Mounted on /dev/hd4 16384 0 100% 1726 22% / /dev/hd2 983040 12036 99% 29402 12% /usr /dev/hd9var 163840 143364 13% 702 2% /var /dev/hd3 1425408 358640 75% 80 1% /tmp /dev/hd1 16384 15816 4% 26 1% /home /proc - - - - - /proc /dev/hd10opt 32768 24228 27% 356 5% /opt /dev/lv00 2457600 151580 94% 6208 1% /image /dev/lv01 5242880 4449812 16% 740 1% /home/db2inst1 /dev/lv02 458752 13828 97% 8560 8% /usr/opt/db2_08_01 /dev/lv03 65536 54396 17% 79 1% /home/dasusr1 /dev/lv04 16384 15824 4% 18 1% /home/db2fenc1 /dev/lv05 1310720 95044 93% 398 1% /usr/sys/inst.images /dev/lv06 802816 306624 62% 16266 9% /usr/WebSphere/AppServer /dev/lv07 49152 28024 43% 1087 9% /usr/IBMHttpServer /dev/lv08 425984 372992 13% 19 1% /itmnp /dev/lv09 114688 110516 4% 29 1% /usr/ibm/tivoli /dev/lv10 131072 64712 51% 281 1% /opt/IBM/ITMNP21 File system needed for installation Table 4-1 shows the file systems that will be used by the installation process. This includes the file systems for the prerequisite products.Table 4-1 File systems used by the Web application installation Path name Free Our Remark space size /usr/WebSphere/AppServer 117 MB 350 MB WebSphere path for the Web application /opt/IBM/ITMNP21 53 MB 53 MB IBM Tivoli Monitoring for Network Performance supplied files /tmp 115 MB 200 MB Temporary installation files /home/db2inst1 120 MB 512 MB IBM Tivoli Monitoring for Network Performance database files60 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 76. The path names in Table 4-1 on page 60 may be included in other file systems in higher level names. We recommend that you create these file systems in advance before installing WebSphere Application Server and DB2. We include the total size of our file systems as a guide for pre-allocating them. For detailed information about planning database space, see Appendix G, “Database Sizings and Configuration for Tivoli Monitoring for Network Performance,” in IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363. Changing file system allocation In AIX, the procedure for modifying the file systems (and also most of the system administration tasks) are performed using the System Management Interface Tool or smit command. This tool can be launched in a text based interface or an X-Windows based Graphical User Interface. For file system administration, use the command smit jfs. This will bring up the JFS menu, as shown in Figure 4-2. Journaled File Systems Move cursor to desired item and press Enter. Add a Journaled File System Add a Journaled File System on a Previously Defined Logical Volume Change / Show Characteristics of a Journaled File System Remove a Journaled File System Defragment a Journaled File System Figure 4-2 Smit JFS menu You can create or modify a file system through this menu. Remember that the size that you specify is always in 512 byte (0.5 KB) units.4.2.2 Setting up the Java Messaging Service The Java Messaging Service (JMS) is a stripped down version of WebSphere MQ for message exchange. This feature of WebSphere Application Server must be installed on the AIX machine. To be able to install this feature, you need to prepare the following: 1. Create the mqm and mqbrkrs groups. Run the command smit group to do this. The Group management window is shown in Figure 4-3 on page 62. Chapter 4. AIX Web application implementation 61
  • 77. Groups Move cursor to desired item and press Enter. List All Groups Add a Group Change / Show Characteristics of a Group Remove a Group Figure 4-3 Group menu 2. Choose Add a Group and press Enter. The Add Group dialog is shown in Figure 4-4. Add a Group Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Group NAME [mqm] ADMINISTRATIVE group? false + Group ID [] # USER list [root] + ADMINISTRATOR list [root] + Figure 4-4 Add Group dialog 3. Specify the group name of mqm and press Enter. 4. Perform the same thing for group mqbrkrs. 5. Press Esc+0 to exit the smit display. If you have an operational WebSphere Application Server, you should verify that JMS is operational by looking at the SystemOut.log, which is located in <WAS_HOME>/logs/server1. In our case, the SystemOut.log is in /usr/WebSphere/AppServer/logs/server1. Examples of the JMS messages are shown in Example 4-2, “JMS messages in SystemOut.log” on page 63.62 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 78. Example 4-2 JMS messages in SystemOut.log [5/12/04 12:28:51:475 CDT] JMSEmbeddedPr MSGS0050I: Starting the Queue Manager [5/12/04 12:28:59:851 CDT] JMSEmbeddedPr MSGS0051I: Queue Manager open for business [5/12/04 12:28:59:871 CDT] JMSEmbeddedPr MSGS0052I: Starting the Broker [5/12/04 12:29:04:802 CDT] JMSEmbeddedPr MSGS0053I: Broker open for business To make sure that the WebSphere Application Server is up and running, find the following message in SystemOut.log: Server server1 open for e-business4.2.3 User authority The user ID that performs the installation needs to be authorized to the file systems and the DB2 database. Authorization to the WebSphere Application Server is acquired using a user ID and password that is specified in the wizard. Typically, when WebSphere Application Server is installed, security is disabled so that there is no authentication needed. We perform the installation using the root user. The root user needs access to DB2 database, which is owned by the so-called DB2 instance owner. Typically, the instance owner user is called db2inst1 with group db2grp1. Unless you connect root to the DB2 owner group, the database will not be created by the wizard. Perform the following to add root to the DB2 group db2grp1. Note: This should be performed after DB2 has already been installed. 1. In AIX session, log in as root. 2. Run the command smit user, which gives the screen shown in Figure 4-5 on page 64. Chapter 4. AIX Web application implementation 63
  • 79. Users Move cursor to desired item and press Enter. Add a User Change a Users Password Change / Show Characteristics of a User Lock / Unlock a Users Account Reset Users Failed Login Count Remove a User List All Users Figure 4-5 The smit user display 3. Choose Change / Show Characteristics of a User, which gives you the screen shown in Figure 4-6. Change / Show Characteristics of a User Type or select a value for the entry field. Press Enter AFTER making all desired changes. [Entry Fields] * User NAME [root] + Figure 4-6 Change / Show characteristics of a User 4. Put root in the User NAME field and press Enter. This gives you the screen shown in Figure 4-7 on page 65.64 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 80. Change / Show Characteristics of a User Type or select values in entry fields. Press Enter AFTER making all desired changes. [TOP] [Entry Fields] * User NAME root User ID [0] # ADMINISTRATIVE USER? true + Primary GROUP [system] + Group SET <dit,lp,db2grp1,mqm,mq> + ADMINISTRATIVE GROUPS [mqbrkrs,mqm] + ROLES [] + Another user can SU TO USER? true + SU GROUPS [ALL] + HOME directory [/] Initial PROGRAM [/usr/bin/ksh] User INFORMATION [] EXPIRATION date (MMDDhhmmyy) [0] Is this user ACCOUNT LOCKED? false + [MORE...36] Figure 4-7 Root user properties 5. Add db2grp1 to the Group SET and press Enter to apply the changes (shown highlighted in Figure 4-7). When the command status is OK, press Esc+0 to exit from the smit interface.4.2.4 Enabling DB2 password encryption DB2 password encryption level is an instance wide setting. It can be changed using the command line by the DB2 instance owner. The command is: db2iupdt -a authType instanceName where: authType Can be CLIENT, SERVER, or SERVER_ENCRYPT instanceName The instance owner ID, typically db2inst1 We do not use this facility. Chapter 4. AIX Web application implementation 65
  • 81. 4.2.5 WebSphere access to DB2 The WebSphere Application Server setting needs to be modified to allow access to the DB2 systems. Note: The data source modification to enable encryption needs to be performed after the Web application is already installed. 1. Include the DB2 JDBC driver: a. From the WebSphere Administrative Console left-hand navigation, select Environment → Manage WebSphere Variables and choose the variable DB2_JDBC_DRIVER_PATH. b. Verify or set the path to the file db2java.zip file; the file is typically found in /home/db2inst1/sqllib/java12. c. Click OK and Save. Then click Save again to save this value to the Master Configuration. 2. Add the DB2 password encryption: a. From the WebSphere Administrative Console left-hand navigation, select Resources → JDBC Providers. b. Select the DB2 JDBC Provider link and then select the DataSources property. c. Click the itmnpds datasource and choose Custom Properties. d. Click on New to create a new property called securityMechanism. The value is an integer which is: 7 For encrypted password 9 For encrypted user and password e. Click OK and Save. Then click Save again to save this value to the Master Configuration.4.3 Web application implementation This documents our installation experiences for implementing IBM Tivoli Monitoring for Network Performance Web application on AIX. The discussions are divided into: 4.3.1, “Implementation steps overview” on page 67 4.3.2, “Web application installation details” on page 6866 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 82. 4.3.1 Implementation steps overview The installation of the Web application is performed by an Install-Shield Multi Platform (ISMP) wizard. The wizard can install the database and the Web application automatically. The detailed installation options and selection is provided in Chapter 5, “Preparing for and installing the Web application on an AIX system“, of IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363. The database creation wizard performs the following functions to create our database, which is called ITMNPDB: 1. Creates the database and connects to it. 2. Creates bufferpools for the ITMNPDB database from itmnpBufferPools.sql. 3. Creates tablespaces for the ITMNPDB database from itmnpTableSpaces.sql. 4. Runs itm_np_ddl to build the configuration tables in the ITMNPDB database. 5. Runs measurements.sql to build the measurement tables in the ITMNPDB database. 6. Creates triggers using itmnp_triggers.sql in the ITMNPDB database. 7. Imports static metadata from preload.sql into the ITMNPDB database. 8. When the database creation is successful, it displays the following message: FNPI0200I: The IBM Tivoli Monitoring for Network Performance database has been created The Web application installation wizard performs the following functions: 1. Sets up two message queues. A message queue connection factory, a destination, and a listener port are defined for the first message queue (for com.tivoli.ItscNotify). A message queue connection factory and a destination are defined for the second message queue (for com.tivoli.ItmnpRealtimeResponse). 2. Sets up a JDBC datasource called itmnpds, which refers to the ITMNPDB created by the database wizard. 3. Installs itmnp21.ear file as a new Enterprise Application. 4. Configures WebSphere Java Management eXtension (JMX) connector. Once the above steps are performed, the WebSphere Application Server is recycled to make the Web application available. The Web application will not be available unless the WebSphere Application Server is recycled. The Web application installation wizard for AIX also loads a set of files into the main directory /opt/IBM/ITMNP21 and the sub-directories. This directory is only Chapter 4. AIX Web application implementation 67
  • 83. used for maintenance and it is not use for the actual Web application processing. The sub-directories are: _jvm This directory contains the Java Runtime library (jre) and its licenses. _uninst This directory contains the uninstalled program (uninstall_itmnp21aix.bin) and the data to uninstall the Web application, after you have successfully installed the Web application in AIX. bin This directory contains sample programs, the Web application installation log, the digital certificates for the monitor and Web application server, and DB2 SQL files for creating the database. license The directory contains the license for the Web application.4.3.2 Web application installation details The following steps discusses some important options that we selected when installing a secure Web application. Important: The security option is selected with the wizard. It is not a trivial task to change the security model once the Web application is installed. It is easier to uninstall and reinstall the Web application to enable security. Uninstallation needs to be performed from the /opt/IBM/ITMNP21/_uninst. The dialog options in the wizard are listed below: 1. Installation and configuration type dialog specifies the action and security authentication that is used. – The installation action can be the Web application, language pack, or database; we choose to install the Web application and database. Remember that installing the database means any existing database will be destroyed. – The security setting options are LDAP, Local OS, or None. We use LDAP security so that the user IDs using the WebSphere applications are authenticated to the LDAP registry. This allows us to use authentication from our z/OS RACF user registry. This option does not modify the WebSphere security option.68 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 84. 2. The database user dialog contains the database user ID, password, and database name to use. We simply use the instance owner db2inst1 and its password and use the name ITMNPDB for the database name. 3. Database communication dialog defines the port and loopback node. – The port defines the port the DB2 instance is listening to. This port is typically 50000 and defined in the /etc/services file as the cDB2 service. – The loopback node creation is necessary for WebSphere to access DB2 through the network communication. The DB2 Control Center panel will show two nodes: one local and one through TCP. We use the default name for the loopback node of ITMNPND. 4. Path to the WebSphere Application Server, typically /usr/WebSphere/AppServer. 5. WebSphere administrator user ID; typically this is wsadmin. Unless security is already enabled in the WebSphere, you can put anything in this panel. 6. Port numbers: You specify the port number for the itmnpUI application and the itmnpItsc application. See also 2.4.2, “Communication port usage” on page 35. The default secure ports are 9453 and 9454. The port relates to the next panel on configuring a secure communication. These ports need to be consistent with the content of itmnp.properties for monitors and nvexportd.properties for the ITSC interface. 7. WebSphere type: We install this in a base configuration, not at a Node Deployment version of WebSphere. 8. LDAP setting page: Specify the options for the LDAP server host name, user ID, and password. These are the parameters that the WebSphere will initially authenticate to the LDAP server. We set these to the SC61 z/OS system and use the TIVO01 user and password. 9. The installation path is where the distribution files will be installed; typically, it is /opt/IBM/ITMNP21. Then the installation will proceed.4.3.3 Setting up LDAP in z/OS In this section, we show how to configure the Lightweight Directory Access Protocol (LDAP) for z/OS LDAP server to run with the Security Database manager (SDBM) database. In order to use LDAP as the user registry, you need to have a valid administration user name, its password, the server host and port, the base DN, and, if necessary, the bind DN and the bind password. The user chosen can be any valid user in the registry and should be searchable. When security is enabled in the product, this server ID and password will be authenticated with the registry during the product startup. If authentication fails, the server will not come up. Chapter 4. AIX Web application implementation 69
  • 85. If the WebSphere Application Server is configured to do LDAP authentication, then it can take advantage of native authentication. When the challenge to do LDAP authentication is presented, the user can enter the Security Server user ID and password (on the system where the LDAP server is running). The Web server will search the LDAP directory for an entry where the uid attribute equals the input user ID. The Web server will use the returned distinguished name (DN) and the password the user entered to do an LDAP bind operation. When the z/OS LDAP server determines that this entry is subject to native authentication, it will verify the password with the Security Server. The z/OS LDAP server accesses the LDAP Directory data in one of the two places: Normal LDAP directory data is stored in DB2 tables managed by the LDAP server. Initially, the server used an internal protocol called Relational Database Manager (RDBM) to access the directory data. With OS/390 V2R10, a new protocol called Traditional Database Manager (TDBM) has been introduced. It provides an optional alternative and gives better performance and flexibility. To talk to DB2, both TDBM and RDBM exploit the DB2 Call Level Interface (CLI). LDAP can also access data from selected RACF profiles kept in the RACF database, and therefore enable LDAP clients to indirectly exploit RACF. In this case, the RACF database is part of the LDAP directory, and an LDAP protocol called Security Database Manager (SDBM) handles the mapping of LDAP requests between LDAP and RACF. The structure of the LDAP directory entries must be defined in the schema files. IBM supplies a number of schema files (stored in UNIX HFS files) with LDAP, which support a general directory structure as well as a structure for entries required by IBM products exploiting LDAP. In this section, we configure a SDBM only LDAP directory server back end. LDAP on z/OS offers a configuration utility called ldapcnf to assist in the installation and customization of an z/OS LDAP server. To complete the installation process, follow the following steps: 1. Create an LDAP working directory for your new LDAP server. In our case, we use /etc/ldap. 2. Copy the sample profile from /usr/lpp/ldap/etc/ldap.profile to this new directory. In our case, this is /etc/ldap/ldap.profile. 3. Customize the ldap.profile file to reflect your system and the configuration variables by following the detailed descriptions of each attribute in the profile. Some attributes in the ldap.profile file are required, but not given a default value. Make sure you read through the entire file, completing all required variables. Although in our environment we only configure SDBM, you must70 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 86. input dummy values into TDBM in order to successfully run the ldapcnf command. Our sample ldap.profile customization file is available in “z/OS LDAP setup configuration file: ldap.profile” on page 224.4. Run ldapcnf from the UNIX System Services. This utility will generate a set of jobs in the MVS™ data set that were specified in the OUTPUT_DATASET definition in the ldap.profile. In our example, this is TIVO01.LDAP.OUTPUT. (See Example 4-3.)Example 4-3 Output of ldapcnf utilityTIVO03 @ SC62:/u/tivo03>/usr/lpp/ldap/sbin/ldapcnf -i /etc/ldap/ldap.profileThe utility is finished checking for errors.Generating dbCli ....Finished generating dbCli.Generating dbSpufi ....Finished generating dbSpufi.Generating dsnaoini ....Finished generating dsnaoini.Generating ldapSrvProc ....Finished generating ldapSrvProc.Generating slapdcnf ....Finished generating slapdcnf.Generating irr ....Finished generating irr.Generating kerb ....Finished generating kerb.Generating slapdenv ....Finished generating slapdenv.Generating racf ....Finished generating racf.Generating prgmCtrl ....Finished generating prgmCtrl.Generating ocsfApf ....Finished generating ocsfApf.Generating ocsf ....Finished generating ocsf.Generating gldOcsfApf ....Finished generating gldOcsfApf.Generating PROGxx ....Finished generating PROGxx.Generating apf ....Finished generating apf.Exiting with return code 0.5. Copy the LDAP server started task procedure from the output data set member to the system PROCLIB. We renamed the started task to LDAPSRV. Chapter 4. AIX Web application implementation 71
  • 87. 6. The following jobs are created in the output data set. Run each job in sequence. Check the output for successful return codes. a. RACF: This job defines and sets authorization for RACF profiles that are needed to run the LDAP server using the user ID called STC. You need to review and modify the appropriate definition that conforms to your security standard. b. APF: This job defines the APF authorization by activating the PROGxx definition from SDSF. You can also issue the SET PROG=xx command from the system console manually. c. PGRMCTRL: This is an optional job for setting program controlled profiles in RACF. When this facility is enabled, access to the LDAP libraries and other supporting libraries becomes restricted, and a user must be authorized to access these modules. 7. Comment off the definitions for TDBM-specific configuration settings in the SLDAPCNF member. Check to see if the suffix setting for SDBM_specific configuration settings is correct. Our example, we used Suffix “o=itso” and use the default for other settings. Our SLAPDCNF configuration file is available in “z/OS LDAP configuration file: SLAPDCNF” on page 225. 8. Start the LDAP server using the LDAPSRV started task. From SDSF, you can start the server by entering /S LDAPSRV. The LDAP server is ready when the following message appears in JOBLOG for LDAPSRV: GLD0122I Slapd is ready for requests 9. To verify LDAP, the native LDAP commands are quite cumbersome to run from UNIX System Services. You may want to use a simpler way to access LDAP. We use an independently developed LDAP browser client, which can be downloaded from this URL: http://www.iit.edu/~gawojar/ldap/ Follow the installation instructions for the prerequisites from the URL. Figure 4-8 on page 73 shows our connection setup for the LDAP browser to connect to our z/OS LDAP SDBM backend.72 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 88. Figure 4-8 LDAP browser connection setup for SDBM back end Figure 4-9 shows the LDAP browser connected to our z/OS LDAP SDBM back end, which is our RACF database. The profile type of the user, group, and connect are displayed.Figure 4-9 LDAP browser to SDBM back end10.Set up the user ID for the default WebSphere administrator to be authenticated through the z/OS LDAP. The default user WSADMIN needs to be defined in RACF. Note the password of the user ID so you can still access the Administrative Console after you enable WebSphere Security. Chapter 4. AIX Web application implementation 73
  • 89. 4.3.4 Configuring WebSphere Application Server for LDAP on z/OS There are five steps to configuring the WebSphere Application Server to use the z/OS LDAP server. To perform these steps, log on to the WebSphere Administrator console as administrator. 1. Set up the authentication mechanism. The Light-weight Third Party Authentication (LTPA) protocol enables the WebSphere Application Server to provide security in a distributed environment using cryptography. This protocol uses cryptographic keys (LTPA keys) to encrypt and decrypt user data that passes between the servers. a. From the WebSphere Administration console, in the left-hand navigation tree, select Security → Authentication Mechanisms → LTPA. b. Complete the dialog; the password is a newly defined password that is used for LTPA encryption. You can use the default timeout of 120. c. Click the Generate Keys button to generate the keys. d. Click the OK button and the Save button on the message window followed by clicking on another Save button to save the master configuration. 2. Set up the LDAP connection. a. From the WebSphere Administration console, in the left-hand navigation tree, select Security → User Registries → LDAP. b. Complete the dialog, as shown in Figure 4-10 on page 75.74 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 90. Figure 4-10 Values for the LDAP connection c. We have to modify the Advanced LDAP setting on the LDAP User Registry configuration Additional Property. This ensures that the filtering settings for the Group Filter, User ID Map, and Group Number ID Map meet the authentication requirements of the users and groups of existing WebSphere Application Server applications and the z/OS LDAP SDBM back-end search filter. Figure 4-11 on page 76 shows the LDAP filters we used for the z/OS LDAP server SDBM backend. Refer to Chapter 18, “Accessing RACF information”, in z/OS V1R4.0 Security Server: LDAP Server Administration and Use, SC24-5923 for more details. Chapter 4. AIX Web application implementation 75
  • 91. Figure 4-11 LDAP filter settings d. Click OK, Save, and Save again. 3. Enabling global security with LDAP. a. From the WebSphere Administration console, in the left-hand navigation tree, select Security → Global Security. b. Complete the dialog, as shown in Figure 4-12 on page 77.76 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 92. Figure 4-12 Values for the LDAP global security settings c. Click OK, Save, and Save again to save the master configuration.4. Mapping users to roles. There are three security roles defined by the IBM Tivoli Monitoring for Network Performance application. The ITMOperator and ITMAdmin roles are used to define the operator and administrator roles respectively. Users must be mapped to one of these roles in order to access the Tivoli Monitoring for Network Performance user interface. The third role, ItscAllAuthority, is used by the monitor component when it connects to the Web application. It is also used to access configuration data and user preferences. a. From the WebSphere Administrative Console left-hand navigation, select Applications → Enterprise Applications. b. Select the itmnp21 application and choose the property Map security roles to users/groups. c. Check the ITMAdmin option to add new users to the roles of the ITM administrator. d. Click Lookup users button to search for users to be added, as shown in Figure 4-13 on page 78. Select the users to be added and click OK. Chapter 4. AIX Web application implementation 77
  • 93. Figure 4-13 Mapping users to roles for LDAP e. Click OK, Save, and Save again to save the master configuration. 5. Recycling WebSphere Application Server. For WebSphere Application Server to recognize the changes we made to security in this section, we have to recycle WebSphere Application Server.4.3.5 Verification of the distributed implementation We verified our IBM Tivoli Monitoring for Network Performance Web application in a distributed environment in the following sections: “Verification of Web application” on page 79 “Verification of IBM Tivoli Monitoring for Network Performance” on page 8078 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 94. Verification of Web applicationWe used a browser to access the WebSphere Application Server with HTTPprotocol. We logged into the URL: http://jakarta.itsc.austin.ibm.com:9091/adminThe WebSphere Application Server redirected the browser to a HTTPS URL asfollows: https://jakarta.itsc.austin.ibm.com:9043/admin/logon.jspWe logged into the WebSphere Application Server admin console, as shown inFigure 4-14.Figure 4-14 WebSphere Application Server secure logon consoleWe selected Applications → Enterprise Applications to verify that itmnp21 isactive (it should have a green arrow icon under the Status column), as shown inFigure 4-15 on page 80. Chapter 4. AIX Web application implementation 79
  • 95. Figure 4-15 Verification of Web Application: itmnp21 is active Verification of IBM Tivoli Monitoring for Network Performance We used a browser to access the IBM Tivoli Monitoring for Network Performance with HTTP protocol. We logged into the URL: http://jakarta.itsc.austin.ibm.com:9080/itmnp The IBM Tivoli Monitoring for Network Performance redirected the browser to a HTTPS URL as follows: https://jakarta.itsc.austin.ibm.com:9443/itmnp/login.jsp However, the secure port is setup as 9453. We changed the login to 9453; otherwise, some functions may not work without accessing the correct port: https://jakarta.itsc.austin.ibm.com:9453/itmnp/login.jsp Figure 4-16 on page 81 shows the secure login panel of IBM Tivoli Monitoring for Network Performance.80 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 96. Figure 4-16 IBM Tivoli Monitoring for Network Performance secure logon panel Note: The following should be done after you install and start the monitors.After login, we verified that the Web application was connecting the monitors byselecting Manage Monitors → Manage Monitor Configurations, as shown inFigure 4-17 on page 82. Chapter 4. AIX Web application implementation 81
  • 97. Figure 4-17 Verification that the Web application and monitor are connected4.4 Some problems and their solutions In this section, we discuss the problems that we found during our Web application installation and solutions to those problems. The topics we discuss include: 4.4.1, “Problem with console users” on page 83 4.4.2, “LDAP user ID character constraint” on page 84 4.4.3, “Uninstallation from admin console” on page 84 4.4.4, “Set up the X-windows DISPLAY properties to enable graphics” on page 86 4.4.5, “LTPA key generation problem” on page 8782 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 98. 4.4.1 Problem with console users We have selected LDAP for active user registry settings, and selected configure security transport to enable encryption and authentication between the monitors and the WebSphere Application Server. The installation works well when creating the database, but fails at the end with the message shown in Figure 4-18.Figure 4-18 FNPI0352E error We viewed our itmnp_install.log in /opt/IBM/ITMNP21/bin, which shows the explanation for the error shown in Example 4-4. Example 4-4 Error showing AdminControl service not available 22: WASX7017E: Exception received while running file "/opt/IBM/ITMNP21/bin/itmnp.jacl"; exception information: com.ibm.ws.scripting.ScriptingException: AdminControl service not available Do the following to make AdminControl service available: Create a wsadmin user ID on AIX, if it does not have one already, and set the password to the wsadmin password. This password was set during the Web application installation. Log on to the WebSphere Application Server admin console and select System Administration → Console Users. If it does not have wsadmin and the LDAP Server User ID (in our case it was tivo01), add them with the proper roles. In our case, Figure 4-19 on page 84 shows our console users. Chapter 4. AIX Web application implementation 83
  • 99. Figure 4-19 WebSphere Application Server console users4.4.2 LDAP user ID character constraint In the installation wizard, we specified the LDAP user ID and password and the LDAP server host name. User IDs with more than 12 characters will cause the following error message: javax.jms.JMSException: MQJMS1092: userid must be less than 12 characters In our case, the user ID without a good filter is: racfid=tivo01,profiletype=user,o=itso which exceeded 12 characters, hence the error. The LDAP character constraint was resolved when we used proper filtering for z/OS LDAP in WebSphere Application Server. See 4.3.4, “Configuring WebSphere Application Server for LDAP on z/OS” on page 74 for LDAP filter details. After changing filters, we were able to log in to the LDAP server using user ID tivo01.4.4.3 Uninstallation from admin console When you want to reinstall IBM Tivoli Monitoring for Network Performance, follow the steps in IBM Tivoli Monitor for Network Performance: Planning, Installing,84 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 100. and Configuring, SC31-6363. As mentioned in the uninstallation steps, you haveto run the ./uninstall_itmnp21aix.bin command from/opt/IBM/ITMNP21/_uninst.Do not uninstall the Web application from the WebSphere Application Serveradmin console. Uninstalling the Web application from the admin console will notrun the uninstall jacl file in which the configuration changes are saved and youwill get problems while reinstalling.When you reinstall the Web application, and if you have not properly uninstalledthe Web application before, then you will see various errors. The errors are inmessage FNPI0002E; the following lines are the probable errors initmnp_install.log:FNPI0002E: WASQueue com.tivoli.ItscNotify is already definedFNPI0002E: ListenerPort ItscNotify is already definedFNPI0002E: JDBCProvider DB2 JDBC PROVIDER is already definedFNPI0002E: DataSource itmnpds is already definedThe above errors can be corrected by going to the WebSphere admin consoleand selecting the properties for those errors that already exist in the FNPI0002Emessage and deleting them manually.For example, we received the error shown in Example 4-5 about the existence ofcom.tivoli.ItscNotifyCF.Example 4-5 error shown in itmnp_install.log while reinstalling the Web application31: FNPI0002E: WASQueueConnectionFactory com.tivoli.ItscNotifyCF isalready defined.31: <<Exit configureWASQueueConnectionFactory>>37: Exiting with RC=237: FNPI0109E: WebSphere Configuration ErrorWe corrected the error by opening the WebSphere Application Server adminconsole and selecting Resources → WebSphere JMS Provider →WebSphere Queue Connection Factories and deleting the Queue ConnectionFactory com.tivoli.itscNotifyCF from the screen in Figure 4-20 on page 86.Actually, you also need to delete com.tivoli.itmnp.RealtimeResponseCF. Chapter 4. AIX Web application implementation 85
  • 101. Figure 4-20 WebSphere admin console - Queue Connection Factories properties4.4.4 Set up the X-windows DISPLAY properties to enable graphics On the AIX server, run export DISPLAY=AIXhost:0.0 and xhost + to make the AIX server available for graphics for all the clients that connect to this machine. If the x-server properties are not set properly, you could end up with the following message: Error 500: Cant connect to X11 window server using :0.0 as the value of the DISPLAY variable as shown in Figure 4-21 on page 87.86 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 102. Figure 4-21 Error 500 message4.4.5 LTPA key generation problem After installing the IBM Tivoli Monitoring for Network Performance Web application, we stopped the WebSphere Application Server and started it. We get an error in SystemOut.log, as shown in Example 4-5 on page 85. Example 4-6 LTPA error [5/11/04 15:31:43:636 CDT] 35be67d5 LTPAServerObj E SECJ0238E: An unexpected exception occurred when trying to create the initial LTPAServerObject. The exception is java.lang.NullPointerException atcom.ibm.ws.security.ltpa.LTPAPrivateKey.decode(LTPAPrivateKey.java:57) at com.ibm.ws.security.ltpa.LTPAPrivateKey.<init>(LTPAPrivateKey.java:47) at com.ibm.ws.security.ltpa.LTPAServerObject.<init>(LTPAServerObject.java:252) at com.ibm.ws.security.ltpa.LTPAServerObject.initLTPAServer(LTPAServerObject.java: 183) at com.ibm.ws.security.ltpa.LTPAServerObject.getLTPAServer(LTPAServerObject.java:1 33) at com.ibm.ws.security.server.lm.ltpaLoginModule.initialize(ltpaLoginModule.java:1 Chapter 4. AIX Web application implementation 87
  • 103. 29) at com.ibm.ws.security.common.auth.module.proxy.WSLoginModuleProxy.initialize(WSLo ginModuleProxy.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0... To solve the error shown in Example 4-6 on page 87, we need to generate LTPA keys. To generate LTPA keys, we need the WebSphere admin console to be up. Since the WebSphere Application Server startup failed with an LTPA error, we need to disable the global security and then restart the server, which would now be in non-secure mode and generate the LTPA keys. To start WebSphere Application Server in non-secure mode, we just modified security.xml file under /usr/WebSphere/AppServer/config/cells/jakarta. We changed the value of the enabled variable to false. The contents of the original file are shown in Example 4-7, with the enabled variable highlighted. Example 4-7 Modifying security.xml file /websphere/appserver/schemas/5.0/orb.securityprotocol.xmi" xmlns:security="http://www.ibm.com/websphere/appserver/schemas/5.0/security.xmi " xmi:id="Security_1" useLocalSecurityServer="true" useDomainQualifiedUserNames="false" enabled="true" cacheTimeout="1200" issuePermissionWarning="false" activeProtocol="BOTH" enforceJava2Security="false" activeAuthMechanism="LTPA_1" activeUserRegistry="LDAPUserRegistry_1" defaultSSLSettings="SSLConfig_1"> After the server is started in non-secure mode, use the WebSphere Administrative Console, in the left-hand navigation tree, to select Security → Authentication Mechanisms → LTPA and click on the Generate Keys button to generate the keys. See Figure 4-22 on page 89.88 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 104. . Figure 4-22 LTPA keys generation4.4.6 Problem with SSL We encountered the error SSLHandshakeException: unknown certificate in SystemOut.log, as shown in Example 4-8. Example 4-8 SSL error [5/12/04 14:12:10:326 CDT] 5303a99 ItmnpJmxResou A ItmnpJmxResourceAdapterImpl. execute TRAS0014I: The following exception was logged javax.net.ssl.SSLHandshakeException: unknown certificate at com.ibm.jsse.bg.a(Unknown Source) at com.ibm.jsse.bg.startHandshake(Unknown Source) at com.ibm.net.ssl.www.protocol.https.b.n(Unknown Source) at com.ibm.net.ssl.www.protocol.https.p.connect(Unknown Source) at com.ibm.net.ssl.www.protocol.http.bw.getInputStream(Unknown Source) at com.ibm.net.ssl.www.protocol.http.bw.getHeaderField(Unknown Source) at com.ibm.net.ssl.www.protocol.http.bw.getResponseCode(Unknown Source) at com.ibm.net.ssl.internal.www.protocol.https.HttpsURLConnection.getResponseCode( Unknown Source) at Chapter 4. AIX Web application implementation 89
  • 105. com.tivoli.itmnp.jmxconnector.ItmnpJmxResourceAdapterImpl.execute(ItmnpJmxResou rceAdapterImpl.java:180) at com.tivoli.itmnp.jmxconnector.ItmnpJmxInteraction.execute(ItmnpJmxInteraction.j ava:242) at com.tivoli.itmnp.connection.ejb.ItmnpMonitorCommandBean.executeCommand(ItmnpMon itorCommandBean.java:534)at com.tivoli.itmnp.connection.ejb.ItmnpMonitorCommandBean.issueCommand(ItmnpMonit orCommandBean.java:130)... To solve the problem, we need to add the Java run-time properties shown in Example 4-9 to Application Servers → server1 → Process Definition → Java Virtual Machine → Generic JVM arguments from the WebSphere administrative console. Example 4-9 Java SSL run-time properties -Djavax.net.ssl.keyStore= /usr/WebSphere/AppServer/etc/itmnpServerKeyStore.jks -Djavax.net.ssl.keyStorePassword=WebAS -Djavax.net.ssl.trustStore= /usr/WebSphere/AppServer/etc/itmnpServerTrustStore.jks -Djavax.net.ssl.trustStorePassword=WebAS4.5 Start and stop procedures This section discusses the start and stop procedures for the Web application. This process needs to be coordinated with various monitors running on z/OS systems. Since the monitors write data directly to DB2, it is better to stop the monitors before stopping the Web application server. The start and stop processes are performed as root. To start up all components, IBM Tivoli Monitoring for Network Performance provides a script called itmnp_was.sh (found under /opt/IBM/ITMNP21/bin). Manual startup can be performed by using the following procedure: 1. Source the environment for DB2 from /home/db2inst1/sqllib/db2profile; the command to do this is: . /home/db2inst1/sqllib/db2profiles 2. Export the DISPLAY environment variable for the X-windows Java client to draw the charts under WebSphere: export DISPLAY=jakarta.itsc.austin.ibm.com:0.0 3. Start the DB2 processes: /home/db2inst1/sqllib/bin/db2start90 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 106. 4. Start the WebSphere Application Server: /usr/WebSphere/AppServer/bin/startServer.sh Stopping DB2 and WebSphere can be done by using the following commands: 1. To stop WebSphere Application Server, you need to specify the user and password when global security is on; otherwise, the stop command will be ignored: /usr/WebSphere/AppServer/bin/stopServer.sh server1 -user TIVO01 -password xxxx 2. Source the DB2 environment: . /home/db2inst1/sqllib/db2profiles 3. Check the application that accesses DB2; the application db2jcc is a monitor, db2bp is a DB2 command line processor, and Java is typically from WebSphere: db2 list application 4. When no more applications are connected, you can stop db2 using the following command: db2stop or you can force stop DB2 and terminate all applications with the following command: db2stop force4.6 Backup and recovery To avoid losing your data in case of a hardware or software failure or before some type of reinstallation, establish a routine for backing up your IBM Tivoli Monitoring for Network Performance installation. How frequently you back up your database depends on your organization’s policy for backups. This section discusses how to back up the Web application in AIX environment. The following topics will be discussed: 4.6.1, “File system backup” on page 92 4.6.2, “DB2 database backup” on page 93 4.6.3, “DB2 database restore” on page 97 4.6.4, “File system restore” on page 98 Chapter 4. AIX Web application implementation 91
  • 107. 4.6.1 File system backup After you install the Web application, there are several file systems that are affected. You need to back up these file systems. We recommend you back up the following paths: /opt/IBM/ITMNP21: IBM Tivoli Monitoring for Network Performance files and installation information /usr/WebSphere/AppServer: WebSphere Application Server path that contains all the Web applications and its settings /home/db2inst1: DB2 instance files and database files There are several way to perform this backup on AIX systems, such as: The dumpvg command will dump the entire volume group. If all the file systems are residing in the same volume group, this is very effective. This typically dumps the volume group into a removable magnetic media, such as tape cartridges. The cpio command will copy files into tape device for storage and backup. The tar command will collect the files from a certain path and put them in a single file, which makes it easier to dump to tape or write to CD. You need to back up these files once after a successful installation of the IBM Tivoli Monitoring for Network Performance Web application. Unless there is a major upgrade or changes to the system, these backups will be sufficient. Note: Make sure that DB2 and WebSphere Application Server are stopped before trying to back up these directories. We created a Journal File System (JFS) using the smitty jfs command and mounted it to the directory /backup. We specified a size of 2 GB for the JFS. We used the tar command to collect the files from the paths in /backup: tar -cvf /backup/itmnp21.tar /opt/IBM/ITMNP21 tar -cvf /backup/was.tar /usr/WebSphere/AppServer tar -cvf /backup/db2.tar /home/db2inst1 Then we can copy these files into a tape or CD.92 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 108. 4.6.2 DB2 database backup There are two types of backup you can perform: Offline database backup. This type of backup requires an exclusive connection to the database, since all tablespaces in the database will be backed up. Online database backup. This type of backup is especially useful for users who run production databases and need to have one or more of the tablespaces in the database running continuously. When an online backup is used, only the tablespaces being backed up require an exclusive connection by the user. This permits other tablespaces within the database that are not being backed up to remain online for other applications to access. Note: If performing an online backup, make sure the roll forward recovery parameter logretain=ON or userexit is enabled in the database configuration file. An offline backup is required before any online backups can be performed. We intended to use the online backup. We need to perform the following steps: 1. “Setting database configuration” on page 93 2. “Performing an offline backup” on page 94 3. “Listing the tablespaces” on page 95 4. “Performing an online backup” on page 96 Setting database configuration Since we use the instance owner, it has the SYSADM authority already. We check that both logretain and userexit are not set by issuing the db2 get db cfg command to display the configuration of the itmnpdb database, as shown in Example 4-10. Example 4-10 Display of ITMNPDB configuration $ db2 get db cfg for itmnpdb Database Configuration for Database itmnpdb Database configuration release level = 0x0a00 Database release level = 0x0a00 . . . Backup pending = NO Chapter 4. AIX Web application implementation 93
  • 109. Database is consistent = NO Rollforward pending = NO Restore pending = NO Multi-page file allocation enabled = NO Log retain for recovery status = NO User exit for logging status = NO . . . We have to set the log retain for recovery status to YES by issuing the db2 update db cfg command, as shown in Example 4-11. Example 4-11 Enabling logretain $ db2 update db cfg for itmnpdb using logretain on DB20000I The UPDATE DATABASE CONFIGURATION command completed successfully. We need to recycle the database manager to pick the new changes by using the command in Example 4-12. Example 4-12 Command to recycle DB2 $ db2stop force 06-01-2004 16:06:15 0 0 SQL1064N DB2STOP processing was successful. SQL1064N DB2STOP processing was successful. $ db2start 06-01-2004 16:07:52 0 0 SQL1063N DB2START processing was successful. SQL1063N DB2START processing was successful We verify that the changes have been updated by using the db2 get db cfg command. This indicates that there is a backup pending and will require a full offline backup before an online backup can be performed. Performing an offline backup We stop the monitors that are connected to the database and stop all applications connected to the database by using the command in Example 4-13. We have to verify that all applications have stopped. Example 4-13 Command to stop all applications in database $ db2 force application all DB20000I The FORCE APPLICATION command completed successfully. DB21024I This command is asynchronous and may not be effective immediately. $ db2 list application94 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 110. SQL1611W No data was returned by Database System Monitor. SQLSTATE=00000We perform the backup offline by using the command shown in Example 4-14.Example 4-14 Offline backup command for DB2$ db2 backup db itmnpdb to /backupBackup successful. The timestamp for this backup image is : 20040601161754Listing the tablespacesBefore we can perform the online backup, we need to check the tablespacenames by connecting to the database and issuing the list tablespaces, as shownin Example 4-15.Example 4-15 List tablespaces for itmnpdb$ db2 connect to itmnpdbDatabase Connection Information Database server = DB2/6000 8.1.0 SQL authorization ID = DB2INST1 Local database alias = ITMNPDB$ db2 list tablespaces Tablespaces for Current Database Tablespace ID = 0 Name = SYSCATSPACE Type = System managed space Contents = Any data State = 0x0000 Detailed explanation: Normal Tablespace ID = 1 Name = TEMPSPACE1 Type = System managed space Contents = System Temporary data State = 0x0000 Detailed explanation: Normal Tablespace ID = 2 Name = USERSPACE1 Type = System managed space Contents = Any data State = 0x0000 Detailed explanation: Chapter 4. AIX Web application implementation 95
  • 111. Normal Tablespace ID = 3 Name = ITMNPTS1 Type = System managed space Contents = Any data State = 0x0000 Detailed explanation: Normal Tablespace ID = 4 Name = SD_TEMP2 Type = System managed space Contents = Any data State = 0x0000 Detailed explanation: Normal In Example 4-15 on page 95, the only tablespace that we need to back up is ITMNPTS1, which is a system managed space. The other tablespaces are either temporary (TEMPSPACE1 and SD_TEMP2), not used (USERSPACE1), or not changed (SYSCATSPACE). Tip: Tablespace SYSCATSPACE does change in terms of database statistic information, which is required to optimize DB2 performance. However, all this information can be regenerated using the UPDATE STATISTIC command. Performing an online backup We perform the online backup shown in Example 4-16. Example 4-16 Online backup command for DB2 $ db2 “backup db itmnpdb tablespace(itmnpts1) online to /backup” Backup successful. The timestamp for this backup image is : 20040601163344 This online backup may need to be performed regularly based on a schedule. Another backup option is to use Tivoli Storage Manager (TSM), but we will not discuss that option here. This backup may also need to be moved to removable media (tape or CD) for recovery. Creating a backup schedule Online backup requires the restore process to have access to the database log. Offline backup is preferred in order to preserve database integrity. Typically, an96 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 112. installation will perform a scheduled maintenance window and perform an offline backup. These offline backup files are offloaded and kept for full recovery purposes.4.6.3 DB2 database restore Before restoring a database, we need to have one of the following authorities on DB2: SYSADM SYSCTRL SYSMAINT If you are restoring to a new database (not an existing one), SYSADM or SYSCTRL is needed. An offline database restore will acquire an exclusive connection, so no application should be connected during this task. This implies that all monitors connected to this database must be shutdown. Also, make sure no other applications are connected to the database when performing an offline restore. We shut down the monitors in WTSC62 and WTSC66. We verify that there is no connection to the database and do the offline restore of the DB2 database, as shown in Example 4-17. Note that the backup timestamp is the timestamp from Example 4-14 on page 95. Example 4-17 Offline restore command $ db2 restore db itmnpdb from /backup taken at 20040601161754 without rolling forward without prompting DB20000I The RESTORE DATABASE command completed successfully. This command will restore the database image that is saved in /backup with a timestamp of 20040601161754 to the existing itmnpdb database. This command will effectively overwrite the old itmnpdb database files with the backed up database image files. The key phrase "without rolling forward" will keep the database manager from putting the restored database in the rollforward pending state (this phrase is not needed if the database was not enabled for rollforward recovery at the time it was backed up). If you are interested in recovering all database files to the point of the last successful transaction and the database was enabled for rollforward recovery at the time it was backed up, leave out the "without rolling forward" key phrase. We use the online restore command to restore the database, as shown in Example 4-18. Note also that the timestamp that we use is the timestamp from Example 4-16 on page 96. Chapter 4. AIX Web application implementation 97
  • 113. Example 4-18 Online restore command $ db2 "restore db imtnpdb tablespace(itmnpts1) online from /backup taken at 20040601163344 without prompting" DB20000I The RESTORE DATABASE command completed successfully. This command will restore the syscatspace and itmnpts1 tablespaces to the existing itmnpdb database while allowing the uninhibited processing of other tablespaces not involved in the online restore command. Again, the "taken at timestamp" key phrase must be declared if there is more than one backup image in the same folder. You can get the timestamps of the backup file by listing the file in the backup directory. As an example, the backup file called ITMNPDB.0.db2inst1.NODE0000.CATN0000.20040601161754.001 is a backup with a timestamp of 20040601161754.4.6.4 File system restore When the disk fails while moving to a new system, we need to restore the images that we have from the file systems images and database images. The following procedure can restore your AIX based IBM Tivoli Monitoring for Network Performance Web application: 1. Prepare the new machine, install AIX, and input all the required PTFs that are similar to the ones in the original server. 2. Prepare the system in a similar fashion to the installation preparation: a. Create the new file systems for /opt/IBM/ITMNP21, /usr/WebSphere/AppServer, and /home/db2inst1. b. Create groups db2grp1, mqm, and mqbrkrs. c. Connect root to mqm, mqbrkrs, and db2grp1. 3. Install WebSphere Application Server with JMS and DB2 with the required level similar to the original server. 4. Restore the backup from 4.6.1, “File system backup” on page 92 for /opt/IBM/ITMNP21, /usr/WebSphere/AppServer, and /home/db2inst1. 5. Restore the archived offline and online backup for DB2 and its log files: tar -xvof /backup/was.tar tar -xvof /backup/db2.tar tar -xvof /backup/itmnp.tar 6. Perform an offline restore or online restore: . /home/db2inst1/sqllib/db2profile98 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 114. db2 restore db itmnpdb from /backup taken at 20040601161754 without rollingforward without prompting Chapter 4. AIX Web application implementation 99
  • 115. 100 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 116. 5 Chapter 5. Mainframe Web application implementation This chapter documents our installation experiences for implementing IBM Tivoli Monitoring for Network Performance. Here we created a scenario where we set up a z/OS only environment with all the relevant components, using the IBM Tivoli Monitoring for Network Performance Monitor to monitor two z/OS systems. The following topics are covered in this chapter: 5.1, “Scenario overview” on page 102 5.2, “Preparing for the implementation” on page 103 5.3, “Web application implementation” on page 114 5.4, “Start and stop procedures” on page 117 5.5, “Backup and recovery” on page 119© Copyright IBM Corp. 2004. All rights reserved. 101
  • 117. 5.1 Scenario overview This chapter discusses the implementation of the IBM Tivoli Monitoring for Network Performance Web application on the z/OS system. The implemented components of our z/OS scenario are shown in Figure 5-1. We configured DB2, WebSphere Application Server, and the Web application to run on our SC61 system. We then configured the IBM Tivoli Monitoring for Network Performance monitors to run on SC61 and SC52 systems. We installed Tivoli Data Warehouse enablement pack so that our performance data, captured by IBM Tivoli Monitoring for Network Performance, could be summarized and migrated into the Tivoli Data Warehouse for archiving. We then used the Crystal Enterprise product, which was provided as part of Tivoli Data Warehouse, to generate historical reports. z/OS WTSC61 Tivoli Data Crystal Ware Enterprise house product DB2 DB2 ITMNPDB TWH_CDW Netview Integrated TCP/IP Services Component z/OS Itmnp21 itmnpMonitor WTSC52 Web MQ Appication (JMS) WMQX itmnpMonitor WebSphere EAS Netview Figure 5-1 z/OS configuration102 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 118. 5.2 Preparing for the implementation This section describes the tasks that need to be performed before you actually install IBM Tivoli Monitoring for Network Performance Web application. The tasks are: 5.2.1, “Preparing HFS files” on page 103 5.2.2, “Preparing DB2” on page 104 5.2.3, “Graphic package” on page 108 5.2.4, “Preparing WebSphere Application Server” on page 1105.2.1 Preparing HFS files The Web application installation requires some files to be created in the systems. These file systems are implemented as Hierarchical File Systems (HFS) files. The required space for Web application implementation on UNIX System Services is as follows: /u/itmnp This is where the installation files will be loaded. We need 20 MB. /tmp The temporary file system needs to have at least 67 MB of free space. $WAS_HOME The WebSphere Application Server home directory that the Web application will be installed on needs to have at least 117 MB of free space. Use the following commands to check the HFS requirements: Issue the UNIX System Services command df -k /u to show the amount of free space on the /u file system. Use the console command D OMVS,F to display the mounted file systems and their associated mount points. To allocate a new HFS dataset, use the TSO dataset allocate utility or the batch utility IEFBR14, as shown in Figure 5-2. HFS datasets are defined as TYPE=HFS and DSORG=PO. //ALLOCHFS JOB ACCT,USER,CLASS=A,NOTIFY=&SYSUID //* //IEFBR14 EXEC PGM=IEFBR14 //DD1 DD DSN=OMVS.U.ITMNP.HFS,SPACE=(MB,(25,0)),DCB=(DSORG=PO), // DSNTYPE=HFS,DISP=(,CATLG) Figure 5-2 Defining an HFS dataset Chapter 5. Mainframe Web application implementation 103
  • 119. In order for a file system to be used in USS, it has to be mounted at a mount point as follows; mount filesystem(‘OMVS.U.ITMNP.HFS’) mountpoint(‘/u/itmnp’) type(HFS) mode(RDWR) We need to populate the /u/itmnp with the supplied tar files from /usr/lpp/itmnp/V2R1M0/web_app_inst/itmnp21zos.tar. The files in this path must be owned by the WebSphere administrator. The processing is shown in Figure 5-3. The default WebSphere administrator is WSADMIN with group WSCFG1. TIVO01:/ >chown -R wsadmin:wscfg1 /u/itmnp TIVO01:/ >su - wsadmin WSADMIN:/u/wsadmin >cd /u/itmnp WSADMIN:/u/itmnp >tar -xvof /usr/lpp/itmnp/V2R1M0/web_app_inst/itmnp21zos.tar WSADMIN:/u/itmnp >chmod 755 *.sh Figure 5-3 Loading the z/OS tar file5.2.2 Preparing DB2 We performed the steps needed to prepare for installing IBM Tivoli Monitoring for Network Performance Web application on z/OS, as described in IBM Tivoli Monitoring for Network Performance: Planning, Installation, and Configuration, SC31-6363. You need to find the correct DB2 HFS directory to use with IBM Tivoli Monitoring for Network Performance. If your installation has multiple versions of the DB2 HFS files, it is important to ensure that the code level of the DLLs in the DB2 HFS files matches the code level in z/OS dataset SDSNLOD2. The SDSNLOD2 library is included in the STEPLIB of the WebSphere Application Server controller and servant startup procedure. We encountered a problem where our HFS and SDSNLOD2 were mismatched. The error message was displayed in the WebSphere Application Servers servant SYSOUT, as shown in Figure 5-4. ./bborjtr.cpp+820 ... BBOO0220E CNTR0019E: Non-application exception occurred w "findByWildcardName". Exception data: java.lang.NoClassDefFoundError: COM/ibm/db2os390/sqlj/jdbc/DB2SQLJDriver Figure 5-4 JDBC error in WS5611S To find the current level of DB2 code, we searched for the string ‘DB2DriverInfo_native.C’ in member DSNAQLDA in our SDSNLOD2 dataset.104 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 120. The maintenance level and compilation date is shown after this string. Figure 5-5shows the result of our search. In our case, the level is pq84783. /DB2DriverInfo_native.C level:pq84783 compiled:Feb 20 2004 20040220182156020A001 1 0Figure 5-5 DB2DriverInfo in member DSNAQLDATo find the DB2 HFS code level, execute theCOM.ibm.db2os390.sqlj.util.DB2DriverInfo.class file in the db2j2classes.zip, asshown in Figure 5-6. TIVO02:/u/tivo02: >cd /db2/db2v8/UQ86912/classes TIVO02:/db2/db2v8/UQ86912/classes: >ls DSNJDBC_JDBCProfile.ser db2jdbc.cursors IBM db2sqljjdbc.properties db2j2classes.zip db2sqljjdbc.properties.orig TIVO02:/db2/db2v8/UQ86912/classes: >export LIBPATH=../lib:$LIBPATH TIVO02:/db2/db2v8/UQ86912/classes: >java -cp ./db2j2classes.zip COM.ibm.db2os390.sqlj.util.DB2DriverInfo DB2DriverInfo main received an SQLException trying to retrieve the driver build version!! --> ** DB2 ERROR: DB2 for z/OS SQLJ/JDBC Driver level mismatch detected ** --> ** DB2 ERROR: Driver Classes build=DB2 8.1 UQ85385 JDBC 2.0 --> ** DB2 ERROR: Native DLL build=DB2 7.1 PQ69861 JDBC 2.0Figure 5-6 Displaying the DB2 HFS levelThe DB2 configuration component required the following steps:1. Ensure that the Resource Recovery Service (RRS) is started and active in the z/OS system. This Started Task must be active before DB2 that DB2 uses the RRS facility.2. Grant BIND permissions for DB2 using SPUFI. We used a user ID that has SYSADM privileges, so this step is not necessary.3. Get the DB2 communication parameters. We obtained our DB2 location name and port number from the DB2 master address space message log. The location name and port number is displayed in the DSNL004I message, as shown in Figure 5-7 on page 106. Chapter 5. Mainframe Web application implementation 105
  • 121. 11.03.28 STC13266 DSNL004I -DB8D DDF START COMPLETE 166 166 LOCATION DB8D 166 LU USIBMSC.SCPDB8D 166 GENERICLU -NONE 166 DOMAIN -NONE 166 TCPPORT 38030 166 RESPORT 38031 Figure 5-7 DB2 startup jes2 message log 4. Define the specified number of threads and batch connections. This is performed by modifying the sample installation job DSNTIJUZ. You need to check with the DB2 system programmer to find the latest source of this member. The DSNTIJUZ job modifies the initialization data module; this module is typically called DSNZPARM. The modified parameter is shown highlighted in Example 5-1. You need to restart DB2 to activate the changes. Example 5-1 Sample excerpt of DSNTIJUZ . . . DSN6SYSP ACCUMACC=10, ACCUMUID=0, AUDITST=NO, BACKODUR=5, CHKFREQ=500000, CONDBAT=10000, CTHREAD=200, DBPROTCL=DRDA, DLDFREQ=5, DSSTIME=5, DSVCI=YES, EXTRAREQ=100, EXTRASRV=100, EXTSEC=YES, IDBACK=50, IDFORE=50, IDXBPOOL=BP0, IXQTY=0, LBACKOUT=AUTO, LOBVALA=10240, LOBVALS=2048, LOGAPSTG=100, MAXDBAT=200, MGEXTSZ=NO, MON=NO, MONSIZE=262144, PCLOSEN=5,106 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 122. . . .5. Increase the BP16K1 bufferpool size to 1000 buffers. IBM Tivoli Monitoring for Network Performance uses tablespaces that have a 16 KB page size; therefore, we need a 16 KB bufferpool. By default, DB2 already predefined some 16 KB bufferpools but with a size of 0. You can check the bufferpools using the command -DIS BPOOL from the DB2 interactive ISPF interface. We altered the BP16K1 buffer pool using the -DB8D ALT BP16K1(VPSIZE=1000) command. The bufferpool display now shows the presence of the BP16K1 buffer, as shown in Example 5-2.Example 5-2 BP16K1 bufferpool display-DB8D DIS BUFFERPOOL(BP16K1) DSNB401I -DB8D BUFFERPOOL NAME BP16K1, BUFFERPOOL ID 121, USE COUNT 1 DSNB402I -DB8D BUFFER POOL SIZE = 1000 BUFFERS ALLOCATED = 1000 TO BE DELETED = 0 IN-USE/UPDATED = 0 BUFFERS ACTIVE = 20 DSNB406I -DB8D PGFIX ATTRIBUTE - CURRENT = NO PENDING = NO PAGE STEALING METHOD = LRU DSNB404I -DB8D THRESHOLDS - VP SEQUENTIAL = 80 DEFERRED WRITE = 30 VERTICAL DEFERRED WRT = 5, 0 PARALLEL SEQUENTIAL =50 ASSISTING PARALLEL SEQT= 06. Bind the JDBC packages using the job FNPLCBND from the FNPSAMP library. This will bind the DSNJDBC plan. You need to grant the execution of this plan for the user IDs that will run WebSphere or grant it to PUBLIC.7. Edit and run the bindJDBC.sh command from UNIX System Services. This will bind the DB2 JDBC remote access module for the IBM Tivoli Monitoring for Network Performance monitors. Alternatively, from the DB2 HFS file system, you can issue the command shown in Example 5-3 to bind the DB2 JDBC remote access.Example 5-3 JDBC bind commandcd /usr/lpp/db2/db8d/jcc/classesjava -cp ./db2jcc.jar:./db2jcc_license_cisuz.jar com.ibm.db2.jcc.DB2Binder -urljdbc:db2://wtsc61.itso.ibm.com:38030 -user TIVO01 -password xxxxxx Chapter 5. Mainframe Web application implementation 107
  • 123. 8. Create a storage group to assign dataset allocation parameters. We need to have the tablespaces for IBM Tivoli Monitoring for Network Performance created under the high level qualifier of DB8DU on disk TIVO01 and TIVO02. From SPUFI, we run the following SQL statement: CREATE STOGROUP ITMNPSTO VOLUMES(TIVO01, TIVO02) VCAT DB8DU 9. Create the database tables using the job FNPCRDB from the FNPSAMP library. 10.Modify the file db2sqljjdbc.properties to reflect the correct DB2 properties. The file is in the classes sub-directory of the DB2 HFS path. Our modification of the file is shown in Example 5-4. Example 5-4 Content of db2sqljjdbc.properties # Any lines starting with the pound sign # # are comments. Please see the DB2 for z/OS # Application Programming Guide and Reference # for Java for the description of these settings. # DB2SQLJSSID=DB8D db2.jdbc.profile.pathname=/usr/lpp/db2/db8d/classes/DSNJDBC_JDBCProfile.ser DB2SQLJPLANNAME=DSNJDBC DB2SQLJJDBCPROGRAM=DSNJDBC #DB2SQLJ_TRACE_FILENAME=/tmp/mytrc #DB2SQLJ_TRACE_BUFFERSIZE=10245.2.3 Graphic package The z/OS environment does not have native X-Window support similar to the AIX environment. To generate the charts, the IBM Tivoli Monitoring for Network Performance Web application requires a graphic driver from the PJA Toolkit. The PJA Toolkit graphic driver can be retrieved from the Web address http://www.eteks.com/pja/en/#Download. The resulting Web site is shown in Figure 5-8 on page 109.108 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 124. Figure 5-8 PJA Toolkit eteks download siteWe download the toolkit for other platforms as a ZIP file and extract the filepja.jar.From our SC61 UNIX System Services, we create our pja install directories andcopy the pja.jar, all font files, and FnpThonburi.ttf file, as shown in Figure 5-9. Weuse the path /u/itmnp/pja. TIVO02:/u/itmnp: >mkdir pja TIVO02:/u/itmnp: >mkdir /u/itmnp/pja/lib TIVO02:/u/itmnp: >mkdir /u/itmnp/pja/lib/fonts TIVO02:/u/itmnp: >cp /u/itmnp/FnpThonburi.ttf /u/itmnp/pja/lib/fonts TIVO02:/u/itmnp: >cp -r /usr/lpp/java/IBM/J1.3/lib/font* /u/itmnp/pja/lib TIVO02:/u/itmnp: >cp /tmp/pja.jar /u/itmnp/pja/lib/pja.jar TIVO02:/u/itmnp: >chmod -R 755 /u/itmnp/pjaFigure 5-9 Creating the pja environment Chapter 5. Mainframe Web application implementation 109
  • 125. 5.2.4 Preparing WebSphere Application Server We install our WebSphere Application Server on the WTSC61 system, which includes the Java Messaging Services (JMS) component. Java Messaging Services is an embedded messaging component that forms part of WebSphere Application Server. An alternative to using this component is to use the WebSphere MQ product. The tasks that need to be performed on WebSphere Application Server are: “Allocating DB2 libraries to WebSphere” on page 110 “Modifying WebSphere settings” on page 111 “Installing a sample graphic test program” on page 112 Our WebSphere environment In our environment, once the WebSphere Application Server starts, the following started tasks are active: WS611 - WebSphere Application Server control region WS611S - WebSphere Application Server servant WS611D - WebSphere Application Server daemon WMQXMSTR - JMS queue manager WMQXCHIN - JMS channel initiator The WebSphere Application Server is active when the BBOO0222I message, as shown in Figure 5-10, is displayed in the servant’s JES2 message log. BBOO0222I WSVR0001I: Server SERVANT PROCESS ws611sc61 open for e-business BBOO0020I INITIALIZATION COMPLETE FOR WEBSPHERE FOR Z/OS SERVANT PROCESS WS611. Figure 5-10 WebSphere Application Server active message We use the WebSphere Administrative Console to configure the WebSphere variables and enable global security. The SYSPRINT for the WebSphere Application Server servant region address space contains the problem logs for WebSphere applications. It is important to view this log file when troubleshooting WebSphere Application Server or WebSphere application problems. Allocating DB2 libraries to WebSphere The DB2 libraries have to be accessible to the WebSphere Application Server control and servant processes. A STEPLIB (or LINKLIST entry) has to be added110 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 126. to the WebSphere Application Server control STC and WebSphere Application Server servant STC for the following datasets: SDSNEXIT SDSNLOAD SDSNLOD2 Modifying WebSphere settings These WebSphere settings are modified from the WebSphere Administrative Console. The variables that we modified are summarized in Table 5-1. A more detailed procedure is in IBM Tivoli Monitoring for Network Performance: Planning, Installation, and Configuration, SC31-6363.Table 5-1 WebSphere setting modificationMenu path Variable name Our valueEnvironment → DB2390_JDBC_DRIVER_PATH /usr/lpp/db2/db8dManage WebSphereVariables DB2SQLJPROPERTIES /usr/lpp/db2/db8d/classes/db2sqljjdbc. propertiesServers → Application Total transaction lifetime timeout 3600Servers → ws611 Maximum transaction timeout 4200Servers → Application protocol_http_timeout_output_ SESSIONServers → ws611 → recoveryCustom Properties protocol_https_timeout_output_ SESSION recoveryServers → Application ConnectionIOTimeout 3600Servers → ws611 →Web Container → ConnectionResponseTimeout 2400HTTP Transport →9080 → CustomPropertiesServers → Application ConnectionIOTimeout 3600Servers → ws611 →Web Container → ConnectionResponseTimeout 2400HTTP Transport →9443 → CustomPropertiesServers → Application WLM Timeout 2400Servers → ws611 →ORB Service →Advanced Setting Chapter 5. Mainframe Web application implementation 111
  • 127. Menu path Variable name Our valueServers → Application GenericJVMArguments -Xbootclasspath/p:/u/itmnp/pja/lib/pja.jarServers → ws611 →Process Definition →Servant → Java VirtualMachineServers → Application awt.toolkit com.eteks.awt.PJAToolkitServers → ws611 →Process Definition → java.awt.fonts /u/itmnp/pja/lib/fontsServant → Java Virtual java.awt.graphicsenv com.eteks.java2d.PJAGraphicsMachine → Custom EnvironmentProperties java2d.font.usePlatformFont true user.home /u/itmnp/pja user.timezone CSTSecurity → Global Enabled CheckedSecurity Enforce Java 2 Security Not checked Cache timeout 1200 Active Authentication SWAM Active User Registry LocalOKSecurity → Global com.ibm.security.useFIPS Not checkedSecurity → CustomProperties EnableTrustedApplications trueSecurity → User com.ibm.security.SAF.authorization trueRegistries → LocalOS→ Additional Properties com.ibm.security.SAF.EJBROLE. false→ Custom Properties Audit.Messages.Supress After all the variables has been entered, you can click on Save and Save again to modify the Master Configuration. Installing a sample graphic test program The graphic test program is supplied in an ear (Enterprise Application Archive) file. 1. From the administrative console, select Application → Install New Application. Install the application from the /u/itmnp/jctest.ear file, and accept all the default values.112 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 128. 2. Select Application → Enterprise Application and click on jctest, and click on the Configuration tab. Set the following: Classloader Mode PARENT_LAST WAR Classloader Policy Application3. Click OK and Save and Save again.4. Restart the WebSphere Application Server.5. Back on the Administrative Console, select Application → Enterprise Application, check the box beside jctest, and click Start.6. We ran the jctest application by opening a browser and entering the following URL: http://wtsc61.itso.ibm.com:9080/jctest/ The result was the display of the Graphing Setup Test graph, as shown in Figure 5-11. At this stage, we are sure that our graphics driver is working.Figure 5-11 Jctest graph Chapter 5. Mainframe Web application implementation 113
  • 129. 5.3 Web application implementation The Web application implementation in this section discusses the problems and experiences we encountered while doing this installation.5.3.1 Installation procedure overview The implementation steps on z/OS can be summarized in the following steps: 1. Generate the itmnp.env file. When you run the command /u/itmnp/itmnp.sh install the first time, it will generate the itmnp.env file. This file contains the setting that will be used for installing the Web application. 2. Edit the itmnp.env file. The parameters in the itmnp.env file are explained in great detail in IBM Tivoli Monitoring for Network Performance: Planning, Installation, and Configuration, SC31-6363. A sample for our installation is given in Example 5-5. Example 5-5 Out itmnp.env file # v2.1.0.0.57 # For information about setting the values below, refer to the IBM Tivoli # Monitoring for Network Performance: Planning, Installation, and Configuration # Guide chapter about installing the Web application on z/OS. # DB2USER=TIVO01 DB2PASSWD=TIVO01 UI_PORT=9080 MONITOR_PORT=9081 CONFIG_FOR_SECURE_MONITOR=N ACTIVE_USER_REGISTRY_CONFIGURATION=LOCALOS LDAP_SERVER_FQDN_HOST=ldap.myserver.net.com LDAP_SERVERID=cn=root,o=ibm,c=us LDAP_SERVERPW=mypassword WASADMIN_USERNAME=wsadmin WASADMIN_PASSWORD=wsadmin ITMNP_DB_LOCATION_NAME=DB8D ITMNP_DB_TYPE=DB2UDBOS390_V7 NODE= SERVER=ws611sc61 TRACE_INSTALL=OFF WAS_50_BIN=/SC61/WebSphereBD/V5R0M0/BS01/AppServer/bin 3. Test your configuration using the command /u/itmnp/itmnp.sh test.114 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 130. 4. Change to superuser using the su command and run the command /u/itmnp/itmnp.sh install again to actually configure WebSphere Application Server and install the itmnp21 application. 5. Restart WebSphere and log into the administrative console. Go to Application → Enterprise Application and check the status of the itmnp21 application. The status should be green (started). 6. Create the necessary RACF profiles for the EJBROLE class. The commands to do this are shown in Example 5-6. Example 5-6 Defining EJBROLE SETROPTS CLASSACT(EJBROLE) RDEFINE EJBROLE ItscAllAuthority UACC(READ) RDEFINE EJBROLE ITMOperator UACC(NONE) RDEFINE EJBROLE ITMAdmin UACC(NONE) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO01) ACC(READ) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO02) ACC(READ) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO03) ACC(READ) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO04) ACC(READ) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO05) ACC(READ) PERMIT ITMAdmin CLASS(EJBROLE) ID(TIVO06) ACC(READ) PERMIT ITMOperator CLASS(EJBROLE) ID(ITMNPOP) ACC(READ)5.3.2 Verification We verified that our IBM Tivoli Monitoring for Network Performance Web Application installed correctly by going to the WebSphere Administrative Console and selecting Applications → Enterprise Applications. Here we saw that our itmnp21 application has a green arrow beside it, as shown in Figure 5-12 on page 116. This was an indication that our installation was successful. Chapter 5. Mainframe Web application implementation 115
  • 131. Figure 5-12 Web Application installation result To verify that the IBM Tivoli Monitoring for Network Performance Web application was working, we did the following: We started a browser session and entered the following URL: http://wtsc61oe.itso.ibm.com:9080/itmnp The welcome screen was then displayed, as shown in Figure 5-13 on page 117.116 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 132. Figure 5-13 Web application welcome screen5.4 Start and stop procedures This chapter discusses the start and stop procedure for IBM Tivoli Monitoring for Network Performance Web application environment. The monitor’s start and stop process should be coordinated with this process.5.4.1 Start up the Web application IBM Tivoli Monitoring for Network Performance should be started in the following sequence: 1. DB2 has to be started as follows: -db2_system_name start db2 Chapter 5. Mainframe Web application implementation 117
  • 133. 2. Start the WebSphere Application Server from the system console as follows: S controlregionprocname, JOBNAME=server_shortname, ENV=cell_shortname.node_shortname.server_shortname where: controlregionprocname The JCL procedure name in PROCLIB. server_shortname The stepname used to start the procedure. Cell_shortname.Node_shortname.Server_shortname The ENV variable is a concantenation of the cell shortname, the node shortname, and the server shortname. In our environment, we started WebSphere Application Server, as shown in Figure 5-14. S WS5611C,JOBNAME=WS611,ENV=CL611.ND611.WS611 IRR812I PROFILE WS5611C.* (G) IN THE STARTED CLASS WAS USED TO START WS5611C WITH JOBNAME WS611. $HASP100 WS611 ON STCINRDR IEF695I START WS5611C WITH JOBNAME WS611 IS ASSIGNED TO USER ASCR1, GROUP WSCFG1 $HASP373 WS611 STARTED IEF403I WS611 - STARTED - TIME=14.26.59 - ASID=0081 - SC61 BBOO0001I WEBSPHERE FOR Z/OS CONTROL PROCESS CL611/ND611/CLU611/WS611 IS STARTING. Figure 5-14 SC61 WebSphere Application Server startup The controller region automatically starts the daemon. This is the necessary command: START WS5611D,JOBNAME=WS611D,ENV=CL611.CL611.WS611D The JMS queue manager is then started automatically with the command: S WMQXMSTR,SUFFIX=R The JMS channel initiator also starts automatically with the command: S WMQXCHIN Lastly, the servant is started automatically by Workload Manager (WLM) with the command: START WS5611S, JOBNAME=WS611S,ENV=CL611.ND611.WS611118 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 134. 5.4.2 Shut down the Web application IBM Tivoli Monitoring for Network Performance should be stopped in the following sequence: 1. Stop the WebSphere Application Server from the system console as follows: P WS611 where WS611 is the control region step name of the WebSphere Application Server. The JMS queue manager and JMS channel initiator will also be stopped as a result of this command. The JMS queue manager is stopped automatically by the control region, such as by using the command P WMQXMSTR. The JMS channel initiator is also stopped automatically using the command P WMQXCHIN. Lastly, the servant is stopped automatically by WLM with the command P WS611S. 2. Stop the WeSphere Application Server daemon as follows: P WS611D where WS611D is the control region daemon name of the WebSphere Application Server. 3. Stop DB2. You need to ensure all remote threads have disconnected from the output of the DIS THREAD(*) command. The command to stop DB2 is: -DB8D STOP DB2 MODE(QUIESCE) The command prefix in our environment is -DB8D.5.5 Backup and recovery The IBM Tivoli Monitoring for Network Performance database can become unusable because of hardware or software failure or both. A routine for backing up your Tivoli Monitoring for Network Performance database has to be established. How frequently you back up your database depends on how frequently you make changes to your Tivoli Monitoring for Network Performance installation, and on your organization’s policies for backups. The WebSphere Application Server, IBM Tivoli Monitoring for Network Performance Web application, and IBM Tivoli Monitoring for Network Performance monitor contain HFS file systems that also require backing up. HFS file systems can be backed up using utilities, such as the ADRDSSU(DFDSS) dump facility, or UNIX shell commands, such as pax cpio and tar. Chapter 5. Mainframe Web application implementation 119
  • 135. 5.5.1 Backup and restore of file systems There are several HFS files that you may want to back up that are related to the IBM Tivoli Monitoring for Network Performance Web application; they are: WebSphere Application Server configuration IBM Tivoli Monitoring for Network Performance install directory We back up our HFS using the ADRDSSU utility. Before ADRDSSU can successfully DUMP the HFS, the file systems must be unmounted using the UMOUNT command from TSO. The ADRDSSU job is shown in Figure 5-15. //DUMPHFS JOB (ACCOUNT),TIVO02,CLASS=A,NOTIFY=&SYSUID /*JOBPARM S=SC61 //STEP1 EXEC PGM=ADRDSSU,REGION=2048K //SYSPRINT DD SYSOUT=* //OUTDD1 DD DSN=VBUDI.WAS61.DUMP, // SPACE=(CYL,(40,20)), // DISP=(NEW,CATLG,DELETE), // UNIT=3390, // VOL=(,,,,SER=TIVO02) //SYSIN DD * DUMP DATASET( - INCLUDE( - OMVS.SC61.WAS5BD.BS01.CONFIG.HFS - )) - OUTDDNAME(OUTDD1) - TOL(ENQF) /* Figure 5-15 DFDSS HFS dump To restore the HFS dataset, we ran our restore job, which is shown in Figure 5-16 on page 121. This restore job replaced the existing WebSphere Application Server HFS dataset with the backed up dataset. Note that the target HFS file must not be mounted for the restore process to succeed.120 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 136. //RESTHFS JOB (ACCOUNT),TIVO02,CLASS=A,NOTIFY=&SYSUID /*JOBPARM S=SC61 //STEP1 EXEC PGM=ADRDSSU,REGION=2048K //SYSPRINT DD SYSOUT=* //INDD1 DD DSN=VBUDI.WAS61.DUMP, // LABEL=(1,SL), // DISP=(OLD,KEEP,KEEP) //SYSIN DD * RESTORE DATASET( - INCLUDE( - OMVS.SC61.WAS5BD.BS01.CONFIG.HFS - )) - INDDNAME(INDD1) - CANCELERROR - REPLACE CATALOG - TGTALLOC(SOURCE) - WAIT(2,2) /* Figure 5-16 DFDSS HFS restore After the backup or restore process completes, we need to re-mount the HFS dataset. We issue the following mount command to mount the HFS. mount filesystem(‘OMVS.SC61.WAS5BD.BS01.CONFIG.HFS’) mountpoint(‘/SC61/WebSphereBD/V5R0M0/BS01’) type(HFS) mount filesystem(‘OMVS.U.ITMNP.HFS’) mountpoint(‘/u/itmnp’) type(HFS)5.5.2 Backup and restore of DB2 database We used the copy utility to create a full image copy of our ITMNP DB2 tablespaces. To display the ITMNPDB tablespace names, we issued the DB2 summary display command for our ITMNP database, as shown in Figure 5-17 on page 122. Chapter 5. Mainframe Web application implementation 121
  • 137. -DB8D DIS DB(ITMNPDB) DSNT360I -DB8D *********************************** DSNT361I -DB8D * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB8D *********************************** DSNT362I -DB8D DATABASE = ITMNPDB STATUS = RW DBD LENGTH = 165548 DSNT397I -DB8D NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ----- ----------------- -------- -------- -------- ----- WCPT2 TS RW WCPT3 TS RW WCPTS TS RW BP1 IX RW . . . Figure 5-17 Displaying ITMNP database We then added these tablespace names to our backup job, as shown in Figure 5-18. //DB2COPY JOB ACCOUNTING INFO,NOTIFY=&SYSUID,CLASS=A,MSGCLASS=T, // MSGLEVEL=(1,1),REGION=4096K,TIME=1440 /*JOBPARM SYSAFF=SC61,L=9999 //UTIL EXEC DSNUPROC,SYSTEM=DB8D,UID=TEMP,UTPROC= //DSNUPROC.WCPT2 DD DSN=TIVO01.ITMNP.WCPT2.BKP0106,DISP=(,CATLG), // SPACE=(CYL,(250,20),,,ROUND),UNIT=3390,VOL=SER=TIVO04 //DSNUPROC.WCPT3 DD DSN=TIVO01.ITMNP.WCPT3.BKP0106,DISP=(,CATLG), // SPACE=(CYL,(250,20),,,ROUND),UNIT=3390,VOL=SER=TIVO04 //DSNUPROC.WCPTS DD DSN=TIVO01.ITMNP.WCPTS.BKP0106,DISP=(,CATLG), // SPACE=(CYL,(250,20),,,ROUND),UNIT=3390,VOL=SER=TIVO04 //DSNUPROC.SYSIN DD * COPY TABLESPACE ITMNPDB.WCPT2 COPYDDN(WCPT2) TABLESPACE ITMNPDB.WCPT3 COPYDDN(WCPT3) TABLESPACE ITMNPDB.WCPTS COPYDDN(WCPTS) // Figure 5-18 DB2 ITMNP backup job To recover these table spaces from the backup we used before, we used the job shown in Figure 5-19 on page 123.122 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 138. //DB2REST JOB ACCOUNTING INFO,NOTIFY=&SYSUID,CLASS=A,MSGCLASS=T, // MSGLEVEL=(1,1),REGION=4096K,TIME=1440 /*JOBPARM SYSAFF=SC61,L=9999 //UTIL EXEC DSNUPROC,SYSTEM=DB8D,UID=TEMP,UTPROC= //DSNUPROC.WCPT2 DD DSN=TIVO01.ITMNP.WCPT2.BKP0106,DISP=SHR //DSNUPROC.WCPT3 DD DSN=TIVO01.ITMNP.WCPT3.BKP0106,DISP=SHR //DSNUPROC.WCPTS DD DSN=TIVO01.ITMNP.WCPTS.BKP0106,DISP=SHR //DSNUPROC.SYSIN DD * RECOVER TABLESPACE ITMNPDB.WCPT2 RECOVERYDDN(WCPT2) TABLESPACE ITMNPDB.WCPT3 RECOVERYDDN(WCPT3) TABLESPACE ITMNPDB.WCPTS RECOVERYDDN(WCPTS) //Figure 5-19 DB2 ITMNP recover jobYour DB2 database administrator needs to be consulted for a morecomprehensive backup and restore procedure for the database. Chapter 5. Mainframe Web application implementation 123
  • 139. 124 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 140. 6 Chapter 6. Monitor implementation and operation This chapter discusses the implementation of the primary function of IBM Tivoli Monitoring for Network Performance, the monitors. Monitors are agent programs that run on each z/OS systems and collect information on TCP/IP network performance. The discussion in this section is divided into: 6.1, “Monitor installation process” on page 126 6.2, “Some problems and their solutions” on page 130 6.3, “Start and stop procedures” on page 132 6.4, “Sample monitor configuration” on page 134 6.5, “Monitoring best practices” on page 137© Copyright IBM Corp. 2004. All rights reserved. 125
  • 141. 6.1 Monitor installation process This section discusses the installation of the IBM Tivoli Monitoring for Network Performance monitor. We configured the IBM Tivoli Monitoring for Network Performance monitors based on the instructions in IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363 to perform the monitor implementation. The monitor component must be installed on each z/OS system you want to monitor. The discussion is divided into the following topics: 6.1.1, “Before the installation” on page 126 6.1.2, “Preparing the system” on page 127 6.1.3, “Parameters for itmnp.properties” on page 1296.1.1 Before the installation The monitor code is installed, using the System Modification Program/Extended (SMP/E) process, into a UNIX System Services directory. The monitor uses several directory paths for its execution, as discussed in 2.3, “Monitor functions” on page 22. Those directories are listed in Table 6-1.Table 6-1 Path used by the monitor Function Default value Defined from SMP/E target libraries /usr/lpp/itmnp/V2R1M0 DDDEFs Common directory setting /etc/ibm/tivoli/common/cfg/log.prop FNPlogPropertiesLocation in erties itmnpMonitor.sh Tivoli common log directory /var/ibm/tivoli/common/FNP/logs log.properties Configuration files /etc/itmnp CONFIG_DIR in itmnpMonitor.sh Database cache files /var/FNP/dbcache itmnp.properties We felt that we need to coordinate these files into a discrete set of paths so that the maintenance of HFS datasets are easier. We decided to leave the SMP/E installation alone and merge the following: Put the database cache files into the common log directory and create an HFS file for /var/ibm/tivoli/common/FNP with two director logs and dbcache underneath it. We allocate 120 MB on this file system. Consolidate the configuration files under the common directory structure /etc/ibm/tivoli/common with cfg and itmnp underneath it. We allocate 10 MB on this file system.126 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 142. The resulting HFS files that we use are shown in Table 6-2.Table 6-2 HFS allocation HFS file Size Mount point Linked to OMVS.ITMNP21.HFS xx MB /usr/lpp/itmnp /itmnp OMVS.FNP.LOGS.HFS 120 MB /var/ibm/tivoli/common/FNP $SYSNAME/FNP OMVS.TIVOLI.CFG.HFS 10 MB /etc/ibm/tivoli/common $SYSNAME/tivoli Note that the common directories are system specific in the SYSPLEX, while the SMP/E installation HFS are shared across SYSPLEX images.6.1.2 Preparing the system There are some setup actions that must be performed on the following items: “TCP/IP address space” on page 127 “OSNMPD process” on page 127 Figure on page 128 “RACF security profiles” on page 128 “Post installation setup” on page 128 TCP/IP address space There are several tasks to perform for the general Communication Server processes that we need to modify. They are: Enabling the TCP/IP address space for monitoring by adding a NETMONITOR statement to our TCP/IP profile. This definition can also be activated using the V TCPIP,,OBEY,dataset command from the system console: NETMON SMFS TCPCONNS MINLIFET 0 Autolog OSNMPD in the AUTOLOG section of TCP/IP profile. SNMP data collection is important for some performance information and z/OS identification for NetView ITSC. Enabled SNAMGMT in VTAM using the command F NET,VTAMOPTS,SNAMGMT=YES. OSNMPD process The OSNMPD daemon can be autostarted by creating an AUTOLOG entry in the TCP/IP profile definition. This requires the existence of the OSNMPD started task to be defined on the system. Some of the TCP/IP parameter metrics are retrieved Chapter 6. Monitor implementation and operation 127
  • 143. from SNMP. We found that in our environment, the SNMP port 161 is reserved for the SNMPD started task instead of OSNMPD. We also need to change this parameter in the PORT section. We use the obey file shown in Example 6-1. Example 6-1 Deleting SNMP port DELETE PORT 0161 UDP SNMPD DELETE PORT 0162 UDP SNMPQE Activate it using the command V TCPIP,,OBEY,dataset. UNIX System Services parameters To optimize the performance of IBM Tivoli Monitoring for Network Performance in the UNIX System Services, some parameters in the USS need to be changed. These parameters reside in the BPXPRMnn member of SYS1.PARMLIB and are loaded when the USS is started. These are the list of parameter changes: MAXASSIZE 1073741824. The Java environment requires that this limit must be high enough. MAXFILEPROC 1000. MAXTHREADS 10000. MAXTHREADTASKS 5000. MAXPROCSYS 200. MAXPROCUSER 25. RACF security profiles There are some security profiles that RACF can use. Run job FNPDEFID to define users for the IBM Tivoli Monitoring for Network Performance monitor component. Run job FNPUAUTH to define access for a monitor user ID to the z/OS communications server network management application programming interfaces. Post installation setup Run the itmnpPostMon.sh or perform the following manually: Set owner, authorization bit, and setuid fields for the distribution files Copy itmnp.properties and jks files to /etc/itmnp Create the log.properties file that point to the tivoli common directory Check the INSTALL_DIR, CONFIG_DIR, and JAVA_HOME variables in itmnpMonitor.sh. They should point to the correct paths.128 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 144. 6.1.3 Parameters for itmnp.properties The primary configuration for the monitor resides in the itmnp.properties files. We copied the properties file and digital certificate files into the /etc/itmnp directory from /opt/IBM/ITMNP21/config. Changes to the itmnp.properties file that we performed are shown in Table 6-3. The table merged the changes that we perform for z/OS and the AIX Web application. SC61 connects to z/OS Web application and SC62 connects to the AIX Web application. Added our monitor_hostname as the fully qualified host name wtsc62.itso.ibm.com®. Defaulted local_httpport to 9455 for communication between WebSphere Application Server and the monitor. The monitor listens on port 9455. Added the fully qualified host name of the server running our WebSphere Application Server to WAS_hostname. In our case, it was wtsc62.itso.ibm.com. Defaulted WAS_httpport to 9454 for communication between the monitor and WebSphere Application Server. WebSphere Application Server listens on port 9454. Added DBName system name ITMNPDB, as defined in the Web application wizard. Added a valid DB2 user name, password, and host name values. Added DBPort value of 50000, as defined in the Web application wizard.Table 6-3 The itmnp.properties changes Parameter name AIX Web application z/OS Web application monitor_hostname wtsc62.itso.ibm.com wtsc61.itso.ibm.com bind_interface 0.0.0.0 0.0.0.0 local_httpport 9455 9081 WAS_hostname jakarta.itsc.austin.ibm.com wtsc61.itso.ibm.com WAS_httpport 9454 9080 socksServer Not used Not used socksPort Not used Not used DBName ITMNPDB DB8D DBUserName db2inst1 TIVO01 Chapter 6. Monitor implementation and operation 129
  • 145. Parameter name AIX Web application z/OS Web application DBPassword ###### ###### DBHostName jakarta.itsc.austin.ibm.com wtsc61.itso.ibm.com DBPort 50000 38030 DBDriverType UNIVERSAL (default) DBCacheDirectory /var/ibm/tivoli/common/FNP /var/ibm/tivoli/common/FNP DBCacheRowLimit Used default Used default enableCloudscape - - db2Security CLEAR_TEXT_PASSWORD_ CLEAR_TEXT_PASSWORD_ SECURITY SECURITY kerberosServer - - WebSphereServletProtocol HTTPS HTTP trustStoreName itmnpMonitorTrustStore.jks - trustStorePassword WebAS - keyStoreName itmnpMonitorKeyStore.jks - keyStorePassword WebAS - keyManagerPassword ###### - CSAPIport 1670 1670 monitor.trace.level DEBUG_MIN DEBUG_MIN Unlike the AIX implementation, the z/OS implementation does not include security between the monitor and the WebSphere Application Server. The communication transport between these two components are not authenticated and not encrypted in z/OS. Authentication and encryption requires LDAP authentication through WebSphere Application Server, and is only available when running on an AIX system. The parameters regarding security are not applicable in the z/OS implementation.6.2 Some problems and their solutions This section collects some problems that we encounter during the monitor implementation. 6.2.1, “Missing Tivoli common directory” on page 131130 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 146. 6.2.2, “Setting APF authorized attribute” on page 1316.2.1 Missing Tivoli common directory When installing the monitor on WTSC62, we received the message FNPM012E, as shown in Figure 6-1. The monitor did not find the Tivoli common directory specified, which should be defined in the log.properties file. To rectify this problem, we added the tivoli_common_dir=/var/tivoli/common statement to the /etc/ibm/tivoli/common/cfg/log.properties file. +FNPM012E THE TIVOLI COMMON DIRECTORY WAS NOT FOUND IN THE /etc/ibm/ti voli/common/cfg/log.properties FILE; THE DEFAULT VALUE, /var/ibm/tivol i/common, WILL BE USED. Figure 6-1 Monitor Tivoli common directory not found message6.2.2 Setting APF authorized attribute Some programs and files associated with IBM Tivoli Monitoring for Network Performance need to be authorized. In our environment, the monitor was supplied as a tar file. The APF extended attributes are lost during the tar or copy process and have to be set again. We changed the attributes for the files /usr/lpp/itmnp/V2R1M0/bin and /usr/lpp/itmnp/V2R1M0/lib directories using the command: extattr +a bin/monitor.cs390 lib/lib* Figure 6-2 on page 132 shows what the attribute displays should look like using the ls -lE command. The second column of the display should show a-s-. Chapter 6. Monitor implementation and operation 131
  • 147. TIVO02:/: >cd /usr/lpp/itmnp/V2R1M0/bin TIVO02:/itmnp/V2R1M0/bin: >ls -lE monitor_cs390 -rwxr-x--- a-s- 1 AAAAAAA SYS1 1839104 Apr 29 10:33 monitor_cs390 TIVO02:/itmnp/V2R1M0/bin: > TIVO02:/itmnp/V2R1M0/config: >cd ../lib TIVO02:/itmnp/V2R1M0/lib: >ls -lE lib* -rwxr-x--- a-s- 1 AAAAAAA SYS1 720896 Apr 29 10:33 libITMNP.so -rwxr-x--- a-s- 1 AAAAAAA SYS1 106496 Apr 29 10:07 libJLOG.so -rwxr-x--- a-s- 1 AAAAAAA SYS1 626688 Apr 29 10:20 libcclog.dll -rwxr-x--- a-s- 1 AAAAAAA SYS1 827392 Apr 29 10:23 libicmp.so -rwxr-x--- a-s- 1 AAAAAAA SYS1 299008 Apr 29 10:20 libmsg23.so -rwxr-x--- a-s- 1 AAAAAAA SYS1 110592 Apr 29 10:20 libtio.dll -rwxr-x--- a-s- 1 AAAAAAA SYS1 114688 Apr 29 10:20 libtos.dll -rwxr-x--- a-s- 1 AAAAAAA SYS1 86016 Apr 29 10:20 libtproc.dll -rwxr-x--- a-s- 1 AAAAAAA SYS1 106496 Apr 29 10:20 libtthred.dll Figure 6-2 Checking the APF security access attributes for monitor files6.3 Start and stop procedures IBM Tivoli Monitoring for Network Performance monitors are started by the shell script /u/itmnp/V2R1M0/bin/itmnpMonitor.sh. The monitor output will be directed to the standard output of the shell process. You can direct the monitor messages to an output file as follows: /usr/lpp/itmnp/V2R1M0/bin/itmnpMonitor.sh > /tmp/<out file> 2>&1 & We created a procedure that uses BPXBATCH to run shell script itmnpMonitor.sh, which starts the monitor. This procedure enabled us to start the monitor from the system console, as shown in Figure 6-3, instead of having to start it from USS. //ITMNPSTA PROC //ITNMPD EXEC PGM=BPXBATCH,REGION=0M,TIME=NOLIMIT, // PARM=SH cd /usr/lpp/itmnp/V2R1M0/bin;./itmnpMonitor.sh //STEPLIB DD DISP=SHR,DSN=DB8D8.SDSNLOAD // DD DISP=SHR,DSN=DB8D8.SDSNLOD2 //STDERR DD PATH=/var/ibm/tivoli/common/FNP/logs/itmnp.log, // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRUSR,SIWUSR,SIRGRP,SIWGRP) //STDOUT DD PATH=/var/ibm/tivoli/common/FNP/logs/itmnp.log, // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRUSR,SIWUSR,SIRGRP,SIWGRP) Figure 6-3 ITMNPSTA procedure to start the monitor132 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 148. IBM Tivoli Monitoring for Network Performance should be stopped in thefollowing sequence:1. Stop the IBM Tivoli Monitoring for Network Performance monitor as follows: Issue the following command from the USS command line: ps -ef | grep launcher From this output, find the process ID and use it to issue the kill command below: kill <process id> where <process id> is the one belonging to the launcher. We created a procedure to stop the monitor from the system console, as shown in Figure 6-4. //ITMNPSTO PROC //ITNMPD EXEC PGM=BPXBATCH,REGION=0M,TIME=NOLIMIT, // PARM=SH /itmnp/V2R1M0/bin/itmnpStop.sh //STDERR DD PATH=/var/ibm/tivoli/common/FNP/logs/itmnp.log, // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRUSR,SIWUSR,SIRGRP,SIWGRP) //STDOUT DD PATH=/var/ibm/tivoli/common/FNP/logs/itmnp.log, // PATHOPTS=(OWRONLY,OCREAT,OAPPEND), // PATHMODE=(SIRUSR,SIWUSR,SIRGRP,SIWGRP)Figure 6-4 ITMNPSTO procedure to stop the monitor The ITMNPSTO batch procedure executes the shell script, which is shown in Example 6-2. This shell script, which we created, finds the monitor’s process IDs and issues kill commands to stop the monitor.Example 6-2 The itmnpsto.sh script to stop the monitor#!/bin/sh pid=`ps -ef | grep launcher | grep -v grep | awk {print $2}` if [ $pid > 1 ] then kill -9 $pid fi pid=`ps -ef | grep monitor_cs390 | grep -v grep | awk {print $2}` if [ $pid > 1 ] then kill -9 $pid fiexit 0 Chapter 6. Monitor implementation and operation 133
  • 149. 6.4 Sample monitor configuration We started our monitors on WTSC62 and WTSC66 and created monitor definitions for these systems. From the IBM Tivoli Monitoring for Network Performance portfolio, we selected Configure Monitors → z/OS Monitor and clicked Next on the Welcome page. On the Define Monitor Location page, we clicked Add Manually to display the Input Monitor Location page. Here we entered our sysplex name, system name, and fully qualified host name, as shown in Figure 6-5. Figure 6-5 Monitor location page After clicking OK, we were returned to the Monitor Location page, as shown in Figure 6-6 on page 135. We then clicked Autoconfigure. By using the autoconfigure option, we opted to use the IBM default values for our collection schedule, which uses default values for subsequent steps and navigates directly to the Save Monitor definition page.134 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 150. Figure 6-6 Monitor location autoconfigure screenAfter saving and naming our monitor definition, we were returned to theSummary page, as shown in Figure 6-7 on page 136. Chapter 6. Monitor implementation and operation 135
  • 151. Figure 6-7 Monitor Autoconfigure summary On the Summary page, we clicked Finish to save the monitor definition and close the wizard. To get the monitor to use this configuration, we deployed the configuration by clicking on Manage Monitors → Manage Monitor Configuration, clicking on the system name, and then on Deploy Configuration, as shown in Figure 6-8 on page 137.136 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 152. Figure 6-8 Deploying monitor configuration Multiple monitor definitions can be created. One would typically create definitions for each system, a group of systems, or for resources that are shared by multiple systems. Monitor definitions could include exceptions for public holidays and scheduled outages. Refer to the IBM Tivoli Monitoring for Network Performance Administrator Guide, SC31-6364 for a description of how to create z/OS Monitor, SNMP Monitor, and Availability and Response Time Monitor definitions.6.5 Monitoring best practices In the z/OS Communication Server, there is plenty of data that is really useful for problem determination and troubleshooting. The major resources being used by the z/OS Communication Server are CPU, network interfaces, network connections, or applications and storage. The CPU consumption can be monitored using other software, such as RMF™, for example. The other resources can be monitored by IBM Tivoli Monitoring for Network Performance. Chapter 6. Monitor implementation and operation 137
  • 153. The data collected can be used to generate system standards or system profiles. It will define the system behavior in terms of network, such as what applications are up, the total number of connections, what network interfaces are being used, and their utilization. Therefore, those profiles can be used to set thresholds or events whenever something unusual happens. There are some other tools that can be used to generate profiles, such as the Policy Agent that comes with z/OS Communication Server. It is possible to configure it based on the data collected by IBM Tivoli Monitoring for Network Performance. Refer to the following documentation on how to configure the Policy Agent feature: z/OS V1R6 Communications Server: IP Configuration Reference, SC31-8776 z/OS V1R6 Communications Server: IP Configuration Guide, SC31-8775 Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 6: Policy and Network Management, SG24-6839 This section discusses some ideas and suggestions on how to configure the monitors for IBM Tivoli Monitoring for Network Performance. The discussion is divided into: 6.5.1, “Monitored metrics” on page 138 6.5.2, “Monitoring configuration design” on page 140 6.5.3, “Monitoring information” on page 142 6.5.4, “Monitoring storage usage” on page 153 6.5.5, “Monitoring network interfaces” on page 1576.5.1 Monitored metrics In IBM Tivoli Monitoring for Network Performance, there are three types of monitors: z/OS Communication Server monitor. This monitor uses the Communication Server API and SNMP collection to gather performance metrics for the z/OS Communication Server TCP/IP function. Availability and response time monitor. This monitor uses the ping command to collect response time and availability of a remote host from the monitor. SNMP monitor. This monitor collects Management Information Base (MIB) variables from remote nodes and stores them in the database. Each monitoring configuration consists of the following information: The monitor host that will perform this monitoring Monitor type138 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 154. Monitoring target, such as TCP/IP stack (for z/OS monitor) or IP address list (for SNMP and ICMP monitor) Data or metric to collect, depending on the monitor types Threshold and rearm values Schedules on which collection will be performedThe following lists the information that is collected by each type: z/OS Communication Server monitor – TCP stack performance – TCP/IP application performance – TCP connection performance – UDP stack performance – IP stack performance – FTP performance – TN3270 performance – Interface performance data – TCP/IP memory usage – HPR and EE performance data – CSM buffer data – OSA performance data Availability and Response Time monitor – Response time information – Availability statistic SNMP monitor – Cisco router performance statistics – Cisco switch performance statistics – Interface statistics – RMON Ethernet performance Chapter 6. Monitor implementation and operation 139
  • 155. 6.5.2 Monitoring configuration design In our design for monitoring, we consider the following aspects: Schedules of the monitors. These schedules must be consistent with each other and present the least number of interruptions in the production environment. The threshold and rearm setting and event forwarding need to be specified wisely to minimize useless alerts, but still forward all the necessary events. Data selection on what information to collect for which node. Let us consider each aspect one by one. Schedule consideration There are several recommendations that we have after analyzing the collected data: Use a single interval value or a multiplication of that interval for all monitors. The use of the same interval allows consistent reports over all schedules. Usage of the same interval for different monitors also allows cross check of data for different monitors and different hosts. Splitting the schedule for prime-time and off-hours may not be really necessary. We prefer to collect a standard amount of information continuously than a different set of information at discrete times. This is easier to analyze. Emergency schedules for problem determination may be prepared to be activated as needed. Remember that all this information is collected into the same database and the same tables. You may need to analyze the data using SQL commands instead of the Web application to get meaningful insight. Note: Remember that FTP and TN3270 are not collected based on the interval but as the session completed. Threshold and rearm value The consideration for putting in the threshold and rearm values are: Put as much threshold as possible, and supply the rearm value only if it is meaningful. Change these thresholds values as you learn more on the performance of your systems. You may need to create a different threshold set for each z/OS TCP/IP stack, different SNMP target, or different IP addresses. Each monitor will typically be unique. You can start them by using the IBM supplied value, if available.140 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 156. Only forward events that are used by automation to respond; do not flood the automation subsystems with unused events.Collection dataWe recommend you define the collection data or at least make the datacollection plan before you actually define it in the IBM Tivoli Monitoring forNetwork Performance Web application. This will identify the necessary resourcesand possible holes in the collected data.Different monitoring types require different considerations: z/OS Communication Server monitoring – For each z/OS TCP/IP stack, you need to determine whether you need to monitor that stack (for example, test TCP/IP processes). – Consider whether you need to collect FTP and TN3270 data. This information is collected as it becomes available. This may introduce unnecessary traffic on the database. – Open System Adapter (OSA) requires that you run the OSA Service Facility (OSASF) started task. Set this up before activating the OSA monitoring. – Consider whether you have High Performance Routing (HPR) needs. – Consider whether you have Enterprise Extender (EE) monitoring needs. Availability and response time data – You may want to collect response time and availability information for remote subnets that need a connection to the z/OS. Use a node or address that is expected to be always available, such as a router interface or a switch control port. – You may have one or more samples from a remote subnet. – It is not very useful to collect this information on a local subnet unless you need the data for Service Level measurement. – You may want to always forward unavailability events and bad performance events for monitoring to switches and routers. – Other nodes to monitor are distributed servers that connect to the mainframe. – This collection uses the ping protocol; be selective on choosing the node to monitor. SNMP performance data – Typically, you collect this information for routers and switches that are critical for the availability of the network. Chapter 6. Monitor implementation and operation 141
  • 157. – As the SNMP performance data is a polling collection, it may generate a large amount of data a single time, and may obscure other performance metrics when collected at the same time. You may want to have a longer interval for SNMP collection. For example, if you collect z/OS communication server and ICMP data every 15 minutes, you may choose to collect SNMP performance every hour.6.5.3 Monitoring information The following sections display and discuss some sample monitoring results. The discussion is divided into the following topics: “Monitoring TCP/IP stack” on page 142 “Monitoring applications” on page 148 “Monitoring storage usage” on page 153 “Monitoring network interfaces” on page 157 Monitoring TCP/IP stack The TCP/IP stack is the major component of a z/OS image TCP/IP network. It is responsible for providing the IP, TCP, and UDP services to all applications that require those services, such as TN3270 server, CICS®, FTP client and server, DB2, IMS™, WebSphere, and MQSeries®. The IBM Tivoli Monitoring for Network Performance has three different kinds of information on the stack monitoring: TCP protocol UPD protocol IP protocol The TCP protocol set of panels has information on TCP connections, giving a more general view of the stack. Figure 6-9 on page 143 can be used to check the number of active connections, the number of accepted connections, the number of connections dropped, and the connection rate in a specific period of time. Having all of those samples for a specific shift, such as 9:00 AM to 12:00 AM, during a month, only on week days, or from Monday to Friday, it is possible to create a profile for those stacks, for example, based on the number of connections.142 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 158. Figure 6-9 TCP stack availability and responseAfter having a profile for a specific stack, for example, the average number ofconnections for a specific stack is 5000 and the connection rate is 20 per second.This information allows the creation of a threshold to generate alerts when thereare more than 6000 connections and more than 30 connections being acceptedat a given time. This means that something unusual happened in this period andfurther investigation can be performed to find out what is going on (see“Monitoring applications” on page 148 for an example).Figure 6-10 on page 144 showed how to define this threshold (select MonitorConfiguration → Set Thresholds → TCP Stack Performance Data WebApplication). Chapter 6. Monitor implementation and operation 143
  • 159. Figure 6-10 Stack availability and response setting threshold Those monitor configurations can be dynamically changed at anytime. Actually, sometimes there will be some changes on the application or in the system behavior that will require changes on the thresholds to avoid generating some events that are not related to problems. There is some information about the traffic flow regarding the TCP applications running on a z/OS system. Figure 6-11 on page 145 shows, for example, the amount of segments being transmitted and received on a specific system.144 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 160. Figure 6-11 TCP stack throughput and trafficFor the UDP application, there are two pages showing the same information fromthe UPD protocol perspective. One of the major users of the UDP protocol will bethe SNMP daemon. In fact, to implement the IBM Tivoli Monitoring for NetworkPerformance, it is necessary to have the OSMPD daemon running on the z/OSsystem.There are two pages with UDP protocol information: The whole TCP/IP stack UDP traffic overview, as shown on Figure 6-12 on page 146. Chapter 6. Monitor implementation and operation 145
  • 161. Figure 6-12 UDP stack throughput and traffic Per the UDP application overview, as shown on Figure 6-13 on page 147. The columns are reordered to fit on screen; this is not the default view and it is showing three different OSMNPD applications running on three different z/OS images. A filter was applied to show only the job name that contains the smnp string.146 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 162. Figure 6-13 UDP endpoint throughput and trafficFor the IP protocol, the IBM Tivoli Monitoring for Network Performance has onlyone panel, which contains information about the manipulated datagrams for eachone of the systems being monitored. Figure 6-14 on page 148 shows an exampleof this situation. Chapter 6. Monitor implementation and operation 147
  • 163. Figure 6-14 IP stack throughput and traffic Monitoring applications Most of the applications running on a z/OS system support the TCP/IP protocol on some level. Each one has its own application specific protocol built under the TCP/IP protocol. For example, for terminal emulation, we have the UNIX Telnet protocol and the TN3270 protocol. For file transfer, we have the FTP protocol, and for Web applications, we have the HTTP protocol. Those are well-known TCP/IP application protocols, but there are many others. Each one of these protocols is identified by the TCP/IP stack by ports. Each port will be listening, or accepting connections, to a specific application protocol, so there will be a specific address space listening on a specific port. It is possible to have the same address space listening on more than one port at the same time, like the HTTP Server for normal and SSL connections. The TN3270 server can have 256 different ports at the same time for normal and SSL connections. The IBM Tivoli Monitoring for Network Performance cannot understand what is happening within the application protocol, but it can monitor the application148 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 164. behavior in terms of the number of active connections, the number ofconnections accepted, bytes sent and received, and so on.For the TN3270 and FTP applications there are a special set of pages showspecific information for those two application protocols.For the FTP applications, there are three different pages: The active FTP sessions page shows, on a specific time, all server and client sessions from the z/OS images. As an example, see Figure 6-15.Figure 6-15 FTP sessions The z/OS FTP client transfer records page shows only the information about client sessions started from a z/OS system to a remote FTP server. The z/OS FTP server transfer records page show remote clients connections to a z/OS local FTP server. This page shows complete information about file transmissions that occurred on a specific period and can be used, for example, to monitor the activity of specific users. See Figure 6-16 on page 150. Chapter 6. Monitor implementation and operation 149
  • 165. Figure 6-16 FTP server transfer records The TN3270 application is one of the most popular TCP applications running on z/OS systems. There are three pages showing the TN3270 information: The TN3270 server session availability page shows all TN3270 remote clients connected to a TN3270 server running on a z/OS image. Each one of the records will show a complete TN2370 session. As an example, it will show the start time, end time, local and remote IP addresses and ports, and the number of transmitted bytes in a specific TN3270 session. The TN3270 sliding-window average response time page shows the response time for the IP and SNA portions of a TN3270 session. This will only work on z/S V1.5 systems or later when selecting the option to collect the TN3270 performance data on monitor definition. The TN3270 response time counts by time bucket page shows the number of times a response times fits in each response time interval (the buckets). This only works on z/OS V1.5 systems or later when selecting the option to collect the TN3270 performance data on monitor definition.150 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 166. Attention: In z/OS V1.5, the Communication Server TN3270 server has two new configuration statements to define monitor options. These new statements are MONITORGROUP and MONITORMAP. Refer to z/OS V1R6 Communications Server: IP Configuration Reference, SC31-8776 for further information on how to define those parameters and collect more detailed performance data on a TN3270 Server connection. As we are running the monitors on z/OS V1.4, we do not have samples to show.For the other applications, IBM Tivoli Monitoring for Network Performance hasfeatures that can help monitor them.We selected a specific address space, in this case, the DB2 TCP/IP application.By using filters, we can have some information about what is going on with it.The shows the number of active connections, the maximum number ofconnections, the date and time for this number, and the idle time since the lastconnection started. Using this information, it is possible to show that the DB2address space was listening and accepting connections at a given time. SeeFigure 6-17 for an example.Figure 6-17 Other application availability and response Chapter 6. Monitor implementation and operation 151
  • 167. It is possible to do the same thing on the connection availability and response page, which shows the applications response time. See Figure 6-18. Figure 6-18 Other connection availability and response The connection throughput and traffic page shows the traffic rate for the same application. See Figure 6-19 on page 153.152 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 168. Figure 6-19 Other connection throughput and traffic Using this method, it is possible to monitor all TCP/IP applications.6.5.4 Monitoring storage usage To run the applications, the TCP/IP stack has to create control blocks to manage all the connections and to do the network I/O. Those control blocks are used to manage all the connections and they will be built under the TCP/IP address space or in the common storage area, for example, ECSA. There are some guidelines on how to estimate and how to do capacity planning to prevent storage shortages on a z/OS system. For each application, there is a different way to estimate the storage requirement for: The TCP/IP address space The application address space Common Storage Using IBM Tivoli Monitoring for Network Performance, it is possible to monitor: The storage consumption in the private area and the ECSA common storage of the TCP/IP address space. Chapter 6. Monitor implementation and operation 153
  • 169. The storage allocated by the Communications Storage Manager (CSM). For the application address space, look at the application specific information to monitor. The only application that allocates storage in the TCP/IP stack address space is the TN3270 server. Since z/OS Version 1 Release 6, it is possible to run the TN3270 server in a separate address space and the memory will be allocated on it, outside of the TCP/IP stack address space. The CSM is a Communication Server component, share by the TCP/IP and VTAM applications, that permits sharing data between their users, like the FTP server and other applications, avoiding the data movement between them and reducing the overhead. Refer to the z/OS Communications Server CSM Guide Version 1 Release 2, SC31-8808 for more information. Another important task to perform when using TCP/IP applications under z/OS is to do storage capacity planning. Chapter 7, “Performance and tuning“, of Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 5: Availability, Scalability, and Performance, SG24-6517 has information about how to do a storage capacity planning on some applications. It is important to monitor the storage consumption to avoid unexpected TCP/IP stack restart or even initial program loads due to memory leak problems. The storage consumption for a given workload should be stable. If the storage consumption is being increased without any workload growth, like the number of connections, especially TN3270 connections, there is probably something wrong, and you should contact the IBM Support Center to check on the latest z/OS Communication Server service maintenance. Here are some examples from IBM Tivoli Monitoring for Network Performance Web Application pages on how to monitor the storage consumption. In Figure 6-20 on page 155, it is possible to gather information about how much memory is being allocated by the TCP/IP address space. It is possible to match this information against applications running on the z/OS image and create a correlation between them at a given time. The same can be done with the CSM Storage pages.154 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 170. Figure 6-20 TCP/IP storage statisticsIn Figure 6-21 on page 156, it is possible to monitor how much CSM storage isbeing allocated by the CSM, the maximum CSM storage allocation allowed on az/OS image, and the percentage of CSA that is being allocated by the CSM. Chapter 6. Monitor implementation and operation 155
  • 171. Figure 6-21 CSM storage summary If more detailed information is needed, like how much storage is being allocated per pool type, refer to the CM storage monitoring page. See Figure 6-22 on page 157.156 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 172. Figure 6-22 CSM storage monitoring6.5.5 Monitoring network interfaces The network interfaces are used as paths to communicate with the network. A single z/OS image can have different types of interfaces, such as OSA express cards, XCF communication links, and channel attached interfaces, such as Cisco and IBM routers. The most used network interfaces are the OSA adapters. OSA adapters have two different configurations: the QDIO mode and the non-QDIO mode or LCS adapters. To learn more about OSA adapters, refer to the following publications: OSA-Express Customer’s Guide and Reference, SA22-7935 and SA22-7476 OSA-Express Implementation Guide, SG24-5948 OSA-2 Implementation Guide (Update), SG24-4770 The network interfaces have to be monitored to prevent network congestion and to ensure that the network definitions are being followed. One of the ways to verify the network routes and load balancing options is by checking the traffic flow for inbound and outbound directions. Chapter 6. Monitor implementation and operation 157
  • 173. To learn more about network interfaces, look at the following documentation: Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 4: Connectivity and Routing, SG24-6516 z/OS V1R6 Communications Server: IP Configuration Guide, SC31-8775 z/OS V1R6 Communications Server: IP Configuration Reference, SC31-8776 The status page shows the interfaces on all systems. Figure 6-23 shows the interface’s capabilities. There is a filter applied to this page that selects only the interfaces that have osa on their names. Figure 6-23 Interface status In Figure 6-24 on page 159, on the same status page, it is possible, as an example, to see the maximum MTU supported by the interfaces.158 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 174. Figure 6-24 Interface status (continued)In Figure 6-25 on page 160, it is possible to monitor the interface statistics, suchas the traffic rate. It is possible to see, for example, if there are two interfacesconnected to the same lan and the TCP/IP profile is set to use multipath;transmitted octets for both interfaces have to be similar. Chapter 6. Monitor implementation and operation 159
  • 175. Figure 6-25 Interface unicast performance metrics For the multicast and broadcast traffic, there is a specific page, Interface Multicast/Broadcast Performance Metric, which shows information about these protocols. If the OSPF routing protocol is being used, it will generate some data for the multicast traffic.160 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 176. 7 Chapter 7. Discovery and alert interfaces This chapter discusses the interfaces for IBM Tivoli Monitoring for Network Performance for TCP/IP resource discovery and the alerting interface. TCP/IP resource discovery is assisted by the IBM Tivoli NetView Integrated TCP/IP Services Component (ITSC), while the alerting interface uses IBM Tivoli Enterprise Console and IBM Tivoli NetView for z/OS. The discussion is divided into the following topics: 7.1, “NetView Integrated TCP/IP services component” on page 162 7.2, “Event integration with IBM Tivoli Enterprise Console” on page 164 7.3, “Event integration with IBM Tivoli NetView for z/OS” on page 175© Copyright IBM Corp. 2004. All rights reserved. 161
  • 177. 7.1 NetView Integrated TCP/IP services component This section discusses the interface from NetView ITSC to IBM Tivoli Monitoring for Network Performance. The discussion is divided into the following topics: 7.1.1, “Concepts” on page 162 7.1.2, “Configure NetView Integrated TCP/IP Services Component” on page 162 7.1.3, “Verification” on page 1637.1.1 Concepts The discovery functions use the IBM Tivoli NetView product. IBM Tivoli NetView is also called the Integrated TCP/IP Solution Component (ITSC). IBM Tivoli Monitoring for Network Performance requires IBM Tivoli NetView Version 7.1.4, as it has the ability to detect a z/OS node. The discovery function adds a new daemon for IBM Tivoli NetView, which is called nvexportd. This process is a Java daemon started by the spmsur program. The nvexportd daemon connects to the itmnpItsc Web application servlets and uploads subnets, nodes, and smartsets information to IBM Tivoli Monitoring for Network Performance. This daemon can be started using NetView standard commands ovstart nvexportd and ovstop nvexportd. Its status can be queried using the command ovstatus nvexportd. The discovery function uses NetView’s netmon discovery, including the preparation for a seed file. We recommend including the z/OS systems in the netmon seed file for quick discovery. As the communication from NetView ITSC to z/OS uses SNMP, which is UDP based, this typically cannot get through a firewall. We recommend having a separate NetView machine on each local subnet that has z/OS monitored.7.1.2 Configure NetView Integrated TCP/IP Services Component We installed IBM Tivoli NetView on a Windows server and installed the nvexportd component of the IBM Tivoli Monitoring for Network Performance on a Windows server. We follow the instructions in Chapter 8, “Setting Up the NetView Integrated TCP/IP Services Component“, of IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363.162 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 178. Example 7-1 shows that the NetView Integrated TCP/IP Services Component is up and running. Example 7-1 Status of nvexportd C:usrovconf>ovstatus nevexportd object manager name: nvexportd behavior: OVs_WELL_BEHAVED state: RUNNING PID: 2232 last message: Initialization complete. exit status: - Done7.1.3 Verification We checked that the NetView Integrated TCP/IP Services Component sent the network resources to the Web application by opening a browser to the following URL: https://jakarta:9454/itmnpitsc/BrowseContents Figure 7-1 on page 164 shows that the NetView Integrated TCP/IP Services Component sent the network resources to WebSphere Application Server as XML-formatted data. Chapter 7. Discovery and alert interfaces 163
  • 179. Figure 7-1 Display of IP-resources from WebSphere Application Server7.2 Event integration with IBM Tivoli Enterprise Console IBM Tivoli Monitoring for Network Performance monitors can be configured to generate events when specific thresholds are crossed. Events can be received by any application that is designed to receive Event Integration Facility (EIF) events, such as IBM Tivoli Enterprise Console (TEC). An overview of IBM Tivoli Enterprise Console is provided in “IBM Tivoli Enterprise Console Version 3.9” on page 235. The discussion in this section is divided into the following topics: 7.2.1, “Customizing TEC rulebase” on page 165 7.2.2, “Configuring event forwarding” on page 168 7.2.3, “Sample event automation program” on page 173164 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 180. 7.2.1 Customizing TEC rulebase IBM Tivoli Enterprise Console is an automation application based on a rulebase. A rulebase contains a set of baroc classes that defines the event formats and structure and a set of rulesets that defines the processing for those events. In order for IBM Tivoli Enterprise Console to understand the events sent from IBM Tivoli Monitoring for Network Performance, it needs the baroc class definition in its rulebase. This baroc class is supplied by the file itmnp.baroc from the IBM Tivoli Monitoring for Network Performance CD-ROM. This file needs to be copied to a local directory of the IBM Tivoli Enterprise Console server. Customization of the rulebase can be performed using the command line or graphical interface with Tivoli Desktop. The command line processing is explained in detail in IBM Tivoli Monitor for Network Performance: Planning, Installing, and Configuring, SC31-6363. We assume that Tivoli Framework and IBM Tivoli Enterprise Console are installed and running. Command line processing We install the IBM Tivoli Monitoring for Network Performance classes to a new rulebase as follows: 1. We create a new rulebase called itmnp using the command wrb -crtrb itmnp -d C:/Tivoli/bin/w32-ix86/TME/TEC/itmnp_rb. 2. We import the itmnp.baroc into the new rulebase using the command wrb -imprbclass C:/itmnp.baroc itmnp. This command copies the itmnp.baroc file into the TEC_CLASSES directory C:/Tivoli/bin/w32-ix86/TME/TEC/itmnp_rb/TEC_CLASSES. 3. We compiled the itmnp.baroc using the command wrb -comprules itmnp. Example 7-2 shows the output of compiled rule. This will create the load rule base for TEC. Example 7-2 Output of the compile rule base bash$ wrb -comprules itmnp Compiling itmnp Loading Classes Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSESroot.baroc Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSEStec.baroc Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSESitmnp.baroc Loading Rule Sets Loading Rule Packs Loading RB Targets Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_RULESEventServer Translating Rules Compiling Rules Chapter 7. Discovery and alert interfaces 165
  • 181. Building RB target EventServer Compiling C:Tivolibinw32-ix86TMETECitmnp_rb.rbtargetsEventServerTEC_RUL ES.load_rules.pro Compilation Complete. 4. We loaded the rule base using the wrb -loadrb itmnp command, as shown in Example 7-3. Example 7-3 Display of the load rule base. bash$ wrb -loadrb itmnp Loading Classes Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSESroot.baroc Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSEStec.baroc Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_CLASSESitmnp.baroc Loading Rule Sets Loading Rule Packs Loading RB Targets Parsing C:Tivolibinw32-ix86TMETECitmnp_rbTEC_RULESEventServer 5. We stopped the TEC using the wstopesvr command. 6. We restarted the TEC with the wstartesvr command. The following message indicates the TEC server has started: The Tivoli Enterprise Console Server is running. Tivoli Desktop interface The same processing can be performed using the Tivoli Desktop. 1. From the Tivoli Desktop, find the Event Server icon. Double click to open the Event Server and get the list of rule bases, you will see at least the Default rule base. 2. From the menu, select Create → Rule Base, as shown in Figure 7-2 on page 167. In the New rule base dialog, specify the name and directory of the new rule base. Click on Create & Close.166 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 182. Figure 7-2 Create a new rule base3. Right-click on the created rule base and select Import, as shown in Figure 7-3. Check the import class and input the path of the itmnp.baroc file. Click on Import & Close.Figure 7-3 Importing a baroc file4. Right-click on the itmnp rulebase and select Compile. The compilation window is shown in Figure 7-4 on page 168. Chapter 7. Discovery and alert interfaces 167
  • 183. Figure 7-4 Compile rulebase window 5. Right-click on the itmnp rulebase and select Load & Close. The load window is shown in Figure 7-5. Figure 7-5 Load rulebase window 6. Close the rulebase window. 7. Right-click on the Event Server icon and select Shut down. Wait until the red arrow disappears and then right-click again and select Start Up. When the red arrow reappears as a solid object, the process has completed.7.2.2 Configuring event forwarding Now that the IBM Tivoli Enterprise Console has been configured to receive IBM Tivoli Monitoring for Network Performance events, we need to tell the monitors to start sending the events to our IBM Tivoli Enterprise Console server. 1. First, we need to have a configuration that has the forwarding request. We defined the FTP threshold event and checked the generate event field in the configure z/OS monitor, as shown in Figure 7-6 on page 169.168 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 184. Figure 7-6 Setup to generate an event2. We saved and deployed the monitor configuration to the monitor.3. We defined the IBM Tivoli Enterprise Console IP address and port in the IBM Tivoli Monitoring for Network Performance by selecting Set Run-time Preferences → Define Event Receivers. Click on the Add button, as shown in Figure 7-7 on page 170. The default port number for IBM Tivoli Enterprise Console is 5529. Chapter 7. Discovery and alert interfaces 169
  • 185. Figure 7-7 Defining the event receiver 4. We simulated the FTP server transfer records exceeding the set threshold to generate the event. Figure 7-8 on page 171 shows the event generated on IBM Tivoli Monitoring for Network Performance.170 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 186. Figure 7-8 Threshold crossed is marked with a red X circle5. Figure 7-9 on page 172 shows the event received in the TEC event. Chapter 7. Discovery and alert interfaces 171
  • 187. Figure 7-9 Display of TEC event view The detailed event structure is shown in Figure 7-10 on page 173.172 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 188. Figure 7-10 Detailed event structure7.2.3 Sample event automation program In this section, we provide a sample ruleset and program to process the event. One problem with the event from IBM Tivoli Monitoring for Network Performance view is that all events use ITMNP_ThresholdExceeded or ITMNP_ThresholdRearmed. The important information about the measurement is contained in the following slots: Hostname: The host name where the monitor that sends the event executes field_name: The actual threshold that get tripped field_value: The threshold value that causes the event The sample ruleset that we use is shown in Example 7-4 on page 174. Chapter 7. Discovery and alert interfaces 173
  • 189. Example 7-4 Sample itmnp.rls file rule: itmnp_threshold: ( event: _event of_class within [ITMNP_ThresholdExceeded] where [ ], reception_action: invoke_Automation: ( exec_program(_event, scripts/itmnp_tec.sh, , [], YES), commit_set ) ). rule: itmnp_rearmed: ( event: _event of_class within [ITMNP_ThresholdExceeded] where [ field_name: _field_name, monitor_host: _monitor_host ], reception_action: invoke_Automation: ( exec_program(_event, scripts/itmnp_tec.sh, , [], YES), commit_set ) reception_action: close_Threshold_event: ( first_instance(event: _original_event of_class ITMNP_ThresholdExceeded where [ field_name: equals _field_name, monitor_host: equals _monitor_host ] ), set_event_status(_original_event, CLOSED), commit_set ) ). This rule selects all IBM Tivoli Monitoring for Network Performance events and executes the itmnp_tec.sh program. The itmnp_tec.sh tool needs to parse and validate the event information and act appropriately. Our sample script just parses the event and writes to an output file. A real program needs to perform additional functions, such as sending e-mail or a page to an operator. Our itmnp_tec.sh is shown in Example 7-5 on page 175.174 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 190. Example 7-5 The itmnp_tec.sh #!/bin/sh LOGFILE=/tmp/itmnpevents.log echo $EVENT_CLASS $date_reception $monitor_host $field_name $field_value >> $LOGFILE # if you want to send this to TBSM based on the measurement # assuming all event goes to ITMNP 2.1 object class # $TEC_BIN_DIR/../../TDS/EventService/ihstttec -b "ITMNP;2.1" # -i "$monitor_host" -p "$field_name" -s "$severity" -o 20 # -1 "$field_value" -t "EXCEPTION" -q "$origin" -m "$msg" ####################################### # event specific processing goes here if [ "$EVENT_CLASS" = "ITMNP_ThresholdExceeded" ]; then # Here are processing specific to Threshold tripped event $TEC_BIN_DIR/bin/TEC_Send_Mail.sh "ITMNP-TEC interface" tec_itmnp "$msg" else # Here are processing specific to Rearmed events fi7.3 Event integration with IBM Tivoli NetView for z/OS To alert an operator about performance issues and threshold value violations, rearm condition events have to be set in IBM Tivoli Monitoring for Network Performance. These generated events can then be sent to IBM Tivoli NetView for z/OS. IBM Tivoli NetView for z/OS must be configured to receive these events using the event-to-alert conversion facility of the Event Automation Service (EAS). This discussion is similar to 7.2, “Event integration with IBM Tivoli Enterprise Console” on page 164. It consists of the following topics: 7.3.1, “Setting up Event Automation Service” on page 176 7.3.2, “Defining threshold and event generation” on page 177 7.3.3, “Automating NetView alert” on page 181 Chapter 7. Discovery and alert interfaces 175
  • 191. 7.3.1 Setting up Event Automation Service Event Automation Service provides an interface between IBM Tivoli NetView for z/OS and IBM Tivoli Enterprise Console. It can send NMVT alerts as TEC events and receive TEC events as NMVT alerts. Here we use the function that receives a TEC event and converts it to a Generic NMVT alert (GENALERT). The following steps set the Event Automation Service. For more information, refer to Tivoli NetView for z/OS™, Installation: Configuring Additional Components Version 5 Release 1, SC31-8874. 1. We copy the Event Automation Service started task procedure, IHSAEVNT, from NETVIEW.SCNMUXMS to our PROCLIB. 2. We customize the IHSAEVNT member and change the reference to NetView datasets in our environment, SQ1=NETVIEW and UQ1=NETVUSER.SC61N. 3. We set the STEPLIB dataset NETVIEW.SCNMUXLK to be APF authorized by using the SETPROG APF,ADD,DSN=NETVIEW.SCNMUXLK,VOL=****** command. 4. We change the event receiver configuration file in dataset NETVIEW.SCNMUXCL(IHSAECFG) to include the statement PortNumber=9162. If the port number is not specified or the value is zero, then a port number will be dynamically assigned. A static port number is required for the event forwarding from IBM Tivoli Monitoring for Network Performance. 5. We also ensure that the event receiver task is started with the initialization of the EAS. The initialization member is NETVIEW.SCNMUXCL(IHSAINIT). You need to ensure that the line NOSTART TASK=EVENTRCV is commented out. 6. We start the EAS using the command MVS S IHSAEVNT from NetView session. We then check the PPI registration from the EAS using the DISPPI command. It should have results similar to Example 7-6. Example 7-6 DISPPI result * SC61N DISPPI SC61N DWO948I RECEIVER RECEIVER BUFFER QUEUED TOTAL STORAGE RCVR DWO949I IDENTITY STATUS LIMIT BUFFERS BUFFERS ALLOCATED ASID DWO950I -------- -------- ---------- ---------- ---------- ---------- ---- DWO951I NETVALRT ACTIVE 1000 0 0 0 0020 DWO951I DSIMCAT ACTIVE 25 0 0 0 0020 DWO951I ISTMTRCV ACTIVE 500 0 1 0 001C DWO951I SC61NHTM ACTIVE 1000 0 0 0 0020 DWO951I NETVRCV ACTIVE 500 0 2 0 0020 DWO968I END OF DISPLAY176 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 192. 7.3.2 Defining threshold and event generation The process of determining a meaningful set of threshold and rearm values for your environment will most likely be an interactive process. In our environment, we set IBM Tivoli Monitoring for Network Performance to forward alerts to IBM Tivoli NetView for z/OS. The process to set up forwarding and threshold is similar to 7.2.2, “Configuring event forwarding” on page 168. This event was forwarded to IBM Tivoli NetView for z/OS EAS and displayed in the hardware monitor (NPDA) events display. Tip: If it is indicated that events have been generated and you do not see the event in NPDA, you may have active NPDA filters. Use the following REXX program to clear the event filters: /* REXX CLIST to Delete Default Alert Filter Settings */ NPDA SRF AREC DELETE E HELD TREF CTRL NPDA SRF AREC DELETE E PERM TREF CTRL NPDA SRF AREC DELETE E PERF TREF CTRL NPDA SRF AREC DELETE E HELD TREF LCTL NPDA SRF AREC DELETE E PERM TREF LCTL NPDA SRF AREC DELETE E PERF TREF LCTL NPDA SRF AREC DELETE E HELD NPDA SRF AREC DELETE E PERM NPDA SRF AREC DELETE E USER NPDA SRF AREC DELETE E NTFY NPDA SRF AREC DELETE E INST NPDA SRF AREC DELETE E SCUR NPDA SRF AREC DELETE E UNKN NPDA SRF AREC DELETE E PERF NPDA SRF AREC DELETE TREF CPU NPDA SRF AREC PASS DEFAULT exit Figure 7-11 on page 178 shows the event history page in NetView. Chapter 7. Discovery and alert interfaces 177
  • 193. N E T V I E W SESSION DOMAIN: SC61N VBUDI 06/15/04 18:34:22 NPDA-41A * MOST RECENT EVENTS * PAGE 1 OF 1 SC61N WTSC61.I 9.12.4.3 ITMNP +--------+ +--------+ +--------+ DOMAIN | PWS |---| NTID |---| APPL | +--------+ +--------+ +--------+ SEL# DATE/TIME EVENT DESCRIPTION:PROBABLE CAUSE ETYP ( 1) 06/01 00:39 wtsc61.itso.ibm.com: Thre:severity=HARMLESS; TEMP ( 2) 05/31 21:57 wtsc61.itso.ibm.com: Thre:severity=WARNING; IMPD ( 3) 05/31 09:51 wtsc61.itso.ibm.com: Thre:severity=HARMLESS; TEMP ( 4) 05/31 08:36 wtsc61.itso.ibm.com: Thre:severity=WARNING; IMPD ( 5) 05/30 12:22 wtsc61.itso.ibm.com: Thre:severity=HARMLESS; TEMP ( 6) 05/30 11:31 wtsc61.itso.ibm.com: Thre:severity=WARNING; IMPD ( 7) 05/29 17:02 wtsc61.itso.ibm.com: Thre:severity=HARMLESS; TEMP ( 8) 05/29 15:54 wtsc61.itso.ibm.com: Thre:severity=WARNING; IMPD ( 9) 05/29 06:52 wtsc61.itso.ibm.com: Thre:severity=HARMLESS; TEMP (10) 05/29 06:12 wtsc61.itso.ibm.com: Thre:severity=WARNING; IMPD ENTER ST (STAT), SEL# (ACTION), OR SEL# PLUS D (EVENT DETAIL) ??? CMD==> Figure 7-11 Event history for SC61 Selecting one of these WARNING events gives the threshold exceeded event, as shown in Figure 7-12 on page 179.178 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 194. N E T V I E W SESSION DOMAIN: SC61N VBUDI 06/15/04 18:37:56 NPDA-43S * EVENT DETAIL * PAGE 1 OF 2 SC61N WTSC61.I 9.12.4.3 ITMNP +--------+ +--------+ +--------+ DOMAIN | PWS |---| NTID |---| APPL | +--------+ +--------+ +--------+ HIERARCHY NAMES LIST: PWS wtsc61.itso.ibm.com NTID 9.12.4.32 APPL ITMNP DATE/TIME: RECORDED - 05/31 21:57 EVENT TYPE: IMPENDING PROBLEM DESCRIPTION: wtsc61.itso.ibm.com: Threshold tripped [ PROBABLE CAUSES: severity=WARNING; ORIGINAL T/EC EVENT: ITMNP_ThresholdExceeded; monitor_host=wtsc61.itso.ibm.com; source=ITMNP; msg=wtsc61.itso.ibm.com: Threshold tripped [TCP_CONN_AVL_MSMT.PERC_SEG_RETRANS = 3.252033 (> 3)]; field_name=TCP_CONN_AVL_MSMT.PERC_SEG_RETRANS; msmt_details=Category=null;LOCAL_NODE_IP=9.12.4.33;REMOTE_PORT=3628;RETRA NS_RT=0.001674;RESP_TM_VAR=38;RETRANS_SEGS=4;START_TM=2004-05-27-03.29.46. 000000;REMOTE_NODE_IP=9.155.177.145;REM_WINDOW_CNT=0;APP_JOB_NM=DFSKERN ;LAST_ACTIV msmt_id=c8b49449090c0420000000fcd9417732; timestamp=2004-05-30 23:43:42.0; adapter_host=wtsc61; hostname=wtsc61.itso.ibm.com; ENTER A (ACTION) OR DM (DETAIL MENU) ??? CMD==>Figure 7-12 Threshold exceeded eventThe corresponding rearm event is shown in Figure 7-13 on page 180. Chapter 7. Discovery and alert interfaces 179
  • 195. N E T V I E W SESSION DOMAIN: SC61N VBUDI 06/15/04 18:39:53 NPDA-43S * EVENT DETAIL * PAGE 1 OF 2 SC61N WTSC61.I 9.12.4.3 ITMNP +--------+ +--------+ +--------+ DOMAIN | PWS |---| NTID |---| APPL | +--------+ +--------+ +--------+ HIERARCHY NAMES LIST: PWS wtsc61.itso.ibm.com NTID 9.12.4.32 APPL ITMNP DATE/TIME: RECORDED - 06/01 00:39 EVENT TYPE: TEMPORARY DESCRIPTION: wtsc61.itso.ibm.com: Threshold rearmed [ PROBABLE CAUSES: severity=HARMLESS; ORIGINAL T/EC EVENT: ITMNP_ThresholdRearmed; monitor_host=wtsc61.itso.ibm.com; source=ITMNP; msg=wtsc61.itso.ibm.com: Threshold rearmed [TCP_CONN_AVL_MSMT.PERC_SEG_RETRANS = 0.000000 (<= 1)]; field_name=TCP_CONN_AVL_MSMT.PERC_SEG_RETRANS; msmt_details=Category=null;LOCAL_NODE_IP=9.12.4.33;REMOTE_PORT=3628;RETRA NS_RT=0.000000;RESP_TM_VAR=49;RETRANS_SEGS=0;START_TM=2004-05-27-03.29.46. 000000;REMOTE_NODE_IP=9.155.177.145;REM_WINDOW_CNT=1;APP_JOB_NM=DFSKERN ;LAST_ACTIV msmt_id=c1c710dc090c0420000000fcda6eae44; timestamp=2004-05-31 05:09:48.0; adapter_host=wtsc61; hostname=wtsc61.itso.ibm.com; ENTER A (ACTION) OR DM (DETAIL MENU) ??? CMD==> Figure 7-13 Threshold rearmed event180 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 196. 7.3.3 Automating NetView alert This section provides some samples to perform automation on IBM Tivoli Monitoring for Network Performance events. As discussed in 7.2.3, “Sample event automation program” on page 173, there are only two event classes defined for IBM Tivoli Monitoring for Network Performance: threshold exceeded event and threshold rearmed event. For NPDA alert, this event content is further hidden in the sub-vector 31 for the original TEC event, as shown in Figure 7-12 on page 179. This provides a challenge on writing the automation program. We make a sample Message Automation Table (MAT) called ITMNP, as shown in Example 7-7. This MAT is then inserted to the active processing using the AUTOTBL MEMBER=ITMNP,FIRST command. Example 7-7 ITMNP sample MAT * * ITMNP Event Automation starts here * IF MSUSEG(0000) ¬= THEN BEGIN; IF HIER(3) = ITMNP APPL THEN BEGIN; * Common processing for ITMNP events EXEC (CMD(ITMNPCMD COMMON) ROUTE(ONE +AUTO1)) CONTINUE(Y); * * ITMNP_ThresholdExceeded event * IF MSUSEG(0000.31(1).30) = . ITMNP_ThresholdExceeded . THEN EXEC (CMD(ITMNPCMD DOWN) ROUTE(ONE +AUTO1)) CONTINUE(N); * * ITMNP_ThresholdRearmed event * IF MSUSEG(0000.31(1).30) = . ITMNP_ThresholdRearmed . THEN EXEC (CMD(ITMNPCMD UP) ROUTE(ONE +AUTO1)) CONTINUE(N); END; END; The automation called the program ITMNPCMD. This is a REXX program that will parse the actual event content and perform some processing. This sample ITMNPCMD program is provided in Example 7-8 on page 182. Chapter 7. Discovery and alert interfaces 181
  • 197. Example 7-8 ITMNPCMD sample REXX program /*REXX*/ /*ITMNPCMD - Automation command to process ITMNP events */ /* Step 1 - Collect MSU information */ event_class = MSUSEG(0000.31(1).30,2) parse value MSUSEG(0000.31(2).30) with = monitor_host ; parse value MSUSEG(0000.31(3).30) with = source ; parse value MSUSEG(0000.31(4).30) with = message ; parse value MSUSEG(0000.31(5).30) with = field_name ; parse value MSUSEG(0000.31(6).30) with = msmt_detail parse value MSUSEG(0000.31(7).30) with = msmt_id ; parse value MSUSEG(0000.31(8).30) with = timestamp ; parse value MSUSEG(0000.31(9).30) with = adapter_host ; parse value MSUSEG(0000.31(10).30) with = hostname ; parse value MSUSEG(0000.31(11).30) with = adapter_ip ; parse value MSUSEG(0000.31(12).30) with = origin ; parse value MSUSEG(0000.31(13).30) with = thold_id ; parse value MSUSEG(0000.31(14).30) with = sub_source ; parse value MSUSEG(0000.31(15).30) with = field_value ; parse value MSUSEG(0000.31(16).30) with = severity ; /* Step 2 - process argument */ parse arg cmd data select when cmd=COMMON then do /* common processing */ say ITMNP event received for message from monitor_host end when cmd=UP then do /* UP processing */ end when cmd=DOWN then do /* DOWN processing */ end otherwise nop end exit182 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 198. 8 Chapter 8. Historical reporting This chapter discusses the interface of IBM Tivoli Monitoring for Network Performance that is used to store historical information. The interface collects the historical data that is then stored in Tivoli Data Warehouse. The discussion consists of the following topics: 8.1, “Tivoli Data Warehouse overview” on page 184 8.2, “Tivoli Data Warehouse setup” on page 184 8.3, “Installation of the Warehouse Enablement Pack” on page 187 8.4, “ETL processing” on page 197 8.5, “Sample reports” on page 202© Copyright IBM Corp. 2004. All rights reserved. 183
  • 199. 8.1 Tivoli Data Warehouse overview Tivoli Data Warehouse is a strategic Tivoli product that collects system management information and stores it in a central data warehouse. This data is then deployed to specific data marts for reporting purposes. For more information on Tivoli Data Warehouse V1.2, refer to Implementing Tivoli Data Warehouse V1.2, SG24-7100. Tivoli Data Warehouse uses data collected from each Tivoli product and loads it using a set of Extract, Transform, Load (ETL) programs. Each product provides the definition on how Tivoli Data Warehouse should handle their data; this is called the Warehouse Enablement Pack (WEP). The processing is shown in Figure 8-1. ETL1 ETL2 Source CDW Data mart data Figure 8-1 Warehouse processing The WEP contains the following definitions: Database schema to be created in the Central Data Warehouse (CDW) Database schema to be created in the data mart ETL definitions that load data from the source to CDW ETL definitions that load data from CDW to the data mart Tivoli Data Warehouse V1.2 now supports a data source residing on DB2 for z/OS. The discussion in this chapter will see how IBM Tivoli Monitoring for Network Performance data is loaded to Tivoli Data Warehouse.8.2 Tivoli Data Warehouse setup Tivoli Data Warehouse consists of the following components: Tivoli Data Warehouse control server Tivoli Data Warehouse agent site Tivoli Data Warehouse reporting server, with Crystal Enterprise 9 We put the whole warehouse installation into a single machine, as we only want to evaluate the historical data collected from IBM Tivoli Monitoring for Network184 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 200. Performance. We have two scenarios set up; therefore, we have two machinesfor Tivoli Data Warehouse:phoenix This warehouse collects data from the AIX Web application server on jakarta.kingston This warehouse collects data from the z/OS Web application server on SC61.In Tivoli Data Warehouse V1.2, the z/OS data source requires that the CDW anddata mart reside on z/OS. The configuration that we use is shown in Figure 8-2. E T ETL1 L ITMNPDB TWH_CDW 2 TWH_MART IBM Tivoli Monitoring for Network Tivoli Data Warehouse V1.2 Performance V1.2 Crystal Enterprise 9 WebSphere Application Server 5 DB2 UDB V7.2.10 DB2 V8.1 Microsoft IIS server jakarta phoenix E T ETL1 L ITMNPDB TWH_CDW 2 TWH_MART IBM Tivoli Monitoring for Network Tivoli Data Warehouse V1.2 Performance V1.2 Crystal Enterprise 9 WebSphere Application Server 5 DB2 UDB V7.2.10 DB2 V8.1 Microsoft IIS server SC61 kingstonFigure 8-2 Warehouse configurationThe installation process for phoenix and kingston are different. The next sectionsdiscuss the installation on each. We only highlight the important steps on theWarehouse Enablement Pack. Chapter 8. Historical reporting 185
  • 201. 8.2.1 Installation for distributed data source The installation of the Tivoli Data Warehouse on phoenix is performed after we have the prerequisites installed. These are our existing platforms: Windows 2000 Server Service Pack 4 with Microsoft® Internet Information Server (IIS) installed. DB2 Universal Database Version 7.2.10 with the data warehousing tools installed. The default database for DB2 warehouse control database is called DWCTRLDB; if you choose the typical install, this is the database that will be used. Tivoli Data Warehouse uses a control database called TWH_MD. We install Crystal Enterprise 9 first. The installation will customize the Microsoft IIS Web server and creates the necessary Crystal Enterprise 9 resources. Crystal Enterprise needs to be installed before you install Tivoli Data Warehouse. Note: If you use IBM HTTP Server, these customizations are not automatic. You need to use the IBM HTTP Administration Server to customize the configuration. We install Tivoli Data Warehouse using the quick start deployment. This installs all components of the Tivoli Data Warehouse V1.2 into a single machine. While this may not be the recommended configuration for a production environment, it is sufficient for our test environment. The databases TWH_CDW, TWH_MART, and TWH_MD are created and populated in the local machine phoenix. Now we are ready to install the warehouse enablement pack.8.2.2 Installation for mainframe data source To use data sources and databases on the z/OS platform, we need to install Tivoli Data Warehouse V1.2 using a customized install. Again, we need to have the prerequisites all installed. These are our existing platforms: Windows 2000 Server Service Pack 4 with Microsoft Internet Information Server (IIS) installed. DB2 Universal Database Version 7.2.10 with the data warehousing tools installed. The default database for DB2 warehouse control database is called DWCTRLDB; if you choose the typical install, this is the database that will be used. Tivoli Data Warehouse uses a control database called TWH_MD. We install Crystal Enterprise 9 first. The installation will customize the Microsoft IIS Web server and creates the necessary Crystal Enterprise 9 resources. Crystal Enterprise needs to be installed before you install Tivoli Data Warehouse.186 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 202. We prepare the z/OS database using the following steps: 1. Create the necessary storage group and tablespaces for Tivoli Data Warehouse on DB2 for z/OS. We use our DB8D subsystem and invoke a SQL program using file input (SPUFI) from the DB2 interactive menu in ISPF. 2. The statement that we execute is shown in Example 8-1. Example 8-1 Statement to initialize DB2 for z/OS CREATE STOGROUP TDWSG VOLUMES (TIVO01,TIVO02,TIVO03) VCAT DB8DU; CREATE DATABASE TWHTEMP BUFFERPOOL BP0 STOGROUP TDWSG AS TEMP; CREATE TABLESPACE TTEMP IN TWHTEMP USING STOGROUP TDWSG PRIQTY 5000 SECQTY 2000 The Tivoli Data Warehouse installation on kingston uses the Custom or Distributed installation types. In the installation wizard, we need to specify the z/OS databases as the Central Data Warehouse and data mart.8.3 Installation of the Warehouse Enablement Pack This section discusses the Warehouse Enablement Pack installation. The discussion is divided into the following topics: 8.3.1, “Back up TWH databases” on page 187 8.3.2, “Warehouse Enablement Pack installation” on page 1888.3.1 Back up TWH databases Before installing any warehouse enablement pack, we highly recommend backing up all the warehouse databases to ensure that you can return to a known valid state if you encounter an unrecoverable error during installation. We need to back up three databases: TWH_CDW, TWH_MART, and TWH_MD. See 4.6, “Backup and recovery” on page 91 for the DB2 distributed backup procedure and see 5.5, “Backup and recovery” on page 119 for the z/OS DB2 database backup procedure. Remember to stop the Microsoft Internet Information Server, Crystal Enterprise, Warehouse Server, and Warehouse Logger before you perform the backup process. Chapter 8. Historical reporting 187
  • 203. 8.3.2 Warehouse Enablement Pack installation To install the IBM Tivoli Monitoring for Network Performance warehouse pack, we completed the following steps on the control server: 1. Click Start → Programs → TDW → Install a Warehouse Pack; this starts the warehouse pack installation wizard shown in Figure 8-3. In the Welcome window, click Next. Figure 8-3 Install a Warehouse Pack window 2. The window in Figure 8-4 on page 189 shows the location of the Tivoli common logging directory, which will contain all the TDW log files. In our installation, we use the default location C:Program FilesibmTivolicommon. Click Next.188 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 204. Figure 8-4 Tivoli Common Logging Directory window3. In the window shown in Figure 8-5, click Add to add the IBM Tivoli Monitoring for Network Performance warehouse enablement pack.Figure 8-5 Add Warehouse Pack window4. In the location of the installation properties file window, shown in Figure 8-6 on page 190, click Browse to point to the location of the IBM Tivoli Monitoring Chapter 8. Historical reporting 189
  • 205. for Network Performance warehouse enablement pack installation properties file, twh_install_props.cfg. You can find this file in the IBM Tivoli Monitoring for Network Performance media. Click Next. Figure 8-6 Location of installation properties 5. The installer will prompt for the data mart database to be used by the processes of the IBM Tivoli Monitoring for Network Performance warehouse enablement pack. It also prompts for the remote agent site that will run the ETL2 processes, as shown in Figure 8-7 on page 191.190 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 206. Figure 8-7 Data mart and remote agent site settings6. The installer will prompt for the central data warehouse database to be used by the processes of the IBM Tivoli Monitoring for Network Performance warehouse enablement pack. It also prompts for the remote agent site that will run the ETL1 processes At this time, you can opt to perform the scheduling settings for the ETL1 processes. If you opt to do so, the installation process will schedule the processes to run at the specified time and promote the processes to production status. You can also opt not to schedule the ETLs at installation time and perform the above tasks manually. The installer will also define the user authority for each one of the warehouse source and target processes. Click Next. See Figure 8-8 on page 192. Chapter 8. Historical reporting 191
  • 207. Figure 8-8 Central data warehouse and remote agent site settings 7. The installer will investigate the existence of an IBM Tivoli Monitoring for Network Performance ODBC connection. Click on Edit to specify the ODBC settings, as shown in Figure 8-9. Figure 8-9 Editing IBM Tivoli Storage Manager ODBC settings192 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 208. 8. As shown in Figure 8-10, set the User ID of the IBM Tivoli Monitoring for Network Performance database Administrator and Password. Click New to create an ODBC to create a new connection.Figure 8-10 IBM Tivoli Monitoring for Network Performance ODBC Settings9. As shown in Figure 8-11 on page 194, create a connection name to the IBM Tivoli Monitoring for Network Performance sever host name; we used the name itmnp21, the default port of 50000, and the database name of itmndb. Click on OK to create the ODBC connection. Chapter 8. Historical reporting 193
  • 209. Figure 8-11 Installation menu window 10.As shown in Figure 8-12, the installer will test the ODBC connection and return to the ODBC Data Source Properties Panel. Click Next. Figure 8-12 IBM Tivoli Monitoring for Network Performance ODBC Settings194 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 210. 11.The installation menu window shown in Figure 8-13 now lists the IBM Tivoli Monitoring for Network Performance warehouse enablement pack. Select this option and click Next to continue.Figure 8-13 IBM Tivoli Monitoring for Network Performance install summary12.Click Install in the summary window shown in Figure 8-14 on page 196 to start the IBM Tivoli Monitoring for Network Performance warehouse enablement pack installation. Chapter 8. Historical reporting 195
  • 211. Figure 8-14 Install panel 13.The final installation window contains either a successful completion notice or messages describing problems. Make sure the window does not list any warnings or errors, and then click Next. If warnings are listed, check the logs to ensure that the warnings can safely be ignored. Click Finish to exit the wizard. See Figure 8-15. Figure 8-15 Installation completed196 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 212. Your Warehouse Enablement pack is ready.8.4 ETL processing The IBM Tivoli Monitoring for Network Performance extracts historical information from the IBM Tivoli Monitoring for Network Performance DB2 databases and loads it into the Tivoli Data Warehouse V1.2 environment. Subsets of the historical information stored in the central data warehouse database are loaded into data mart databases to enable reporting in specific areas of interest.8.4.1 ETL overview The IBM Tivoli Monitoring for Network Performance enablement pack includes report information related to the following: FTP performance statistics Open System Adapter Performance performance statistics and interface statistics TCP Application performance Reports TN3270 Performance Reports that use the SNMP measurements are not provided; however, the information can be displayed by the Web application. IBM Tivoli Monitoring for Network Performance generates reports for 22 categories of collected data. Sub reports of each category can be generated. There are over 100 sub reports that can be generated and approximately 10 configurable parameters can be specified for each sub report. You can also use the default values for all parameters except the measurement data start and stop dates. All reports can be generated for hourly, daily, weekly, monthly, quarterly, and yearly time frames. Reports are generated for all monitored resources. Resources are displayed as lines in a time-series graph. Alternatively, you can generate the top 10 sub reports that use a bar graph to compare the top 10 monitored resources. Table 8-1 provides a list of the warehouse objects installed with the IBM Tivoli Monitoring for Network Performance WEP. Table 8-1 FNP WEP object names Warehouse Targets FNP_TWH_CDW_Target FNP_TWH_MART_Target Warehouse Sources FNP_TWH_CDW_Source FNP_TWH_MART_Source Chapter 8. Historical reporting 197
  • 213. Subject Area FNP_IBM_Tivoli_MonitoringNetworkPerformance_V2.1.0.0 _Subject_Area Processes FNP_c05_ETL1_Process FNP_m05_ETL2_Process Steps FNP_c05_s010_extractzosdata FNP_c05_s020_transformzosdata FNP_c05_s030_loadzosdata FNP_c05_s040_extracticmpdata FNP_c05_s050_transformicmpdata FNP_c05_s060_loadicmpdata FNP_c05_s070_extractsnmpdata FNP_c05_s080_transformsnmpdata FNP_c05_s090_loadsnmpdata FNP_c05_s100_lastetlrun Steps FNP_m05_s010_extract FNP_m05_s020_load FNP_m05_s*_<function>_rollup (step 30 - 280) FNP_m05_s300_prune_rollup Table 8-1 on page 197 shows that the IBM Tivoli Monitoring for Network Performance warehouse enablement pack has the following processes: FNP_c05_ETL1_Process FNP_m05_ETL2_Process These processes are in the main central data warehouse ETL. It extracts information from an IBM Tivoli Monitoring for Network Performance server and loads it into the central data warehouse database. We scheduled the process to run once each 24 hour period to collect information at 02:00. You should choose a time of low activity on the IBM Tivoli Monitoring for Network Performance server in which to run this process. For example, you might choose to schedule the process in the early morning, after your nightly backups have completed but before the daily server maintenance processes begin. In order to obtain the steps and dependencies of the ETL processes, perform the following steps: 1. Start the IBM DB2 Control Center utility by selecting Start → Programs → IBM DB2 → Control Center. 2. On the IBM DB2 Control Center utility, start the IBM DB2 Data Warehouse Center utility by selecting Tools → Data Warehouse Center. The Data Warehouse Center logon window appears.198 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 214. 3. Log into the IBM DB2 Data Warehouse Center utility using the local DB2 administrator user ID, in our case, db2admin. 4. In the Data Warehouse Center Window, expand Subject Areas → Processes and double click on FNP_c05_ETL1_Process. A similar process can be followed for the FNP_m05_ETL2_Process.8.4.2 Testing, scheduling, and promoting the ETLs Tivoli Data Warehouse V1.2 has simplified the installation process of warehouse enablement packs to the extent that the scheduling and promoting of ETLs developed for Version 1.2 are set during the warehouse enablement pack installation time. We verified the ETL processes manually by processing each step in sequence. The installation of the warehouse enablement pack installed all the processes in production mode. In data warehouse you can only run processes manually in test mode. This is how we verified that the ETL processes would run successfully; you can verify data collection on the data mart database as follows: 1. On the control server, start the DB2 Control Center tool by selecting Start → Programs → DB2 → Control Center. On the DB2 Control Center, select Tools → Data Warehouse, as shown in Figure 8-16. Figure 8-16 Logon to Data Warehouse 2. The Data Warehouse logon screen will be presented. Click on the Advanced button to ensure that you are pointing to the correct database alias and warehouse server. Enter your DB2 administrator user ID and password. See Figure 8-17 on page 200. Chapter 8. Historical reporting 199
  • 215. Figure 8-17 Data Warehouse logon screen 3. The Data warehouse center will show the Subject Areas, Source Data warehouse databases, Target warehouse databases, Warehouse schemas, and the Administration options. Expand the Subject Areas → FNP_IBM_Tivoli_MonitoringforNetworkPerformance_V2.1.0.0_Subject_ Area -> FNP_c05_ETL1_Process. 4. The steps for the FNP_c05_ETL1_Process will be listed in name order, which is also the order the steps will process when they run in production. The Mode column reflects that each step is in Production mode. We manually changed the steps to test mode by clicking on the first step and holding down the shift and the down tab arrow to select the FNP_c05* steps we were going to test. Right-click and select Mode → Test, which will change all the steps to test mode, as shown in Figure 8-18. Figure 8-18 Change FNP_c05_ETL1_Process steps to test mode200 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 216. 5. The mode for each step will change to Test. We right-click on the first process, FNP_c05_s010_extractzosdata, and click on Test, which will test the step. A process panel will appear and then disappear if the step executes successfully.6. If the step executes unsuccessfully, you must investigate the error by going to the Warehouse → Work in Progress menu.7. Find the process that you run, right-click, and select Show log to diagnose the problem with each step. See Figure 8-19.Figure 8-19 Show log of steps8. Follow this process for each step to ensure the steps would run successfully when they were run in Production mode.9. Once all the steps are verified to normal completion for the FNP_c05_ETL1_Process, the same process is used for the FNP_m05_ETL2_Process.10.At this point, we change all the processes back to Production mode by manually changing the steps to production mode by clicking on the first step and holding down the Shift key and the down tab arrow key to select all 10 steps we are going to promote to production. Once the steps are highlighted, we expand Selected → Mode → Production, which will change all the steps back to Production mode. Chapter 8. Historical reporting 201
  • 217. 8.5 Sample reports The reports provided by IBM Tivoli Monitoring for Network Performance Warehouse Enablement Pack are installed directly in the Crystal Enterprise server. This section shows the additional configuration and use of these reports. The discussion is divided into the following topics: 8.5.1, “Configuring Crystal Enterprise” on page 202 8.5.2, “Available reports” on page 208 8.5.3, “Accessing the Crystal ePortfolio” on page 2098.5.1 Configuring Crystal Enterprise First, you need to have the data mart database TWH_MART be available as an ODBC data source at the Crystal Enterprise machine. In our environment, the Tivoli Data Warehouse and Crystal Enterprise are installed on the same machine, so that had already been set. In the IBM® Tivoli® Monitoring for Network Performance Version 2.1 README File (the known limitation section), there is a set of steps to set up the connection to the data mart from Crystal Enterprise. We include the steps here. If these steps are not completed, you will encounter errors while viewing the reports. The following instructions tell you how to set the Data Mart database connection in Crystal Management Console after installing and running the IBM Tivoli Monitoring for Network Performance Warehouse Enablement Pack. 1. Open the Crystal Management Console. Select Start → Programs → Crystal Enterprise 9 → Crystal Launchpad, as shown in Figure 8-20 on page 203. Click on Crystal Management Console.202 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 218. Figure 8-20 Crystal Enterprise Launchpad2. Enter the Crystal administrator user ID and password to log onto the Crystal Management Console, as shown in Figure 8-21 on page 204. Use the Windows’ administrator user ID and its password. Click Log On. Chapter 8. Historical reporting 203
  • 219. Figure 8-21 Crystal Management logon 3. In the Crystal Management Console, click on the Manage Objects icon, as shown in Figure 8-22 on page 205.204 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 220. Figure 8-22 Select Manage Objects4. A list of 220 reports will be presented, which consists of 22 reports in 10 different languages. In our case, we used the English version of the reports, so we modified only the English reports. We clicked on the English CSMStorageReports first. The reports attributes are shown in Figure 8-23 on page 206; click the Database tab. Chapter 8. Historical reporting 205
  • 221. Figure 8-23 CSMStorageReport attributes 5. Distributed report only: Set the default information for logging onto the data source. We check the Automatically whenever the report is run option and entered the DB2 administrator user ID and password and then clicked Update. See Figure 8-24 on page 207.206 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 222. Figure 8-24 Set the default information for logging onto the data source6. Mainframe reports only: Set the default information for logging onto the data source. We checked Use Custom Database logon information specified here and entered the user ID and password for the Data Mart database on z/OS and then clicked Update. See Figure 8-24. Chapter 8. Historical reporting 207
  • 223. Figure 8-25 Set the default information for logging onto the data source 7. This process must be performed for each of the 22 reports that you will be viewing. Click on your browser’s Back tab and follow this process for the rest of the reports. Once this is complete, you will be able to view the reports.8.5.2 Available reports Here is a list of predefined reports shipped with the IBM Tivoli Monitoring for Network Performance V1.2 warehouse enablement pack provides the following reports that can be used for workload analysis, capacity planning, and trend analysis: TCP Applications Workload TCP Connections Workload208 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 224. UDP Applications Workload Response Time TN3270 Server TN3270 Client TN3270 Application OSA Adapter Port Status OSA Adapter Processor Utilization and Throughput OSA Ethernet Throughput Interface FTP Server FTP Server User FTP Client FTP Client User Enterprise Extender Availability Enterprise Extender Throughput and Traffic TCP Layer Stack IP Layer Stack UDP Layer Stack TCPIP Stack Memory CSM Storage Reports can be viewed in a variety of ways. For example, you can view bar or line graphs calculated on an hourly, daily, monthly, or yearly basis. Crystal Enterprise Professional Version 9 for Tivoli has, in comparison to a full Crystal license, reduced configuration options. If the reports shipped with IBM Tivoli Monitoring for Network Performance warehouse enablement pack do not match your needs and you want to develop additional reports, you have to upgrade your Crystal Enterprise installation.8.5.3 Accessing the Crystal ePortfolio All reports provided by Tivoli Data Warehouse V1.2 must be accessed using the ePortfolio feature of the Crystal Enterprise. The ePortfolio can be accessed from the Crystal Enterprise launchpad. 1. In order to access the Crystal Enterprise launchpad, we started an Internet browser and opened the following URL: http://phoenix/crystal/enterprise9 The resulting Web page is shown in Figure 8-20 on page 203. 2. Select ePortfolio link and log in using the user ID of tivoli and a blank password, as shown in Figure 8-26 on page 210. Chapter 8. Historical reporting 209
  • 225. Figure 8-26 Crystal Enterprise - ePortfolio log on 3. After entering the required data, select Log On to proceed. Instead of No Folders in the guest users ePortfolio window in Figure 8-27 on page 211, there is now a folder for IBM Tivoli Monitoring for Network Performance reports.210 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 226. Figure 8-27 Crystal Enterprise 9 - Folders4. Go into the IBM Tivoli Monitoring for Network Performance folder and see the reports listed there, as shown in Figure 8-28 on page 212. Chapter 8. Historical reporting 211
  • 227. Figure 8-28 Crystal Enterprise 9 - available reports for ITSM 5. In order to obtain a report, select the desired report, for example, TCP_LayerStackReports, and select View, as shown in Figure 8-29 on page 213.212 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 228. Figure 8-29 View TCP_LayerStackReports6. The view report will launch and provide The report you requested requires further information panel. See Figure 8-30 on page 214. Chapter 8. Historical reporting 213
  • 229. Figure 8-30 Crystal Enterprise 9 - parameters 7. The time parameters must be entered in the exact format of DateTime(yyyy,mm,dd,hh,mm,ss). Enter the appropriate values and press Enter to see the TCP_LayerStackReports report. See Figure 8-31 on page 215.214 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 230. Figure 8-31 Viewing reports 8. In the upper right corner of the screen, there will be a list of the z/OS operating systems that are collecting data. You can click on any of the operating systems and it will provide the details for the instances on the graph. See Figure 8-32 on page 216. Chapter 8. Historical reporting 215
  • 231. Figure 8-32 Details for TCP_LayerStackReports graph216 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 232. A Appendix A. ITMNP configuration files and test programs This appendix provides the configuration files we customized for our environment to set up IBM Tivoli Monitoring for Network Performance. The configuration files are provided for both AIX and z/OS Web application server. This appendix consists of four files: “AIX itmnp.properties” on page 218 “z/OS itmnp.properties” on page 218 “Sample server TSO REXX program” on page 219 “Sample object REXX client program” on page 220© Copyright IBM Corp. 2004. All rights reserved. 217
  • 233. AIX itmnp.properties Example A-1 shows a sample itmnp.properties file for a monitor in the SC62 host connecting to AIX Web application server. We have stripped out the comments in the file. Example: A-1 AIX itmnp.properties monitor_hostname=wtsc62.itso.ibm.com local_httpport=9455 WAS_hostname=jakarta.itsc.austin.ibm.com WAS_httpport=9454 DBName=ITMNPDB DBUserName=db2inst1 DBPassword=****** DBHostName=jakarta.itsc.austin.ibm.com DBPort=50000 DBCacheDirectory=/var/ibm/tivoli/common/FNP db2Security=CLEAR_TEXT_PASSWORD_SECURITY WebsphereServletProtocol=https trustStoreName=/etc/itmnp/itmnpMonitorTrustStore.jks trustStorePassword=WebAS keyStoreName=/etc/itmnp/itmnpMonitorKeyStore.jks keyStorePassword=WebAS keyManagerPassword=WebAS monitor.trace.level=DEBUG_MINz/OS itmnp.properties Example A-2 shows a sample itmnp.properties file for a monitor in the SC61 host connecting to z/OS Web application server. We have stripped out the comments in the file. Example: A-2 z/OS itmnp.properties monitor_hostname=wtsc61.itso.ibm.com WAS_hostname=wtsc61.itso.ibm.com DBName=DB8D DBUserName=TIVO01 DBPassword=TIVO01 DBHostName=wtsc61.itso.ibm.com DBPort=38030 DBCacheDirectory=/var/ibm/tivoli/common/FNP db2Security=CLEAR_TEXT_PASSWORD_SECURITY monitor.trace.level=DEBUG_MIN218 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 234. Sample server TSO REXX program Example A-3 shows our sample REXX server program that generates TCP/IP traffic to be monitored. Example: A-3 Sample server TSO REXX program /*rexx*/ call Time(r) say Cab() “Initializing sockets ...” Time(e) say Cab() “Socket Version” Socket(‘VERSION’) parse value Socket(‘INITIALIZE’,’RTMSRV’,10,’TCPIP’) with rcinit message if rcinit <> 0 then call Error “Error” rcinit “on sockinit function.” say Cab() ‘Socket initialized:’ message Time(e) parse value Socket(“SOCKET”,”AF_INET”,”SOCK_STREAM”,”IPPROTO_TCP”), with rcsocket socket if rcsocket < 0 then call Error “Error on socksocket call” rcsocket, socket say Cab() “Socket” socket “initialized ...” Time(e) x=Socket(‘SetSockOpt’,socket,’Sol_Socket’,’SO_REUSEADDR’,on) address.family = “AF_INET” address.port = “7777” address.addr = “INADDR_ANY” rcbind = Socket(‘BIND’,socket,address.family address.port address.addr) if rcbind <> 0 then call Error “Error on bind” rcbind say Cab() “Bind on port” address.port “for socket” socket “ok” rcbind, time(e) rclisten = Socket(“LISTEN”,socket,10) if rclisten <> 0 then call Error “Error on listen” rclisten say Cab() “Listen on port” address.port “for socket” socket “ok”, rclisten time(e) count = 1 t = 60 call Time(r) socketr = socket do forever say Cab() “Selecting socket” socket “for” t “seconds ...” count, time(e) parse value Socket(“SELECT”,”READ” socketr,t) with, rcselect nsock ‘READ’ socket ‘WRITE’ 1 message socket = Strip(socket) say Cab() “Select rc” rcselect nsock socket select when nsock = 0 then say Cab() “Timeout ocurred ...” Time(e) when nsock > 0 then call PROCESS otherwise call Error “Error on select” rcselect nsock end Appendix A. ITMNP configuration files and test programs 219
  • 235. end CLOSE: rcclose = Socket(‘CLOSE’,socket) say Cab() “Closed socket” socket rcclose Time(e) exit 0 ERROR: parse arg message say Cab() message signal close CAB: return date() Time() PROCESS: parse value Socket(‘ACCEPT’,socket) with rcaccept socket, addressaccept.domain addressaccept.port addressaccept.addr 1 message if rcaccept < 0 then call ERROR “Error on accept” message say Cab() “Connected to” addressaccept.addr addressaccept.port parse value Socket(“RECV”,socket,1024) with rcreceive len bufrecv if rcreceive <> 0 then call ERROR “Error on receive” rcreceive, len bufrecv say Cab() “Received” rcreceive rc len count=count+1 parse value Socket(‘SEND’,socket,bufrecv) with message say Cab() “Closing the socket” socket rcclose = Socket(‘CLOSE’,socket) returnSample object REXX client program Example A-4 shows our sample REXX client program that generates TCP/IP traffic to be monitored. Example: A-4 Sample object REXX client program /* */ rc1 = RxFuncAdd(“SockLoadFuncs”,”rxsock”,”SockLoadFuncs”) rc2 = SockLoadFuncs() say Cab() “Initializing sockets ...” rc1 rc2 rcinit = SockInit() if rcinit <> 0 then call Error “Error” rcinit “on sockinit function.” do forever220 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 236. socket = SockSocket(“AF_INET”,”SOCK_STREAM”,”IPPROTO_TCP”) if socket < 0 then call Error “Error on socksocket call” socket say Cab() “Socket” socket “initialized ...” address.family = “AF_INET” address.port = ““ address.addr = “INADDR_ANY” rcbind = SockBind(socket,”address.”) say Cab() “Binding on port” address.port “for socket” socket if rcbind <> 0 then call Error “Error on bind” rcbind say Cab() “Binded on port” address.port “for socket” socket Server_List = “wtsc52 wtsc61 wtsc62 wtsc66” Server_Name =Word(Server_List,Random(1,Words(Server_list)))”.itso.ibm.com” rc = SockGetHostByName(Server_Name,host.) address.family = “AF_INET” address.port = “7777” address.addr = host.addr say Cab() “Connecting to port” address.port “for socket” socket “toaddress” address.addr Server_Name rcconnect = SockConnect(socket,”address.”) if rcconnect <> 0 then say Cab() Error “Error on connect” rcconnect else do say Cab() “Connected to port” address.port “for socket” socket “toaddress” address.addr Server_Name dados = Substr(Copies(Date() Time(),50),100,Random(101,1000)) rcsend = SockSend(socket,dados) say Cab() “Number of bytes sent on socket” socket”:” rcsend end rcclose = SockSoClose(socket) say Cab() “Closed socket” socket rcclose time(e) timetowait=Format(1/Random(1,10),1,1) say Cab() “Waiting for” timetowait “seconds” Call SysSleep timetowait end Signal Drop ERROR: parse arg message errno = SockSock_Errno() say Cab() “Error Number” errno say Cab() message call SockPSock_Errno rcclose = SockSoClose(socket) DROP: call SysDropFuncs exit 0 Appendix A. ITMNP configuration files and test programs 221
  • 237. CAB: return date() time()222 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 238. B Appendix B. z/OS LDAP SDBM back end configuration files This appendix provides the configuration files we customized for our environment to set up a z/OS LDAP server using the SDBM back end for WebShere Application Server on AIX. This appendix consists of three files: “z/OS LDAP setup configuration file: ldap.profile” on page 224 “z/OS LDAP configuration file: SLAPDCNF” on page 225 “z/OS LDAP started procedure: LDAPSRV” on page 226© Copyright IBM Corp. 2004. All rights reserved. 223
  • 239. z/OS LDAP setup configuration file: ldap.profile This is the ldap.profile for our environment. This file includes the setup parameters for the SDBM and the TDBM back end. We only configured for the SDBM back end. We put a dummy value in the TDBM parameters to enable ldadcnf to run with return code 0. Example B-1 shows the content of this file. We removed the comments from the listing. Example: B-1 Sample of z/OS LDAP setup file USR_LPP_ROOT=/ OUTPUT_DATASET=TIVO01.LDAP.OUTPUT1 OUTPUT_DATASET_VOLUME=TIVO01 TEMPORARY_DIR=/tmp GLDHLQ=GLD SGLDLNKVOL=SYSRES GSKHLQ=GSK SGSKLOADVOL=SYSRES DSN_SDSNEXITHLQ=DB2 SDSNEXITVOL=TESTZZ DSN_SDSNLOADHLQ=DB8D8 SDSNLOADVOL=TESTZZ DSN_SDSNDBRMHLQ=DB8DU DSN_SSID=DB8D CEEHLQ=CEE SCEERUNVOL=SYSRES CBCHLQ=CBC SCLBDLLVOL=SYSRES CSSLIBVOL=SYSRES LINKLIBVOL=SYSRES LDAPUSRID=STC LDAPUSRGRP=SYS1 TDBM_SUFFIX=o=ITSO SDBM_SUFFIX=o=itso ADMINDN=cn=LDAP Administrator ADMINPW=secret PROG_SUFFIX=62 APF_JOBCARD_1="//LDAPAPF JOB ACCOUNTING INFO,NOTIFY=&SYSUID," APF_JOBCARD_2="// CLASS=A,MSGCLASS=T,MSGLEVEL=(1,1)," APF_JOBCARD_3="// REGION=4096K,TIME=1440" APF_JOBCARD_4="/*JOBPARM SYSAFF=SC62,L=9999" APF_JOBCARD_5="" PRGCTRL_JOBCARD_1="//LDAPPRG JOB ACCOUNTING INFO,NOTIFY=&SYSUID," PRGCTRL_JOBCARD_2="// CLASS=A,MSGCLASS=T,MSGLEVEL=(1,1)," PRGCTRL_JOBCARD_3="// REGION=4096K,TIME=1440" PRGCTRL_JOBCARD_4="/*JOBPARM SYSAFF=SC62,L=9999"224 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 240. PRGCTRL_JOBCARD_5="" DB2_JOBCARD_1="//LDAPDB2 JOB ACCOUNTING INFO,NOTIFY=&SYSUID," DB2_JOBCARD_2="// CLASS=A,MSGCLASS=T,MSGLEVEL=(1,1)," DB2_JOBCARD_3="// REGION=4096K,TIME=1440" DB2_JOBCARD_4="/*JOBPARM SYSAFF=SC62,L=9999" DB2_JOBCARD_5="" RACF_JOBCARD_1="//LDAPRAC JOB ACCOUNTING INFO,NOTIFY=&SYSUID," RACF_JOBCARD_2="// CLASS=A,MSGCLASS=T,MSGLEVEL=(1,1)," RACF_JOBCARD_3="// REGION=4096K,TIME=1440" RACF_JOBCARD_4="/*JOBPARM SYSAFF=SC62,L=9999" RACF_JOBCARD_5="" # The following include the advanced profile files. # Update the following lines to properly import advanced # configuration settings ${SOURCE_CMD} ${USR_LPP_ROOT}/usr/lpp/ldap/etc/ldap.slapd.profile ${SOURCE_CMD} ${USR_LPP_ROOT}/usr/lpp/ldap/etc/ldap.db2.profile ${SOURCE_CMD} ${USR_LPP_ROOT}/usr/lpp/ldap/etc/ldap.racf.profilez/OS LDAP configuration file: SLAPDCNF This is the HLQ.LDAP.OUTPUT(SLAPDCNF) file for our environment. Example B-2 shows the example of our configuration. This file includes the configuration for the SDBM back end, but it does not include the SSL security configuration. Example: B-2 Sample of the SLAPDCNF file listen ldap://:3389 maxConnections 60 adminDN "cn=LDAP Administrator" adminPW "secret" logfile //TIVO01.LDAP.OUTPUT(LOG) # # SDBM-specific CONFIGURATION SETTINGS # database sdbm GLDBSDBM suffix "o=itso" sizeLimit 2000 timeLimit 180 # # TDBM-specific CONFIGURATION SETTINGS # Extended Operations (EXOP) Backend CONFIGURATION SETTINGS Appendix B. z/OS LDAP SDBM back end configuration files 225
  • 241. z/OS LDAP started procedure: LDAPSRV This is a started procedure: LDAPSRV for our z/OS LDAP server. The DB2 load library has been removed from the STEPLIB of the procedure, as our environment does not use the TDBM back end, as shown in Example B-3. Example: B-3 Sample of the LDAPSRV started procedure //LDAPSRV PROC REGSIZE=0M, //*-------------------------------------------------------------------- //* CUSTOMIZABLE SYMBOLIC PARAMETERS //*-------------------------------------------------------------------- // PARMS=, // PGLDHLQ=GLD, // PCNFOUT=TIVO01.LDAP.OUTPUT, // OUTCLASS=A //*-------------------------------------------------------------------- //GO EXEC PGM=GLDSLAPD,REGION=&REGSIZE,TIME=1440, // PARM=(/&PARMS >DD:SLAPDOUT 2>&1) //*-------------------------------------------------------------------- //STEPLIB DD DSN=&PGLDHLQ..SGLDLNK,DISP=SHR //* //*-------------------------------------------------------------------- //* CONFIG can be used to specify the LDAP server config file. //*-------------------------------------------------------------------- //CONFIG DD DSN=&PCNFOUT.(SLAPDCNF),DISP=SHR //*-------------------------------------------------------------------- //* ENVVAR can be used to specify any environment variables //* If this DD is used, the name of the data set must be customized //* for the installation. //*-------------------------------------------------------------------- //ENVVAR DD DSN=&PCNFOUT.(SLAPDENV),DISP=SHR //SLAPDOUT DD SYSOUT=&OUTCLASS //SYSOUT DD SYSOUT=&OUTCLASS //SYSUDUMP DD SYSOUT=&OUTCLASS //CEEDUMP DD SYSOUT=&OUTCLASS226 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 242. C Appendix C. Tips collection This appendix provides overview information, a collection of useful commands, tools, and utilities for products related to IBM Tivoli Monitoring for Network Performance. As discussed in Chapter 2, “Components and architecture” on page 7, IBM Tivoli Monitoring for Network Performance utilizes various IBM software products and interfaces to other software. This appendix can serve as a “cheat sheet” for implementers and administrators that want to work with IBM Tivoli Monitoring for Network Performance without needing to be an expert on the various products. The products discussed in this appendix are: “WebSphere Application Server Version 5” on page 228 “DB2 Universal Database Version 8” on page 229 “z/OS Communication Server TCP/IP” on page 233 “IBM Tivoli NetView for z/OS Version 5” on page 233 “IBM Tivoli Enterprise Console Version 3.9” on page 235 “IBM Tivoli Data Warehouse Version 1.2” on page 237© Copyright IBM Corp. 2004. All rights reserved. 227
  • 243. WebSphere Application Server Version 5 WebSphere Application Server is IBM’s premier Web application server. WebSphere Application Server Version 5 provides a single Java Virtual Machine (JVM) environment for all the Web applications. It conforms to the Java 2 Enterprise Edition (J2EE) specification. This appendix provides some quick tips and facts about WebSphere Application Server usage and operation. Some of this information applies to both z/OS and AIX based WebSphere Application Server; some information is different on those platforms.Administration WebSphere Application Server Version 5 is administered using a Web based application. This application conforms to the J2EE specification and manages the setting of the WebSphere Application Server. The application manages the enterprise applications under the WebSphere directory structure and also a set of configuration XML files. The administration is a J2EE application and runs inside the WebSphere Application Server’s Java Virtual Machine. Sometimes, a configuration error causes WebSphere to abend in startup. This results in an inaccessible WebSphere Application Server. You have to either restore the WebSphere configuration file system or manually edit the configuration file to fix the error. This requires you to understand some important configuration files. The configuration files are stored in XML format. These files reside under the AppServer/config directory of the WebSphere root directory. WebSphere is organized in a cell - node - server structure. A base WebSphere configuration cell only consists of a single node, while a node deployment structure allows multiple nodes in a cell. A node is typically mapped to a single server machine. This structure allows multiple servers to reside in a single machine. The configuration scope conforms to this structure; each setting of an XML file applies to a configuration level. The following are some important configuration files that may need to be edited to recover from a failure in a WebSphere start up: security.xml (cell level) server.xml (server level) variables.xml (all levels)228 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 244. Attention: In z/OS WebSphere server, these XML files are stored in ASCII, although the system is in EBCDIC. You cannot edit these files from the UNIX System Services. You need to transfer the files to an ASCII based system to edit them and transfer them back.J2EE based application There are some file types that are related to the J2EE based application; they are: Java archive file (jar) Web archive file (war) Enterprise application archive file (ear) Resource archive file (rar) An installed application resides under the AppServer/installedApps directory.Startup and shutdown procedure The startup and shutdown of WebSphere on a distributed platform is performed by using scripts from WebSphere/AppServer/bin. The scripts are: To start the server, use startServer.sh. To stop a server with security active, use stopServer.sh -username <uid> -password <pwd>. On z/OS, WebSphere is governed by a set of started tasks that runs UNIX System Services processes. These processes are typically grouped into the following: Controller process: This is the start up process for WebSphere for z/OS. Daemon process is the primary process that starts up the application server. Servant process is the Java Virtual Machine engine that serves the clients.DB2 Universal Database Version 8 DB2 is IBM’s premier relational database system. It resides on both the z/OS and distributed systems. This appendix discusses some quick tips on using DB2, either on z/OS or distributed. Appendix C. Tips collection 229
  • 245. Relational database system DB2, as a relational database management system (RDBMS), organizes its information in tables, with their indexes. These tables are physically stored in files in the file systems. These file containers, where the table resides, are called tablespaces. On a z/OS platform, tablespaces are implemented as Linear VSAM files, while on distributed tablespaces that are associated with a path or file system. For performance reasons, tablespaces are logically organized in pages. Each page has a fixed size that can contain one or more rows of the table. A row must reside in a single page. Typically, the default page size is 4096 bytes or 4 KB. However, some tables has rows that are larger than this size, so there is a need to get larger pages, such as 16 KB or 32 KB. The access to the pages is cached using memory areas called the bufferpools. The bufferpools store pages and reuse them as request come in, in order to minimize actual disk I/O. Bufferpools have segments at the same size as the pages, so if the pages are 16 KB, the bufferpool must use 16 KB segments too. Programs access the database tables must be known to DB2. The program’s access profile is stored in plans and packages. Plans for DB2 for z/OS contain a single executable or a collection of packages. Packages specify a module access profile to DB2. Most of the processing in DB2 programs are based on packages. The access profile is important for understanding the necessary access required for the program.DB2 on z/OS systems DB2 on z/OS acts as a subsystem. It consists of several started tasks: MSTR: The master database process DBM1: The database access process DIST: The distributed access thread manager SPAS: The stored procedure address space The following commands are some console commands that you can use. These commands need to be prefixed with the command identifier set for the DB2 system; we represent this prefix with a hyphen (-). You need to substitute this with the real prefix when issuing these commands. START DB2: This starts the DB2 system. The syntax of the command is -START DB2. STOP DB2: This stops the DB2 system. The syntax of the command is -STOP DB2 MODE(QUIESCE).230 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 246. DISPLAY THREAD: This displays all the connected applications to the DB2 subsystem. The syntax of the command is -DIS THD(*). DISPLAY BUFFERPOOL: This displays the size and usage statistics of the buffer pools in DB2. The primary usage of this command is -DIS BPOOL(<bpname>). ALTER BUFFERPOOL: This command activates or resizes a DB2 bufferpool size. The primary usage of this command is -ALT BPOOL(<bpname>) VPSIZE(size).DB2 on AIX or Window systems On AIX systems, DB2 resides in instances. Each instance is owned by the DB2 instance owner. The instance is the individual DB2 process that can be started or stopped. The following are some useful commands that you can use: Starting the DB2 environment: – In Windows, issue the DB2CMD command to establish the command window for DB2. – In AIX, use the environment setting in $DB2idhome/sqllib/.db2profile. Running the DB2 command. Use the db2 command. SQL and DB2 commands can be executed using this command. On a UNIX platform, you may need to enclose the SQL statement between double quotes. Running the graphical administration tool using the db2cc command. This starts the DB2 Control Center Java application that lets you manage and configure DB2 databases. Managing the ODBC database connection using the db2cca command. This allows creation of an ODBC setting for DB2 databases.Differences between z/OS and distributed DB2 processing There are some differences between the processing of the DB2 database for z/OS and distributed. We discuss these differences in this section. Instances In DB2, an instance contains one or more databases. Each database in turn consists of tables and indexes. On z/OS, an instance is accessed in whole; no duplicates on tables or indexes are allowed. On distributed systems, a database is the unit of access. A remote connection in z/OS connects to an instance, while in distributed it connects to a database. Appendix C. Tips collection 231
  • 247. DB2 subsystem (z/OS) DB2 instance (AIX/Windows) database database database tablespace tablespace table tablespace tablespace tablespace tablespace table table table table table database Distributed connection Distributed connection tablespace tablespace table table Distributed connection Figure C-1 DB2 concepts Tablespaces A tablespace is a container that stores one or more tables. In z/OS, a tablespace is made from a single linear VSAM dataset. The organization of tables in the dataset is managed by DB2. In distributed, a tablespace is a container, path, or a file system. Distributed tables are files in that tablespace path. In z/OS, indexes are created in indexspaces, which are separate linear VSAM datasets; in distributed, indexes are files in the tablespace path. Bufferpools Bufferpools are used to buffer information from disk to boost performance. In z/OS, a set of bufferpools are already allocated; when we need to use any bufferpools, we change the size of an existing one (which is typically set to zero). In distributed, we can create new bufferpools with a specific page size and size.Important tables When working with a DB2 database, there are several tables that contain system information. These tables are called catalog tables. They are important in understanding the structure of the database. Some of these tables are: SYSIBM.SYSTABLES. Use the SQL statement: SELECT CREATOR, NAME FROM SYSIBM.SYSTABLES232 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 248. SYSIBM.SYSTABLESPACES. Use the SQL statement: SELECT DBNAME, NAME FROM SYSIBM.SYSTABLESPACES SYSIBM.SYSDATABASES. Use the SQL statement: SELECT DBNAME FROM SYSIBM.SYSDATABASES SYSIBM.SYSSTOGROUP and SYSIBM.SYSVOLUMES. Use the SQL statement: SELECT A.NAME, B.VOLUMES FROM SYSIBM.SYSSTOGROUP A, SYSIBM.SYSVOLUMES WHERE A.NAME = B.NAMEz/OS Communication Server TCP/IP This section provides some commands that you can issue from the z/OS operator console for modifying TCP/IP processing: Showing TCP/IP connection and port usage: D TCPIP,,NETSTAT,CONN. This command allows you to see which connections are made. Running an obey file: V TCPIP,,OBEYFILE,datasetname. This command allows a temporary override of the TCP/IP setting. Additional commands in a booklet format can be downloaded from: http://www.redbooks.ibm.com/tips/TIPS0091/tips0091.pdfIBM Tivoli NetView for z/OS Version 5 IBM Tivoli NetView for z/OS is the automation engine for z/OS systems. It support both systems and network automation. IBM Tivoli NetView for z/OS has the following features: Hardware monitor (NPDA) Session monitor (NLDM) Status monitor (STATMON) Program to program interface (PPI) Automation support Command interface Event automation system Graphical interface Appendix C. Tips collection 233
  • 249. Alerts structure Alerts are historically used as an unsolicited information trigger when a network hardware detects a potential problem. The subsystem that works with alerts in IBM Tivoli NetView for z/OS is the hardware monitor, which is also called the network problem determination assistant (NPDA). Alerts are structured as a Network Management Vector Table (NMVT) (see SNA Formats, GA27-3136 for more information about NMVT). It consists of a primary vector, sub-vectors, and sub-sub vectors. The following are several of the well-known sub-vectors: Sub-vector 5: This sub-vector contains information about the hierarchy of the device that triggers this alert. Each hierarchy element includes the name and type of device that it represents. This sub-vector is typically used to identify the source of the alert. Sub-vector 00: Text message. Sub-vector 92: Generic alert data. Sub-vector 93: Probable cause. Sub-vector 33: User data. Sub-vector 31: Self-defining text messages, which are used for representing TEC slots values.Automation support Automation in IBM Tivoli NetView for z/OS is implemented using automation tables. The automation system is invoked whenever a message or alert is received by NetView, so the automation table is often referred to as the message automation table (MAT). The automation table consists of a set of IF - THEN rules that govern how message or alerts must be handled. Typically, a statement has the following format: IF condition THEN action The action can be a single command or a group of commands or statements between a BEGIN END block. Automation tables are managed by the AUTOTBL command. The AUTOTBL command allows testing, activation, insertion, and removal of automation table members.234 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 250. Event automation service The event automation service was introduced to provide an interface between IBM Tivoli NetView for z/OS and IBM Tivoli Enterprise Console. It provides three services: Converting hardware monitor alerts to TEC events Converting console messages to TEC events Converting TEC events to hardware monitor alerts The event automation service is one of the earliest NetView programs that require UNIX System Services. It converts TEC slots into alert sub-vectors and vice-versa. This service runs in its own address space, typically called IHSAEVNT. The event automation service communicates with IBM Tivoli NetView for z/OS using PPI channels. When the task starts, we can see that new PPI channels are established with the DISPPI command.IBM Tivoli Enterprise Console Version 3.9 The IBM Tivoli Enterprise Console is an event processing application that is based on Tivoli Management Framework. Conceptually, the operation of any Framework based application can use the Tivoli Desktop or command line (CLI). The CLI operation must have the environment set first. The environment is set using the /etc/Tivoli/setup_env.sh (UNIX platform) or %WINDIR%System32driversetcTivolisetup_env.cmd (Windows). This command needs to be invoked before any Tivoli CLI command can be run.IBM Tivoli Enterprise Console structure The conceptual structure of IBM Tivoli Enterprise Console is shown in Figure 8-33 on page 236. Appendix C. Tips collection 235
  • 251. Rulebase TEC Event Server Reception Event Adapter Event Adapter Event Adapter Event source Event source Event source Figure 8-33 Structure of IBM Tivoli Enterprise Console In Figure 8-33, the main part is the event server. It receives events from various event sources. It uses a database using the Tivoli Framework’s RDBMS InterfaceModule (RIM) to access the TEC database. It processes events using a rulebase. Event sources generate events that are formatted by the event adapters, which send them to the tec_reception process. The rulebase consists of two main parts: the event definition and the event processing parts. The event definition consists of a set of baroc files. These files define events classes and the fields that they have. The fields are called slots. These events definitions are object oriented, which means that event definitions can be sub-classes, and the sub-class inherits the super-class’ attributes. The event processing part is defined as a set of ruleset files. These files are written in the prolog programming language. Each ruleset defines a set of rules that may be applied to incoming events and acts on them accordingly. Important Tivoli Management Framework commands The following are some commonly used commands: wlookup The wlookup command is used to get a list of all Tivoli object types using wlookup -R or to get all the objects of a specific type using wlookup -ar <objtype>.236 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 252. wbkupdb Tivoli database backup is performed using the command wbkupdb -d <path>, while restore is performed using the command wbkup -r -d <path>. odadmin This is a general command to interact with the Tivoli object dispatcher. For example, odadmin odlist retrieves all managed nodes and their statuses, while odadmin itself retrieves the current Tivoli server process information. odstat This command retrieves all running Tivoli object methods and lists the last 200 object methods. The odstat command is very useful for debugging errors. Important IBM Tivoli Enterprise Console commands The following are some CLI that we use with IBM Tivoli Enterprise Console. First, IBM Tivoli Enterprise Console is based on a rulebase. It governs how it reacts and processes events. wlsrb This command lists any defined rulebases in the system. The command wlsrb -d will show where the rulebases are stored. wlscurrb This command shows the currently active rulebase. wrb The wrb is a shell script that invoke various rulebase processing commands. Some of the processing options are: wrb -imprbclass This imports a new baroc file into the rulebase. wrb -imprbrules This imports a new ruleset file into the rulebase. wrb -comprules This compiles the rulebase into an internal form. wrb -loadrb This loads the rulebase as the current rulebase. Other commands that are useful are: wstartesvr The wstartesvr command starts the event server. wstopesvr The wstopesvr command stops the event server. wtdumprl The wtdumprl command dumps the reception log. wtdumptr The wtdumptr command shows the automation processing log.IBM Tivoli Data Warehouse Version 1.2 IBM Tivoli Data Warehouse is a historical data collection system that collects system management information from various IBM Tivoli applications. IBM Tivoli Appendix C. Tips collection 237
  • 253. Data Warehouse provides a consolidated platform for data collection in IBM Tivoli products. From this platform, it is very convenient to perform historical analysis, trend calculation, and other bulk processing.Data warehouse concepts IBM Tivoli Data Warehouse is based on the DB2 Version 7 warehouse system. In Version 1.2, the reporting portion is based on Crystal Enterprise 9. The conceptual structure of the IBM Tivoli Data Warehouse is shown in Figure 8-34. TWH_MD Warehouse server Reporting server DB2 server (Crystal Enterprise) source TWH_CDW TWH_MART Figure 8-34 IBM Tivoli Data Warehouse structure DB2 Version 7 warehouse runs on the Windows platform only (DB2 Version 8 supports AIX and other UNIX platforms, but IBM Tivoli Data Warehouse only supports DB2 Version 7). The warehouse server runs as two Windows services; Warehouse server (vwkernel) and Warehouse logger (vwlogger). In the warehouse processing, there are several databases that are involved. The names shown in Figure 8-34 are the typical names for the databases. TWH_MD This is the control database. It contains metadata information on how to process data from the source to the target database. The processing itself is often referred as the extract, transform, load (ETL) process. TWH_CDW This is the central data warehouse. It stores all data from the source as indicated by the appropriate ETL program. The data in the warehouse is raw.238 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 254. TWH_MART This is the reporting database. It is written by the warehouse server from data extracted from the data warehouse. The data is extracted based on a certain reporting requirement. The reporting server accesses this database. Typically the tables are arranged in a star schema.Warehouse enablement pack Each IBM Tivoli application has data and information stored in a database. The warehouse enablement pack (WEP) contains the support information to integrate the application into IBM Tivoli Data Warehouse. Typically, the WEP consists of the following: Additional tables needed in the central data warehouse Additional tables needed in the data mart ETL programs to transfer data into the central data warehouse ETL programs to extract data from the central data warehouse into a reporting data mart Report definition for Crystal Enterprise The installation program will put each component into the IBM Tivoli Data Warehouse server. Appendix C. Tips collection 239
  • 255. 240 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 256. D Appendix D. Underlying technology This appendix discusses the basics of some underlying technologies and their application to IBM Tivoli Monitoring for Network Performance. These standards based technologies are considered to be nontraditional in z/OS. The discussion consists of the following topics: “Light-weight Directory Access Protocol (LDAP)” on page 242 “eXtensible Markup Language (XML)” on page 247 “Certificates and encryption” on page 251 “Secure Sockets Layer protocol” on page 261© Copyright IBM Corp. 2004. All rights reserved. 241
  • 257. Light-weight Directory Access Protocol (LDAP) Information describing the various users, applications, files, printers, and other resources accessible from a network is often collected into a special database that is sometimes called a directory. As the number of different networks and applications has grown, the number of specialized directories of information has also grown. This growth results in islands of information that are difficult to share and manage. If all of this information could be maintained and accessed in a consistent and controlled manner, it would provide a focal point for integrating a distributed environment into a consistent and seamless system. LDAP is an open industry standard that has evolved to meet these needs. LDAP defines a standard method for accessing and updating information in a directory. LDAP is gaining wide acceptance as the directory access method of the Internet and is therefore also becoming strategic within corporate intranets. It is being supported by a growing number of software vendors and is being increasingly incorporated into applications. LDAP defines a message protocol used by directory clients and directory servers. The LDAP Protocol uses a variety of messages. For example, a bindRequest may be sent from the client to the LDAP server at the beginning of a connection. A searchRequest is used to search for a specific entry in the directory. There are also associated LDAP APIs for the C language and ways to access LDAP from within a Java application. Additionally, within the Microsoft development environment, you can access LDAP directories through its Active Directory Service Interface (ADSI). In general with LDAP, the client is not dependent upon a particular implementation of the server; the server can implement the directory however it chooses. LDAP defines a communication protocol. That is, it defines the transport and format of messages used by a client to access data in an X.500-like directory. LDAP does not define the directory service itself. When people talk about the LDAP directory, they refer to the information that is stored and can be retrieved by the LDAP protocol. The Comite Consultatif International Telephonique et Telegraphique or Consultative Committee on International Telephony and Telegraphy (CCITT), which is now called the International Telecommunications Union - Telecommunication Standardization Sector (ITU-T), defined the X.500 standard in 1988. The X.500 standard then became International Standards Organization (ISO) 9594, Data Communications Network Directory, Recommendations X.500/X.521 in 1990, though it is still commonly referred to as X.500.242 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 258. X.500 organizes directory entries in a hierarchical name space capable ofsupporting large amounts of information. It also defines powerful searchcapabilities to make information retrieval easier. Because of its functionality andscalability, X.500 is often used together with add-on modules for interoperationbetween incompatible directory services.X.500 specifies that communication between the directory client and thedirectory server use the Directory Access Protocol (DAP). However, as anapplication layer protocol, the DAP requires the entire Open SystemsInterconnection (OSI) protocol stack to operate. Supporting the OSI stackrequires more resources than are available in many small environments.Therefore, an interface to an X.500 directory server using a lessresource-intensive or lightweight protocol (in this case, LDAP) is desired.An application client program initiates an LDAP message by calling an LDAP API.But an X.500 directory server does not understand LDAP messages. In fact, theLDAP client and X.500 server even use different communication protocols(TCP/IP versus OSI). The LDAP client actually communicates with a gatewayprocess (also called a proxy or front end) that forwards requests to the X.500directory server. This gateway is known as an LDAP server. It services requestsfrom the LDAP client. It does this by becoming a client of the X.500 server. At thebeginning, the LDAP server implementations supported both OSI and TCP/IP tobe able to translate requests received by LDAP clients to DAP requests requiredto access X.500 directories. Newer LDAP server implementations, such as theIBM SecureWay Directory server, support only the LDAP protocol to access thedirectory. The LDAP server on the iSeries™ server is called Directory Servicesand implements the IBM SecureWay Directory.All modern LDAP directory servers are based on LDAP Version 3. You can use aVersion 2 client with a Version 3 server. However, you cannot use a Version 3client with a Version 2 server unless you bind as a Version 2 client and use onlyVersion 2 APIs.All LDAP servers share many basic characteristics, since they are based onindustry-standard Request for Comments (RFCs). However, due toimplementation differences, they are not all completely compatible with eachother when there is not a standard defined.If application developers could be assured of the existence of a directory service,then application-specific directories would not be necessary. However, acommon directory must address the problems mentioned above. It must bebased on an open standard that is supported by many vendors on manyplatforms. It must be accessible through a standard API. It must be extensible sothat it can hold the types of data needed by arbitrary applications. Also, it mustprovide full functionality without requiring excessive resources on smaller Appendix D. Underlying technology 243
  • 259. systems. Since more users and applications will access and depend on the common directory, it must also be robust, secure, and scalable. When such a directory infrastructure is in place, application developers can devote their time to developing applications instead of application-specific directories. In the same way that developers rely on the communications infrastructure of TCP/IP and remote procedure call (RPC) to free them from low-level communication issues, they must be able to rely on powerful, full-function directory services. LDAP is the protocol to be used to access this common directory infrastructure. Like HTTP (hypertext transfer protocol) and FTP (file transfer protocol), LDAP has become an indispensable part of the Internets protocol suite. When applications access a standard common directory that is designed in a proper way instead of using application-specific directories, redundant and costly administration can be eliminated and security risks are more controllable. For example, the telephone directory, e-mail, and Web applications shown below can all access the same directory to retrieve an e-mail address or other information stored in a single directory entry. The advantage is that the data is kept and maintained in one place. Various applications can use individual attributes of an entry for different purposes (assuming that the they have the correct authority). New uses for directory information will be realized, and a synergy will develop as more applications take advantage of the common directory. Figure D-1 on page 245 depicts the advantages of this arrangement.244 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 260. Telephone WebSphere Directory Application Application Server B WebSphere E-mail Application Application Server A Directory HTTP Web Server Objects 0=IBM CN=John CN=Wendy CN=Wolfgang sn (surname): Eckert telephoneNumber=2022 givenName (Firstname): Wolfgang uid (UserID): weckert userPassword: ******** mail (e-mail): wolf@iseries.ibm.com CN=TomFigure D-1 Several applications using attributes of the same entry Storing data in a directory and sharing it among applications saves you time and money by keeping administration effort and system resources down. Many IBM applications also utilize directories to centrally store and share information. The number of applications that support LDAP directories is constantly increasing. For example, LDAP directory support, such as for authentication and configuration management, is provided in various IBM Operating Systems, IBM WebSphere Application Server, WebSphere Portal Server, Tivoli Access Manager, Directory Server (Directory Server), IBM HTTP server, Lotus® Domino®, and so on. A directory contains a collection of objects organized in a tree structure. The LDAP naming model defines how entries are identified and organized. Entries are organized in a tree-like structure called the Directory Information Tree (DIT). Entries are arranged within the DIT based on their distinguished name (DN). A DN is a unique name that unambiguously identifies a single entry. DNs are made up of a sequence of relative distinguished names (RDNs). Each RDN™ in a DN corresponds to a branch in the DIT leading from the root of the DIT to the directory entry. A DN is composed of a sequence of RDNs separated by commas, such as cn=thomas,ou=itso,o=ibm. Appendix D. Underlying technology 245
  • 261. You can identify common names (CNs) within DNs. You also can organize entries, for example, after organizations. You can further split the tree into organizational units within a single organization as needed. You can define your DIT based on your organizational needs, as in the simple example below (Figure D-2). If you have, for example, one company with different divisions, you may want to start with your company name under the root as the organization (o) and then branch into organizational units (ou) for the individual divisions. In case you store data for multiple organizations within a country, you may want to start with a country (c) and then branch into organizations. Figure D-2 provides an example of this approach. Directory Root (top) o=IBM c=US ou=Marketing ou=Support o=ACMESupply o=iSeriesShop cn=mbarlen cn=tbarlen objectClass=Person objectClass=Person objectClass=ePerson objectClass=ePerson mail=marion@ibm.com mail=thomas@acme.com sn=Barlen sn=Barlen givenName=Marion telephoneNumber=112 deviceID=PrinterSales objectClass=cimPrinter cn=Klaus objectClass=ePrinter objectClass=Person location=Printer room 3rd floor objectClass=ePerson owner=John Doe mail=Ktebbe@ibm.com queuename=lsprt01 sn=Tebbe maxCopies=10Figure D-2 Example of a directory information tree Each object also referred to as an entry in a directory belongs to one or more object classes. An object class describes the content and purpose of the object. It also contains a list of attributes, such as a telephone number or surname, that can be defined in an object of that class. You can publish entries of different object classes under another object. The object class also defines which of the attributes must be defined (are required) when creating an object of this class and which attributes are optional.246 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 262. Object classes also can inherit characteristics, such as attributes from other object classes. In the example of the ePrinter, the class inherits all of the attributes that are defined in the class cimPrinter. Therefore, you must define the deviceID when you create an ePrinter object. Optionally, you can specify the location, owner, and queuePtr attribute of ePerson and all of the attributes of cimPrinter. Attributes themselves also have certain characteristics. The surname attribute name, for example, is defined as sn and surName and describes a persons family name. The attribute definition also specifies the syntax rules for the attribute value. A telephone number may only contain numbers and hyphens, while the surname consists of alpha characters. Other specifications include whether this attribute can contain only one or many values, the matching rules, the Object Identifier (OID), and so on. The IBM Tivoli Directory Server (ITDS) product also includes some IBM proprietary extensions for each attribute. Other manufacturers, such as Microsoft, have similar extensions. The IBM extensions also include an access class, which is used in combination with access control lists (ACLs) to control who can perform a certain action on the attribute value (such as read, write, search, or compare operations). All the objects and attributes with their characteristics are defined in schemas. The schema specifies what can be stored in the directory. Schema-checking ensures that all required attributes for an entry are present before an entry is stored. Schema-checking also ensures that attributes not in the schema are not stored in the entry. Optional attributes can be filled in at any time. A schema also defines the following: Inheritance Subclassing of objects Where in the DIT structure (hierarchy) objects may appeareXtensible Markup Language (XML) XML is a proposed standard by the World Wide Web Communities (W3C). It contains an extensible data structure for communicating structured information through a standard protocol, such as HTTP. The format of an XML document highly resembles a HTML or SGML document; however, the tags that are used are usually user defined. These user defined tag are defined in a Document Type Definition or DTD file. An XML document contains fields that is delimited with begin and end tags of the fields. A field can also contain a set of other fields. The XML is usually read by an XML parser. There are some commonly available XML parsers, such as SAX (Simple API for XML). Appendix D. Underlying technology 247
  • 263. Figure D-3 shows an example of an XML document. <?xml version="1.0" standalone="no"?> <HTTP-TRANSACTION> <REQUEST> <REQUEST-LINE NAME="STATIC"> <METHOD>GET</METHOD> <URI>/</URI> <PROT-VER>HTTP/1.0</PROT-VER> </REQUEST-LINE> <HEADERS> <HL>Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*</HL> <HL>Accept-Charset: iso-8859-1,*,utf-8</HL> <HL>Accept-Language: en</HL> <COOKIE> <INVOCATION> <FUNCTION NAME="retrieve_cookie"></FUNCTION> </INVOCATION> </COOKIE> <HL>Host: tokyo.itsc.austin.ibm.com</HL> <HL>User-Agent: Mozilla/4.73 [en] (WinNT; U)</HL> <HL>Referer: https://tokyo.itsc.austin.ibm.com:443/</HL> </HEADERS> </REQUEST> </HTTP-TRANSACTION> Figure D-3 Sample XML document In today’s technology, XML is starting to becoming a key piece of software infrastructure. The main idea is extremely simple. It is a language like HTML and is text based, but is rigidly enforced, and therefore can be built upon easily. XML documents may use a Document Type Definition (DTD) or an XML Schema. XML was designed to describe data and to focus on the data, unlike HTML, which was designed to display data. It was created to structure and store data. XML has these main applications: Sharing of information: The main problem integrating data between any two business organizations is the interface between them. If they can at least agree upon the standard of their common meeting point and its usage, they can build upon this to start building their applications. If there is already an existing interface or infrastructure provided by industry or government standard or infrastructure, the business cost of developing it is extinguished. Storage, transmission and interpretation of data: If the storage of information is adaptable to many mediums, its cost will be driven to the lowest248 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 264. cost of all mediums. XML is based on text, which obviously is accepted by all. It can be read. Its storage is cheap, compared to the storage requirements of graphics. And because the size of this text based object is small, its transmission, not withstanding cost, is cheap as well. And because it is commonly accepted, since it adheres to world wide standards, it can be easily interpreted. Security: With the explosion of confidential information being transmitted across the Internet, there is now a need for sophisticated levels of security. Companies need to protect their documents and e-mail, banks need to allow their depositors to download their accounts, and merchants need to be available to their customers to enter their credit card details without compromising their security and privacy. The more secure the transmission medium, the more confidentiality it provides, the more it can be used to advertise a competitive advantage. With the evolution of XML digital signatures, the ability to protect or hide parts of a document, while sitting on a PC, server, or mainframe, now covers up a security patch. Speed and amount of content delivery: With the rapid evolution of network technologies, speed and delivery of any object has gained importance. Network companies now advertise download times of movies and CDs. Again, the first companies discovering the abilities of delivering something at an increasing speed without compromising their content will gain the competitive advantage.The working draft for XML 1.1 was published on the W3C Web pages in April2002. XML 1.1 was formerly known as XML Blueberry.The XML 1.0 specifications were based on the Unicode Standard. However, theUnicode standards have evolved from Version 2.0 to 3.1 and beyond. XML 1.0relied on the standard for character specifications. Characters that are notpresent in Version 2.0 would probably have be used in XML documents andcharacter data. Developers would have developed workarounds for charactersthat were not supported in Unicode Version 2.0. These characters are notallowed in XML names, such as element type, names, and attribute names, justto name a few. Also, some characters should have been permitted in XML 1.0,but were not due to oversights and inconsistencies in Unicode 2.0.The philosophy for names has been reversed since XML 1.0. XML 1.1 nameshave been designed such that everything that is not forbidden is permitted. InXML 1.0, the philosophy was everything that was not permitted was forbidden.For example, under XML 1.0, if only ‘a’, ‘b’, ‘c’, ‘d’ and ‘e’ were allowed as names,that was all that could be used. In XML 1.1, we could say that we would not allow‘a’ and ‘b’. Therefore, we could use ‘b’, ‘c’, ‘e’, ‘f’, ...’g’, ...’$’, and so on. AsUnicode grows past Version 3.1, changes to the XML can be avoided if nearly allcharacters are allowed. This will allow any kind of characters in a name. Appendix D. Underlying technology 249
  • 265. A new version of XML is being created, rather than a set of errata, because the changes affect the definition of well-formed documents. XML Processors will recognize XML 1.1 documents from XML 1.0 documents by the declaration at the start of each document. For more details, visit the W3C Web site: http://www.w3.org/TR/xml11/Microsoft Excel for browsing XML files One of the easiest way to browse XML log files from IBM Tivoli Monitoring for Network Performance is using Microsoft Excel. When you open an XML file with Microsoft Excel, it formats the XML document so that each XML field resides in a column. This is quite convenient for log files, as we can selectively hide unwanted columns or change its width or ever reorder the sort order. An IBM Tivoli Monitoring for Network Performance log file is not well formatted; in a sense, it needs to have an additional header and footer. We use a file called xmlheader, as shown in Example D-1. Example: D-1 The xmlheader file <?xml version="1.0"?> <LOG> We also use an xmlfooter file, which contain a single line: </LOG> We merge the files with the following command from a Windows workstation: COPY xmlheader/b+msg_fnp_monitorc.log/b+xmlfooter msg1.xml When we open msg1.xml using Microsoft Excel, we get a display similar to Figure D-4 on page 251.250 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 266. Figure D-4 Sample XML log viewed using Microsoft ExcelCertificates and encryption On unsecured networks like TCP/IP, there is concern for both the sender and the receiver about the security of the data that is sent over the network. The network protocol itself does not provide any protection against tampering with the data whatsoever. In order to transport sensitive data over the network, the following issues need to be resolved: Confidentiality: Only the parties involved in the data transfer should be able to read the contents of the data. Authentication: The parties should be able to identify each other beyond doubt. Data integrity: Data that has been altered during transmission should be detectable. Non-repudiation: The sender of the data should not be able to deny the data he sends. Appendix D. Underlying technology 251
  • 267. The following sections discusses some approaches that provide resolution for these issues. More information on secure communication and infrastructure can be found in Deploying a Public Key Infrastructure, SG24-5512.Secret Key Cryptography The oldest cryptographic technique is known as Secret Key Cryptography. It is a way of achieving data confidentiality by encrypting and decrypting the data using a single key (the symmetric or secret key). Both the sender and the receiver of the data must have the secret key. In Figure D-5, A is sending a message to B, and both are using the same secret key to encrypt and decrypt the message. A E nc ryp t with C le artext m essag e E ncryp ted m essag e s e c re t ke y "Hello Alic e" xc % s lk& ls kf B Trans fe r o f m e s s ag e D e c ryp t with C lea rte xt m essag e s e c re t ke y E ncryp ted m essag e "Hello A lic e" xc % s lk& ls kf Figure D-5 Secret Key Cryptography No one else can encrypt or decrypt the message, as they do not have access to the secret key. The advantage of Secret Key Cryptography is that it is fast. It is also less complex than Public Key Cryptography, which is discussed in “Public Key Cryptography” on page 253. The disadvantages of Secret Key Cryptography are: The distribution of the secret keys. When one party creates a secret key, how is the other party going to get the secret key? Send it in e-mail? That is insecure. Send it with normal mail? That can take a long time and is hard to automate.252 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 268. Secret key administration. Another disadvantage is that for each party you want to communicate with, you must create a different secret key. If you do not, then the different parties would be able to read the communications of all the parties. You will end up with a lot of keys, which can become cumbersome to manage. Some well-known secret key algorithms are DES, Triple DES, Blow fish, IDEA, and RC5.Public Key Cryptography Public Key Cryptography is another widely used cryptographic technique that does not have the problems of multiple shared secret keys and the distribution of secret keys. The most well-known public key algorithm is RSA, but a technique called Elliptic Curves is becoming more widespread. Public Key Cryptography is a way of encrypting of data using an asymmetric key pair. One key, the Public key, is used for encrypting the data; the other key, the Private key, is used for decrypting the data. Both keys are different, and the key used for encrypting data cannot be used for decrypting the same data. The public key is made available to the world, while the private key should never be revealed. This technique is used for: Data encryption to provide confidentiality As an example, in Figure D-6 on page 254, A is sending a message to B. The message is encrypted with B’s public key, so that only B can decrypt the message using B’s private key. Appendix D. Underlying technology 253
  • 269. A E n c ry p t w ith C le a rte xt m e s s a g e E nc ryp te d m e s s a g e B s p u b lic k e y "H e llo A lic e " f m k lw # ls d k 0 T ra n s f e r o f m e s s a g e B D e c ry p t w ith C le a rte xt m e s s a g e E nc ryp te d m e s s a g e B s p riv a te k e y "H e llo A lic e " f m k lw # ls d k 0 Figure D-6 Public Key Cryptography Data signing for authentication of the sender A, as the sender, wants to send data to B. A encrypts data with A’s private key, then B can decrypt the data with A’s public key. This way, B is sure that the data came from A, since no one else can create the message that can be decrypted with A’s public key. This process is shown in Figure D-7 on page 255. Note that although signing encrypts the data, it is not giving confidentiality, as anybody can decrypt the data by decrypting it with the senders’ public key. What signing provides is non-repudiation and data integrity (the sender cannot deny sending the message). The message cannot be changed, as that would make it impossible to decode it.254 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 270. A S ign with Cleartext message A s private key Signed message "Hello Alice" xc% slk&lskf Transfer of m essage B unsign with Cleartext message A s public key Signed message "Hello Alice" xc% slk& lskfFigure D-7 Signing of data with Public Key CryptographyThe combination of both processes can achieve confidentiality andnon-repudiation. As shown in Figure D-8 on page 256, when A sends a messageto B, the encryption/decryption process is as follows:1. A signs the data with A’s private key.2. A encrypts it with B’s public key.3. When B receives the signed and encrypted data, B decrypts it using B’s private key4. B checks the signature by unsigning (decrypting) it with A’s public key. Appendix D. Underlying technology 255
  • 271. A Sign with As Encrypt with private key Bs public key Cleartext message Signed message Encrypted message "H Alice" ello xc%slk&lskf cdl$d)sl&f Transfer of message B Unsign with Decrypt with Cleartext message As public key Signed message Bs private key Encrypted message "Hello Alice" cdl$d)sl&f xc%slk&lskf Figure D-8 Signing and encryption using Public Key Cryptography Public Key Cryptography has its own problems: The Public Key Cryptography process is computation intensive. Signing and encrypting the whole data stream implies a long encryption time. The difference in speed between Secret Key Cryptography and Public Key Cryptography is more than a factor of 1000. Man-in-the-middle attacks. Who guarantees us that the public key of Alice is really the public key of Alice? Somebody may publish a public key that is claimed to be someone else’s public key and access communications encrypted with that fraudulent public key.Refinements on cryptographic techniques At the moment, we have a technique that is fast, but has problems from the management point of view, and a technique that has good manageability, but is slow, and has problems concerning authenticating the other party. Let us now discuss a number of techniques that can alleviate these problems. Using hashes in Digital Signatures We saw earlier that by encrypting a message with the private key of the sender, the message was signed, but also that this was a slow process due to the256 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 272. inherent slowness of Public Key Cryptography. One way to overcome thisproblem is by creating a Message Digest through a hash function andsubsequently sign the Message Digest. This process is shown in Figure D-9. Cleartext message Encrypt with Message Digest Digital signature private key "H Alice" ello Hash wd$s*lsd^ cx3s&dsk)Figure D-9 Generating a Digital SignatureA hash function is a tool that takes a message of any size and generates a smallfixed size block of data (a Message Digest). The Message Digest has thefollowing characteristics: The Message Digest is always the same for the same block of data. If the original message is altered by even one bit, the resulting digest of the changed message is very different. Message Digest creation through a hash function is very fast. It is impossible to generate the original message from the digest.So instead of signing the whole message and encrypting it afterwards (employinga Digital Signature), the encryption/decryption process shown in Figure D-10 onpage 258 uses the following steps:1. The sender, A, create a Message Digest. A encrypts the digest with A’s private key to create a Digital Signature.2. The Digital Signature is appended to the original message, and the whole is encrypted with B’s public key.3. The receiver, B, decrypts the whole message with B’s private key and separates the Digital Signature from the message.4. B decrypts the Digital Signature with A’s public key to obtain the Message Digest. B also generates a Message Digest from the whole message with the same hash function.5. B compares the two Message Digests. If both digests are the same, then B is sure who the message comes from and that the message is unchanged. Appendix D. Underlying technology 257
  • 273. A Sign with Message Digest Digital Signature Cleartext message As private key "H Alice" ello hash c4$dil f6k#d Cleartext message Encrypt with Bs public key Encrypted message & digest "H Alice" ello xc5kf&lf f6k#d Transfer of message B Unsign with Digital Signature Message Digest As public key f6k#d Decrypt with c4$dil Bs private key Encrypted message & digest Compare Cleartext message xc5kf&lf Message Digest c4$dil hash "H Alice" ello Figure D-10 Public Key Cryptography using Digital Signatures Some well known hash algorithms are SHA-1, MD5, and RIPEMD. Digital Certificates The problem of authentication of the public key can be solved by the use of Digital Certificates. A Digital Certificate binds the owner of the public key to the public key itself. It is a data structure that contains a public key, necessary details about its owner and some other information. All this information is signed by a trusted third party called a Certificate Authority (CA). Some important details about CAs are outlined below: 1. When a public/private key pair is generated, the public key, together with the identity of the owner, must be submitted to a CA. 2. The CA signs the data with his own private key. The data becomes a Digital Certificate, and returns it to the owner.258 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 274. 3. A certificate does not contain any confidential data, and should be made available to the world, so that other people can use this certificate for sending data to the owner of the certificate and decrypt data from the owner.There are many commercial CAs. Some examples, with their Web sites, areVerisign (http://www.verisign.com), Thawte (http://www.thawte.com), andEntrust (http://www.entrust.com). There are also some local CAs for eachcountry. Commercial CA’s certificates are often included in products like Webbrowsers. For Netscape, you can view the certificates of the different CAs (that itrecognizes) by selecting Communicator → Tools → Security Info andclicking on the Signers link, as shown in Figure D-11.Figure D-11 CA’s Digital Certificates that Netscape recognizesThe use of the digital certificate in secure communication is shown inFigure D-12 on page 260. Appendix D. Underlying technology 259
  • 275. Transfer of message A B Sign with Signed challenge compare "message" 1 "challenge" secret key xc%slk&lskf Unsign challenge Signed challenge with Bs public key and Bs certificate Signed challenge "challenge" xc%slk&lskf and Bs certificate 2 xc%slk&lskf 3 Bs certificate Bs certificate Unsign certificate Message Bs public key compare Hash with CAs public key Digest 4 Decrypt with Bs private key 5 "message" "message" we4^&4kls we4^&4kls sldk$0-2+ As certificate signature sldk$0-2+ Unsign certificate Encrypt whole with CAs public key signature As certificate message with Bs public key As public key Unsign signature with As public key Message Digest Figure D-12 Digital Certificates in Public Key Cryptography 1. A must obtain the certificate of B. If A cannot get it from a public directory, then A could send a challenge (a piece of arbitrary data) to the perceived address of B. 2. B signs the challenge using its private key, attaches B’s certificate to the challenge and sends it back to A. 3. A unsigns the certificate using a CA’s public key, extract B’s public key and unsigns the challenge. If the original challenge is received, then A is certain about the identity of B. 4. A then generates a message with a digital signature, attaches A’s Digital Certificate and encrypts the whole with B’s public key. The result is sent to B. 5. After receiving the encrypted message, B decrypts it with B’s private key. The receiver then has access to the certificate of the sender, the message, and the signature of the message. a. B unsigns A’s certificate using the CA’s public key and retrieve A’s public key.260 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 276. b. B unsigns A’s signature using A’s public key and retrieves the Message Digest. c. B generates a digest from the original message and compares it with the unsigned message digest. Adding Secret Key Cryptography to the mix We now have established a secure way to transfer messages through Public Key Cryptography without having the authentication problems associated with it. The problem that remains is performance. It would be nice to marry the performance of Secret Key Cryptography with the management scalability of Public Key Cryptography. Here is how it works: 1. When two entities want to communicate, they set up communication channels using Public Key Cryptography. 2. During this set up, a secret key is created and transferred to the parties involved. This secret key will only be used for this session and will be destroyed afterwards. 3. Once all parties have the secret key, they can send information to each other with Secret Key Cryptography. 4. When the communication channel is closed, the secret key is destroyed.Secure Sockets Layer protocol In this section, we take a closer look on how the Secure Sockets Layer (SSL) protocol works. This protocol is used by the Tivoli Web Services Manager components to communicate securely over untrusted networks. SSL was conceived by Netscape Communications. It became the de facto standard to authenticate and encrypt communication between client and servers on TCP/IP networks. SSL performs the following functions: It authenticates the server to the client. Optionally, it authenticates the client to the server. It creates an encrypted connection between both machines. The authentication of the server to the client and vice versa happens through the exchange of certificates. The Certificate Authority that signed the certificate can be a different CA for the server than for the client. They must be trusted by the client and the server respectively. Appendix D. Underlying technology 261
  • 277. The encryption of the connection allows you to keep the data sent over the connection confidential. On top of that, it also checks whether the data has been changed during the transfer. All the above functions of SSL depends strongly on the cryptographic principles described in “Certificates and encryption” on page 251. SSL sits between the TCP/IP protocols, who are responsible for the transport and routing of data over the Internet, and the application protocols, like Hypertext Transport Protocol (HTTP) or other application protocols. This is shown in Figure D-13. It consists of two protocol levels: Record layer protocol Communication protocols: – Handshake protocol – Change Cipher Specification protocol – Alert protocol – Application protocol HTTP LD AP IM A P H andshake C h a n g e C ip h e r A p p lic a tio n A le r t P ro to c o l P ro to c o l P ro to c o l P ro to c o l SSL R e c o r d L a y e r P r o to c o l TCP IP Figure D-13 SSL structure and place in the protocol stack262 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 278. The Record Layer All messages coming from the higher level protocols go through the Record Layer before going to the transport layer. The Record Layer sends blocks of data called records, which are of fixed length. A record contains the content type, the protocol version number, the length and the data, which is compressed and encrypted. Each message passes the following three functions: Fragmentation of data: The message is divided or combined to fit into a record. Remember that a record has a fixed length. Compression before sending the data. Encryption of the data part of the record.The communication protocols There are four communication protocols: The Handshake protocol defines the sequence of events to establish an SSL session between two entities. The Change Cipher Specification protocol is actually a subset of the handshake protocol. Its primary function is to indicate to the other party that there has been a change in the cryptographic options. The Alert protocol deals with errors. An alert message contains two parts: the actual error description and the severity level of the error. There are two levels of errors: – Warning: This indicates a potential problem. An example is the close_notify error, which specifies that the sender will not send any more messages in the current session. – Fatal: This interrupts the current session, and also means that the current session cannot be resumed in the future. An example of this is the bad_record_mac error, which indicates that the message or its hash has been tampered with. The Application protocol is responsible for passing messages from the application layer protocol to the record layer protocol. SSL communication is set up using the handshake protocol. Both entities negotiate the version of the protocol to be used (2.0 or 3.0), the cryptographic algorithms, and the setup of the keys. It is also possible to include entity authentication in this step (one-way or mutual). If this happens, the server will authenticate itself to the client using Public Key Cryptography, after which Secret Key Cryptography is used for performance, as discussed in “Adding Secret Key Cryptography to the mix” on page 261. The sequence of messages exchanged during the handshaking is illustrated in Figure D-14 on page 264. Appendix D. Underlying technology 263
  • 279. C lie n t S e rv e r Request 1st C lient P hase P rotoc ol version S ession ID C ry ptographic options C ompression methods 1st S er ver P hase Certific ate S erver K ey E xc hange C ertific ate Request S erver Hello D one 2n d Client P hase C ertific ate C lient K ey E xc hange C ertific ate V erify c hangec ipherspec 2nd S er ver P hase Finished c hangec ipherspec Finished Applic ation D ata A pplic ation Data Figure D-14 SSL 3.0 handshake protocol 1. A request is sent by the server to start the handshake process. This implies that the client has instigated the connection by requesting the appropriate URL from the server. 2. The first client phase reply contains: a. A protocol version. b. A random number, used for the generation of the session keys. c. A session ID, to check whether a new session needs to be set up. d. A list of cryptographic options: Key exchange algorithm, hash algorithm, and so on. e. A list of the compression methods supported by the client. After sending this message, the client waits for the first server phase. If it receives any other type of message, this results in a fatal error, and the handshake must be restarted. 3. During the first server phase, the server sends the following messages: a. It acknowledges which version of the SSL protocol is supported (lower than or equal to the version of the client). The server also generates a random number and returns the session ID from the first client phase, if it agrees to pick up the old session. A choice is also made from the set of cryptographic options and compression methods that were proposed by the client.264 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 280. b. If the server authenticates to the client, the certificate of its public key is included. c. With the Server Key Exchange, the key exchange algorithm is specified. It can either be Diffie-Hellmann, RSA, or Fortezza. d. If the server wants the client to authenticate (mutual authentication), it sends a Certificate Request.4. The client responds with the second client phase: a. If the server requested it, it sends its certificate. b. With the Certificate Verify, the client proves to the server that it is in possession of the private key that corresponds to the public key in its certificate by digitally signing a specific message (called a challenge). See “Digital Certificates” on page 258 for more details on Digital Certificates. c. The Client Key Exchange contains all the necessary information that both parties need to calculate all the session keys (the ephemeral secret keys). See “Adding Secret Key Cryptography to the mix” on page 261 for more details on this step.5. Both parties now send, in turn, the finished message, preceded by the changecipherspec message, as shown in Figure D-14 on page 264. These are the first messages that use the newly agreed upon key material.SSL is a session based protocol and not a connection based protocol. Aconnection is set up every time there is data to be transferred between the clientand the server. It is not necessary to go through the handshake protocol everytime a new connection is made. Therefore, if previously arranged keys andalgorithms are used, the session is resumed. Appendix D. Underlying technology 265
  • 281. 266 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 282. Abbreviations and acronymsAIX Advanced Interactive FTP File Transfer Protocol eXecutive HFS Hierarchical File SystemsAPF Authorized Program Facility HPR High Performance RoutingAPI Application Programming HTML HyperText Markup Language Interface HTTP HyperText Transport ProtocolASID Address Space Identifier HTTPS HTTP SecureAUTOTBL Automation Table I/O Input OutputBPOOL Bufferpool IBM International BusinessCCITT Comite Consultatif Machines Corp. International Telegraphique et Telephonique (now ITU-T) ICMP Internet Control Message ProtocolCD-ROM Compact Disk Read Only Memory IIS Internet Information ServerCDW Central Data Warehouse IMS Information Management SystemsCICS Customer Information Control Systems ISMP Install Shield Multi PlatformCLI Command Line Interface ISO International Standard OrganizationCPU Central Processing Unit ISPF Interactive SystemCS390 Communication Server for Productivity Facility OS/390 ITSM IBM Tivoli Storage ManagerCSA Common System Area ITSO International TechnicalCSM Communication Storage Support Organization Manager ITU-T InternationalDB2 Database 2™ TelecommunicationDES Data Encryption Standard Union-Telephony divisionDFDSS Data Facility Data Set J2EE Java 2 Enterprise Edition Services JCL Job Control LanguageDTD Document Type Definition JDBC Java Database ConnectivityEAS Event Automation Services JDK Java Development KitECSA Extended Common System JES2 Job Entry Subsystem 2 Area JFS Journaled File SystemsEIF Event Integration Facility JMS Java Messaging ServicesEJB Enterprise Java Bean JMX Java Management eXtensionETL Extract, Transform, Load JVM Java Virtual Machine© Copyright IBM Corp. 2004. All rights reserved. 267
  • 283. LDAP Light-weight Directory Access RPC Remote Procedure Call Protocol RRS Resource Recovery ServicesLRU Least Recently Used SAX Simple API for XMLLTPA Light-weight Third Party SDBM Security Database Manager Authentication SDSF System Display and SearchMAT Message Automation Table FacilityMD5 Message Digest 5 SGML Simple Generalized MarkupMIB Management Information Language Base SMP/E System Modification ProgramMSU Management Service Unit ExtendedMTU Message Transport Unit SNA Systems NetworkMVS Multiple Virtual Storage ArchitectureNLDM Network Logical Data SNMP Simple Network Management Manager ProtocolNMVT Network Management Vector SPUFI SQL Processing Using File Table InputNPDA Network Problem SQL Structured Query Language Determination Assistant SSL Secured Socket LayerNPM NetView Performance STC Started Task Manager TBSM Tivoli Business SystemsODBC Open Database Connectivity ManagerOID Object Identifier TCP/IP Transmission ControlOSA Open Systems Adapter Protocol / Internet ProtocolOSASF OSA Service Functions TDBM Traditional Database ManagerOSI Open Systems Interconnection TDW Tivoli Data WarehouseOSPF Open Shortest Path First TEC Tivoli Enterprise ConsolePID Process ID TSM Tivoli Storage ManagerPPI Program to Program Interface TSO Time Sharing OptionRACF Resource Access Control UDP User Datagram Protocol Facility URL Universal Resource LocatorRDBM Relational Database Manager USS UNIX Systems ServicesRDBMS Relational Database VSAM Virtual Storage Access Management Systems MethodREXX Restructured Executive VTAM Virtual Telecommunication eXtended eXecutor Access ManagerRIM RDBMS Interface Module WAR Web ArchiveRMF Resource Measurement WEP Warehouse Enablement Pack Facility268 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 284. WLM Workload ManagerXCF Cross-System Coupling FacilityXML eXtensible Markup LanguageZFS z Filesystem Abbreviations and acronyms 269
  • 285. 270 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 286. Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.IBM Redbooks For information on ordering these publications, see “How to get IBM Redbooks” on page 273. Note that some of the documents referenced here may be available in softcopy only. Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 4: Connectivity and Routing, SG24-6516 Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 5: Availability, Scalability, and Performance, SG24-6517 Communication Server for z/OS V1R2 TCP/IP Implementation Guide Volume 6: Policy and Network Management, SG24-6839 Deploying a Public Key Infrastructure, SG24-5512 OSA-Express Implementation Guide, SG24-5948 OSA-2 Implementation Guide (Update), SG24-4770 Useful TCP/IP and FTP Commands, TIPS0091Other publications These publications are also relevant as further information sources: IBM Tivoli Monitoring for Network Performance Administrator Guide, SC31-6364 IBM Tivoli Monitoring for Network Performance: Messages and Troubleshooting, SC31-6366 IBM Tivoli Monitoring for Network Performance Operator Guide, SC31-6365 IBM Tivoli Monitoring for Network Performance: Planning, Installation, and Configuration, SC31-6363 IBM Tivoli Monitoring for Network Performance Version 2.1 README File May 19, 2004, GI10-3255© Copyright IBM Corp. 2004. All rights reserved. 271
  • 287. IBM Tivoli Monitoring for Network Performance Version 2.1 Warehouse Enablement Pack, Version 1.1.0.0 Implementation Guide for Tivoli Data Warehouse, Version 1.2, SC31-6793 Implementing Tivoli Data Warehouse V1.2, SG24-7100 Open Systems Adapter-Express Customer’s Guide and Reference, SA22-7935 and SA22-7476 SNA Formats, GA27-3136 Tivoli NetView for z/OS Installation: Configuring Additional Components Version 5 Release 1, SC31-8874 z/OS Communications Server CSM Guide Version 1 Release 2, SC31-8808 z/OS V1R4.0 Security Server: LDAP Server Administration and Use, SC24-5923 z/OS V1R6 Communications Server: IP Configuration Guide, SC31-8775 z/OS V1R6 Communications Server: IP Configuration Reference, SC31-8776Online resources These Web sites and URLs are also relevant as further information sources: Entrust, Inc. http://www.entrust.com Free LDAP browser http://www.iit.edu/~gawojar/ldap Freeware for graphic toolkit http://www.eteks.com/pja/en/#Download Thawte, Inc. http://www.thawte.com Useful TCP/IP and FTP commands http://www.redbooks.ibm.com/tips/TIPS0091/tips0091.pdf VeriSign, Inc. http://www.verisign.com w3C: Extensible Markup Language (XML) 1.1 http://www.w3.org/TR/xml11272 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 288. How to get IBM Redbooks You can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooksHelp from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services Related publications 273
  • 289. 274 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 290. Index automation specialist 56Symbols$CONFIG_DIR 25$WAS_HOME 103 B/etc/ibm/tivoli/common 126 backup command 95/etc/ibm/tivoli/common/cfg 25 backup schedule 96/etc/ibm/tivoli/common/cfg/log.properties 131 baroc file 165/etc/itmnp 128 BEST_PRACTICES table 43/etc/itmnp/ 28 bind_interface 28/etc/ldap 70/etc/services 69/home/db2inst1/sqllib/bin/db2start command 90 C CA/home/db2inst1/sqllib/db2profile 90 Call Level Interface, see CLI/home/db2inst1/sqllib/java12 66 capacity planner 56/opt/IBM/ITMNP21 67 CCITT 242/opt/IBM/ITMNP21/_uninst, 84 CDW/opt/IBM/ITMNP21/bin/itmnp.jacl 83 Central Data Warehouse, see CDW/tmp 103 certificate authority 258/u/itmnp 103–104 Certificate Authority, see CA/u/itmnp/jctest.ear 112 CLI/u/itmnp/pja 109 CLIENT 65/usr/lpp/db2/db8d/jcc/classes 107 COLL_ATTR_GRP table 43/usr/lpp/itmnp/V2R1M0/bin/itmnpMonitor.sh 132 COLL_TBL table 43/usr/lpp/itm- COM.ibm.db2os390.sqlj.util.DB2DriverInfo.classnp/V2R1M0/web_app_inst/itmnp21zos.tar 104 105/usr/lpp/ldap/etc/ldap.profile 70 commands/usr/WebSphere 19 /home/db2inst1/sqllib/bin/db2start 90/usr/WebSphere/AppServer 69 backup 95/usr/WebSphere/AppServer/logs/server1 62 db2 91, 93/var/ibm/tivoli/common/FNP 126 db2iupdt 65/var/ibm/tivoli/common/FNP/logs 31 db2start 94 db2stop 91A df 60access control list 247 DIS BUFFERPOOL 107Active Directory Service Interface 242 DISPPI 176ADRDSSU 119 extattr 131APF authorized attribute 131 itmnp_was.sh 90API 242 itmnpMonitor.sh 132asymmetric key pair 253 kill 133 private key 253 ldapcnf 71 public key 253 mount 104, 121attribute 246 NETMON 127authType 65 ovstatus 163automation blueprint 2–3 ps 133© Copyright IBM Corp. 2004. All rights reserved. 275
  • 291. restore 98 DBHostName 29 startServer.sh 91 DBName 29 stopServer.sh 91 DBPassword 29 tar 92, 98 DBPort 29 wrb 165 DBUserName 29common directory 244 df command 60Communication port usage 35 DFDSS 119Communications Storage Manager, see CSM digital certificate 68console users 83 digital certificates 258COPY utility 121 digital signature 257cryptographic principles 251 Directory Access Protocol 243cryptographic technique Directory Information Tree 245 public key cryptography 253 DIS BUFFERPOOL command 107 secret key cryptography 252 DISPPI command 176Crystal Enterprise 9 186, 209 distibuted consideration 51Crystal Enterprise configuration 202 distinguished name 245Crystal Enterprise Server 59 distinguished name, see DNCSAPIport 28 DNCSM document organization 5CSM_MON_MSMT table 43 driverCSM_SUMM_MSMT table 43 UNIVERSAL 29 DSNAQLDA 104 DSNL004I 29Ddata source 85data sources 26 Edatabase administrator 56 EASdatabase configuration 93 EEDB2 10, 58 EE_AVL_MSMT table 44db2 command 91, 93 EE_TT_DET_MSMT table 44DB2 HFS code level 105 EE_TT_MSMT table 44DB2 HFS directory 104 EIFDB2 level 104 enableCloudscape 30DB2 Universal Database Version 8 59 Enterprise Extender, see EEDB2Binder 107 ENUM_TYPES table 43DB2DriverInfo_native 104 environment 5db2iupdt command 65 ePortfolio 209db2j2classes.zip 105 ETLdb2Security 30 ETL processing 197DB2SQLJJDBCPROGRAM 108 event automation program 173DB2SQLJPLANNAME 108 Event Automation Service, see EASDB2SQLJSSID 108 Event Integration Facility, see EIFdb2start command 94 event receiver 48db2stop command 91 EVENT_DEST table 43dbcache 25 extattr command 131DBCacheDirectory 25, 29 extract, transform, and load, see ETLDBCacheRowLimit 30 Extract-Transform-Load, see ETLDBCacheTimeout 29DBDriverType 29276 IBM Tivoli Monitoring for Network Performance V2.1: The Mainframe Network Management Solution
  • 292. F HPR_PIPE_MSMT table 44files HPR_TT_MSMT table 44 /etc/services 69 HTTP /home/db2inst1/sqllib/db2profile, 90 HTTP secure, see HTTPS /opt/IBM/ITMNP21/_uninst, 84 HTTPS /u/itmnp 104 Hyper Text Transfer Protocol, see HTTP /u/itmnp/jctest.ear 112 /u/itmnp/pja 109 I /usr/lpp/db2/db8d/jcc/classes 107 IBM automation blueprint 3 /usr/lpp/itmnp/V2R1M0/bin/itmnpMonitor.sh IBM automation platform 2 132 IBM Global Security Kit, see GSKIT /usr/lpp/ldap/etc/ldap.profile 70 IBM Tivoli Enterprise Console 16, 58, 161 /var/ibm/tivoli/common/FNP 126 IBM Tivoli Monitoring for Network Parformance fnp_config.xml 25 startup 117 itmnp IBM Tivoli Monitoring for Network Performance 2 baroc 165 alert interface 161 itmnp.jacl 83 architecture 7 itmnp.properties 25, 128 automation sample 181 itmnp_install.log 83 backup and receovery 91 itmnp_was.sh 90 backup and recovery 119 itmnpServerKeyStore.jks 90 client certificates 39 itmnpServerTrustStore.jks 90 communication 34 log.properties 25 components 8 msg_fnp_monitor*.log 25 data collection 15, 141 trace_fnp_monitor*.log 25 database communication 40fnp_config.xml 25, 43 database structure 41FNPI0002E 85 discovery 161FNPlogPropertiesLocation 25 environment 5FNPM012E 131 ETL processing 197FTP_CTRANS_MSMT table 44 files 24FTP_SESS_MSMT table 44 functions 8FTP_STRANS_MSMT table 44 graphic package 108full image copy 121 historical reporting 183 implementation components 48G implementation scenarios 47GENALERT introduction 1Generic NMVT alert, see GENALERT monitor 22graphic test program 112 monitor configuration 134GSKIT monitor installation 126 monitor problem determination 31 monitor start and stop 132H monitored metrics 138hardware monitor 177 monitoring best practices 137hash function 257 native authentication 35HFS port usage 35Hierarchical File Systems, see HFS pre-defined reports 208High Performance Routing, see HPR preparation 103HPR reinstallation 84HPR_AVL_MSMT table 44 Index 277
  • 2