Your SlideShare is downloading. ×
Migrating to netcool precision for ip networks --best practices for migrating from ibm tivoli net view sg247375
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Migrating to netcool precision for ip networks --best practices for migrating from ibm tivoli net view sg247375


Published on

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Front coverMigrating to Netcool/Precisionfor IP NetworksBest Practices for Migrating fromIBM Tivoli NetViewCompare capabilities and solutionarchitecturesMigrate IBM Tivoli SwitchAnalyzerPerform the migration andconfigure the new features Stephen Hochstetler Donald Hart Leslie Clark Mathias Scharfenberg Pádraig Byrne Rob Clark Bob
  • 2. International Technical Support OrganizationMigrating to Netcool/Precision for IP NetworksBest Practices for Migrating from IBM Tivoli NetViewFebruary 2007 SG24-7375-00
  • 3. Note: Before using this information and the product it supports, read the information in “Notices” on page ix.First Edition (February 2007)This edition applies to Version 7, Release 1, modification 5 of IBM Tivoli NetView (productnumber 5698-NTV) and Version 1, Release 3 of IBM Tivoli Switch Analyzer (product number5724-C72) and Version 3, Release 6 of Netcool/PrecisionIP Discovery and Root Cause Analysis(product number 5724-O52) and Version 3, Release 6 of Netcool/PrecisionIP Topology Server(product number 5724-O60) and Version 3, Release 6 of Netcool/PrecisionIP Topology DiscoveryTier 1 (product number 5724-O85) and Version 3, Release 6 of Netcool/PrecisionIP TopologyDiscovery Tier 2(product number 5724-O86) and Version 3, Release 6 of Netcool/PrecisionIPFault Discovery and Asset Tier 1 (product number 5724-O87) and Version 3, Release 6 ofNetcool/PrecisionIP Fault Discovery and Asset Tier 2(product number 5724-O88)© Copyright International Business Machines Corporation 2007. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp.
  • 4. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiPart 1. Product comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 IBM Service Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Next Generation Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Netcool/Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 NetView customer choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5 The purpose of this book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 2. Product review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 Discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Network visualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.5 Event management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.6 SNMP tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.7 Diagnostic tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.8 User consoles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.9 Product administration and configuration . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.10 Integration with other products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Chapter 3. Benefits of migrating to Precision . . . . . . . . . . . . . . . . . . . . . . 37 3.1 Full layer 2 discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.1.1 The OSI seven layer model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2 Filling in gaps in the discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2.1 Inserting missing connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3 MPLS networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.1 Example MPLS discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.2 MPLS edge view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.3.3 MPLS core view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.3.4 More information on MPLS capabilities in Netcool/Precision . . . . . . 47© Copyright IBM Corp. 2007. All rights reserved. iii
  • 5. 3.4 Topology-based root cause analysis (RCA) . . . . . . . . . . . . . . . . . . . . . . . 48 3.4.1 Netcool Knowledge Library example. . . . . . . . . . . . . . . . . . . . . . . . . 48 3.4.2 Netcool/Precision root cause analysis . . . . . . . . . . . . . . . . . . . . . . . 50 3.5 Multiple domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.5.1 Managed service provider (MSP) environments . . . . . . . . . . . . . . . . 53 3.5.2 Distinct administrative areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.6 Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.7 Extending your discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.8 Event enrichment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.8.1 Event enrichment in the Netcool suite. . . . . . . . . . . . . . . . . . . . . . . . 56 3.8.2 Event enrichment in Netcool/Precision . . . . . . . . . . . . . . . . . . . . . . . 57 3.9 Asset management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.9.1 Basic asset information in standard installation . . . . . . . . . . . . . . . . 58 3.9.2 Netcool for Asset Management - NfAM. . . . . . . . . . . . . . . . . . . . . . . 59 Chapter 4. Solution architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.1 Netcool overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.1.1 Netcool OMNIbus ObjectServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.1.2 Netcool probes and monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.1.3 Netcool/Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.1.4 Netcool/RAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.1.5 Netcool/Reporter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.1.6 Netcool/Webtop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2 A first look at Netcool/Precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.2.1 Netcool/Precision components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.2.2 Inter component communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.2.3 Precision services and OQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.3 Event flow through Netcool/Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.4 Example Netcool/Precision deployments . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.4.1 Small scale Netcool/Precision deployment . . . . . . . . . . . . . . . . . . . . 71 4.4.2 Large scale Netcool/Precision deployment . . . . . . . . . . . . . . . . . . . . 73 4.5 Netcool/Precision in failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5.1 OMNIbus failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.5.2 Webtop and NCSM failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.5.3 Netcool/Precision failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77Part 2. Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Chapter 5. Preparing the server for migration and installing the Netcool components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.1 Planning for migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.2 Prepare the new monitoring servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.2.1 Our lab server environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.2.2 Operating system preparation and checks . . . . . . . . . . . . . . . . . . . . 84iv Migrating to Netcool/Precision for IP Networks
  • 6. 5.3 Required Netcool components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.4 Installation of Netcool components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.4.1 Install and verify Netcool License Server . . . . . . . . . . . . . . . . . . . . . 86 5.4.2 Install and verify Netcool OMNIbus 7.1.0 . . . . . . . . . . . . . . . . . . . . . 87 5.4.3 Install and verify Netcool Knowledge Library . . . . . . . . . . . . . . . . . . 91 5.4.4 Install and verify Netcool Mttrapd Probe . . . . . . . . . . . . . . . . . . . . . . 91 5.4.5 Install and verify Netcool Security Manager 1.3 . . . . . . . . . . . . . . . . 93 5.4.6 Install and verify Netcool Precision IP 3.6 . . . . . . . . . . . . . . . . . . . . . 935.5 Starting Netcool products at server boot . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.1 Running the OMNIbus script to create startup files. . . . . . . . . . . . . . 97 5.5.2 Running the Precision script to create startup files . . . . . . . . . . . . . . 98 5.5.3 Creating a startup script for Netcool License Manager . . . . . . . . . . . 99 5.5.4 Creating a startup script for Netcool GUI Foundation . . . . . . . . . . . 100 5.5.5 Creating a startup script for Netcool Security Manager . . . . . . . . . 100 5.5.6 Symbolic link creation to auto-start applications . . . . . . . . . . . . . . . 101Chapter 6. Migrating NetView and Switch Analyzer. . . . . . . . . . . . . . . . . 1036.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.1.1 NetView architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.1.2 Netcool architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.2 Gathering information from the NetView server . . . . . . . . . . . . . . . . . . . 1056.3 Migrating the discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.3.1 First pass at discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.3.2 Second pass at discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.3.3 Third pass at discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.3.4 Fourth pass at discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.3.5 Migrating discovery rules and adding agents . . . . . . . . . . . . . . . . . 120 6.3.6 Discovering extra information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1256.4 Migrating the network map visualization . . . . . . . . . . . . . . . . . . . . . . . . . 131 6.4.1 Migrating SmartSets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 6.4.2 Migrating the network view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1436.5 Migrating network monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 6.5.1 Tivoli NetView preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 6.5.2 Netcool/Precision preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 6.5.3 Configure ping polling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 6.5.4 Configure SNMP link polling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 6.5.5 Configure SNMP threshold polling . . . . . . . . . . . . . . . . . . . . . . . . . 154 6.5.6 Activating the changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 6.5.7 Passive monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6.5.8 Understanding how interfaces are managed . . . . . . . . . . . . . . . . . 156 6.5.9 Enabling new node events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 6.5.10 Examples of the monitoring events . . . . . . . . . . . . . . . . . . . . . . . . 1586.6 Netcool OMNIbus automations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Contents v
  • 7. 6.6.1 Mail on critical automation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.6.2 Event enrichment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.7 Creating users for Netcool components . . . . . . . . . . . . . . . . . . . . . . . . . 170 6.7.1 User creation in Netcool/OMNIbus . . . . . . . . . . . . . . . . . . . . . . . . . 172 6.7.2 Creating user in NGF with admin permissions . . . . . . . . . . . . . . . . 176 6.7.3 Assign user roles and groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.7.4 Creating a user with operator access . . . . . . . . . . . . . . . . . . . . . . . 179 6.7.5 Creating the operator user in the NGF . . . . . . . . . . . . . . . . . . . . . . 180 6.7.6 Creating a limited access executive view in the NGF . . . . . . . . . . . 183 6.7.7 Summary of new Netcool/OMNIbus and NGF users . . . . . . . . . . . 186 6.8 Adding tools to the user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 6.8.1 The Ping tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 6.8.2 Adding a MIB application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.8.3 Adding an http management tool . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Chapter 7. Migration topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 7.1 Scheduled discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 7.2 Provisioning Netcool/Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 7.3 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 7.4 Populating the user interface by roles . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 7.4.1 Create the network operators group . . . . . . . . . . . . . . . . . . . . . . . . 211 7.4.2 Create the tabbed page for the operators view . . . . . . . . . . . . . . . . 213 7.4.3 Create the network topology view . . . . . . . . . . . . . . . . . . . . . . . . . . 215 7.4.4 Build the Operators tabbed view . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 7.5 The menus in Omnibus and Netcool/Precision . . . . . . . . . . . . . . . . . . . . 220 7.5.1 Omnibus X11 menus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 7.5.2 NGF/Webtop menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.5.3 NGF/Topoviz menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 7.6 Enriching interface events with chassis object attributes . . . . . . . . . . . . 226 Appendix A. Useful information for Netcool installation and maintenance 227 A.1 Environment settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 A.2 License Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 A.3 ObjectServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 A.4 OMNIbus probes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 A.5 OMNIbus gateways. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 A.6 Process control (PA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 A.7 Menus, tools, and prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 A.8 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 A.9 Automation triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 A.10 Security Manager 1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 A.11 Webtop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234vi Migrating to Netcool/Precision for IP Networks
  • 8. A.12 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 A.12.1 Precision server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 A.12.2 Precision monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 A.12.3 Precision monitor probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 A.12.4 Precision discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 A.12.5 Precision bidirectional gateway . . . . . . . . . . . . . . . . . . . . . . . . . . 237 A.12.6 Precision Failover: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238A.13 mySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238Appendix B. Scripts and commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241B.1 Commands and scripts used to extract information from the NetView installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 B.1.1 Devices that are discovered and managed by NetView . . . . . . . . . 242 B.1.2 Custom fields information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 B.1.3 User account information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 B.1.4 Polling information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 B.1.5 Trap and event processing information . . . . . . . . . . . . . . . . . . . . . 261 B.1.6 Event processing information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 B.1.7 Other automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263B.2 Scripts and commands for validating and customizing the Precision installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 B.2.1 Perl script to extract all unknown OIDs from Precision . . . . . . . . . . 264 B.2.2 Script to compare discovered nodes in NetView and Precision . . . 267 B.2.3 Perl script to handle unmanaged nodes or interfaces . . . . . . . . . . 270 B.2.4 Sample of threshold polling definition to be put into *.aoc file . . . . 275B.3 Precision agents we modified . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277B.4 Startup scripts modified to run at boot time . . . . . . . . . . . . . . . . . . . . . . 280B.5 NGF menu configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285B.6 Stitchers for event enrichment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292Appendix C. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 System requirements for downloading the Web material . . . . . . . . . . . . . 298 How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Contents vii
  • 9. viii Migrating to Netcool/Precision for IP Networks
  • 10. NoticesThis information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the usersresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send licenseinquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.The following paragraph does not apply to the United Kingdom or any other country where suchprovisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATIONPROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS ORIMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM maymake improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirmthe accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrate programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.© Copyright IBM Corp. 2007. All rights reserved. ix
  • 11. TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both: Redbooks (logo) ™ Netcool® Tivoli Enterprise Console® DB2® NetView® Tivoli® IBM® Redbooks™ Viewpoint™ MQSeries® System p™ WebSphere® Netcool/Omnibus® Tivoli Enterprise™The following terms are trademarks of other companies:IT Infrastructure Library, IT Infrastructure Library is a registered trademark of the Central Computer andTelecommunications Agency which is now part of the Office of Government Commerce.ITIL is a registered trademark, and a registered community trademark of the Office of GovernmentCommerce, and is registered in the U.S. Patent and Trademark Office.Java, JRE, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States,other countries, or both.Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,other countries, or both.Pentium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of IntelCorporation or its subsidiaries in the United States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.x Migrating to Netcool/Precision for IP Networks
  • 12. Preface This IBM® Redbook will help you determine if you want to migrate from IBM Tivoli® NetView® version 7 and IBM Tivoli Switch Analyzer to Netcool/Precision for IP Networks version 3.6. The first part of the book is written to help you understand the changes and benefits that Netcool/Precision for IP Networks can bring to your environment. The intent is to help you evaluate your own usage of the NetView features and see how they map to the Netcool/Precision features. You can also learn about the additional features that Netcool/Precision offers to help determine if a migration is right for your company at this time. The second part of the book takes a systematic and detailed approach to the process of planning and performing the migration from NetView to Netcool/Precision. Drawing on the authors’ many years of experience with both NetView and Netcool/Precision, as well as on extensive work in the Redbooks™ lab, this part is intended for the technical leaders and specialists who will be performing the migration and who have the appropriate education or experience to deploy Netcool/Precision. The scripts we developed to help with the migration tasks are documented in appendixes and are available for download from the redbook Web site.The team that wrote this redbook This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, Austin Center. Stephen Hochstetler is a project leader at the International Technical Support Organization, Austin Center. He writes extensively and teaches IBM classes worldwide on all areas of system management, Linux®, and System p™. Before joining the ITSO 6 years ago, Stephen worked in Tivoli Services, USA as a network management architect. Donald Hart is a Solutions Architect in the USA. He has 10 years of experience in managing networks with the Netcool® product suite. He has traveled extensively around the world providing network architecture consulting and training for the past 6 years. Leslie Clark is a Senior Services Specialist with IBM Global Services USA. She holds a BSc from the University of Michigan. She has helped customers© Copyright IBM Corp. 2007. All rights reserved. xi
  • 13. implement and customize Tivoli NetView across the US and Canada over the last fifteen years. Mathias Scharfenberg is a Senior IT Architect in Germany. He has 10 years of experience in networking. He holds a BSc degree in Computer Science from the University Of Hertfordshire. His areas of expertise include networks and network management. Pádraig Byrne is a Netcool Specialist for IBM Australia. He has six years of experience working with telco and network management software. Prior to joining the pre-sales team in Australia he worked with the Precision development team in London. He holds a degree in Mathematics from the University of Cambridge. His areas of expertise include networks, Precision and the Netcool suite. Rob Clark is a software developer in the USA. He has 20 years of experience in software development and 10 years with NetView development. He holds an MS degree in Computer Science from Northeastern University. His areas of expertise include software engineering, and all aspects of Tivoli NetView. Bob Louden is a Consulting IT Specialist on the Tivoli Sales Enablement team responsible for training and supporting worldwide sales teams on Tivoli products. He holds a BS in Computer Science from Virginia Tech, and an MS in Computer and Communications Science from the University of Michigan. Bob has enjoyed twenty-four years with IBM – in roles ranging from product development, to sales, to technical sales support, to consulting – helping clients apply technology solutions to their business problems. Thanks to the following people for their contributions to this project: Special thanks to Andrew Hepburn with IBM, United Kingdom. His technical guidance is reflected in many sections of the book. All of the authors learned several things from Andrew. Arzu Gucer International Technical Support Organization, Austin Center Jonathan Baggott, Bhrat Patel, Dave Roberts IBM, United Kingdom Nick Ho, Bob Louden, Raymond Sun IBM USAxii Migrating to Netcool/Precision for IP Networks
  • 14. Become a published author Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. Youll have the opportunity to team with IBM technical professionals, Business Partners, and Clients. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, youll develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: welcome Your comments are important to us! We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at: Send your comments in an email to: Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 Preface xiii
  • 15. xiv Migrating to Netcool/Precision for IP Networks
  • 16. Part 1Part 1 Product comparisons In this part we discuss the reasons that IBM bought Micromuse and the value that the new products bring to the Tivoli portfolio. After looking at the Netcool®/Precision for IP Networks™ features that map to IBM Tivoli NetView and IBM Tivoli Switch Analyzer, we also look at additional benefits that Netcool/Precision brings to customers.© Copyright IBM Corp. 2007. All rights reserved. 1
  • 17. 2 Migrating to Netcool/Precision for IP Networks
  • 18. 1 Chapter 1. Introduction The IBM acquisition of Micromuse Inc., on February 14, 2006, marks a major milestone for IBM Tivoli software because it significantly strengthens the end-to-end IBM Service Management software portfolio. The acquisition comes at a time when important new networking technologies have emerged and very high network availability has become mission critical for most organizations. While the IBM Tivoli NetView product has a long history of industry-leading out-of-the-box utility, the addition of Netcool/Precision to our portfolio extends our network management capabilities to include extensive automated network discovery and best-of-breed topology-based root cause analysis – providing customers with comprehensive, real-time understanding of their network infrastructures and the fastest possible resolution of network problems. While significant focus is being placed on enhancing the ease of installation and use of coming versions of Netcool/Precision, IBM will continue to protect NetView customers’ investments and also intends to provide a smooth upgrade path to a future converged network management product offering.© Copyright IBM Corp. 2007. All rights reserved. 3
  • 19. 1.1 IBM Service Management Today, IT environments are under tremendous pressure. This pressure can be traced to four key sources: complexity, change, compliance, and cost. Businesses must be able to quickly respond to market changes in order to maximize revenue. As businesses increasingly rely on technology to improve their ability to deliver products and services, additional pressure is put on the IT department to adapt their services to these changes. 1. Change - Fast-changing external and internal forces, and unpredictable variations in workloads make meeting service levels difficult. 2. Complexity - Organizations manage complex IT environment to support business processes. 3. Compliance - The changing global regulatory and business environment requires security, privacy, and ongoing audit capabilities. 4. Cost - To meet business service-level expectations, infrastructure costs have been outpaced by spending on management and administration. For example, a new product launch and promotion may stress the order fulfillment process, which relies on a Supply Chain Management application. The IT department must be able to provide capacity to support the application during this period of high demand, but purchasing additional hardware that will not be utilized during normal periods is not the most effective solution. It is dealing with these changes in increasingly complex environments while under constrained budgets that truly challenges IT. By combining the Netcool and Tivoli portfolios, IBM enables customers to take a more comprehensive approach to aligning IT operations and processes with their organizations business needs - an approach that leverages best practices such as those of the IT Infrastructure Library® (ITIL®). IBM calls this approach IBM Service Management (Figure 1-1).4 Migrating to Netcool/Precision for IP Networks
  • 20. Figure 1-1 IBM Service Management IBM Service Management includes a uniquely broad and modular set of capabilities that help customers better manage the business of IT: Operational management helps organizations deliver services across the infrastructure effectively and efficiently. Tivoli operational management products span networking, business applications, servers, devices, storage, and security to provide an end-to-end service perspective. Service management platform is built on IBM Tivoli Change and Configuration Management Database (CCMDB), which standardizes and shares information across the enterprise to help align operations with business context and enable customers to manage change. Tivoli CCMDB includes automated, configurable best-practice workflows for the change and configuration processes. It also serves as a platform for integrated process management. Process management integrates and automates service management processes to increase operational efficiency. Best practices learned from thousands of successful customer engagements serve as the foundation for IBM Service Management. Network management is key to Tivolis comprehensive service management strategy. Awareness of network devices, configuration, and faults is required for Service Deployment, Business Resilience, and Service Delivery processes. By joining the Tivoli leadership and experience managing data center environments with those of Netcool in the network operations center, IBM enables customers to benefit from fully integrated management software that shares event and performance management, visualization, and automated workflow capabilities across the enterprise. The combined Netcool and Tivoli portfolio will help users Chapter 1. Introduction 5
  • 21. manage any data related to infrastructure elements such as networks, systems, security devices, storage components, and applications to gain full visibility into the health and performance of infrastructure-dependent services. IBM is as committed to Netcool customers and products as it is to customers who have invested in Tivoli solutions. The companys strategy is to enable all Netcool and Tivoli users to protect, optimize, and extend their investments in the combined product portfolio. Protect: IBM seeks to protect customer investments of not only resources, but also knowledge accumulated over years of building ever more advanced IT operations infrastructures. As the Netcool and Tivoli product portfolios converge, IBM intends to provide smooth upgrade paths that facilitate adoption of the best capabilities across the combined portfolio while preserving and unlocking customers knowledge investments. Optimize: IBM is helping customers leverage expanded capabilities today, even as work progresses toward the converged Tivoli portfolio. In product categories where the combined portfolio capabilities overlap, customers can "trade up" to the more feature-rich product in the category. For example, IBM Tivoli NetView and IBM Tivoli Switch Analyzer users can trade up to Netcool/Precision for integrated Layers 2 and 3 network discovery and management. Extend: Whether a customer currently uses Netcool products, Tivoli products, or both, the combined portfolio offers many additional products and capabilities the organization can leverage. Specifically, the Netcool portfolio offers Tivoli users a wide range of capabilities for security operations management, performance management, and network management. The Netcool portfolio further extends the Tivoli portfolio with next-generation management solutions for telecommunications infrastructures. IBM is dedicated to every customers success. As the company works to deliver a converged portfolio, it is taking numerous steps to enable the investments customers have made in IBM and Micromuse products over the years to continue to benefit their organizations. Furthermore, the smooth upgrade paths IBM is putting in place are meant to help customers derive even greater value from these investments moving forward.1.2 Next Generation Networking For years, the networking industry has been heralding the emergence of Next Generation Networks (NGNs) - networks where new TCP/IP-based technologies leverage extraordinary (wireless, wired, and optical) transport network capabilities to deliver voice, video, data, and multimedia traffic across a common6 Migrating to Netcool/Precision for IP Networks
  • 22. networking infrastructure (and, in many cases, the Internet). NGNs are heretoday, but increasing dependence upon them brings with it significantly greaterrequirements for high-quality, secure, and highly-available communicationservices. Likewise, network management technologies and protocols haveevolved (such as the Simple Network Management Protocol, most recently,SNMPv3) to provide ever greater security and functional capabilities. The rate ofchange in networking technologies and requirements has strained the ability ofmany network management products to keep up – with the consequent inabilityof network managers to see, understand, and troubleshoot problems within theirnetworking infrastructures.Network management challenges for NGNs include: NGNs are normally heterogeneous (multi-vendor), requiring broad management support for network equipment that is vendor-specific. NGNs normally involve a combination of network technologies for delivery, including: – Transport protocols such as SONET/SDH, ATM, Frame Relay, and wireless – Dynamic networking and high availability technologies such as OSPF, HSRP, VRRP, and BGP – TCP/IP transport technologies such as Voice over IP, IP Multimedia Services, and MPLS – Security technologies such as Virtual Private Networking, firewalls, and Network Address Translation NGNs often involve more complex meshed network architectures, including: – Traffic engineering to optimize traffic flows, as well as ensuring service availability in the event of a network failure – Potentially overlapping IP address spaces – often due to mergers and acquisitionsThe traditional network management approach of "discover all the boxes andping (ICMP) the devices" no longer provides sufficient coverage to ensureservice availability. Crucial time may be spent chasing alarms that are merelysymptomatic of deeper, underlying problems. Tools, such as Netcool/Precision,are required to enable an end-to-end view across the IP and Transmissionnetwork components. Chapter 1. Introduction 7
  • 23. 1.3 Netcool/Precision The addition of Netcool/Precision to the IBM network management portfolio extends our network management capabilities to include extensive automated network discovery and best-of-breed topology-based root cause analysis, providing customers the best possible, real-time understanding of their network infrastructures and the fastest possible resolution of network problems. Key features of the Netcool/Precision product are the following: Netcool/Precisions automated network discovery uses advanced techniques to gather in-depth information about the contents and structure of the network, including: – Layer 2: the data-link layer, including switched networks and Virtual LANs – Layer 3: the network layer, including dynamic routing protocols, Virtual Private Networking, and multi-protocol label switching (MPLS) services The network is then modeled within Netcool/Precision to create a highly-accurate representation of the true network fabric. Collecting extensive information directly from the network devices provides the most complete and up-to-date details of the network assets and their connectivity. This discovery information is maintained ("persists") across restarts of the Netcool/Precision system, thereby eliminating the need for extensive network rediscovery after restarts. Netcool/Precision helps network management teams visualize and understand the layout of complex networks and the impact of network events and failures upon them – and, more importantly, the services delivered across them. Within Netcool/Precision, the topology-based event correlation engine uses the model of the discovered network to understand the relationships between network events based upon the connectivity and containment (various groupings) of network devices. This enables Netcool/Precision to quickly and accurately identify root cause events (to the node and port level) and their associated symptoms, thereby reducing the time needed to restore the network and ensuring that customer-facing network operations staff has meaningful contextual information at their fingertips. Integration with Netcool/OMNIbus allows the Netcool/Precision topology-based event correlation engine to process events obtained from both network devices and other management systems using a broad range of available integrations. Netcool/Precision easily integrates with operational support systems (OSS) and other mission-critical workflow applications.8 Migrating to Netcool/Precision for IP Networks
  • 24. The Precision product will anchor future Tivoli network management offerings, including the planned support for enhanced next-generation networks and IPv6. The next planned release of Netcool/Precision aims to blend the capabilities of Precision for IP Networks (Precision IP) and Precision for Transmission Networks (Precision TN) to facilitate integrated discovery and management of all layers of the network infrastructure. A future version of Precision is planned to provide fast and easy problem identification and resolution for small and midsize businesses.1.4 NetView customer choices While significant focus is being placed on enhancing the ease of installation and use of coming versions of Netcool/Precision, IBM will continue to protect our NetView customers investments and intends to provide a smooth upgrade path to a future converged network management product offering. Customers who do not yet need the enhanced device discovery and layer 2 support offered by Precision, and who are concerned about disrupting their environment, can continue to use NetView 7.1.4. Customers who need enhanced SNMP support, duplicate IP address support, or NetView monitoring capabilities, can upgrade to NetView 7.1.5. Customers who have an immediate need for the deep discovery (including layer 2 support), advanced protocol support, and topology-based root cause analysis offered by Precision IP, can upgrade immediately to Precision IP.1.5 The purpose of this book This book was written primarily for customers who are thinking about upgrading to Netcool/Precision. We have established a team of experts from NetView Development, Network Management Services, Netcool Services, and IBM Services. Together, we have documented the best practices for upgrading customer environments from NetView to Netcool/Precision. This book will help you identify the additional features that Netcool/Precision brings to your environment to help you determine which strategy is better for you. Chapter 1. Introduction 9
  • 25. 10 Migrating to Netcool/Precision for IP Networks
  • 26. 2 Chapter 2. Product review In this chapter we discuss the major features of Tivoli NetView and match them with the equivalent Netcool®/Precision for IP Networks™ features. This will give you a good idea of how your current network management functionality can be provided with Netcool/Precision. The features and capabilities discussed are: Discovery Monitoring Network visualization Event management SNMP tools Diagnostic tools User consoles Product administration and configuration Integration with other products© Copyright IBM Corp. 2007. All rights reserved. 11
  • 27. 2.1 Overview The Tivoli NetView users often centers their activity around the topology maps from where they can see status changes and access diagnostic tools and device information. To make this task easy, many users customize the maps to organize the information visually and to make navigation easier. Events offer useful information including status changes, but to do any serious event management, NetView users typically integrate with Tivoli Event Console (TEC) and manage the events from there. From the TEC event view they can launch the NetView topology maps via the Web Console to access network-related information and visual orientation. With Netcool, the components focus on contributing events or enriching events. Netcool/Precision discovers and monitors the network devices. It contributes topology information to the events and uses this for further enrichment by topology-based Root Cause Analysis (RCA). The GUI uses the network topology information to construct network views based on object attribute criteria and hop views based on connectivity information. Because the event management is central to the Netcool suite, operators tend to watch the filtered events and can navigate seamlessly to the maps for contextual information or orientation. Tivoli NetView’s single-server architecture makes it simpler to administer and generally has GUIs for routine maintenance. Netcool/Precision, on the other hand, gains much of its scalability and flexibility from the multi-tiered architecture, and low-level access to data as well as program controls in the form of SQL tables and scripting. It is closely integrated with the other Netcool products as discussed in Chapter 4, “Solution architecture” on page 61. The trend in recent releases is to provide more GUI control to administrative tasks, as seen in the discovery configuration GUI in Precision 3.6. This low-level control also makes it possible to customize the product in the field to handle unique devices or unique network management requirements, things which often require a new release of Tivoli NetView.2.2 Discovery This section provides an overview of network discovery with Tivoli NetView and IBM Tivoli Switch Analyzer (ITSA), followed by a comparison with Netcool/Precision.12 Migrating to Netcool/Precision for IP Networks
  • 28. Tivoli NetViewTivoli NetView discovers and monitors at the layer 3 OSI level using largelystandard MIBs (management information bases). This is a relatively simplediscovery process that builds a network representation based on IP hierarchy.The discovery is fast and continuous: a new node discovery poll, by default, runsevery 15 minutes. The main NetView process that handles layer 3 discovery isnetmon.Tivoli NetView supports specific technologies such as Cisco HSRP, ISDNfailover, Cisco PIX Firewall failover, and unnumbered serial links. The netmondaemon automatically creates objects for subnets, segments, nodes, andinterfaces in both the Object database (ovwdb) and the Topology database(ovtopmd). The subnet and segment container model is automatically based onIP addresses and the corresponding subnet masks. Netmon issues SNMP trapsfor all topology changes on each object.You can scope the Tivoli NetView network discovery based on IP address,hostname, and device type. For SNMP access, you can either provide a list ofalternate community names for netmon to try during discovery, or configure thecommunity names per node or IP address range. Tivoli NetView maintains anSNMP configuration database that is used by other Tivoli NetView applicationsfor SNMP queries.IBM Tivoli Switch Analyzer (ITSA) is a closely integrated product used todiscover, monitor, and visualize the layer 2 level. Layer 2 requires asophisticated process to build the layer 2 connections and model the VLANs.ITSA has basic support for switches through the standard Bridge MIB andprovides VLAN support for Cisco devices. ITSA holds the layer 2 topology inmemory, which requires a full layer 2 discovery on every restart. ITSAreschedules a full layer 2 discovery typically on a daily or weekly basis.Tivoli NetView can also discover and monitor services available on the network,based on port sniffing or custom tests using the Servmon daemon. Thiscapability is not in Netcool/Precision, but monitoring network services can beaddressed with Netcool/OMNIbus Application Service Managers (ASM) andTivoli Application Dependency Discovery Manager.Netcool/PrecisionNetcool/Precision itself is the approximate equivalent to netmon in TivoliNetView. It discovers the network devices, queries for layer 2 and 3 information(including specialized technology information), and then builds the connectionsbetween objects, both intra-node and network connections. Depending on thedevice, Netcool/Precision can gather a wide variety of information primarily bySNMP, but telnet/ssh can also be used. Netcool/Precision’s discovery time is Chapter 2. Product review 13
  • 29. therefore more comparable with ITSA than Tivoli NetView’s layer 3 only discovery. Netcool/Precision’s discovery process consists of regularly scheduled full network discovery passes along with the ability to incrementally add new nodes later triggered by SNMP traps received, such as Warm Start/Link Up. With IBM Tivoli Switch Analyzer (ITSA) the application does not generate status events while ITSA is processing a layer 2 topology discovery. Unlike ITSA, however, the layer 2 topology of Netcool/Precision remains in use by the application until the next full discovery has completed, whereupon the new discovery information becomes available. As with Tivoli NetView, you can scope the discovery by IP address and further filter devices by SNMP sysObjectID. Netcool/Precision can use ping spray to find nodes within subnets, or use a list of seeds, or both. You can configure the Netcool/Precision discovery to try a set of alternative community names and associate the list by IP address range, or associate specific community names per IP address. Netcool/Precision supports a much wider range of devices and technologies than Tivoli NetView does, as the list in Figure 2-1 shows. In addition to supporting Cisco, Juniper, Nortel, and Alcatel for layer 2, it also supports technologies such as MPLS, ATM (ILMI & PNNI), Cisco Frame Relay and static NAT. There are some technologies Tivoli NetView supports that Netcool/Precision does not at this time, specifically unnumbered serial links and Cisco PIX firewall failovers out of the box.14 Migrating to Netcool/Precision for IP Networks
  • 30. Figure 2-1 Precision agents for device support Chapter 2. Product review 15
  • 31. By default Netcool/Precision does not send events for topology changes in the network like Tivoli NetView does, but you can configure it to send events when new nodes are found. Just like ITSA, Netcool/Precision’s topology-based Root Cause Analysis (RCA) needs to know the path back to the Point of Reference, normally the Netcool/Precision server. If there is an undiscovered router or undiscovered WAN network along that path, topology-based RCA will be affected due to the gap created by the undiscovered devices. Tivoli NetView is able to use a custom link to bridge the gap for ITSA and similarly with Netcool/Precision you can create an artificial link.2.3 Monitoring This section compares how network device monitoring is done on Netcool/Precision and Tivoli NetView. This includes polling, availability status, root cause and impact determination. Tivoli NetView Tivoli NetView actively polls all managed network interfaces at regular intervals. The intervals can vary based on IP address or SmartSet. The poll can be ICMP or SNMP (adminStatus and operStatus from the Interface table). The IP Status attribute for each interface is set depending on the result of the poll. Status for higher order objects, such as node, segment, and subnet, are propagated from the interface and are persistent. Netmon issues SNMP traps for each status change on an object to inform both the network management operator and other applications, such as the maps. Tivoli NetView calculates root causes. At the layer 3 level, Tivoli NetView’s Router Fault Isolation (RFI) algorithm determines the root cause and issues a trap for the causal router or node. If the problem is with a router, the Tivoli NetView program issues a Router Status trap and calculates the impact. Subnets and routers in the impacted partition are set to the Unreachable status by netmon. Netmon has an option to suppress generating critical events for nodes in unreachable areas (the default). However, some users consider those critical events important so they can do their own event correlation in TEC for impacted services and trouble ticket prioritization. ITSA provides layer 2 monitoring and root cause. ITSA can monitor the switch ports actively and also listens for status traps from Tivoli NetView, which prompt it to begin the algorithm to determine the root cause at the layer 2 or 3 levels. Tivoli Switch Analyzer discovers the ports of layer 2 devices and integrates this16 Migrating to Netcool/Precision for IP Networks
  • 32. information into the known layer 3 topology, creating a complete layer 2 andlayer 3 network topology. In addition, Tivoli Switch Analyzer creates a networksegment for each port to represent the connection between the port and thedevices connected directly to it. This means that correlation can be to a switchport, rather than a device downstream from that port. The Tivoli Switch Analyzercorrelator is a process that uses this integrated topology to determine the rootcause of a network outage, either confirming the Tivoli NetView RFI result (atlayer 3) or identifying a layer 2 root cause. ITSA issues SNMP traps to alert thesystem management operator of root cause changes. Tivoli NetView changesthe maps to reflect port status changes on switches. Note that the ITSA rootcause algorithm and events are independent from the netmon RFI algorithm.This can result in redundant events, which can then be correlated in TivoliEnterprise™ Console or Netcool/OMNIbus.Completely separate from availability monitoring, Tivoli NetView can monitorSNMP MIB variables for threshold triggers using the SNMP Data Collector(snmpcollect). Threshold and Rearm SNMP traps are issued, but do notcontribute to the map status for the object, unlike in Netcool/Precision.Netcool/PrecisionNetcool/Precision has a highly integrated monitoring capability coupled withtopology-based Root Cause Analysis (RCA) that does a nice job of signalling theproblem events versus the symptom events from any suitable source based onthe discovered layer 2 network topology and intra-device containership.The Netcool/Precision IP component AMOS performs topology-based RootCause Analysis (RCA). It does this by correlating events with each other, andwith the network topology, to determine which ones are the root causes, andwhich are symptoms that disappear when the root cause is resolved. BecauseAMOS knows how devices in the network are connected, it can use a techniquecalled downstream suppression to determine which devices are temporarilyinaccessible due to other network failures. It suppresses the events on thesetemporarily inaccessible devices. Suppressed events are still visible to the user;however, they are marked as symptomatic, rather than root cause.Active monitoring consists of defining pollers, which can be ICMP or SNMP.SNMP pollers are configured to trigger events when a threshold is exceeded.The pollers and the polling intervals are assigned to classes of devices based onthe Active Object Class structure. This is sufficiently different from how you setup polling frequencies and types in Tivoli NetView that it will require a completereconfiguration for Netcool/Precision.Passive polling consists of listening for SNMP traps (Link Down and others) andsyslog events. These events are automatically enriched with topology Chapter 2. Product review 17
  • 33. information and feed into topology-based RCA just like the active monitoring events. The color of map symbols represents the severity of problem events for the device or devices represented by the symbol. Because events represent more than just availability problems, this is a useful state of health indication. There are six severity states based on the events, with the most severe being propagated up to the container object. Unlike Tivoli NetView, Netcool/Precision does not maintain status fields for objects. Instead, current and historical status can be seen by clicking the object to see a filtered list of events providing you with the current state of the device. Because of the richness of the events, operators typically create filtered event lists to cover the environment they are interested in. This is analogous to the SmartSet submaps in Tivoli NetView. From these event lists the operator can easily jump to the topology map views in context to examine the environment of the problem and the possible impact. You can create Netcool GUI Foundation (NGF) views, complete with background maps, that consist of symbols representing, for instance, each of the data centers. These symbols, including artificial connection symbols, reflect the most serious status represented by filtered events for that symbol. There are no diagnostic tools, equivalent to NetView’s demandpoll, that query the SNMP MIB on the device and update the management database with changed information. However, there are many real-time tools that allow the operator to learn the current state of a device and its underlying technologies for diagnostic purposes, such as Ping, Trace Route, Whois, DNS, and Cisco and Juniper tools. There is no capability to unmanage devices from the GUI out of the box. This can be achieved by running an OQL command to update the polling.suspended table.2.4 Network visualization This section compares the typical map usage in Tivoli NetView and Netcool/Precision. Tivoli NetView By default, NetView displays a hierarchical set of submaps for the IP layer 3 network. The symbols can differentiate device types by their shape and image. See Figure 2-2 on page 19 for an example. The symbol color represents one of 918 Migrating to Netcool/Precision for IP Networks
  • 34. status states of that symbol. Status from the interfaces is propagated up the hierarchy depending on context and the algorithm you select.Figure 2-2 NetView IP Network hierarchy In addition to the IP Internet hierarchy, there are submaps to visualize dynamic SmartSets. The SmartSets consist of a set of objects that match boolean expressions based on object fields. The SmartSet becomes an object that can also be used in other parts of NetView to define SNMP parameters and SNMP data collections, and for event filtering. The user can also create custom submaps consisting of objects and connections. Typically these ad hoc submaps are manually constructed as physical representations of the network per site, or a custom collection of devices and objects meaningful to the operator. When ITSA is installed, the layer 2 views are available on the Web console only. You can navigate from the regular layer 3 views to the layer 2 views in context. Chapter 2. Product review 19
  • 35. The layer 2 views consist of a physical hop view, point to point view, and a VLAN membership list. There are a number of contextual options available to navigate around the network, perform diagnostics on devices, trigger updates on devices, view details for a device, and observe current status for the device and all its interface objects. This rich source of information available from the map compared with, for example, the event display, is why users often customize heavily and rely on the maps for daily operations. Netcool/Precision Netcool/Precision uses NGF/Webtop for visualizing the network and hosting the MIB Browser. All functionality is controlled by the Security Manager with user accounts, groups, and roles. There are two types of topology views: network views and hop views. Both are available from event lists and topology views in context. When multiple views are available, you are prompted for a selection. Alternatively, you can select from the tree view of all the network views you have created. Network views Network topology views can be created via filters on any attribute. You can partition the network automatically on some attribute, which will create container objects for each variation. Drill into each container to see the devices with the common attribute. For instance, let’s say all devices have a location attribute and fall into one of two locations: New York and Texas. Auto-partitioning on location would yield two container objects, one for each of the locations, as shown in Figure 2-3 on page 21.20 Migrating to Netcool/Precision for IP Networks
  • 36. Figure 2-3 Auto-partitioning views Drilling into the New York container will show all the devices with the location value of New York. Any connections that exist between devices will be drawn. This feature allows the network to be partitioned automatically, or you can create a custom filter for a single view. This is equivalent to the dynamic SmartSet submaps in Tivoli NetView. For example, you could create the following: An auto-partition based on location A single view based on technology - MPLS or OSPF devices per autonomous region A view based on a combination, such as core Cisco switches (based on OID) in New York Chapter 2. Product review 21
  • 37. Any attribute can be used for partitioning or filtering purposes. Unlike SmartSets, these views will show any connections that exist between objects. These views are basically filters, so new devices are automatically added to the appropriate views and little map maintenance is required. Hop views The Hop view, shown in Figure 2-4, shows the selected device and all devices connected to it within a configurable number of hops. It is useful for viewing the impacted area of an outage or state of each network connection on a core network device, for instance. Unlike NetView, Netcool/Precision does not show symbols for interface objects with their status. Instead, the interfaces of a selected device appear in a frame under the Hop view. You can choose to show a layer 2 Hop View or a layer 3 Hop View.22 Migrating to Netcool/Precision for IP Networks
  • 38. Figure 2-4 Hop view showing interfaces2.5 Event management This section describes event management in Tivoli NetView and Tivoli Event Console (TEC) and contrasts the capabilities in the Netcool suite. Chapter 2. Product review 23
  • 39. Tivoli NetView While NetView has a complete event management feature set, TEC is often used as a central manager of managers (MOM) because of its stronger feature set for historical analysis, correlation rules, and event grouping and filtering per operator. TEC ships with a ruleset that has basic correlation of NetView and ITSA events preconfigured into the ruleset. NetView can receive SNMP traps from the network and also generate internal events based on status changes, topology changes, and configuration changes. NetView processes these traps and events using the same standard SNMP format. Each event can be configured with additional information such as severity, category, formatted description, and actions. If NetView is installed within a Tivoli Framework environment, the events can be exported to a relational database. Users sometimes build custom applications or tools to parse the trapd.log as a convenient way of processing traps further. Within NetView you can view events, correlate them using complex rulesets, trigger actions, and forward them as SNMP traps to other managers, such as TEC. Events are persisted on disk and NetView can receive SNMP traps from other network devices. However, users typically forward important events to a central TEC server where event management is stronger. In TEC, you can correlate events against those from other products, group and filter events per operator, automatically clear events, automate notifications and other actions. From TEC you can also launch the NetView Web console to view devices in context with the event to get more details on the device and perform diagnostics, and view other devices in the vicinity to determine the cause and impact. NetView has a single configuration file for handling SNMP traps. Here you can specify additional information such as severity and category, format the event description to include varbind information, trigger actions and notifications, and whether to forward to other event managers, including TEC. NetView has a graphical ruleset builder that you can use to build complex rulesets based on correlation and time sequencing. A default ruleset filters events and forwards them to TEC. Netcool/OMNIbus Netcool/Precision is one management application among several that feed events to Netcool/OMNIbus. Each application is called an event source. Your solution may include many event sources, including a number of Netcool suite components such as Netcool/OMNIbus Internet Service Monitors (ISM), Application Service Monitors (ASM), and System Service Monitors (SSM); Netcool/RAD; and other Event Management Systems. Other Netcool components exist to enrich events received by Netcool/OMNIbus, such as Netcool/Precision’s topology-based Root Cause Analysis (RCA), which adds24 Migrating to Netcool/Precision for IP Networks
  • 40. topology information and calculates root cause information to classify events into either problem or symptom categories. Netcool/Impact is another product that enriches events with information potentially from any existing data source, as shown in Figure 4-1 on page 62. Each Netcool/OMNIbus installation must have at least one ObjectServer to store and manage alert information. The events are viewed in event lists in Webtop according to the configuration and filtering for each user. Since the current status of devices is reflected in the event database, you can construct event lists to monitor the health of specific areas, mimicking the functionality of Tivoli NetView’s SmartSet submaps. The event lists are a central place for the operator to access a rich source of information. A right click takes you to any topology view in context, where you can see the relationship of the device in the network, access a wealth of stored information about the device, and use a wide variety of real-time focused SNMP tools to diagnose the problem further. Probes connect to an event source, detect and acquire event data, and forward the data to the ObjectServer as alerts. Probes use the logic specified in a rules file to manipulate the event elements before converting them into fields of an alert that is sent to Netcool/OMNIbus. The mttrapd probe receives and feeds unsolicited traps from the network into Netcool/OMNIbus. Using Netcool/OMNIbus, you can configure the same actions on traps that were set in Tivoli NetView, such as E-mail/pager notifications and executing scripts. During the transition period, if you have a TEC server, you may want to continue using it for central event management if you are moving network management to the Netcool suite while maintaining a Tivoli server management solution. Just like Tivoli NetView, Netcool/OMNIbus can forward events to TEC. There is a white paper and configuration files for Tivoli and Netcool Event Flow Integration available in the IBM Tivoli Open Process Automation Library: SNMP tools This section describes SNMP data collection capability in Tivoli NetView and contrasts the capabilities in the Netcool suite. Tivoli NetView NetView provides a set of SNMP tools. These tools all use the central SNMP configuration database for community names (with the exception of the Web Chapter 2. Product review 25
  • 41. console MIB Browser). NetView 7.1.4 supports SNMPv1 for all functions except the MIB browsers, which support SNMPv2 as well, while version 7.1.5 has a new SNMP library that extends support to SNMPv2 in general. SNMP data collection Tivoli NetView includes an application that can be configured to collect SNMP data, store it, and trigger threshold events. The data is typically stored in proprietary flat files and users often write custom applications to access this data to augment reporting. Users can view the stored data in both tabular and graphical format from the NetView native console. If NetView is installed in a Tivoli Framework environment, the data can be exported to a relational database for easier custom access and reporting. NetView can store collected data in Tivoli Data Warehouse (TDW) v1.3, if it is installed. However, it is left to the user to create reports from TDW. You can configure each data collection to store the data, or evaluate against threshold and rearm values, or both. If a threshold is triggered, the snmpcollect daemon issues a NetView threshold event. SNMP data can be collected and displayed in a real-time graph that is useful for diagnosing or evaluating an ongoing network problem. NetView 7.1.5 introduced a new SNMP Collector that stores data in DB2® and can handle SNMPv2 including 64-bit counters. SNMP MIB browser In NetView 7.1.4, there are two native MIB browsers – one for SNMPv1 and the other for SNMPv1/v2. Each has its own MIB loader. These browsers can be launched in context from the map, and will use the centralized SNMP configuration data for access. The Web console has a Java™ MIB browser (SNMPv1/v2) that has a built-in loader on startup. It does not share the centralized SNMP configuration. SNMP MIB Application Builder NetView also includes a graphical tool to create small custom applications that collect and display specific SNMP MIB variables or tables. These SNMP MIB Application Builder applications are then available from the native console menu to be run in context with selected devices. SNMP command line tools The standard snmpwalk, snmpget, snmpgetnext, snmpset, and snmptrap commands are available from the command line. These commands, if not overridden, will use the community names from the centralized SNMP26 Migrating to Netcool/Precision for IP Networks
  • 42. configuration. In NetView 7.1.5 an additional set of equivalent commands are available that also support SNMPv2. Netcool/Precision During discovery Netcool/Precision determines and stores the SNMP community names for each device, including any SNMPv3 authentication settings. These settings can then be used transparently by the SNMP MIB Browser in the Netcool GUI Foundation (NGF) or overridden if necessary. The MIB Browser is available as a Netcool/Precision application in NGF. It uses the SNMP access data provided centrally by Netcool/Precision and you can perform SNMP walks, SNMP gets, and SNMP get tables (no SNMP sets). The MIBs are loaded automatically; there is no separate process to load them once they reside in the MIB directory. Netcool does not provide command line versions of the SNMP tools. There are custom MIB Browser diagnostic tools available from the topology maps. These tools are equivalent to Tivoli NetView’s MIB Application Builder tools. They gather and display specific MIB data in context and can be extended to include custom tools. As we saw in the Monitoring section, Netcool/Precision incorporates threshold monitoring as an integral part of its network polling. The Netcool/Proviso product is designed for heavy duty performance metric collection and analysis - Netcool/Precision does not have an equivalent function to gather and store SNMP data or a real time graph for MIB variables at this time.2.7 Diagnostic tools This section describes the diagnostic tools available in Tivoli NetView and contrasts the capabilities in the Netcool suite. These tools typically access data in real time rather than rely on previously collected data. They enable you to quickly explore connectivity, configuration, and performance information while diagnosing a problem. Tivoli NetView The Tivoli NetView native console has a number of menu-driven options in context for diagnosis: Connectivity tests using ping, Quicktest/Demandpoll, Locate Route UNIX® commands such as netstat Custom SNMP MIB-based graphical or tabular reports Chapter 2. Product review 27
  • 43. In addition, you can create custom reports based on command line output shown in the appmon display window, or using the SNMP Application Builder. The Web console includes the following options: Connectivity tests using ping, Quicktest/Demandpoll, Locate Route Canned SNMP MIB-based tabular reports on system and networking data, including basic MPLS data and layer 2 forwarding data Custom reports can be added using HTML or text-based output from applications or commands run on the NetView server. Netcool/Precision Netcool/Precision has a wide variety of diagnostic tools and reports available from right-click menus, as shown in Figure 2-5 on page 29. These WebTools include: Ping, including a subnet ping Traceroute DNS lookup Whois lookup A set of Cisco tools A set of Juniper tools The Cisco and Juniper reports require telnet access to the devices and run native commands to display information such as routing, BGP, OSPF, MPLS, ISIS, Cisco ping, and so forth. Custom menu items can be added that use a cgi-based script. As an example, we added the SNMP MIB browser similar to the NetView SNMP Application Builder.28 Migrating to Netcool/Precision for IP Networks
  • 44. Figure 2-5 Right click diagnostic tools2.8 User consoles This section describes the consoles available in Tivoli NetView and contrasts the capabilities in the Netcool products. Tivoli NetView NetView supports two different consoles: an X-based native console on the NetView server and a Web-based Java console for remote access. Chapter 2. Product review 29
  • 45. Native console Tivoli NetView has a native console with full functionality for the operator and administrator. The administrator can optionally enable the native security system and implement NetView user security roles for user groups and individuals. The native console can be distributed to other machines as heavy X-based clients. Web console The Web console, as shown in Figure 2-6 on page 31, is an HTTP-based Java console that can run either as a Java application or as an applet in a browser. The proprietary Web server supports users, roles, and scopes, which are independent from the security system of the native console. The Web console is basically an operator or help desk console; it does not provide the administrator functions to control the maps, discovery, or other NetView configuration tasks. It contains the following components: Submap Explorer Here you see the network topology in tabular or graphical form with a right-hand tree frame for navigation. Object Properties This is a central place to view attribute and event information for an object. Diagnostics This component provides a set of real-time displays for ICMP and SNMP data. MIB Browser This MIB Browser is different from the one available in the native console. Event Browser Read-only display of filtered events.30 Migrating to Netcool/Precision for IP Networks
  • 46. Figure 2-6 Tivoli NetView Web console Netcool suite Netcool/Precision uses the Netcool GUI Foundation (NGF) for a Web-based console. The NGF uses the Netcool Security Manager for single sign-on user accounts for authorization and authentication across products using the NGF. The security system supports users, groups, and roles. For authentication, it can use the native ObjectServer, NIS, or LDAP. The NGF is a common GUI for the Netcool products within a Web browser. TopoViz provides the Netcool/Precision relevant views for NGF, which consist of: Hop views Network views Chapter 2. Product review 31
  • 47. MIB Browser Configuration wizard for discovery Discovery progress and status views Webtop provides the event views, consisting of: Active Event Lists based on customized filtering Light Event Lists based on customized filtering (read-only) Custom portal views (URL-based) At the top there is a drop-down box (Figure 2-7) that contains a list of roles available for the account you logged in under. These roles cover administration tasks and desktop views. For each account you can create home pages with the views for that user. Figure 2-7 NGF roles available for current user2.9 Product administration and configuration Tivoli NetView The initial setup and configuration of Tivoli NetView is done via the installation. After installation the user can expect the product to be running, the initial discovery underway, configuration completed for databases including Tivoli Data Warehouse, if installed, and connection to TEC, if installed. After installation, it may be necessary to modify the best practices applied out of the box for discovery, monitoring, event management, and daemons configurations, using the GUIs mentioned in this section.32 Migrating to Netcool/Precision for IP Networks
  • 48. Tivoli NetView provides a complete set of GUIs and some property files to manage the on-going changes to the product, including: Discovery - Discovery scope changes, SNMP settings, SmartSet definitions Monitoring - Monitor policies Databases - Clear databases, plus CLI utilities for querying and maintenance Daemons - Daemon control and configuration Events - Define varbinds, attributes, actions. A graphical rule builder for correlating traps and taking actions. Maps - Map properties, add/delete/manage/unmanage/acknowledge objects Web Console Security - Set up user accounts and roles Native Security - Set up user accounts and roles Tools are available from the command line to make it easy and safe to perform various administration or setup tasks, including: Dump or query the various data or configuration stores in Tivoli NetView. Register/deregister daemons. Configure traps from MIBs. Netcool/Precision Due to the multi-server architecture possible with Netcool/Precision, Netcool/OMNIbus, and NGF, including the ability to enable hot failover, the installation and deployment needs more planning and design than Tivoli NetView. An understanding of the Netcool architecture, data flow within the product and script programming knowledge of the various rules and configuration files is necessary for a successful deployment. Important: It is recommended that personnel with the proper Netcool/Precision experience and education perform the initial installation, customization, and mentoring of other personnel. Due to the solution’s flexibility, many choices will be made during initial customization that can best be done by experienced, trained personnel.2.10 Integration with other products This section describes the typical products that are used with Tivoli NetView and the APIs available to 3rd party integrators and compares this with Netcool/Precision. Chapter 2. Product review 33
  • 49. Tivoli NetView NetView has strong capabilities for products to closely integrate with the GUI (including custom maps and menu options), the event stream, database access, including registering daemons under NetView’s daemon control structure (ovspmd). IBM Tivoli Switch Analyzer (ITSA) This is an integrated application that provides layer 2 management for NetView. It is integrated into the Web console maps, and updates NetView’s database with status updates for switches. Status events are fed into NetView’s event stream. These events can be forwarded to TEC, where NetView rules correlate them with layer 3 events and provide clearing functions. ITSA makes use of NetView’s public APIs to download the layer 3 topology and update NetView’s object database with status changes. Tivoli Enterprise Console (TEC) Many NetView customers implement TEC as a manager of managers with NetView as one of the event feeds. If TEC is integrated, NetView will provide network-related information including service impact events triggered when network outages impact important services (in NetView 7.1.4 these are WebSphere® servers, MQSeries® servers, and DB2 servers). TEC includes a NetView ruleset that correlates network events with events such as ITM heartbeat events, implements housekeeping functions such as clearing events, and correlates layer 2 events with layer 3 events for the same device. IBM Tivoli Monitoring (ITM) Prior to Tivoli NetView 7.1.5, NetView integrated with ITM only to provide service impact events for network outages. NetView queried ITM for important servers and then raised the severity for outage events sent to TEC for those servers. In addition to this, NetView 7.1.5 provides an ITM NetView Health agent to monitor NetView processes and provide a dashboard to the health of the network. Tivoli Business Systems Manager (TBSM) NetView provides network data, including topology, object, and availability events to TBSM using an Intelligent Monitor for NetView. This enables TBSM to augment the business views with network visualization and real-time network outage and impact information. Ciscoworks Users can install and integrate Ciscoworks. This updates the maps to use special Cisco icons for Cisco devices, and menu options to launch the Ciscoworks element managers in context.34 Migrating to Netcool/Precision for IP Networks
  • 50. Application programming interfacesIn addition to being able to register applications with the GUI and the daemoncontrol structure, third party applications can take advantage of the APIs to haveread/write access to the object database (OVwDB API), event stream (OVSNMPAPI), SNMP configuration information (OVSNMPCONF API), and map database(OVW and NVOT APIs) in order to extend the capabilities of the product.Netcool/PrecisionNetcool/Precision includes complete layer 2 and event management capabilitiesout of the box. However, in a TEC environment, especially during the transitionperiod, it may be useful to continue sending events to TEC to take advantage ofthe existing processes and procedures. Refer to 2.5, “Event management” onpage 23 for a reference to the Tivoli and Netcool event flows.Like Tivoli NetView, you can customize the console interface to includecontextual launch points to 3rd party products such as element managers andother diagnostic tools from event and topology views. You can also augment theconsole with CGI-served portlet views by the NGF server or from the Internet.Netcool/Precision includes a powerful Perl API for access to all the data in thevarious components of Precision via Object Query Language (OQL) commands.This is widely used as a scripting language for extending the product, such ascreating new discovery agents, custom reports, and so forth. The NetcoolPrecision IP Discovery Configuration Guide describes how to write or modifyagents and stitchers with which you can augment discovery for specializeddevices or purposes.Netcool/OMNIbus ObjectServerOMNIbus is the heart of the Netcool system. It delivers real-time centralizedmonitoring of complex networks and IT domains. Many customers useNetcool/OMNIbus to manage tens of millions of events daily. The software canbe deployed in a distributed, parallel, or hierarchical fashion to support complexoperations environments that span diverse geographic boundaries.When Netcool/Precision detects faults, the events are processed in the OMNIbusObjectServer, a high-speed, in-memory database that collects events fromacross the infrastructure in real time. Netcool/OMNIbus then eliminates duplicateevents and filters events through an advanced problem escalation engine.Netcool ImpactImpact extends the functionality of the Netcool suite by allowing automation tocorrelate, calculate, enrich, deliver, notify, escalate, visualize and perform a widerange of automated actions by accessing data from virtually any source. Chapter 2. Product review 35
  • 51. Netcool RAD Netcool/Realtime Active Dashboards (RAD) helps business and operations staff understand the complex relationships between business services and supporting technology. It gives organizations advanced, real-time visualization of services and processes in a comprehensive service dependency model.36 Migrating to Netcool/Precision for IP Networks
  • 52. 3 Chapter 3. Benefits of migrating to Precision In this chapter we look at some of the extra features that migrating to Precision will bring to existing NetView customers. This is not designed to be an exhaustive list of the capabilities of Precision, but instead to demonstrate the flexibility of Precision and the Netcool suite. The following topics are included: Full layer 2 discovery Extending your discovery MPLS networks Topological root cause analysis Multiple domains Failover Event enrichment Asset management© Copyright IBM Corp. 2007. All rights reserved. 37
  • 53. 3.1 Full layer 2 discovery NetView is limited in its discovery capabilities in that it will only discover connectivity between devices at Layer 3. It is possible to expand the capability of NetView by adding IBM Tivoli Switch Analyzer (ITSA), but support is limited to Cisco devices. We begin this section by defining what we mean by Layer 2 and Layer 3 discovery, and identifying the differences between them.3.1.1 The OSI seven layer model During the eighties, multiple vendors came together to develop a set of rules that would allow separate products to communicate independent of the platform they were on. The result was what we now call the Open Systems Interconnection (OSI) model. The OSI model consists of seven layers. Each layer of the model is responsible for a function of the communication protocol, and each layer on a device only interact with the layers directly above and below itself. Between devices a certain layer will communicate with its counterpart. Figure 3-1 on page 39 shows a representation of an OSI model.38 Migrating to Netcool/Precision for IP Networks
  • 54. Figure 3-1 The OSI ModelEven though the OSI model was never fully implemented (it was overtaken bythe de facto standard TCP/IP), the way communication is split into distinct layersprovides a simple method of discussing network technologies.Table 3-1 identifies the layers of the OSI model and provides a brief descriptionof the function of each layer and an example.Table 3-1 OSI Model Layer Functions Layer Function Example protocol (7) Application Interact with the end user HTTP, FTP (6) Presentation Encodes and converts data for the MPEG, ASCII application (5) Session Starts, controls and stops communications NetBIOS (4) Transport Providing end to end communication, error TCP, UDP control Chapter 3. Benefits of migrating to Precision 39
  • 55. Layer Function Example protocol (3) Network Logical network addressing, route IP, IPv6 determination (2) DataLink Reliable transfer across links, physical HDLC, Ethernet addressing (1) Physical Transmission protocols 10baseT, 802.11g We are interested in Layers 2 and 3. In the following sections we describe them in more detail. How layer 3 works As mentioned previously, layer 3 looks after the logical network addressing and the path that a packet will take through a network. This is the level at which routers operate. IP addresses these days are usually quotes in CIDR format, which consists of a 32-bit number and subnet mask, for example The subnet mask is used by IP platforms to determine which subnet the device is on; in our example the subnet is Routers determine the path that a packet takes by building up a list of subnets and the “next hop” which is nearest to the subnet. When a packet enters it, the router looks at the IP address of the packet and then performs a lookup on the list. The router then forwards the packet to the correct next hop. This makes a discovery at layer 3 simple: the only information you need to be able to collect is IP address and subnet, and the next hop data from the routers. With this you can build up a whole layer 3 map by linking up all of the next hop information. Problems with layer 3 discoveries There are many problems with layer 3 only discoveries, including: Switches are shown as node devices. There is no connectivity from hosts to switches to routers. It prevents true topological root cause analysis because only layer 3 network relations are known. You get incorrect analysis because you are missing other network relations (non layer 3) that affect root cause analysis. Some devices (such as Cisco 6509s) perform layer 2 and 3 functionality and are not discovered correctly, sometimes appearing as two devices.40 Migrating to Netcool/Precision for IP Networks
  • 56. These problems can be solved by performing a layer 2 discovery to uncover themissing information.Layer 2 discovery difficultiesLayer 2 discoveries are much more complex than those at layer 3 for manyreasons, including: The range of technologies involved. While layer 3 only has IP, layer 2 can include Ethernet, Serial HDLC, Frame Relay, FDDI, ATM, and others. Lack of vendor support for, or poorly implemented, industry standards. Networks often contain “dumb” or unmanaged devices like hubs. As mentioned previously, some modern devices are hybrid layer 2 and 3 machines, making it more complex to analyze their role in the network.It is obvious, therefore, that much more thought must be put into how to discoverand accurately represent a network at layer 2.How a layer 2 discovery worksThere are many ways to approach layer 2 discoveries. The simplest way wouldbe if all devices supported the industry Management Information Bases (MIBs),which can be discovered via SNMP.There are a number of standard MIBs that are useful for layer 2 mappings. Inethernet networks it is common to query the BRIDGE-MIB, for example. Thisreturns the MAC addresses of machines that are directly connected to anethernet.Of course things are not as simple as this in real life. Some vendors do notcorrectly implement industry standards, or they may be incorrectly implemented.There are also problems inherent within standard MIBs—excessive timeouts, forexample—which mean that the information is not always reliable.Luckily the information about the connections often exists somewhere; it is just acase of knowing where to look for it. So in a Cisco network that is running theCisco Discovery Protocol (CDP) it is possible to get information back aboutneighboring devices by querying the CDP MIB. Or you could pull the informationfrom a proprietary MIB on the device and use that to link against neighborinformation.Even if they are correctly implemented, it is possible to retrieve conflictinginformation about links. If you try to discover a Local Area Network Emulation(LANE) connection over an ATM network, it can throw up two neighbors over onelink because one protocol is being tunnelled. In this case the ATM connection Chapter 3. Benefits of migrating to Precision 41
  • 57. would be the true link because it is the physical connection, and the LAN connection is traveling over the ATM. Once the information has been retrieved you then have to decide which has precedence. As mentioned previously, it is possible to retrieve conflicting information about links, so the network management will have to decide which information is likely to be more correct. Layer 2 discovery with Netcool/Precision Netcool/Precision was built with these problems in mind, and was designed to dig deep into the network to obtain the connectivity information from layer 2 upwards. When Netcool/Precision is initiated, it will sweep the network to find out what is out there. It then launches targeted interrogations based on the type of devices it found. The same device may be questioned many times by different Netcool/Precision agents, which means that it will have the most comprehensive information about the device and its neighbors. The results of these interrogations are then pulled together and Netcool/Precision builds the network model from this information. In Netcool/Precision terminology this is called stitching the network. This network model contains all the information in the network - a complete layer 2 and 3 topological map that greatly increases the ability of the network management team to report on network availability, full topology-based RCA, comprehensive monitoring, and asset reporting.3.2 Filling in gaps in the discovery One of the key mantras of discovery is the following: You can’t discover what you can’t see. What this means is that if the network discovery tool cannot “see” or communicate with a device, there is no way it can automatically add that device into the topology model. Discovery is not a magical process. However, it is very common in networks to come across types of devices that do not reply to ICMP (for example, firewalls) or don’t have SNMP access enabled (for example, workstations). Network administrators will still want to manage these devices with polling.42 Migrating to Netcool/Precision for IP Networks
  • 58. Another scenario frequently encountered is when a company has various branch offices that are connected by a third party service provider. The company has no ability to see the internals of the third party network. This means that even in the case that the discovery tool can see across to the branch offices, they will appear as disparate networks in the discovered topology. The problem with this is that it will break the capability of the modeling tool to perform root cause analysis with problems that occur in the network. Netcool/Precision has the ability to overcome these problems inherit with automatic discovery. Administrators can perform the following: insert connections that will connect disparate networks. Create objects to represent non-SNMP devices. Create objects to represent undiscovered devices.3.2.1 Inserting missing connections Networks that connect over a third party provider will often have gaps in their network discovery. Netcool/Precision can fix this by creating a special “stitcher” that runs at discovery time and will create a link between the two networks. Figure 3-2 on page 44 is an example of what a network looks like before and after a missing link has been added. The initial discovery was unable to discover the link between the two 1700 routers because they are connected over a frame relay link. The administrator is able to create a new stitcher which tells Netcool/Precision a port on C1700-2 to connect to a port on C1700-1. When the discovery is rerun the new information in the stitcher is picked up and merged with the rest of the discovery information. The resultant network topology is shown on the right-hand side of Figure 3-2. Chapter 3. Benefits of migrating to Precision 43
  • 59. Figure 3-2 On the left, a network has been discovered with a missing link, which has been added on theright3.3 MPLS networks MPLS technology has grown in many networks in recent years. Initially MPLS was only found with large service providers with massive MPLS core networks offering VPNs to enterprise networks who are looking to take advantage of its QoS and traffic-shaping features. Now you find MPLS in large corporate or governmental networks, leveraging MPLS technology for enhanced security and availability. MPLS represents a number of challenges not present when dealing with pure IP networks. In particular the logical topology of an MPLS network can be distinct from that of the underlying Layer 2 and 3 maps. In response to this there is a need for companies to discover, model, monitor, and visualize these networks. Netcool/Precision can perform all four functions by using special discovery agents to interrogate devices and get information about the MPLS network.3.3.1 Example MPLS discovery Figure 3-3 on page 45 is an example of a typical service provider network, which is offering a VPN service over their MPLS core.44 Migrating to Netcool/Precision for IP Networks
  • 60. Figure 3-3 Layout of a typical MPLS network In Figure 3-3 you can see three types of routers that occur in an MPLS VPN network: the P devices, which are the Provider core devices; the PE or Provider Edge devices marking the edge of the provider network; and the CE or Customer Edge devices, which mark the start of the customer network. Tip: There are numerous resources on line for more information on MPLS networks, in particular on the Web sites of major router vendors. Search for MPLS overview. After a successful MPLS discovery, Precision can produce two types of visualization of the network: an edge view and a core view. We next describe each of these views for Precision.3.3.2 MPLS edge view An edge view will only show the PE and CE devices. The core of the network is represented by a cloud in Figure 3-4 on page 46. The advantage of this method is that customer-related problems tend to happen at the edge of the network and this view allows operators to see these devices without displaying large numbers of unimportant devices. Chapter 3. Benefits of migrating to Precision 45
  • 61. Figure 3-4 MPLS Edge View The operator is able to see at a glance the status of the machines at the edge of the service provider network. Double-clicking the MPLS core cloud icon will automatically bring up the MPLS core view of the network.3.3.3 MPLS core view The core view shows all the devices that are in the service provider’s MPLS network, including P routers. It also has information about the Label Switched Path (LSP) data within the MPLS core. This visualization presents a complete view of the MPLS network so that operators are able to analyze the status of the machines across the network as shown in Figure 3-5 on page 47.46 Migrating to Netcool/Precision for IP Networks
  • 62. Figure 3-5 MPLS Core View3.3.4 More information on MPLS capabilities in Netcool/Precision For more information about MPLS discoveries in Netcool/Precision, consult the Netcool/Precision for IP Networks Discovery Configuration Guide, available online at: Chapter 3. Benefits of migrating to Precision 47
  • 63. 3.4 Topology-based root cause analysis (RCA) Note: Root cause analysis or RCA can have different meanings depending on the context of the problem. For example, a business consultant will apply RCA techniques such as Ishikawa Diagrams to solve business problems. With respect to Netcool/Precision we use RCA only to mean topological root cause analysis, which is using knowledge of the network to establish a single point of failure and identify those events that are symptomatic. Multiple root causes are generated when the network is redundant. Root cause analysis is the process of determining the primary problem within one or more alerts. Netcool/Precision performs topology-based RCA by analyzing the events in the context of the discovered topology. This helps Netcool/Precision to decide which devices may be inaccessible due to other network failures. Alerts that are determined to be symptomatic are suppressed until the root cause event is resolved. Netcool/Precision topology-based RCA is more accurate than NetView’s RFI because Netcool/Precision has better information about the topology of the network. Netcool/Precision knows about the layer 2 links that are transparent to NetView, and therefore can perform a more comprehensive root cause analysis. As an additional bonus, recent improvements have been made to the Netcool Knowledge Library (NCKL) that allow it to perform some root cause analysis functions in Netcool/OMNIbus, before the events are received into Netcool/Precision. This multi-layered approach to event correlation increases the power of Netcool to find root cause events.3.4.1 Netcool Knowledge Library example The Netcool suite as a whole has a multi-layered approach to event correlation, of which Netcool/Precision is one part. One other key module, the Netcool Knowledge Library (NCKL) contains a number of built-in rules that perform some advanced correlation. Figure 3-6 on page 49 shows the different levels of event correlation in Netcool.48 Migrating to Netcool/Precision for IP Networks
  • 64. Figure 3-6 Multi layered approach to event correlationSome events can be pre-classified before they reach the ObjectServer. Forexample, a power outage will be flagged as a root cause event because it will nothave a related upstream event, and certain OSPF events will always besymptomatic because they it will always have an associated root cause.NCKL can also perform basic intra-device correlation with events in theObjectServer, for example, when a sub-interface event is caused by a fault onthe parent interface. This is illustrated in Figure 3-7 on page 50. Chapter 3. Benefits of migrating to Precision 49
  • 65. Figure 3-7 Example intra-device correlation in NCKL3.4.2 Netcool/Precision root cause analysis Since Netcool/Precision contains full topological information about the network, it is able to suppress symptomatic events that occur in the network. This can be represented either on the event list, where suppressed events appear in a separate color (purple by default), or on the map view. Figure 3-8 on page 51 shows a map view when topology-based RCA has isolated the root cause problem in a network.50 Migrating to Netcool/Precision for IP Networks
  • 66. Figure 3-8 Precision map view after RCA Figure 3-9 on page 52 shows the related event list for the root cause analysis. Chapter 3. Benefits of migrating to Precision 51
  • 67. Figure 3-9 The event list for Precision RCA (colors adjusted for B/W printing)3.5 Multiple domains Netcool/Precision can manage a number of distinct multiple networks, or split a large network into smaller administrative areas. These separate areas are called domains within Netcool/Precision. One of the key features of Netcool/Precision is to be able to manage all of these domains from a single user interface and from a single ObjectServer. There are a number of reasons to use multiple domains within a network management tool. We take a look at two scenarios in the following sections: Managed service provider (MSP) Separate administrative areas52 Migrating to Netcool/Precision for IP Networks
  • 68. 3.5.1 Managed service provider (MSP) environments In an MSP environment, one provider will be managing networks for different customers. The MSP has a choice of either discovering the whole network with customers in one domain, or splitting up the discovery into separate administrative policies. Each approach has benefits. The single discovery means that there is less maintenance, but separate views would have to be created for each customer. Polling on a per customer basis would require more effort to implement. Creating one domain per customer would mean that customer maps would be automatically created and per customer polling could be turned on and off or changed without affecting other customers (accidently or otherwise).3.5.2 Distinct administrative areas Even though a company may appear to be one contiguous network, it often makes sense to divide the network up into administrative partitions. This allows per region custom polls and discovery cycles. For example, a company may have offices in America, Europe, and Asia, but the areas are not directly connected (perhaps linked via the Internet or a third party provider). In this case the areas are logically separated, so it might make sense to have a separate Netcool/Precision server in each region. There might also be distinct ObjectServers in each region, which would lessen the amount of traffic sent over WAN links. Figure 3-10 on page 54 shows an example setup for such an environment. Chapter 3. Benefits of migrating to Precision 53
  • 69. Figure 3-10 Sample separate administrative areas deployment with multiple domains3.6 Failover High availability of network management systems is a crucial concern for modern networks. The network management tool is central for monitoring and helping to administer the infrastructure. Failover opportunities in NetView are complex to implement and maintain. Netcool/Precision comes with built-in features for automatic failover for key management elements by the use of Virtual Domains as shown in Figure 3-11 on page 55. When these features are combined with the native redundancy features of Netcool/ OMNIbus and Webtop, it is possible to build a high availability network management suite.54 Migrating to Netcool/Precision for IP Networks
  • 70. Figure 3-11 Sample Precision Failover Architecture We look at the architecture of Precision failover in more detail in 4.5, “Netcool/Precision in failover” on page 75.3.7 Extending your discovery NetView discoveries will only retrieve certain types of pre-defined information like IP address, description, and so forth. This can be restrictive in circumstances where the administrator wants to gather specific information from a device. Adding extra fields to NetView generally requires raising an enhancement request with development. Precision’s open and extensible discovery agents and stitchers allow administrators to gather information that they have an interest in. They may want to retrieve serial numbers from devices to assist with asset management. Extra information about a device’s location or contact details may also be listed. This can help with event enrichment. In 6.3.6, “Discovering extra information” on page 125 we give an example of how to create these extra fields and how to use Precision to populate them at discovery time. Chapter 3. Benefits of migrating to Precision 55
  • 71. 3.8 Event enrichment Network management centers around event management. Based on event information, administrators or network management tools can open trouble tickets, contact affected customers, perform root cause analysis, assess how the business has been impacted, and perform many other functions. The more information that is contained within the event the easier it is to locate what caused the event and who it is impacting. However, when an event is received from a device on the network it typically only has minimal data associated it with. The process of adding additional information to the event is called event enrichment.3.8.1 Event enrichment in the Netcool suite Event enrichment happens at several points within the Netcool suite. When an event arrives at a probe it will often contain only an IP address and an event code, as shown in Table 3-2. Table 3-2 Example of an event received by a Netcool probe IP address Summary Error 773 The first thing the probe will do is examine its rules file to translate the error code into something meaningful. So now the event looks like Table 3-3. Table 3-3 Example of an event received by a Netcool probe IP address Summary Link Failed Once the event is received by the ObjectServer, it can undergo a number of other enrichments, depending on which Netcool components are installed. If Netcool/Impact is present, it would be possible to enrich information with data from third-party databases. For example, you could relate the event IP address to a customer, and then associate that customer information with the relevant SLA. Netcool/OMNIbus could then use this to create a new Trouble Ticket. The end event might look like Table 3-4 on page 57.56 Migrating to Netcool/Precision for IP Networks
  • 72. Table 3-4 Example of an event after enrichment by Netcool Impact IP address Summary Customer Contact SLA London MajorBank John Smith 10:35pm branch down Corp (Gold) (1- 555-1234) Compare this event to the one that initially arrived into the Netcool probe; within seconds of the event being received into the Netcool system the administrators and engineers have valuable information to help them analyze the problem.3.8.2 Event enrichment in Netcool/Precision Netcool/Precision gathers a lot of extra information about devices and topology in the network. You can use this information to enrich events with data such as system location, contact information, product serial number, and so forth, as shown in Figure 3-12. We go into this in greater detail in 6.6.2, “Event enrichment” on page 165.Figure 3-12 Example of event enrichment with Precision3.9 Asset management Almost as a by-product of the in-depth network discovery of Netcool/Precision, it pulls back comprehensive information about network infrastructure that can be used as asset management information. This information can be vital to IT departments that are tracking the location and movement of assets within a company. Chapter 3. Benefits of migrating to Precision 57
  • 73. 3.9.1 Basic asset information in standard installation By default, all asset information obtained by Netcool/Precision is readily available in the central model. This information can be passed to the Netcool/Precision Web GUI, where it can be viewed by operators, as shown in Figure 3-13. Figure 3-13 Asset information viewed in the Precision GUI The information can also be viewed directly from the Netcool/Precision central database with Netcool/Precision OQL (similar to SQL) queries. This is useful if administrators want to run a quick check on equipment, or they can integrate it into their own scripts.58 Migrating to Netcool/Precision for IP Networks
  • 74. Example 3-1 shows a snippet from a query to obtain the name, operating system, and serial number from all devices. Example 3-1 Simplified output from a simple query of Precision asset information |host>select EntityName, Description, ExtraInfo->m_ciscoSerial from master.entityByName where EntityType=1;" ( 11 record(s) to follow ) { EntityName=S2900-1; Description=Cisco Internetwork Operating System Software IOS (tm) C2900XL Software (C2900XL-C3H2S-M), Version 12.0(5)WC16, RELEASE SOFTWARE (fc1) Copyright (c) 1986-2006 by cisco Systems, Inc. Compiled Thu 21-Sep-06 13:00 by antonino; m_ciscoSerial=0x0E; } .....3.9.2 Netcool for Asset Management - NfAM Event though it is possible to extract all the asset information with your own OQL queries, this can become repetitive. This information can be placed directly into a third-party system. A few years ago Micromuse recognized the value of a pre-built product with standardized reports and a standards-compliant database structure. This would save administrators time and effort replicating what had been done elsewhere. The resulting product is Netcool for Asset Management or NfAM. NfAM automatically takes the information contained within the Precision model and puts the information into out-of-the-box reports. Some of the reporting capabilities of NfAM include: Networks and Devices (both quantity and type) Device non-compliance Stranded assets Location of unauthorized equipment within the network Address space utilization Figure Figure 3-14 on page 60 is a sample report screen from NfAM showing the status of interfaces for each node discovered. Chapter 3. Benefits of migrating to Precision 59
  • 75. Figure 3-14 NfAM view showing operational status of interfaces on discovered nodes60 Migrating to Netcool/Precision for IP Networks
  • 76. 4 Chapter 4. Solution architecture This chapter presents an overview of the Netcool suite and a detailed description of the architecture of a Netcool/Precision solution. The following topics are covered: Overview of the Netcool architecture Architecture of Netcool/Precision Detailed look at the components of Netcool/Precision© Copyright IBM Corp. 2007. All rights reserved. 61
  • 77. 4.1 Netcool overview Figure 4-1 shows the architecture of the Tivoli Netcool components. It shows how the different pieces of the suite interconnect with each other and with third-party products.Figure 4-1 Netcool architecture Before we get into a detailed discussion of Netcool/Precision itself, we spend a few moments looking at the other components that make up the Netcool Suite.4.1.1 Netcool OMNIbus ObjectServer OMNIbus is the heart of the Netcool system. It delivers real-time centralized monitoring of complex networks and IT domains. Many customers use Netcool/OMNIbus to manage tens of millions of events daily. The software can be deployed in a distributed, parallel, or hierarchical fashion to support complex operations environments that span diverse geographic boundaries.62 Migrating to Netcool/Precision for IP Networks
  • 78. When the software detects faults, they are processed in the OMNIbus ObjectServer, a high-speed, in-memory database that collects events from across the infrastructure in real time. Netcool/OMNIbus then eliminates duplicate events and filters events through an advanced problem escalation engine.4.1.2 Netcool probes and monitors Netcool probes actively collect business and technology events from thousands of sources in real time. These lightweight agents and applications look for events and traps, and monitor network devices across the business. You can also develop and customize Netcool probes to support virtually any kind of “event,” such as those generated by proprietary business applications. In addition to the Netcool probes, you can deploy monitors from the IBM Tivoli Monitoring family that integrate with Netcool/OMNIbus to proactively measure user experiences and performance across applications – and generate alarms based on thresholds you establish.4.1.3 Netcool/Impact Netcool/Impact extends the functionality of the Netcool suite by allowing automation to correlate, calculate, enrich, deliver, notify, escalate, visualize, and perform a wide range of automated actions by accessing data from virtually any source.4.1.4 Netcool/RAD Netcool/Realtime Active Dashboards (RAD) helps business and operations staff understand the complex relationships between business services and supporting technology. It gives organizations advanced, real-time visualization of services and processes in a comprehensive service dependency model as shown in Figure 4-2 on page 64. Chapter 4. Solution architecture 63
  • 79. Figure 4-2 Example of a Realtime Active Dashboard4.1.5 Netcool/Reporter Netcool/Reporter complements the Netcool/OMNIbus application by capturing, analyzing, and presenting event data generated over various time frames. Increasingly, IT managers want long-term, retrospective information about the behavior of devices, links, and services within their networks. Netcool/ Reporter supplements the real-time information provided by Netcool/Webtop with historical and trend information by capturing, analyzing, and presenting data generated over time into meaningful reports.4.1.6 Netcool/Webtop Netcool/Webtop delivers graphical maps, tables, and event lists to the remote operator via HTML and Java. Netcool users can also manage Netcool alerts with the Netcool/Webtops flexible interface and advanced management capabilities. The Netcool/Webtop application extends Netcool/OMNIbus capabilities by adding a new set of graphical views as well as flexible management and64 Migrating to Netcool/Precision for IP Networks
  • 80. administrative functions. This Web-enabled interface allows monitoring and viewing of high volumes of management data from Netcool/OMNIbus.4.2 A first look at Netcool/Precision In this section we take a closer look at Netcool/Precision itself. We start by identifying the components that come with Netcool/Precision, then look at how these components link together, and conclude with a look at how an example event flows through the system.4.2.1 Netcool/Precision components If you look at the /bin directory of a Netcool/Precision installation, you will see more than twenty-five executables. In this section we describe the most commonly used executables and their functions (Table 4-1). Table 4-1 Netcool/Precision components, executables, and description Component Executable Description DISCO, ncp_disco Controls the network discovery. Agents and helpers ncp_agent Kicks off the relevant agents, ncp_dh_* helpers and stitchers to build the topology model. MODEL ncp_model Contains Precision’s current topological model. MONITOR, Timed ncp_monitor Parent monitoring process. Poll Agents ncp_m_timedstitcher AMOS ncp_f_amos Performs root cause analysis on events related to the discovered network. Event Gateway ncp_ncogate Passes events between Precision and the ObjectServer. Monitor Probe nco_p_ncpmonitor Feeds events from Precision into the ObjectServer. CLASS ncp_class Dynamic library management system responsible for managing the AOCs. OQL (Object Query ncp_oql Provides a user interface to the Language) OQL databases. Chapter 4. Solution architecture 65
  • 81. Component Executable Description CTRL ncp_ctrl Starts, stops and controls precision processes. STORE ncp_store Persistent storage engine for all Precision information. MySQL mysql.server Stores network topology information for access by Topoviz. Topoviz - The Precision web JAVA GUI. Model to MySQL ncp_model_to_mysql Responsible for populating MySQL from Model. Rendezvous Bus rvd ; rva Inter component communication.4.2.2 Inter component communication The “glue” that holds all the components together is the TIBCO rendezvous bus. When a component starts it will register itself as a primary on the bus, so that all other components know that it exists and can communicate with it. Figure 4-3 on page 67 shows the main components linked on a TIBCO bus.66 Migrating to Netcool/Precision for IP Networks
  • 82. Figure 4-3 Inter component communication on the TIBCO Rendezvous bus By default the TIBCO bus only listens for connections on the local machine, but it can be configured to listen across a subnet or multicast domain. Further information can be found in the Netcool/Precision Installation and Deployment Guide.4.2.3 Precision services and OQL A very powerful feature of Netcool/Precision is the ability to interface directly with the underlying components via the Object Query Language (OQL), an SQL-like interface. Many of the Netcool/Precision components have an associated database, or “service.” These databases contain information on many aspects of Netcool/Precision, for example, which discovery agents are still waiting for information from the network, which components are running under control, and so forth. The powerful aspect of this is that an experienced administrator can check whether Netcool/Precision is functioning okay, and take steps to troubleshoot and fix low level problems. You can also directly interface with the network model to create or change links, or script commands to pull out asset information. Chapter 4. Solution architecture 67
  • 83. Service startup When one of the components is started, one of the first things that it will do is start its own service by reading in a file from the $NCHOME/etc/precision directory. It will read its own database schema file, which tells the database which tables and fields it contains. It will then read in files that initiate the database. Once started, these fields will change and can be written back to a file. As an example, let us take a look at how CTRL does this. When CTRL starts up, it will read the CtrlSchema.cfg file. This will create the tables and fields that CTRL uses to operate, for example the database service, then the tables services.inTray and the fields within like serviceName and servicePath. Once the database is created, it then reads in our domain-specific file CtrlServices.REDBOOK_P.cfg. This file contains information on how to populate the fields on startup. In this example the file contains information to start up all the precision components. Using OQL to query the services Once the service is up and running, you can use OQL to log into the database and have a look at its contents. Anyone who is familiar with SQL will have no problem with OQL queries since the syntax is very similar. For more information on using OQL to perform queries, refer to the Tivoli Netcool Precision IP Discovery Configuration Guide.4.3 Event flow through Netcool/Precision Figure 4-4 on page 69 depicts the architecture of Netcool/Precision and demonstrates the event flow through the system.68 Migrating to Netcool/Precision for IP Networks
  • 84. Figure 4-4 Precision Components showing flow through the product You can see an ObjectServer in Figure 4-4; all instances of Netcool/Precision are deployed with an ObjectServer because it is central to the handling of event and polling information from the network. Netcool/Precision might appear to be a complex product because of the number of different components that it contains, but it is this modularity that gives it its power and flexibility. In the following discussion, we describe the flow of the event as it arrives from the network into the Netcool system. Chapter 4. Solution architecture 69
  • 85. Discovery 1. Netcool/Precision interrogates network devices using a variety of protocols, including SNMP, ICMP, and command line tools. 2. Information gathered in step 1 is passed to the Netcool/Precision auto-discovery engine, named “DISCO.” 3. The auto discovery engine takes all the information and “stitches” the information into a network topology. This topology is then passed across to a central topological repository “MODEL.” 4. The root cause analysis engine “AMOS” contains its own version of the topology which is populated from MODEL. MODEL also sends the topology to the Event Gateway so that the gateway can enrich events in the ObjectServer with topological information. Monitoring 5. Netcool probes receive events from the network. 6. The events are passed to the ObjectServer. 7. MONITOR starts and kicks off the necessary polling agents, which periodically test for items like network connectivity and SNMP threshold breaches. 8. If a problem or problem resolution is detected, the polling agents will generate one or more events and send them to the monitor probe. 9. The monitor probe puts the event into the standard format and passes it up to the ObjectServer. 10.The ObjectServer performs event correlation and de-duplication on the events that it receives. The Precision Event Gateway requests a subset of these events that it is interested in (that is, only the events relating to devices discovered by Netcool/Precision). Netcool/Precision can enrich these events with information obtained during discovery. (For more information see 6.6.2, “Event enrichment” on page 165.) 11.The Event Gateway sends these events to AMOS, which performs a root cause analysis and sends back the enriched events to the ObjectServer via the Event Gateway.4.4 Example Netcool/Precision deployments This section looks at how Netcool/Precision might be deployed in different scenarios, from a small scale enterprise to a large managed service provider environment. These scenarios are taken from the Netcool Precision Installation and Deployment Guide.70 Migrating to Netcool/Precision for IP Networks
  • 86. Important: This section does not provide sizing guidelines for Precision. There are many factors that affect sizing, including number of interfaces, type of device, number of polls, bandwidth available, and others. Always consult a Netcool architect before deciding on a deployment solution.4.4.1 Small scale Netcool/Precision deployment The first scenario we look as it a typical small to medium sized environment. This customer might be a national level enterprise with a number of branch offices across the country. It has a few hundred devices across its infrastructure, none of which are particularly large switches or routers. After consulting with a Netcool specialist, it is decided that this scenario can be handled by the following deployment: One ObjectServer Virtual Pair One Netcool GUI Foundation server One Netcool Security Manager One Precision server, single domain, running with failover One licensing server The architecture for this is shown as Figure 4-5 on page 72. Chapter 4. Solution architecture 71
  • 87. Figure 4-5 Simple Precision deployment architecture The actual installations of the Netcool suite are as shown in Figure 4-6 on page 73.72 Migrating to Netcool/Precision for IP Networks
  • 88. Figure 4-6 Host machine allocation for Precision simple deployment with failover capabilities Note: If failover was not necessary, all of these components could be installed into a single server. It is recommended that a server with a minimum of 2 CPUs be used in a small production environment. For a detailed description of the installation steps involved, see the Netcool Precision Installation and Deployment Guide.4.4.2 Large scale Netcool/Precision deployment This example looks at a larger deployment of Netcool/Precision. This customer might be a global enterprise with offices in London and New York, and a head network operations center (NOC) in San Francisco. They want Netcool to manage their global network, and they also want operators to be able to access the Web interface from across the world. Chapter 4. Solution architecture 73
  • 89. The Netcool specialist decides on the following architecture: One ObjectServer and Precision server in London. One ObjectServer and Precision server in New York. One ObjectSever consolidates events from across the globe. Events from London and New York are sent to this server in San Francisco.The NGF can access the topologies, which provides visibility for the Precision GUI. Operators can connect to the GUI from anywhere in the world. Figure 4-7 shows this architecture.Figure 4-7 Large Netcool/Precision deployment architecture Figure 4-8 on page 75 shows how the Netcool components are split across the machines at each location.74 Migrating to Netcool/Precision for IP Networks
  • 90. Figure 4-8 Host machine allocation for Precision large deployment4.5 Netcool/Precision in failover As discussed in the previous chapters, one of the major advantages of Netcool/Precision over NetView is the ability to deploy in a high availability architecture with automatic failover between systems. Chapter 4. Solution architecture 75
  • 91. One of the critical functions of Netcool/Precision is monitoring the network. Customers rely on the constant polling to detect network faults, see changes in the network, and perform root cause analysis. This section looks at how Netcool/Precision handles monitoring failover. Before looking at Netcool/Precision in failover, we briefly talk about using OMNIbus in failover. Then we look at how failover works for Netcool/Precision, and finally, we consider failover at the visualization layer.4.5.1 OMNIbus failover To ensure high availability of the Netcool ObjectServer, it is possible to configure OMNIbus to run as a failover pair. The Primary ObjectServer regularly will sync with the backup ObjectServer. If the Primary fails, the backup will seamlessly take over, as shown in Figure 4-9. Figure 4-9 OMNIbus Failover This works because instead of telling components to connect directly to a specific ObjectServer, it instead connects to a virtual interface on the ObjectServer. This virtual interface automatically sends the connection to the current primary ObjectServer.76 Migrating to Netcool/Precision for IP Networks
  • 92. 4.5.2 Webtop and NCSM failover Each of these parts of the Netcool suite have their own failover methods; however, they are beyond the scope of this book. You can find more details in the relevant manuals, or in the online documentation at: Netcool/Precision failover To obtain failover in Netcool/Precision, a new component is introduced: the virtual domain. It is this component that looks after checking to see which Netcool/Precision domain is currently the Primary. Figure 4-10 is a copy of the architecture of failover in Netcool/Precision.Figure 4-10 Netcool/Precision failover architecture Figure 4-10 contains three domains: a primary, secondary, and a virtual domain. This is analogous to the situation with OMNIbus. The virtual domain allows Chapter 4. Solution architecture 77
  • 93. seamless failover to Netcool/Precision because any connection to the virtual domain is routed to the current primary domain. One of the first things to note is that there is only one instance of DISCO. Because discoveries are carried out infrequently—typically at intervals of days or weeks—it is not considered necessary to have a failover for discovery processes. If failover happens and there is a major hardware fault on the primary machine, it is trivial to kick off a new discovery on the backup machine. The network topology data that is stored in MODEL is shared between the primary and backup. The virtual domain controller ensures that any updates on the primary MODEL are copied across into the secondary. We now look at what happens when the system is functioning correctly, and what happens when problems arise. Netcool/Precision failover: Normal operation Figure 4-11 shows the event flow in Netcool/Precision when the system is functioning correctly.Figure 4-11 Precision failover system in a healthy state The following steps are illustrated in Figure 4-11. 1. CTRL checks that all processes are functioning okay. 2. This information is passed to the Virtual Domain controller, which decides if the system is in good health. In this case everything is okay, so it passes a Health Check resolution to the monitor probe.78 Migrating to Netcool/Precision for IP Networks
  • 94. 3. The monitor probe forwards the okay onto the ObjectServer. 4. The Event Gateway on the primary Netcool/Precision domain picks up the Health Check from the ObjectServer. 5. The Event Gateway passes this to the Virtual domain controller, which checks that the timestamp on the event is within the last five minutes. In this case everything is okay. Also at this stage the Event Gateway is checking for events from the backup server so that the status of both domains is known. 6. The Event Gateway on the backup Precision server also receives the Health Check resolution (same as step 4 but on the backup server). 7. The Event Gateway on the backup passes this to its Virtual Domain controller, which checks that the timestamp on the event is within the last five minutes. The same process happens on the backup server, so the servers are in sync. Netcool/Precision failover: Fault on the primary We now look at what happens when there is a problem on the primary server. This is illustrated in Figure 4-12.Figure 4-12 Precision failover system in a faultily state The following steps are shown in Figure 4-12: 1. CTRL checks that all processes are functioning okay. In this case there is a problem with one of the processes that it manages. Chapter 4. Solution architecture 79
  • 95. 2. This information is passed to the Virtual Domain controller, which decides if the system is in good health. In this case there is a problem, so it passes a Health Check problem event to the monitor probe. It also switches the Primary precision server to backup mode. 3. The monitor probe forwards the fault onto the ObjectServer. 4. The Event Gateway on the primary Precision domain picks up the Health Check from the ObjectServer. 5. The Event Gateway passes the problem to the Virtual Domain on the primary. 6. The Event Gateway on the backup picks up the problem from the ObjectServer. 7. On the backup server the Event Gateway forwards the event to the Virtual Domain controller. The controller switches the backup Precision server to primary mode and takes over monitoring and polling the network. Failing back Netcool/Precision handles automatic fallback to the primary Precision server when the problem is resolved. When the Primary server comes up, the Virtual domain sends a Health Check resolution to the probe, which is passed through the system as described previously. The virtual domain controller on both domains will pick up the resolution and the primary Precision server will resume operation as normal, and the backup will return to its standby mode. Further reading For more information about failover in Precision, including a full description of how to implement failover for large and small architectures, read the failover chapter in the Precision Install and Deployment Guide, available online at: Migrating to Netcool/Precision for IP Networks
  • 96. Part 2Part 2 Migrations In the first part we looked at the architecture and features of the system management products. In this part we look at the specific best practices and processes for moving the management from NetView to Netcool/Precision.© Copyright IBM Corp. 2007. All rights reserved. 81
  • 97. 82 Migrating to Netcool/Precision for IP Networks
  • 98. 5 Chapter 5. Preparing the server for migration and installing the Netcool components This chapter describes the pre-installation system checks that we made to our server as well as the modifications we made to the default installation of the Netcool components.© Copyright IBM Corp. 2007. All rights reserved. 83
  • 99. 5.1 Planning for migration We decided to install and run our products as user netcool, which does not have root permissions. It is important to note, however, that root access is required in order to properly install the Netcool components and perform some of the server configurations. While the Netcool components can be run as a non-root user, certain processes in Netcool must be run as root. Initial creation of UNIX users and some server configuration steps also require root access.5.2 Prepare the new monitoring servers The first step to installing the Netcool software is to prepare the new server and verify some basic operating system settings.5.2.1 Our lab server environment For our test server, we used the following: Red Hat Enterprise Linux AS release 3 (Taroon Update 8) x86 Server with 1 Pentium® 4, 2.4 GHz CPU 1.5 GB RAM5.2.2 Operating system preparation and checks A few basic operating system checks were performed before we began installing the Netcool software. UNIX users and groups We made the following changes to our local UNIX users and groups: A user named netcool was created. A group named ncoadmin was created. We added both root and netcool users to the ncoadmin group. Name service verification Name resolution can affect several aspects of a discovery. Primarily, the naming convention of the discovered devices relies on the system name services for84 Migrating to Netcool/Precision for IP Networks
  • 100. consistency. Misconfigured name services can also have a negative effect on the speed and efficiency of a network discovery. For our lab servers we verified the following naming services: DNS nslookup or dig commands were used to verify a few sample devices in the network. /etc/hosts The lab devices had multiple interfaces. Since our test lab IP addresses were not configured in DNS, the additional device IPs and names were configured in the local /etc/hosts file. Any entries in the /etc/hosts file should match the information in other naming services such as DNS. File space verification Once installed, our Netcool solution used 1.5 GB of disk space. Pre-installation verification of at least 5 GB of free disk space will ensure plenty of space for the Netcool components as well as working space. Changed permissions on /opt We changed the permissions on the /opt directory so that the netcool user had write permissions. This allowed us to run the Netcool installers as a non-root user and create the required subdirectories. Set the LANG environment variable The Netcool components expect that LANG will be set to C. We set this and exported it in /etc/profile. The installation manual for each component includes instructions for the setting of additional environment variables. See Appendix A.1, “Environment settings” on page 228 for the /etc/profile that we ended up creating.5.3 Required Netcool components The following components were used in our deployment of Netcool/Precision: OMNIbus 7.1.0 Precision Webtop 2.0.74 Netcool GUI Foundation (NFG) 1.0.129 Security Manager 1.3.939 Chapter 5. Preparing the server for migration and installing the Netcool components 85
  • 101. Netcool Knowledge Library (NCKL) 1.2 Netcool License Server 1.0.31 Mttrapd probe Note: The Precision IP 3.6 installation is bundled with the NGF and Webtop 2.0. Separate installations for these components is not necessary. In addition to the software installation files, we verified that we had access to all of the Netcool installation manuals and release notes, which are available from: Reviewing the release notes, we verified that all of the components we planned to install were supported on the version of Red Hat running on our servers. We also had the Red Hat media handy in case any additional RPMs were called for, paying particular attention to the JRE™ requirements and browser requirements.5.4 Installation of Netcool components In our lab, we installed all of the Netcool components as a non-root user named netcool.5.4.1 Install and verify Netcool License Server We installed the Netcool License Server as instructed in the Netcool License Server Installation Manual. During the installation, we selected all of the default options. No additional steps were required to do this as a non-root user. After installing the license server, we copied our license file to $NETCOOL_LICENSE_FILE. The top line of the license file is specific to the hostname of the server, so we modified the file to reflect our server’s hostname of precision2 as shown in Example 5-1. Example 5-1 Verifying correct hostname in license file [netcool@precision2 etc]# head -3 AnyHost.lic SERVER precision2 ANY 27000 VENDOR netcool USE_SERVER [netcool@precision2 etc]#86 Migrating to Netcool/Precision for IP Networks
  • 102. 5.4.2 Install and verify Netcool OMNIbus 7.1.0 We installed Netcool/OMNIbus as instructed in chapter 2 of the Netcool OMNIbus v7.1 Installation and Deployment Guide. We used all defaults in the installation. Configuring Process Control (PA) to run ObjectServer as non-root user Since we were running Netcool/OMNIbus as the user netcool, which is a non-root user, we edited $OMNIHOME/etc/nco_pa.conf file so that the ObjectServer is launched as netcool instead of root. These changes can be seen in Example 5-2. Example 5-2 Configuring nco_pa to run Netcool OMNIbus as non-root user Note: nco_process MasterObjectServer { Command $OMNIHOME/bin/nco_objserv -name NCOMS -pa NCO_PA run as 501 Host = precision2 Managed = True RestartMsg = ${NAME} running as ${EUID} has been restored on ${HOST}. AlertMsg = ${NAME} running as ${EUID} has died on ${HOST}. RetryCount = 0 ProcessType = PaPA_AWARE Configuring PAM for OMNIbus For the Redbook we used Red Hat Linux as an operating system. Therefore, we needed to use PAM authentication with Netcool OMNIbus. We performed the following steps to configure OMNIbus’ Process Control so that it uses PAM: 1. We ran a script named nco_install_ospam from the $OMNIHOME/bin directory as seen in Example 5-3. Example 5-3 Installing OS PAM $ cd /opt/netcool/omnibus/bin/ $ ./nco_install_ospam Netcool/OMNIbus 7.1 ObjectServer PAM Installation Ready to install and setup the PAM module. What is the name of the authentication server? [NCOMS] Chapter 5. Preparing the server for migration and installing the Netcool components 87
  • 103. Do you wish to view the updates to be added to /etc/pam.d? (y/n)? [y] y # # PAM Configuration for the Netcool/OMNIbus ObjectServer. # auth required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam account required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam password required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam Do you wish to install these changes? (y/n)? [y] y Installation complete. NOTE: You now need to enable the use of PAM by the ObjectServer itself by setting or adding the property Sec.UsePam : TRUE to the ObjectServers properties file. 2. We edited ObjectServer properties to use PAM. As the Note at the end of the nco_install_ospam script states, we next had to edit our $OMNIHOME/etc/NCOMS.props file so that the ObjectServer would use PAM. The line in the file should then look like the one in Example 5-4. Example 5-4 Modified line in NCOMS.props to use PAM Sec.UsePam : TRUE 3. We created a new PAM file. The nco_install_ospam script created a new file named /etc/pam.d/nco_objserv. There is a known issue, documented in that section of the release notes, that requires the different configuration on Linux servers. This file must be called netcool and it goes in the same directory, /etc/pam.d. The default text for /etc/pam.d/nco_objserv is shown in Example 5-5.88 Migrating to Netcool/Precision for IP Networks
  • 104. Example 5-5 Original text of /etc/pam.d/nco_objserv # PAM Configuration for the Netcool/OMNIbus ObjectServer. # auth required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam account required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam password required /opt/netcool/omnibus/platform/linux2x86/module/pam/omnibus_os/libpam We created a new file as instructed in the release notes, /etc/pam.d/netcool containing the text in Example 5-6. Example 5-6 /etc/pam.d/netcool # # PAM Configuration for the Netcool/OMNIbus ObjectServer. # auth required /lib/security/ shadow nullok account required /lib/security/ use_authtok nullok shadow password required /lib/security/ use_authtok nullok shadowVerifying PA operationOnce this is correctly configured, you can start nco_pad as root and it will startthe objectserver, which will execute as the netcool uid. Then, as the netcool user,you can stop it, start it, and show its status. The results should be as shown inExample 5-7. The password specified is the UNIX login password of the netcooluserid.Example 5-7 Controlling processes under PA control from a non-root user# As root user...[root@precision2 pam.d]# nco_pad -name NCO_PA -authenticate PAMNetcool/OMNIbus : Starting Process Control... [ OK ]Netcool/OMNIbus Process Agent Daemon - Version 7.1 Netcool/OMNIbus PA API Library Version 7.1 Sybase Server-Library Release: 12.5.0Server Settings :Chapter 5. Preparing the server for migration and installing the Netcool components 89
  • 105. Name of server : NCO_PA Path of used log file : /opt/netcool/omnibus/log/NCO_PA.log Configuration File : /opt/netcool/omnibus/etc/nco_pa.conf Child Output File : /dev/null Maximum logfile size : 1024 Thread stack size : 69632 Message Pool size : 45568 PID Message Pool size : 50 Rogue Process Timeout : 30 Truncate Log : False Instantiate server to daemon : True Internal API Checking : False No Configuration File : False Start Auto-start services : True Authentication System : Pluggable Authentication Modules Trace Net library : False Trace message queues : False Trace event queues : False Trace TDS packets : False Trace mutex locks : False Host DNS name : PID file (from $OMNIHOME) : ./var/ Kill Process group : False Secure Mode : False Administration Group Name. : ncoadmin Forking to a Daemon Process............. # As non-root user netcool... [netcool]$ nco_pa_status -server NCO_PA -user root -password xxxx --------------------------------------------------------------------- Service Name Process Name Hostname User Status PID --------------------------------------------------------------------- Core MasterObjectServer precision2 netcool RUNNING 15607 ---------------------------------------------------------------- [netcool]$ nco_pa_stop -server NCO_PA -user root -password xxxxx -service Core [netcool]$ nco_pa_status -server NCO_PA -user root -password xxxx ---------------------------------------------------------------------90 Migrating to Netcool/Precision for IP Networks
  • 106. Service Name Process Name Hostname User Status PID --------------------------------------------------------------------- Core MasterObjectServer precision2 netcool DEAD 0 --------------------------------------------------------------------- [netcool]$ nco_pa_start -server NCO_PA -user root -password xxxx -service Core [netcool]$ nco_pa_status -server NCO_PA -user root -password xxxx --------------------------------------------------------------------- Service Name Process Name Hostname User Status PID --------------------------------------------------------------------- Core MasterObjectServer precision2 netcool RUNNING 15776 ---------------------------------------------------------------------5.4.3 Install and verify Netcool Knowledge Library We installed the Netcool Knowledge Library as instructed in the Netcool Knowledge Library Installation manual.5.4.4 Install and verify Netcool Mttrapd Probe We installed the Netcool Mttrapd Probe as instructed in the Netcool OMNIbus 7.1 installation and Deployment Guide. Modifying the probe to run as non-root user Since we are running the probe as a non-root user, there were a few additional steps that we had to take. The probe needs to open up port 162, which is a privileged port. We added the following line to our /etc/ /opt/netcool/platform/linux2x86/lib We then reloaded the dynamic linker run-time bindings by running: precision2# /sbin/ldconfig Next, we changed the owner of the mttrapd binary to root and set the file for SUID. precision2# cd $OMNIHOME/probes/linux2x86/ precision2# chown root nco_p_mttrapd precision2# chown +s nco_p_mttrapd Chapter 5. Preparing the server for migration and installing the Netcool components 91
  • 107. After making these changes, we were able to start the Mttrapd Probe as the non-root user, netcool. Note: Running the Mttrapd Probe as non-root on other operating systems requires different configurations. Refer to the probe installation manual for exact directions for other operating systems. Modifying mttrapd.props to use NCKL rules Since we used NCKL in our test lab, we edited our $OMNIHOME/probes/linux2x86/mttrapd.props file so that our Mttrapd Probe would use the NCKL rules instead of using the default rules file. This was done by adding the following line to our mttrapd.props file: RulesFile : /opt/netcool/rules/snmptrap.rules Modifying $OMNIHOME/etc/nco_pa.conf We modified our $OMNIHOME/etc/nco_pa.conf file in order to have the Mttrapd Probe start automatically and be managed using PA. Example 5-8 shows the additional modifications to this file. Example 5-8 Adding mttrapd probe settings to nco_pa.conf nco_process MTTrapdProbe { Command $OMNIHOME/probes/nco_p_mttrapd -server NCOMS run as 501 Host = precision2 Managed = True RestartMsg = MTTrapd Probe running as ${EUID} has been restored on ${HOST}. AlertMsg = MTTrapd Probe running as ${EUID} has died on ${HOST}. RetryCount = 0 ProcessType = PaPA_AWARE } # # List of Services # nco_service Core { ServiceType = Master ServiceStart = Auto process MasterObjectServer NONE process MTTrapdProbe NONE }92 Migrating to Netcool/Precision for IP Networks
  • 108. Note: The command for the Mttrapd Probe was set to run as 501, which is the UID for netcool. This ensures that the probe runs as the non-root user instead of the root, which is the default.5.4.5 Install and verify Netcool Security Manager 1.3 We installed Netcool Security Manager 1.3 as instructed in chapter 2 of the Netcool Security Manager 1.3 Installation Guide. For the installation we left all default values and no additional changes were made to Netcool Security Manager.5.4.6 Install and verify Netcool Precision IP 3.6 We installed Netcool Precision IP 3.6 as instructed in the Netcool Precision for IP Networks 3.6 Installation and Deployment Guide. The Precision IP installation also installs the Netcool GUI Foundation server, as well as Webtop 2.0. These products do not need to be installed separately. Selecting domain name for Precision server For the Redbook, we decided to name our Precision server REDBOOK_P as shown in Figure 5-1. Figure 5-1 Entering Precision domain name Chapter 5. Preparing the server for migration and installing the Netcool components 93
  • 109. Running script to allow Precision to run as non-root Since we installed Precision IP as a non-root user, the installation GUI prompted us with the message shown in Figure 5-2, informing us of a final step to take in order for Precision to run properly.Figure 5-2 Precision installation GUI showing steps to perform as root Modifying Precision to run as non-root user We wanted to run Precision IP as a non-root user, so we decided to run the $NCHOME/precision/scripts/ script. Example 5-9 shows the output from the script.94 Migrating to Netcool/Precision for IP Networks
  • 110. Example 5-9 Script to configure Precision IP to run as a non-root userprecision2# cd $PRECISION_HOME/scriptsprecision2]# ./setup_run_as_setuid_root.shCertain processes in Netcool/Precision for IP Networks need to be runas root.If you installed Netcool/Precision for IP Networks as a non-root user,you can use this script to alter permissions so that you can run itwhilst logged on as the non-root user who installed the product, oranother user in the same group. The processes that need to run as rootwill have their setuid permission changed so that they execute as rooteven when invoked by a non-root user.In order for this script to work correctly, you must be logged on asroot when you run it.Press return to continue, or <CTRL> + C to abortIncreasing trustworthiness of shared libraries...Changing ownership of the following executables:ncp_df_ping ncp_df_sniffer ncp_df_trap ncp_dh_arp ncp_dh_pingncp_m_timedstitcher ncp_m_trapstitcher ncp_trapmuxEnabling setuid permission on the following executables:ncp_df_ping ncp_df_sniffer ncp_df_trap ncp_dh_arp ncp_dh_pingncp_m_timedstitcher ncp_m_trapstitcher ncp_trapmuxprecision2#Creating symbolic linksPrecision IP 3.6 uses a new directory structure compared to earlier versions ofPrecision IP. We created symbolic links so that our Precision 3.6 directorystructure would resemble the structure of earlier versions. This is helpful whenreferencing older Precision documentation as well as when using scripts fromprevious versions of Precision IP. Example 5-10 shows how we created thesymbolic linksChapter 5. Preparing the server for migration and installing the Netcool components 95
  • 111. Example 5-10 Creating symbolic links for Precision 3.6 cd /opt/netcool/precision ln -s /opt/netcool/log/precision ./log ln -s /opt/netcool/var/precision ./cache ln -s /opt/netcool/etc/precision ./etc ln -s /opt/netcool/platform/solaris2/mysql4.1 ./mysql ln -s /opt/netcool/platform/solaris2/mysql4.1/data ./mysql_data Once the symbolic links have been created, the directory structure for $PRECISION_HOME should look like the output shown in Example 5-11. Example 5-11 $PRECISION_HOME after creating symbolic links [netcool@objserver2 precision]$ ls -o total 64 drwxr-xr-x 4 netcool 4096 Oct 30 15:27 aoc drwxr-xr-x 2 netcool 4096 Oct 30 14:16 bin lrwxrwxrwx 1 netcool 27 Oct 30 14:17 cache -> /opt/netcool/var/precision/ drwxr-xr-x 5 netcool 4096 Oct 30 14:16 contrib drwxr-xr-x 6 netcool 4096 Oct 30 14:16 desktop drwxr-xr-x 4 netcool 4096 Oct 30 14:16 disco lrwxrwxrwx 1 netcool 27 Oct 30 14:17 etc -> /opt/netcool/etc/precision/ drwxr-xr-x 5 netcool 4096 Oct 30 14:16 images drwxr-xr-x 2 netcool 4096 Oct 30 14:15 java_api lrwxrwxrwx 1 netcool 27 Oct 30 14:17 log -> /opt/netcool/log/precision/ drwxr-xr-x 2 netcool 8192 Oct 30 14:16 mibs drwxr-xr-x 3 netcool 4096 Oct 30 14:16 monitor lrwxrwxrwx 1 netcool 39 Oct 31 09:18 mysql -> /opt/netcool/platform/solaris2/mysql4.1 lrwxrwxrwx 1 netcool 44 Oct 31 09:18 mysql_data -> /opt/netcool/platform/solaris2/mysql4.1/data drwxrwxr-x 2 netcool 8192 Oct 30 15:55 newaoc drwxr-xr-x 6 netcool 4096 Oct 30 14:16 perl drwxr-xr-x 4 netcool 4096 Oct 30 14:15 platform drwxr-xr-x 3 netcool 4096 Oct 30 14:16 scripts drwxr-xr-x 4 netcool 4096 Oct 30 14:16 standalone_aoc Changing default latency for processes The default latency for Precision processes as configured in the CtrlServices.<DOMAIN>.cfg file is 60000. This should be increased to 100000 to96 Migrating to Netcool/Precision for IP Networks
  • 112. accommodate latency caused by larger domains, busy servers, or heavily utilized networks. We used vi to make a global change in our $PRECISION_HOME/etc/CtrlServices.REDBOOK_P.cfg file using the following command: :%s/60000/100000/g5.5 Starting Netcool products at server boot Since each customer has different preferences regarding the desired behavior for applications when a system starts, the Netcool installation does not automatically create startup links. For the Redbook lab, we wanted to have all of our applications start automatically once the server first booted, so we created the necessary startup scripts. On Linux systems, these startup scripts are placed in /etc/init.d, with links to them in the desired run level directories.5.5.1 Running the OMNIbus script to create startup files Netcool OMNIbus ships with a script that will automatically create the necessary startup file for the nco_pad process and any other daemons under its control, such as the objectserver. To install the startup script we changed to the $OMNIHOME/install/startup directory and ran a script named linux2x86install. The results of this can be seen in Example 5-12. Example 5-12 Running the Netcool OMNIbus script to create startup scripts cd /opt/netcool/omnibus/install/startup [netcool@precision2startup]# ./linux2x86install This script copies a startup script into the /etc/rc.d/init.d directory to enable you to automatically start and stop Netcool/OMNIbus processes. It does this by: Copying linux2x86/etc/rc.d/init.d/nco to /etc/rc.d/init.d/nco Linking /etc/rc.d/init.d/nco to /etc/rc.d/rc1.d/K65nco Linking /etc/rc.d/nit.d/nco to /etc/rc.d/rc2.d/S81nco Do you wish to continue (y/n)? [y] y Name of the Process Agent Daemon [NCO_PA]: Should NCO_PA run in secure mode (y/n)? [y] n Chapter 5. Preparing the server for migration and installing the Netcool components 97
  • 113. Enter value for environment variable NETCOOL_LICENSE_FILE [/opt/netcool/license/etc]: Scripts installed. This results in a script, /etc/init.d/nco, that accepts as parameters start, stop, restart, or reload. It will be run automatically at system boot, with the start option, and at shutdown, with the stop option, by virtue of the links created in the directories for run levels 1 and 2. This one must be executed by root because it starts the nco_pad daemon, which provides control for the processes under process control. It starts its child processes as the non-root user that we configured in nco_pa.conf, so the non-root user, netcool, can take it from there.5.5.2 Running the Precision script to create startup files Netcool Precision also ships with a shell script that can be run to create startup scripts. After creating our OMNIbus startup scripts we changed to the $PRECISION_HOME/scripts and ran a script named The results of this can be seen in Example 5-13. Example 5-13 Running shell script to create Precision startup scripts cd /opt/netcool/precision/scripts ./ This script creates scripts that the system will use to automatically start Netcool/Precision for IP Networks when the system is started. Additionally, the scripts can be used as a means of starting and stopping Netcool/Precision for IP Networks without resorting to "kill" commands. In order for this script to work correctly, you must be logged on as root when you run it. Press return to continue, or <CTRL> + C to abort Creating control script /etc/rc.d/init.d/ncp Creating startup/shutdown links ncp 0:off 1:off 2:on 3:on 4:on 5:on 6:off The final line of output from the startup script shows that links were installed to start Precision for run levels 2, 3, 4 and 5.98 Migrating to Netcool/Precision for IP Networks
  • 114. This results in a script, /etc/init.d/ncp, that accepts as parameters start, stop, restart, or reload. It will be run automatically at system boot, with the start option, and at shutdown, with the stop option. Its job is to start the ncp_ctrl process, which starts all other Precision processes. This one can be run by the Netcool administrator, since we have installed the products with that end in mind, but at boot time this script is going to be executed by root. The switching of ownership will cause problems. So in environments where we want the product to run as non-root, and be administered by a non-root user, we modify the startup script to start ncp_ctrl as non-root by using the su command. The changed script we used is in Example B-19 on page Creating a startup script for Netcool License Manager Netcool License Manager does not ship with a startup script, so we created a small script to start and stop the license server with the system. Example 5-14 contains the /etc/init.d/nclicense script that we created. Example 5-14 Startup script for Netcool License Manager #!/bin/bash # # Red Hat Linux 6.0+ startup script # # Source function library and global profile . /etc/rc.d/init.d/functions . /etc/profile case "$1" in start) $NCLICENSE/bin/nc_start_license ;; stop) $NCLICENSE/bin/nc_stop_license ;; *) echo "Usage: /etc/init.d/nclicense { start | stop }" ;; esac This is the basic case. In our case, since we are planning to run as non-root, we added a check in the start option to determine the non-root user, and used su to issue the start. The complete script is in Example B-20 on page 282. Chapter 5. Preparing the server for migration and installing the Netcool components 99
  • 115. 5.5.4 Creating a startup script for Netcool GUI Foundation The Netcool GUI Foundation does not ship with a script for starting the application with the server. There are various ways to start the NGF; we decided to create a small shell script named ngf and place it in the /etc/init.d directory. This script can be seen in Example 5-15. Example 5-15 Script to autostart Netcool GUI Foundation #!/bin/bash # # Red Hat Linux 6.0+ startup script # Source function library and global profile . /etc/rc.d/init.d/functions . /etc/profile case "$1" in start) $NCHOME/bin/ngf_server start ;; status) $NCHOME/bin/ngf_server status ;; stop) $NCHOME/bin/ngf_server stop ;; *) echo "Usage: /etc/init.d/ngf { start | stop | status }" ;; esac This too is the basic case. In our case, since we are planning to run as non-root, we added a check in the start option to determine the non-root user, and used su to issue the start. The complete script is in Example B-21 on page Creating a startup script for Netcool Security Manager The Netcool Security Manager does not ship with a startup script, so we decided to create a small shell script named ncsm and place it in the /etc/init.d directory. This script can be seen in Example 5-16.100 Migrating to Netcool/Precision for IP Networks
  • 116. Example 5-16 Script to autostart Netcool Security Manager #!/bin/bash # # RedHat Linux 6.0+ startup script # # Start up the Netcool Security Manager. # Source function library and global profile . /etc/rc.d/init.d/functions . /etc/profile #set -x case "$1" in start) "$NCSM_HOME/bin/ncsm_server &" ;; status) $NCSM_HOME/bin/ncsm_status ;; stop) $NCSM_HOME/bin/ncsm_shutdown ;; *) echo "Usage: /etc/init.d/ncsm { start | stop | status }" ;; esac This too is the basic case. In our case, since we are planning to run as non-root, we added a check in the start option to determine the non-root user, and used su. The complete script is in Example B-22 on page Symbolic link creation to auto-start applications OMNIbus created a startup script in the /etc/init.d directory. However, it does not create links to the various rc.d directories. In order to make each of the products start with the various run levels, we created symbolic links to the /etc/init.d scripts. These links are ordered so that the products will start in the proper order. An example of these links is shown in Example 5-17. Example 5-17 Created symbolic links for run levels ln -s /etc/init.d/nclicense /etc/rc2.d/S71nclicense ln -s /etc/init.d/nco /etc/rc2.d/S75nco ln -s /etc/init.d.ncsm /etc/rc2.d/S76ncsm ln -s /etc/init.d/ngf /etc/rc2.d/S79ngf Chapter 5. Preparing the server for migration and installing the Netcool components 101
  • 117. ln -s /etc/init.d/nclicense /etc/rc3.d/S71nclicense ln -s /etc/init.d/nco /etc/rc3.d/S75nco ln -s /etc/init.d/ncsm /etc/rc3.d/S76ncsm ln -s /etc/init.d/ngf /etc/rc3.d/S79ngf ln -s /etc/init.d/nclicense /etc/rc4.d/S71nclicense ln -s /etc/init.d/nco /etc/rc4.d/S75nco ln -s /etc/init.d/ncsm /etc/rc4.d/S76ncsm ln -s /etc/init.d/ngf /etc/rc4.d/S79ngf ln -s /etc/init.d/nclicense /etc/rc5.d/S71nclicense ln -s /etc/init.d/nco /etc/rc5.d/S75nco ln -s /etc/init.d/ncsm /etc/rc4.d/S76ncsm ln -s /etc/init.d/ngf /etc/rc5.d/S79ngf Note: Netcool Precision links were created by the installation script so no additional symbolic links were created. The links created by the Netcool Omnibus installation script were removed in favor of these new links to improve component coordination on our systems. Those links were /etc/rc1.d/K65nco and /etc/rc2.d/S81nco.102 Migrating to Netcool/Precision for IP Networks
  • 118. 6 Chapter 6. Migrating NetView and Switch Analyzer In this chapter we examine the basic case of replacing a NetView implementation with the appropriate components from the Netcool suite. The NetView implementation may or may not include IT Switch Analyzer, the Web Client, or a failover NetView. The resulting Netcool implementation will include layer2 discovery and monitoring, a Web-based user interface, and an optional failover server. After a period of running in parallel, the NetView server or servers can be retired. This chapter is intended as a workflow guide for configuring the Netcool components in a manner that leverages your investment in configuring NetView. It is not intended to explore or define all of the capabilities of the Netcool suite of products, and the exclusion of topics from this book should not be interpreted as limitations in the product. The topics chosen for inclusion fall into two categories: Those features that should be customized to provide the basic capabilities expected by current NetView customers Those features that are exclusive to Netcool and that we think will be of great interest to NetView customers© Copyright IBM Corp. 2007. All rights reserved. 103
  • 119. You will sometimes be directed to the appropriate sections of product manuals, or to other sections of this document for more details. While reading this chapter, have these manuals on hand: Netcool/Precision IP Discovery Configuration Guide 3.6 Netcool/Precision IP Installation and Deployment Guide 3.6 Netcool/Precision lP Topology Visualization Guide 3.66.1 Architecture The architecture for our lab machines is described in this section.6.1.1 NetView architecture The existing NetView implementation consists of two Red Hat servers, one for production and one for test and backup as shown in Table 6-1. Failover to the backup is done manually, as is the synchronization of the two systems. There is no Tivoli Enterprise Console® in this environment. Event automation and notification is done either in trapd.conf or with NetView rulesets in ESE.automation. RFI is enabled for layer 3 root cause correlation. This is a fairly common arrangement. We also have Tivoli Switch Analyzer installed and running for layer-2 root cause correlation. Table 6-1 NetView servers: Products installed in hosts NetView1 NetView2 NetView V7.1.4 x x ITSA 1.3 x x6.1.2 Netcool architecture The target environment for this example can be arranged in a number of ways. The product manual to review is Netcool/Precision IP Installation and Deployment Guide 3.6, Chapter 1, “Netcool/Precision IP Deployment.” For our lab, we chose to use one multi-cpu server for all of the components of the solution, and to repeat that arrangement on a second server for failover. This arrangement is fine for many networks, depending on the size of the network and the resources of the server. The target servers were set up as described in Chapter 5, “Preparing the server for migration and installing the Netcool components” on page 83, including preparation, product installation, base configuration, and verification.104 Migrating to Netcool/Precision for IP Networks
  • 120. Subsequent sections assume that the products listed in Table 6-2 were installed by a non-root user, and that the Netcool administrator is the UNIX userid netcool. Table 6-2 Netcool servers: Products installed objectserver1 precision2 OMNIbus objectserver 7.1 x x Precision 3.6 x x mttrapd probe x x Webtop 2.0 x x Security Manager 1.3 x x License Server 1.0b31 x x NC Knowledge Library 1.1 x x In the course of our work in the lab, most work was done on the precision2 server. However, some was done on the objectserver1 server. This allowed parallel development by different team members. Ultimately, of course, all customization would be moved to both servers and tested together. This explanation is provided to avoid any confusion over examples or screen shots that include the name of one server or the other.6.2 Gathering information from the NetView server This information should be collected from the NetView server. Most of it can be gathered programmatically using simple UNIX or NetView commands. It will be used to help configure the Netcool components, and it will be used again for verification of the results. Refer to Appendix B.1, “Commands and scripts used to extract information from the NetView installation” on page 242 for details on how we collected this information. Discovery and mapping information – List of routers and route-switches from nvUtil – List of all other layer2 devices from nvUtil – List of all other nodes from nvUtil – List of those which are presently non-SNMP from nvUtil – List of those which are presently unk-oids from nvUtil – Default and alternate community strings from xnmsnmp.conf and communityNames.conf – name resolution (netsvc.conf, resolv.conf, nsswitch, /etc/hosts, and so forth) Chapter 6. Migrating NetView and Switch Analyzer 105
  • 121. – Seedfile rules: ranges, exclusions, oids, and so forth – Smartset definitions from nvUtil G – List of unmanaged nodes and interfaces from nvUtil – List of disconnected areas of the map – location.conf if used and current – generated location.conf if none exists – Layer2 report and discovery report if ITSA is in use. – Devices recognized as layer2 from ovtopodump -X Custom fields information – Fields added from custom fields registration file User account information – Web user accounts, roles, and scopes – UNIX user accounts for the server interface – Native security feature accounts, groups, and so forth Polling information – Frequency, time-outs, retries from xnmsnmpconf – Threshold polls from snmpCol.conf – netmon number of pollers from netmon.lrf (-q and -Q) Trap processing information – EXEC statements from the trapd.conf – Correlation rulesets for traps from ESE.automation – Automation rulesets for traps from ESE.automation Event processing information – Automation rulesets for NetView status and threshold events from ESE.automation – Event Displays from $HOME/NvEnvironment/Workspaces Other automation – Functions scheduled in cron – Functions scheduled in Tivoli Scheduler Other custom functions – Additions to the Motif menu or Web client menu – Additional command line functions and tools – Custom functions that use the programming APIs Note: All of the automations may not be needed in the migration, but they should be collected and reviewed.6.3 Migrating the discovery This section covers the initial discovery of the network based on the NetView discovery. There are multiple goals: Discover and manage the same set of devices in Precision as is currently being managed by NetView.106 Migrating to Netcool/Precision for IP Networks
  • 122. Apply, where possible, policies to control future discoveries that match those in place for NetView. Ensure that the discovery is complete enough to support accurate root cause analysis. Take advantage of some of the extended discovery capabilities available with Netcool/Precision.The product manual to review is: Netcool/Precision IP Discovery ConfigurationGuide 3.6, Chapter 4, “Network Discovery Using the Network Discovery GUI.”Discovery is performed by the DISCO component of Netcool/Precision, and anumber of processes get involved before the completed topology is exported bythe MODEL component to MySql for viewing in Webtop. The configuration of thecontrols on the discovery are for the most part GUI-driven. All of the work in thissection was performed using the UNIX userid netcool.Some NetView administrators strictly control the discovery of nodes bymaintaining an explicit list in the seedfile, either by name or by IP address, of thenodes that they want to monitor. This is the equivalent of the filefinders method inNetcool/Precision, and will work fine as long as the set of devices comprises acomplete, connected topology. If there are gaps, then the root cause analysis willbe imperfect. There is a process in Netcool/Precision to close those gaps.Administrators who are using Switch Analyzer will probably already have trackeddown all of the layer2 devices required to complete their layer2 topology.Administrators who have not implemented Switch Analyzer may possibly beunaware of missing layer2 devices, so care should be taken when relyingcompletely on the list of currently managed devices. When the results of adiscovery are reviewed, it is likely that gaps will be identified. Once those missingdevices are found, configured for SNMP access, and added to the discovery,then those who have a strong preference for a strictly controlled discovery cancontinue to use the filefinders approach for the ongoing maintenance ofdiscovery.Other administrators prefer a more dynamic discovery based on rules because itinvolves less maintenance or because it alerts them to unknown devices thathave joined the network. This is the preferred method in most cases, andNetcool/Precision provides controls similar to NetView’s for excluding unwanteddevices from the discovery. These customers should use the filefinders methodfor initial discovery during the migration, and then switch to a rules-baseddiscovery for ongoing maintenance, using the initial list or the router part of theinitial list as seeds.A complete rediscovery will be done on a regular basis, depending on how oftenchanges are made to network equipment. This is similar to a combination ofNetViews configuration poll and new node discovery poll and a restart of Switch Chapter 6. Migrating NetView and Switch Analyzer 107
  • 123. Analyzer. There is no concept in Netcool/Precision that is the equivalent of a discovery poll or a configuration poll. This will have implications for ongoing operations, such as a regularly scheduled rediscovery, adding and deleting devices, and for unmanaging things. These issues are considered in Chapter 7, “Migration topics” on page 199. For now we focus on an orderly initial discovery. For beginners, the discovery should be done in multiple passes, with verification after each pass so problems can be identified and resolved early on. This will save time in the long run. The recommended order of discovery is as follows: First pass: Discover the Netcool/Precision server and its local switches and routers. This provides a quick familiarization exercise and verifies basic functionality. Second pass: Add the router backbone. Include the routers and layer3 switches (route switches). Third pass: Add the layer2 switches and layer2 discovery agents. Fourth pass: Add the rest of the nodes and the rest of the discovery agents. Important: Root cause analysis requires that the Netcool/Precision server be discovered, managed, and connected to the rest of the network map. If it is not connected, as sometimes happens with firewalls, then you must define an alternate NcpServerEntity, preferably something very near the Precision server. This is configured in the $NCHOME/etc/precision/NcoGateSchema.cfg file. If the Netcool/Precision server has multiple IP addresses, it is a good idea to enter the preferred one in this file as well. See the comments in that file for more information.6.3.1 First pass at discovery Discover the local Netcool/Precision server and its local switch and router and verify the views. Begin by logging into the Webtop as the Netcool/Precision administrator. Choose the Precision Admin page. Choose the Discovery Configuration tab, and make sure your current domain is selected. Ours was REDBOOK_P. Configure the discovery Scope: Chose the Scope tab. Put one or more address ranges in here, enough to allow the initial set of devices. Specify Include. Deselect the Add to Ping Seed List box because we don’t want to ping the range. This is the equivalent to a wildcard or range entry in the NetView seedfile, in that it allows but does not force the discovery of matching addresses. Define the ranges as tightly as possible while still keeping them easy to maintain. The wider the range the bigger the routing tables that will be pulled from routers during discovery. See Figure 6-1 on page 109.108 Migrating to Netcool/Precision for IP Networks
  • 124. Figure 6-1 Setting the scope Seed: Choose the Seed tab. In this dialog we deselected the Ping Finder method and selected the File Finder method. We made a directory for holding our files, /opt/netcool/local. We created a file there called pass1.ff.list that contained just a list of three IP addresses: for the Netcool/Precision server, the local switch, and the local router, as shown in Example 6-1.Example 6-1 pass1.ff.list# cat /opt/netcool/local/pass1.ff.list10.1.0.1210.1.0.25410.1.0.1 In the Seeds tab we specified the path and filename, a null, column 1 for the IP address, and column 0 for the name, as shown in Figure 6-2 on page 110. Chapter 6. Migrating NetView and Switch Analyzer 109
  • 125. Figure 6-2 Setting the Seed entry Passwords: Choose the Passwords tab. This is where you configure the community strings to try, and also the version of SNMP to try. Order them so the most likely are tried first. Use the communities from NetView’s communityNames.conf file, and also the default string from xnmsnmpconf. For the ranges of addresses to which they apply, you can use null for now. NetView only used SNMP V1 for inquiries. Netcool/Precision can use SNMP V2 if the network equipment supports it. Some of the threshold polls, for instance, will use the SNMPv2 getbulk form when retrieving tables, which is more efficient for the devices being polled. Therefore, you should specify SNMPv2 where that capability exists. You can actually calculate the frequency with which strings were used in NetView by the method shown in Example 6-2. Note that the default community string will be counted for all non-SNMP-capable nodes, inflating that number; therefore, you might want to reduce the number attributed to the default community by the number of non-SNMP nodes in the discovery. In our case it does not change the result. Example 6-2 Calculating the frequency of community string usage # tail -1 /usr/OV/conf/communityNames.conf router switch server # xnmsnmpconf -dumpCache | cut -f2 -d: | sort | uniq -c | sort -rn 211 public 15 server110 Migrating to Netcool/Precision for IP Networks
  • 126. 6 switch 6 router# nvUtil e (isSNMPSupported=False) | wc -l 78 Device Support: Choose the Device Support tab. This controls which discovery agents are run. Make sure Full Layer 2 and Layer 3 Discovery is checked. Other device support can wait until later runs. DNS: Choose the DNS tab. This controls name resolution. If nothing is specified, Netcool/Precision will do no name resolution. To get it to use the operating system name resolution, as NetView does, add a service as shown in Figure 6-3. There are a number of other options for configuring the name. For more details see Netcool/Precision IP Discovery Configuration Guide 3.6, Chapter 4.2, “Configuring DNM Helpers.” You can also override the system settings by configuring one more methods here. Most of our network devices are in /etc/hosts, and the non-network things are in DNS.Figure 6-3 Configuring DNS Advanced: Read the documentation carefully before modifying anything here. This is where you can instruct Netcool/Precision to use sysNames for labels instead of name resolution. It is also where you instruct it to ignore nodes in the filefinders list that are not presently up. By default, the filefinders Chapter 6. Migrating NetView and Switch Analyzer 111
  • 127. list acts like NetView’s loadhost command, adding them regardless of status. Checking the Enable File Finder Verification box makes it behave similar to loadhost with the -p option. That is, it will ping them first, and only add them if they are up. If your NetView has a lot of old devices that are no longer on the network, you might want to enable this. We did not. We checked these boxes in the Advanced Discovery Configuration section of the Advanced tab: Enable VLAN Modelling Enable Rediscovery Rebuild Layers Enable ifName/ifDescr Interface Naming Disable the Feedback stitcher: This function adds addresses that it finds to the ping-finders list, like the HINTS that appear in NetView’s database during discovery. Those hints would be bounded by the Scope, but for now we wanted the discovery to be bounded by the filefinders list, so we turned this off altogether by renaming the file $NCHOME/precision/disco/stitchers/Feedback.stch to Feedback.opt. Save: This is easy to overlook. Save the configuration by clicking the diskette-shaped icon. Run the discovery Switch to the Discovery Status tab. The routine here, for full rediscovery, is: 1. Stop the current discovery by clicking the red square. 2. Wait for it to turn gray, and for the triangle (the play button) to turn green. The text on the upper right should display Discovery Type: Full Discovery. 3. Click the green play button. Verify completion of the discovery The discovery process writes to $NCHOME/logs/precision/ncp_disco.<>.log, where <> is replaced by the name of your Netcool/Precision domain. When you click the red STOP button, this is placed at the end of the file: ncp_disco is dead. This file is erased when you restart the discovery, and it is completely replaced with each rediscovery, so check the timestamp to see if it is current. This pass should take only a few seconds. The Discovery Status tab will show the agents that the discovery uses, and the numbers will show how many nodes or interfaces they run against. Those numbers start at 0, then increase and decrease to 0 again. Between discoveries they will report Awaiting Status. The Discovery Devices tab does not update dynamically like the others do. Refresh it with the circling arrows icon and it should show a list of three addresses or names for this run.112 Migrating to Netcool/Precision for IP Networks
  • 128. Verify the results of the discovery First check the Discovery Devices tab on the Discovery Status tab of the Precision Admin page of the NGF; refresh it if necessary. You should see listed the three nodes that you included in the file finders list. Next, switch from the Precision Admin page to the Precision Desktop page in the login box in the top right corner of the screen. In the View tree, select a set of views you want to work with, for instance admin Views. There will probably be no views yet in that set. Create a view called All, for which the mainNodeDetails field EntityName is not null, Type is Layer 2, and End Nodes are included. You should see your three nodes in the right panel, and they should be connected. The status icon (the round button) for the All view on the Network View Tree panel should show the highest severity of any events which have occurred for these nodes. Try the right-mouse menu option Show Details. Read Chapter 3 in Netcool/Precision IP Topology Visualization Guide 3.6, and also see 6.4, “Migrating the network map visualization” on page 131 in this book for help creating and navigating the Precision views. Do not proceed until you have everything working up to this point, whether it is the actual discovery, or the visualization of the discovery so far in the Precision views. If you have problems, verify that you are using a supported browser and a supported level of Java as described in 5.2.2, “Operating system preparation and checks” on page Second pass at discovery Now that you know that the basic features are functioning, discover those same three nodes plus the entire layer3 backbone of the network. This important pass will flush out many problems with access to the network devices such as access lists and firewalls. If access for the Netcool/Precision server has been made equivalent to that of the NetView server, it should go pretty smoothly. Return to the Precision Admin page of the NGF for these steps. Configure the discovery Scope: If ranges or wildcards were specified in the NetView seedfile, use those as your guide. Whereas NetView discovered all interfaces on a device if one allowable interface was found, Netcool/Precision will exclude individual interfaces if they are not in range. So you might need to broaden them. The list of all networks exported from NetView is shown in Example 6-3. We are still saying no on Add to Ping Seed List. Chapter 6. Migrating NetView and Switch Analyzer 113
  • 129. Example 6-3 Exporting network addresses from NetView for scoping # nvUtil e (isNetwork=True) %Network Address%,%IP Subnet Mask%| sort | head,,,,,,,,, Seeds: Add another file here for your routers and layer3 switches. You should have extracted a list of these from NetView. Ours was called /opt/netcool/pass2.ff.list and contained three fields separated by commas: Selection Name, SNMP ipAddress, vendor. That is because we extracted a list of everything NetView called a router and included the vendor so we could exclude the servers that sometimes show up in that kind of list, as shown in Example 6-4. In the Seeds tab, we added this second file in the Filefinders section, and specified comma as the delimiter, column 2 as the IP address, and column 0 as the name because we want to use name resolution of the new Netcool/Precision server for labeling. Example 6-4 Generating pass2.ff.list # nvUtil e (isIPRouter=True) %Selection Name%,%SNMP ipAddress%,%vendor% | grep cisco > pass2.ff.list # head pass2.ff.list C1700-1,,cisco Systems C1700-2,,cisco Systems,,cisco Systems Passwords: Nothing new should be needed here if you set them in discovery pass 1. Device Support: Here you have two options. If you want to make a fast test of all of the routers to verify reachability and SNMP access, you can disable everything except Details, AssocAddress, and Interfaces agents. Turn off even the Layer2/3 entry. This will result in the fastest, least intrusive check of access. The results will not be connected nicely, but you will be able to tell if they are SNMP-accessible, and go back and fix things that are not. The more complete approach involves those same three plus the layer 3 agents114 Migrating to Netcool/Precision for IP Networks
  • 130. (IpRoutingTable; IpBackupRoutes; and IpForwarding). Do that style of discovery to verify connectedness after the SNMP issues are cleared up. DNS: Nothing new should be needed here. Disable the Feedback stitcher: Leave it off for this run. We are still working with the filefinders lists. Save: Don’t forget to do this. Click the diskette icon.Run the discoverySwitch to the Discovery Status tab and kick off the discovery as before.Verify the completion of the discoveryWatch the ncp_disco.<>.log. This time it will take a bit longer for the new log tobe created. Once that happens, you should also see activity in the DiscoveryDetails and the Discovery Tables tabs. Watch here to see what is waiting to beprocessed. The end of the log will be the same as before.Verify the results of the discoveryReturn to the Precision Desktop page and check your All view. You shouldexpect to see all of the routers connected to each other except where there areunavoidable breaks. You should be able to identify the groups of connecteddevices as being similar to a newly discovered NetView map. Any disconnectedsections should be reviewed for missing devices that should join them to theother areas. Try different map arrangements using the Layout buttons across thetop of the view. These offer a variety of automatic layout algorithms, unlike theautomatic layout in NetView which gave only one choice. You can also movethings manually, and you can save the arranged view.There is a difference to consider. Netcool/Precision does not rely, as NetViewdoes, on subnet addressing to draw its connections. It inspects a wide variety ofMIB variables to determine connectedness. Therefore it is more sensitive thanNetView is to problems with the SNMP agent on the devices. Some connectionsmay not be drawn at certain device code levels that would be drawn at morecurrent levels.You might find that you need to repeat this pass a number of times. It is whereyou are most likely to find out that you do not have the access you require todiscover the network. Review the non-SNMP view and the no class view,reporting any problems to the network analyst for resolution. Compare the list ofrouters found to the list of routers in your filefinders list. Are there firewallsblocking access to any of them? Resolve as many as you can before continuingto the next pass because the next pass will probably take quite a bit longer thanthis pass does. Chapter 6. Migrating NetView and Switch Analyzer 115
  • 131. 6.3.3 Third pass at discovery Now that the basic structure of the network has been discovered, and we know that the Netcool/Precision server has SNMP access to all areas of the network, we can move on to the layer2 devices. When these devices are discovered by NetView, they are drawn below the subnet level, and do not contribute to the complexity of the top level map. With Netcool/Precision, they will be displayed differently depending on whether the view was defined as layer2 or layer3, and the current setting of the “toggle end nodes” button. These views can get quite complex, and changing those settings on the view definition filters them for you. Configure the discovery Scope: Nothing changes here for now as long as the management IP address range for the switches is covered by the existing scopes. Seed: Add another file here for the list of layer2 network devices that you extracted from NetView. Example 6-5 shows how we exported the list of layer2 devices from NetView. Ours was called /opt/netcool/pass3.ff.list and contained three fields separated by commas: Selection Name, SNMP ipAddress and IP Status. We specified comma as the delimiter, column 2 as the IP address, and column 0 as the name because we want to use name resolution of the new Netcool/Precision server for labeling, not the names used by NetView. Example 6-5 pass3.ff.list # ovtopodump -X | awk {print $2","$5","$3} | grep -v Status S2900-1,,Up S1900-2,,Up S2900-2,,Up S1900-1,,Up S2700-3,,Up S1900-3,,Up Passwords: If any more are needed for these devices, add them. Device Support: Here again you have two options. You can do a fast check for IP and SNMP reachability by using only the Details, AccocAddress, and Interfaces agents, or you can go for a real discovery of connectedness. If your SNMP access is all good, and you are ready for a more thorough discovery, specify the Layer2/3 agents. That section includes the 3 IP routing agents we used in the last pass, and also the layer2 agents. Disable the Feedback stitcher: Leave it off for this run. We are still working off the filefinders lists.116 Migrating to Netcool/Precision for IP Networks
  • 132. Save the configuration by clicking the diskette icon. Run the discovery Do this as before. Verify the completion of the discovery Do this as before. Expect this pass to take longer than the first two. Verify the results of the discovery As before, return to the Precision Desktop page to analyze the results. In the All view, check the option to show Layer 2. Try the hop views. Spot check some areas for the expected layer2 view. The layer2 view will be unfamiliar to NetView customers unless they have used IT Switch Analyzer 1.3 with its layer2 views. Compare some of these views to the known configuration of a few devices. The All View of our little test network looked like Figure 6-4.Figure 6-4 All View after pass 3 Chapter 6. Migrating NetView and Switch Analyzer 117
  • 133. This is a good time to add a view for non-SNMP devices, similar to the Non-SNMP Smartset we often create in NetView. Use the new view function in the Network View Tree panel. These are the settings we used: Name: Non-SNMP Parent: NONE Domain: REDBOOK_P Table: mainNodeDetails Filter: EntityOid is null and topoVizType = node Include: End Nodes Layout: Circular At this point in the discovery, there should be nothing in this view. If any devices appear here, you should get them corrected before proceeding. This is also a good time to check whether Netcool/Precision recognizes all of the SNMP sysObjectIDs that it has discovered. To group devices by Class, run the auto-partition wizard on Device Class. This is covered in Chapter 3 in Netcool/Precision IP Topology Visualization Guide 3.6. There is also more on view creation in 6.4, “Migrating the network map visualization” on page 131 in this book. For now, it is enough to launch the AutoPartition wizard and select these options: Auto-Partition Type: Select pre-defined auto-partitions, Next Domain: REDBOOK_P Select Partitions: Device Classes, Next Create a new view named: ClassView Under: NONE, Next Layout: Circular Approve the preview, and click Go. Devices will be grouped by Class based on SNMP sysObjectId. If a device has a specific Class, such as Cisco17xx or CiscoCat29xx, then Netcool/Precision knows whether it is a router or a switch or a server of some sort. There are a couple of catch-all groups to be reviewed and dealt with. One of these is Device. Any devices that are in this view need further refinement. This view might also contain devices that could not be reached via SNMP, if you have not yet cleared those up.118 Migrating to Netcool/Precision for IP Networks
  • 134. In addition, you might find views that have been created for just a vendor, not specifying any type of device. For instance, you may see a view called Cisco or Nortel. Attention: Auto-partitioned Device Class views named simply for a vendor do not contain all nodes from that vendor. They contain nodes that Precision was able to match to that vendor, but not to any child Class below it. These also need refinement, since those Class definitions do not know whether the device is a router or a switch. Such devices will be represented by a simple computer symbol, referred to as a node, rather than by a router or switch symbol. This would be similar to the Unknown_OIDs Smartset we often make for things that are SNMP supported but for which the vendor is unset. You can look at their fields with Show Details on the right mouse button menu. You can then use this data to create additional Class files and re-run the discovery. This is the equivalent of updating the oid_to_type file in NetView. We used a custom script,, to collect information on these. See the discussion of class scope expressions and Active Object Classes in 6.5, “Migrating network monitoring” on page 148 for more details on clearing up these nodes. Also see Netcool/Precision Discovery Configuration Guide 3.6, Chapter 15, “The Active Object Class Server.” Most of the network equipment should be discovered and properly represented at this point.6.3.4 Fourth pass at discovery Add the miscellaneous nodes such as servers, printers, and non-SNMP things to the discovery. You can get a list of those nodes from NetView in a variety of ways similar to the previous runs, or you can use a list of all nodes in NetView and replace the first 3 filefinder lists with the one big one. The instructions are the same as for the earlier passes and so are not repeated here. The Device Support entries checked should include the full Layer 2/3 list of agents as well as the basic ones: Details, AssocAddress, and Interfaces. Verify the results of the discovery Delete the Class Views hierarchy and re-run the auto-partition so the views reflect the miscellaneous types of devices found in the last discovery. Check the membership of the Non-SNMP view and the Device view and any Vendor-only views to see if the results are acceptable. Rerun the script to generate a fresh list if OIDs that are not recognized, and consider whether any further refinement is required. Spot check some hop views that include layer2 connections, especially of end nodes such as critical servers. You can export a Chapter 6. Migrating NetView and Switch Analyzer 119
  • 135. list of IP Addressable interfaces from NetView server, as shown in Example 6-6, and compare it to the final discovery. We included in this list the name of the parent node for each interface address, and the current IP status. This information might help with the reconciliation of any differences. Transfer this list to the Netcool/Precision server and run the script. It will extract a similar list from Netcool/Precision and compare it to your NetView list. Example 6-6 Extracting a complete address list from NetView # nvUtil e ‘(isInterface=True)’ ‘%IP Address%,%Selection Name%, %IP Status%’ | grep -v Layer2Status | sort > NetView.addresses.all #head NetView.addresses.all,,Normal,C1700-1:Loopback0,Normal,C1700-2:Loopback0,Normal,C1700-1:Serial0,Critical,C1700-2:FastEthernet0,Normal6.3.5 Migrating discovery rules and adding agents Once you have verified your ability to discover all of the devices previously discovered by NetView as basic layer 2 or layer 3 devices, you will probably want to switch from an explicit list of devices to just rules. Using the Discovery Configuration GUI, you can specify ranges for inclusion and exclusion based on the rules that were specified in NetView’s seedfile. Your goal is to achieve the same discovery results using this method as with the filefinders method. The ideal arrangement is to define the scope in a fairly detailed manner, use the routers in the pingfinders seed section, and enable the Feedback.stch stitcher, which we previously disabled. The right combination will be different for every network. Example 6-7 shows our NetView seedfile. Example 6-7 NetView seedfile # <- a seed for a NetView server # <- a seed for a NetView server 1. ! # <- and exclusion range !@oid* <- and exclusion oid # <- a seed C1700-2 # <- a seed S1900-3 # <- a seed S2700-3 # <- a seed120 Migrating to Netcool/Precision for IP Networks
  • 136. ScopeWe configured positive ranges as shown in Figure 6-5 to accommodate theaddresses discovered by NetView. Like the ranges or wildcards in a netmonseedfile, these ranges or wildcards allow the discovery of nodes but do not forcethem to be discovered. The ranges are actually subnets, and not the same styleof ranges that NetView uses. More complex ranges can be defined in filters,described later. In each scope entry you also have the option to check the Add toPing Seed List box. This is the equivalent of explicitly listing the addresses in thenetmon seedfile, which will actively ping the addresses to force discovery.Whether that is necessary or not depends on how well you chose your seednodes. This behavior is also similar to NetView.Figure 6-5 Defined discovery rangesSeedsWe included the same seed devices as were in the NetView seedfile as shown inFigure 6-6 on page 122. This list included the management servers and a fewnearby network devices including the local router. Notice that we unchecked theFileFinders method that we used in the previous runs. Chapter 6. Migrating NetView and Switch Analyzer 121
  • 137. Figure 6-6 Defining the seeds Feedback Stitcher Now that we are using ranges and seeds to control the discovery, we re-enable the Feedback Stitcher by uncommenting the lines in the trigger stanza of the $NCHOME/precision/disco/stitchers/Feedback.stch file. This will cause new hints to be added to the ping finders based on the discovery of the seeds that we entered. Filters Our NetView seedfile contained a negative address range of a type that did not easily map to a subnet and mask, ! We used a Pre-Discovery filter to exclude those addresses. The same mechanism allows us to exclude by SNMP sysObjectID, and we had one of those in the NetView seedfile as well. See the manual Netcool/Precision IP Discovery Configuration 3.6, Chapter 4.2, “Setting Discovery Filters” for more information on discovery filters. To exclude a range of addresses between and, we added a new Pre-Discovery filter called Filter Out Range 9_3_5_100-150 to the Pre-Discovery filter library as shown in Figure 6-7 on page 123.122 Migrating to Netcool/Precision for IP Networks
  • 138. Figure 6-7 Defining a filter to exclude a range To exclude the Microsoft® OID, we added a new Pre-Discovery filter called Filter Out Microsoft to the Pre-Discovery filter library. The criteria for that filter was m_ObjectId not like We applied both of these in the Pre-Discovery Filter dialog. Netcool/Precision also allows us to exclude selected interfaces on an device such as a router that is otherwise in scope. This was not possible with NetView, and the usual recourse was to manually unmanage those interfaces after discovery. For those cases where we do not want to monitor the status of selected IP addresses on devices even though they are up at discovery time, we can filter them out of the discovery. Review the lists of unmanaged router interfaces in the NetView map and consider specifying them for exclusion by configuring a Post-Discovery filter. As an example, we chose one of the interfaces on a server that had two interfaces, which were both up. Use the Advance tab in the filter dialog to reference the m_Addresses(2) field since it is not selectable in the Basic tab. The example in Figure 6-8 on page 124 worked to exclude the specified IP address from rediscovery. The extra parentheses were required. Chapter 6. Migrating NetView and Switch Analyzer 123
  • 139. Figure 6-8 Defining a filter to exclude an interface Tip: The pre-discovery filters and the post-discovery filters configured via the GUI end up in the file $NCHOME/etc/precision/DiscoSchema.<domain>.cfg. Precision filters are counter intuitive. Things that match the filter will be included. Historically people have tended to make the filters match what they want to exclude – not what they want to keep. Netcool/Precision will also exclude or unmanage interfaces automatically for a variety of reasons, and there is more about this in 7.2, “Provisioning Netcool/Precision” on page 201 and in 6.5, “Migrating network monitoring” on page 148. Device Support In addition to verifying that all of the devices and interfaces are discovered, you should review the agents list in the Device Support section of the Discovery Configuration tab. Be sure to turn on any that apply, if they were turned off during the early runs of the discovery. For instance, if there are ATM elements in the network, check that box.124 Migrating to Netcool/Precision for IP Networks
  • 140. Run the discovery If you discover nodes and then decide to exclude them, you can either rediscover three times, or set the linger time for those nodes to 0 and rediscover once, or you can flush the cache files from previous discoveries and rediscover. To test that our new discovery specifications produce the same results as the previous discovery, we decided to clear out all traces of the previous discovery and restart it from scratch. That involved these steps: 1. Stop ncp_ctrl, which stops all of its child processes (kill it). 2. Back up the cache files for the current discovery, /opt/netcool/precision/cache/*, and remove them. 3. Clear all events from the objectserver to prevent mismatched ObjectIDs: a. From a map view, right-click Show Events. b. Choose the Default filter in the top left pull-down menu. c. Edit: All. d. Alerts: Delete. 4. Restart ncp_ctrl. 5. Manually restart the discovery as before, using the GUI. 6. Verify completion as before. 7. Verify the results as before. 8. Export the list of resulting addresses and compare them to the previous runs. Tip: The command to restart our ncp_ctrl is: nohup ncp_ctrl -domain REDBOOK_P -logdir $NCHOME/log/precision >$NCHOME/log/precision/ctrl.out 2>&1 &6.3.6 Discovering extra information Custom fields can be added and populated at discovery time, and then used as the basis for individual views, similar to Smartsets, or for the auto-partitioning of views. More importantly, those fields can also be used later to enrich events about those nodes. The additional data might be helpful to operators who must act on those events, or useful in automation triggered by the events. The simplest kind of data to add at discovery is SNMP MIB values, which can be retrieved directly from the device. Netcool/Precision already retrieves many standard MIB variables, such as sysContact, sysLocation, and sysName; and includes them in the ExtraInfo column of the database. The ExtraInfo column is also the standard place for adding your favorite vendor-specific fields such as serial number, model, or Chapter 6. Migrating NetView and Switch Analyzer 125
  • 141. version. This is described in the manual Netcool/Precision IP Discovery Configuration Guide 3.6, Chapter 9.8 “Retrieving Extra Information.” In our case we added the serial number for Cisco equipment. Using a custom agent to add SNMP fields The steps for adding a custom discovery agent are summarized as follows: 1. Create the custom agents to populate a new element in the ExtraInfo field. 2. Add the custom agents to the agents list in the Device Support section of the discovery configuration dialog. 3. Enable the agents in the Device Support section of the discovery configuration dialog. 4. Add the field to the mysql database and optionally to the for viewing. Creating the custom agents The agent files are under $NCHOME/precision/disco/agents. You can read more about discovery agents in Netcool/Precision IP Discovery Configuration Guide 3.6, Chapter 9, “Discovery Agents.” If you are retrieving a MIB variable that exists in the same place on all devices, such as a variable under the MIB II mgmt tree, then you can add it to the ExtraDetails.agnt configuration file. This agent already retrieves variables such as sysContact and sysLocation from all SNMP-capable devices. We know from experience that nearly all Cisco equipment stores a serial number in one of two different MIB variables, depending on the type of device it is, as shown in Table 6-3. Table 6-3 Cisco serial number OIDs Agent sysObjectID Serial number OID Variable name* chassisSerialNumberString* chassisId* chassisId Those whose SNMP sysObjectID (or EntityOID) begins with keep it in a variable under the cisco workgroup branch of the MIB tree. Those that begin with1. or respond to a variable under the cisco temporary branch of the MIB tree. So rather than use the ExtraDetails.agnt for this job, we created two custom agents to handle the job separately for these two groups of devices.126 Migrating to Netcool/Precision for IP Networks
  • 142. We made two copies of the ExtraDetails.agnt file, in the same directory,$NCHOME/precision/disco/agents, and named them like this: CiscoSerialWorkgroup.agnt CiscoSerialOld.agntWe edited those files and replaced the existing entries with our own, beingcareful to preserve the formatting. The updated files are included inAppendix B.3, “Precision agents we modified” on page 277. The changes wemade to each clause are described here.The DiscoAgentSupportedDevices clause specifies the selection criteria for thedevices that this agent will be run against, and we used the SNMP sysObjectID.The DiscoAgentSupportedDevices clause specifies the selection criteria for thedevices that this agent will be run against, and we used the SNMP sysObjectIDsshown in table Table 6-3 on page 126. You could add any number of such agentswithout too much worry about wasted processing because this clause acts as afilter and processing stops quickly for devices that are not of this type.The DiscoSnmpRequests clause specifies which variable to retrieve. Thisvariable may be referenced by the last word in the OID name, if it is unique, or bya more qualified name, or the dotted decimal identifier if necessary. In our case,the variable for workgroup switches is called chassisSerialNumberString, whichwas unique enough. In this context, unique means that it is the only occurrenceof a MIB variable by that name in any MIB file stored in$NCHOME/precision/mibs. For the other devices, the MIB was not on the systemalready and we had to add it. That meant finding theOLD-CISCO-CHASSIS-MIB-V1SMI.mib on the Cisco Web site and storing it in$NCHOME/precision/mibs. The variable name for the serial number in that MIBis chassisId, and that was also unique enough to use without qualification.The DiscoSnmpRequests clause also defines the field that the variable shouldbe stored in, and the DiscoAgentProcLayerAddTags clause also references thisfield name. It is customary to name it m_YourNameHere. We called oursm_ciscoSerial. This is a new sub-field in the ExtraInfo field that already exists,and it created just by referencing it here.This same approach could be taken for any vendor’s equipment which provides aserial number via SNMP. If you have that information, you might want to use avendor-neutral name for your field rather than m_ciscoSerial.Add the custom agents to the agents listThe agents list is $NCHOME/etc/precision/DiscoAgents.<domain>.cfg. Wecopied the entry for ExtraDetails, twice, and added our agent names. Note that Chapter 6. Migrating NetView and Switch Analyzer 127
  • 143. you can enable and disable these entries by setting the m_Valid value to 1 or 0. See how we set ours in Example 6-8. Example 6-8 Custom agents in the agents lists insert into disco.agents ( m_AgentName, m_Valid, m_AgentClass, m_IsIndirect, m_Precedence ) values ( "CiscoSerialOld", 1, 0, 0, 2 ); insert into disco.agents ( m_AgentName, m_Valid, m_AgentClass, m_IsIndirect, m_Precedence ) values ( "CiscoSerialWorkgroup", 1, 0, 0, 2 ); Select the agents in the discovery You will need to stop/start ncp_ctrl to get the revised agents list read. Once the agents show up in the Device Support tab of the discovery configuration, check the boxes to have them run on your next discovery.128 Migrating to Netcool/Precision for IP Networks
  • 144. Making use of added fieldsThe field we added can be seen in the Show Details display that is launched fromthe context menu from any node. Expanding the ExtraInfo section showed ouradded field, m_ciscoSerial, along with a number of other attributes as shown inFigure 6-9.Figure 6-9 Added field in ExtraInfoBecause it is a sub-field of ExtraInfo, it is not offered as a field that you couldauto-partition on, or use in a view definition. Of course we would not use this fieldin that way, but we would certainly use other ExtraInfo fields in that way. Toenable that, an ExtraInfo field can be associated directly with a node field byadding that assignment to the $NCHOME/etc/precision/ModelToMySQL.cfg bycopying the original file to$NCHOME/etc/precision/ModelToMySQL.REDBOOK_P.cfg and making ourchanges. We added our new field here, and we also added a linkage for theSNMP sysLocation field as shown in Example 6-9. That field is collectedautomatically, but was not automatically mapped. Figure 6-9 shows how to mapadditional node fields in ModelToMySQL.cfg.Example 6-9 Adding a custom field to ModelToMySQL.REDBOOK_P.cfg########optional#######mainNode:Description, text, Description;mainNode:EntityOid, text, EntityOID;mainNode:SysName, text, ExtraInfo->m_SysName;mainNode:BGPAs, text, ExtraInfo->m_As;mainNode:OSPFBdrRtr, int, ExtraInfo->m_OSPF->m_IsBdrRtr;mainNode:ASBdrRtr, int, ExtraInfo->m_OSPF->m_IsAsBdrRtr;mainNode:ASMsRunning, text, ExtraInfo->m_ASMsRunning;# Mappings added for Redbook Chapter 6. Migrating NetView and Switch Analyzer 129
  • 145. mainNode:Location, text, ExtraInfo->m_SysLocation; mainNode:ciscoSerial, text, ExtraInfo->m_ciscoSerial; This will cause Location and ciscoSerial to show up in the field list for the creation of views. To get it to take effect, kill the npc_model_to_mysql process. It will be restarted automatically by ncp_ctrl. It also adds the fields to the content of the More Info pop-up that appears when you select a device in the Device Info tab of the table at the bottom of a network view. Another file that you may want to update when you discover new fields is $NCHOME/precision/etc/ This allows you to add the fields to the tables that appear at the bottom of a network view for the selected devices. Since our new field applies to the devices as opposed to the interfaces, we added it to the list of fields displayed on the Device Info tab. While we were at it, we also added the Location field to this display. This display should be tailored to suit your needs. Example 6-10 shows the changes we made to for these two device fields.Example 6-10 Adding a custom field to the file## Device Info columns to display.## The list can contain any of the column names from the mainNodeDetails table## from the MySQL database used by TopoViz.#topoviz.client.deviceInfoList=EntityName,IPAddress,EntityOid,ClassName,SysName# Added columns for the Redbooktopoviz.client.deviceInfoList=EntityName,IPAddress,Location,EntityOid,ClassName,SysName,ciscoSerial Once that change was made, the results were visible on the next reload of the browser as shown in Figure 6-10.130 Migrating to Netcool/Precision for IP Networks
  • 146. Figure 6-10 Device Info table with custom columns added6.4 Migrating the network map visualization The user of NetView is accustomed to different views on the network, specifically: Network maps SmartSets All the network visualization in Netcool/Precision is done via views where you can accomplish similar results. To work on views, select within the Precision Desktop the tab Topoviz Network View, as shown in Figure 6-11 on page 132. Chapter 6. Migrating NetView and Switch Analyzer 131
  • 147. Figure 6-11 Topoviz Network View from the Precision Desktop6.4.1 Migrating SmartSets SmartSets in NetView are based on rules. All Objects that meet the rule’s criteria are displayed in a certain smartset. We extracted all smartsets with their defining rules from the existing NetView installation earlier on. See 6.2, “Gathering information from the NetView server” on page 105 and Appendix B.1, “Commands and scripts used to extract information from the NetView installation” on page 242 for details. Often SmartSets are used during discovery to collect devices which are not completely or correctly discovered, for example the unknown OIDs SmartSet, or the non-SNMP Smartset. Implementing these as Network Views in Topoviz was described in 6.3, “Migrating the discovery” on page 106. You can also extract this information from Netcool/Precision using an OQL query as shown in Example 6-11 on page 133.132 Migrating to Netcool/Precision for IP Networks
  • 148. Example 6-11 Extract all unknown OID devices from OQL[netcool@precision2 netcool]$ ncp_oql -domain REDBOOK_P -service Model-username admin -password -query "selectEntityName,Address,ClassName,EntityOID from master.entityByName whereEntityType=1 and ClassName = Device and EntityOID is not null;"ncp_oql ( Netcool/Precision OQL Interface )Copyright (C) Micromuse Ltd., 1997-2006. All Rights Reserved.Netcool/Precision Version 3.6 (Build 13) created by lfredric at18:11:46 Tue Aug 15 BST 2006Executing query:select EntityName,Address,ClassName,EntityOID from master.entityByNamewhere EntityType=1 and ClassName = Device and EntityOID is not null;.{ EntityName=precision1; Address=[,00:02:55:7B:02:B1,]; ClassName=Device; EntityOID=;}{ EntityName=objserver2; Address=[,00:06:29:19:DF:61,]; ClassName=Device; EntityOID=;}{ EntityName=NetView1; Address=[,00:80:C8:64:28:5D,]; ClassName=Device; EntityOID=;}{ EntityName=NetView2; Address=[,00:60:08:A6:E4:0A,]; ClassName=Device; EntityOID=;}( 4 record(s) : Transaction complete )ncp_oql is dead.[netcool@precision2 netcool]$ Chapter 6. Migrating NetView and Switch Analyzer 133
  • 149. Note: There are slight differences between the syntax in the GUI and on the OQL command line. For example, EntityOid like in the GUI would be the equivalent of EntityOID like in the OQL command line. If there are only a small number of devices with unknown OIDs the graphical approach described might be sufficient. We also created a perl script that will extract all unknown OIDs together with the system description from the precision database (see Appendix B.2.1, “Perl script to extract all unknown OIDs from Precision” on page 264); however, it is a good idea to keep the view “unknown OIDs” in case new devices with unknown OIDs are discovered later. Altering class definition In order for Netcool/Precision to correctly classify devices, there must be a class whose rule matches this specific OID. Class definitions are stored in *.aoc files. These can be edited manually or by using a GUI. We recommend changing the class definitions by using the GUI. Note: *.aoc files not only contain class definitions, but also, for example, polling definitions. There are certain times when the *.aoc files have to be edited directly, for instance when adding or changing monitoring policies. See “Migrating network monitoring” on page 148 for details. Issue the following command to start the monitor configuration GUI: ncp_monitorconfig & Figure 6-12 on page 135 shows the login screen and Figure 6-13 on page 136 shows the hierarchical class structure.134 Migrating to Netcool/Precision for IP Networks
  • 150. Figure 6-12 Monitor configurator login screen Chapter 6. Migrating NetView and Switch Analyzer 135
  • 151. Figure 6-13 Hierarchical class structure Note: There are many more classes in a new installation. In our lab environment we removed all classes that we did not need in order to come up with a simple structure. Tip: The class structure is evaluated top to bottom and left to right until the most precise match is found. Be careful when defining new or altering existing classes to make sure that they do not overlap in the same line and that the rules of parent classes always include those of the child classes. Otherwise, these will not be reached. We first edited the Linux class by adding the EntityOID to the rule (Figure 6-14 on page 137).136 Migrating to Netcool/Precision for IP Networks
  • 152. Figure 6-14 Filter of the Linux class By doing this, the next time the devices are modeled the Red Hat servers would be classified as “Linux.” To make it even more specific we created a subclass of Linux, called “Red Hat,” which has only EntityOID in the rule (Figure 6-15 on page 138). Chapter 6. Migrating NetView and Switch Analyzer 137
  • 153. Figure 6-15 Filter of the Red Hat class After we do this, the GUI creates or alters *.aoc files in its output directory with the classname and domainname. For the Linux class the file shown in Example 6-12 is created. Example 6-12 Editing Linux.REDBOOK_P.aoc //************************************************************* // File : Linux.aoc // // Automatically generated at: Thu Nov 9 17:38:26 2006 // by the ClassFileManager. //*************************************************************138 Migrating to Netcool/Precision for IP Networks
  • 154. active object Linux{super_class = Device; data_dictionary = []; instantiate_rule = "EntityOID = OR EntityOID = EntityOID ="; extension for Fault = { rules = [] , poll_list = [] }; visual_icon = ; menu_rules = []; visual_menu = [];};For the Red Hat class the file shown in Example 6-13 is created.Example 6-13 Editing Red Hat .aoc//*************************************************************//// File : Red Hat .aoc//// Automatically generated at: Thu Nov 9 17:57:27 2006// by the ClassFileManager.////*************************************************************active object Red Hat { super_class = Linux; Chapter 6. Migrating NetView and Switch Analyzer 139
  • 155. data_dictionary = []; instantiate_rule = "EntityOID ="; extension for Fault = { rules = [] , poll_list = [] }; visual_icon = ; menu_rules = []; visual_menu = []; }; [netcool@objserver2 aoc]$ You would repeat this process until all OIDs of your network are included in a class. To have changes to AOC files occur before Netcool/Precision you must stop ncp_model and ncp_class. Make sure ncp_class is restarted before restarting ncp_model. Important: Defining a class for a certain device does not automatically mean that this device is correctly discovered. It is now only classified. For example, if there is an unknown switch, assigning an appropriate class for this switch would make it possible to show this in a certain view with a special icon, but it still may not be possible to correctly discover all its interfaces because there is no appropriate agent. Auto partitioning A big advantage in Netcool/Precision is the auto-partitioning wizard. You can use it to create a series of views once all devices are classified. If you run the wizard and select “predefined auto-partitions” it will create a selection of views that depend on the data found during discovery. Since the wizard creates a view for each attribute it can group on, and is limited to only the data it finds, you will have an instant overview of the type of network devices of your network. In other words, there will be a “cisco 29xx” view only if cisco 2900 routers were discovered in your network. The result of running the auto-partitioning wizard is shown in Figure 6-16. Note that there is a view created for device class “Redhat.”140 Migrating to Netcool/Precision for IP Networks
  • 156. Figure 6-16 Auto partitioning Note: The auto-partitioning wizard creates views based on the information that is present at the time it is run. If a device is discovered later on that does not match any of the existing criteria, it will not show up in any view. This is a good reason for always having an “All” view. If devices are discovered later that would fall into different views, the auto-partitioning wizard could be run again or new views could be added manually. Creating various views Similar to smartsets in NetView, views can be created based on any criteria. Using the definition of smartsets extracted from NetView it should not be difficult to create an equivalent view in precision. Note that the attribute names and the filter syntax is different in NetView and Netcool/Precision. We created a view to match the “cisco devices” smartset from NetView. The NetView rule SNMP sysObjectID ~ would translate to the filter EntityOid like as shown in Figure 6-17 on page 142. Chapter 6. Migrating NetView and Switch Analyzer 141
  • 157. Figure 6-17 Create the cisco device view The view that results from this filter is shown in Figure 6-18 on page 143.142 Migrating to Netcool/Precision for IP Networks
  • 158. Figure 6-18 Cisco device view Note: The newly created view will display all cisco devices, regardless of whether they are also in the views created by the partitioning wizard. However, the auto-partitioning wizard might create a view “cisco” parallel to the other device classes. In this view you will find only cisco devices that are not further classified. This is because of the hierarchical structure of the classification: these devices match the criteria of “cisco” but no criteria further down. If the partitioning wizard creates such a view you should investigate further into these devices.6.4.2 Migrating the network view Many customers have customized the network view in NetView by including locations. This can be done manually or it can be accomplished automatically Chapter 6. Migrating NetView and Switch Analyzer 143
  • 159. during discovery by providing rules for creating locations and assigning subnets and devices to them. There is not yet a comparable feature in Netcool/Precision. However, you can achieve similar results using different techniques. Creating views based on location information A good practice is to create views based on either the fully qualified domain name (if DNS is set up this way) or the setting of SysLocation. We decided to group by SysLocation because we can use the auto-partitioning wizard to do this. Note: You cannot use the auto-partitioning wizard to group by DNS name because the wizard can only group by the whole attribute. In the case of DNS names each name is different and this would lead to one view per device. However, it is possible to group by DNS name manually by creating views with filter criteria such as EntityName like and so on for each location. We started the auto-partitioning wizard but selected “Specify custom auto-partitions” this time, which leads to the screen shown in Figure 6-19.Figure 6-19 Auto partitioning based on SysLocation144 Migrating to Netcool/Precision for IP Networks
  • 160. Select the Table mainNodeDetails and the Field sysLocation. Click Next, give it a name, and select the parent view. Click Finish and Go. You can see the result of the wizard in Figure 6-20. Note that you see in the tree any value that occurs in the device set. You can immediately see errors in the settings in the devices. In our case there are some locations of “Unknown ...”. You can use this information to correct the settings on the devices.Figure 6-20 result of auto-partitioning based on SysLocation Creating hierarchical views You can build a hierarchical representation manually, as shown in Figure 6-21, by creating a set of views manually as type “container only” and selecting the appropriate parent view, for example, world - usa - texas. The views that should contain the devices (for example, austin with parent texas) must be created with appropriate filters, such as syslocation. You can use the views created by the Chapter 6. Migrating NetView and Switch Analyzer 145
  • 161. auto-partitioning wizard and just change the parent view to put the views in the place where they belong.Figure 6-21 Hierarchical view with background map Note: Connections are not drawn between sub-views, or between devices and sub-views, in this version of precision. Only connections between devices are drawn. Fine tuning the layout with maps and icons Like in NetView it is possible to improve the layout of a view by assigning a background map. Background images must be in the folder $NCHOME/etc/precision/resource, and they should be in PNG, JPEG, or GIF formats. The images of devices or sub-views can be positioned on the map by dragging them with the cursor.146 Migrating to Netcool/Precision for IP Networks
  • 162. You can specify icons for devices based on the device class. Place the icons in $NCHOME/etc/precision/resource. Edit the file $NCHOME/etc/precision/ by adding a line like: topoviz.deviceicon.Redhat=server7.png We customized the appearance of the Red Hat devices as you can see in Figure 6-22.Figure 6-22 Customized icons Note: Because the views are cached in the GUI you may have to log out and log in again to see these settings take effect. Chapter 6. Migrating NetView and Switch Analyzer 147
  • 163. 6.5 Migrating network monitoring This section covers the configuration of polling in Netcool/Precision based on the status polling configuration for NetView, and the SNMP threshold polling configuration for NetView. Validate the results by reviewing the events that are generated by the monitoring. Expect some differences. The goal of this section is to set up active availability monitoring of the discovered network using ping and SNMP link requests, and for monitoring network performance by setting up threshold polls for bandwidth utilization and avgBusy5 as examples. Review the manual Netcool/Precision for IP Networks Monitoring and RCA Guide for more information. NetView users will see there are several differences in monitoring compared with Netcool/Precision. The monitoring component is responsible for detecting specific network changes, whether they are availability or performance related, and issuing events to the ObjectServer. These events are enriched with topology information and root cause classification. To evaluate the state of an object you view the events associated with it. The state is not recorded as a attribute of the object as in NetView, but it is readily available from the event view.6.5.1 Tivoli NetView preparation The first step is to review polling within NetView. Ping 1. Determine the standard polling frequency, time-out, and retry counts that are used. 2. Make a note of non-standard areas and how you might classify the target groups in Netcool/Precision. In our example we polled the routers at 2 minute intervals and everything else at 5 minutes. Although NetView timeouts and retries were different, the Netcool/Precision defaults were equally good. SNMP link status These polls equate to NetView’s SNMP status and should be set for all devices with unnumbered ports. Determine frequency, time-out, and retry count, and how you might group the targets in Netcool/Precision. In our example, we polled the switches for link status every 5 minutes.148 Migrating to Netcool/Precision for IP Networks
  • 164. SNMP thresholds Review the data collections configured in NetView’s snmpCol.conf file for threshold evaluation. Note the MIB variable or expression, threshold and rearm values, frequency and targets. In our example, we set thresholds on the bandwidth utilization and avgBusy5 for Cisco devices.6.5.2 Netcool/Precision preparation In Netcool/Precision we chose to edit the Active Object Class files ($NCHOME/precision/aoc/) to define the monitoring policies. To do this successfully, you must understand the class hierarchy and the file structure of these files. We could have used the MONITOR Configuration tool to make changes to existing policies, but we needed to edit the files to add the bandwidth utilization poll definition. Refer to the manual Netcool/Precision IP Discovery Configuration Guide 3.6, Chapter 15, for details about the AOC file and class structure. As part of the thought process to plan the policies for monitoring, we considered the following steps to devise a scheme that would be easy to maintain in the future for our network. 1. Start with a clearly defined outline of the polling policies and the groups you want them to apply to. In our case it looks like this: – Ping routers every 2 minutes. – Ping everything else every 5 minutes. – SNMP link status for switches every 5 minutes. – Apply threshold polls for avgBusy5 to all Cisco devices. – Apply threshold polls bandwidth utilization to all Cisco devices. 2. Consider the class structure as a whole and where to put the polling policies to minimize duplication. Balance this against the complexity of the Scope expressions for inclusion and exclusion. We re-parented the Linux class under Device instead of DataEntity, as shown in Example 6-23, so the polling policies would be available to Linux class objects as well. Note: When modifying Scope expressions for the classes, remember that the matching algorithm evaluates classes in the same peer group alphabetically. Therefore, DataEntity would be tried before Device. Chapter 6. Migrating NetView and Switch Analyzer 149
  • 165. Note: Make sure an object matches the class Scope expressions in each class all the way up the chain.Figure 6-23 Class hierarchy change for Linux 3. Minimize the complexity of the Scope expressions to match the objects to the policies. Generally speaking, put the polling policy high in the hierarchy to avoid having to duplicate it. Balance this against the complexity of excluding objects that match the class, but that you do not want in the policy.6.5.3 Configure ping polling An event is issued on every poll that matches the failure criteria. Unlike NetView, subsequent failure events are not suppressed. Instead, the duplicate events150 Migrating to Netcool/Precision for IP Networks
  • 166. increment the tally in the event display, giving a sense of how long it has failed. Aclear event is then issued the first time the failure criteria is not met.The CHASSISPING pings the management interface, typically the loopback onrouters. Events are triggered independently, so for instance when a router goesdown there will be an event for each interface as it fails its PING poll. In order toeasily recognize that the router is down, and not just some of its interfaces, youshould configure the CHASSISPING to poll more frequently to provide noticationof the node down state. Topology-based RCA will mark the CHASSISPING asthe root cause when both events fire.See 6.5.8, “Understanding how interfaces are managed” on page 156 for adiscussion of how Netcool/Precision automatically manages and unmanagesinterfaces and how you can change it.We edited the $NCHOME/precision/aoc/Device.aoc file to modify the two pingpolls.1. We created the interface pings. These will ping all the IP addresses associated with the node. For each target group, we cut and pasted the defaultInterfacePing template and renamed the PollName to identify the target group, as in Example 6-14. We used the default values for the retry count and time-out. We set the PollStatus to 0 so that the default polls do not occur and double-poll the interfaces. Our naming scheme allowed us to differentiate the target groups based on the hostname to provide a simple example, but you can use any suitable attribute. The second target group was for everything else, so we achieved this by changing the Scope expression to use the inverse of the hostname like rtr.*. Example 6-14 Interface ping for routers { /------------------------------------- // // defaultInterfacePing pollDef // // Use a timed stitcher agent (key PING) // to perform the pinging // //------------------------------------- PollName="RouterInterfacePing", PollStatus=0, AgentName="ncp_m_timedstitcher", AgentKey="PING", Chapter 6. Migrating NetView and Switch Analyzer 151
  • 167. StitcherName="DefaultPing", Frequency=120, Scope = "IsActive = 1 AND EntityType = 2 AND ExtraInfo->m_IsManaged <> 0 AND ExtraInfo->m_HasMainEntityIp <> 1 AND ExtraInfo->m_BaseName like rtr.* ", StitcherInfo={ EventName=NmosPingFail, Severity=3, TimeOut=3000, Retries=2 } }, 2. We created the chassis pings to test the state of the node itself. We cut and pasted the defaultChassisPing template block and renamed the PollName to identify the target group as in Example 6-15. Note that the Frequency was set to a value much less than the interface ping so that if the node goes down, this alert will be sent and topology-based RCA can determine the root cause early, before most of the interface alerts are sent. We used the default values for the retry count and time-out. Example 6-15 Chassis ping for routers { //------------------------------------- // // defaultChassisPing pollDef // // Use a timed stitcher agent (key CHASSISPING) // to perform the pinging // //------------------------------------- PollName="RouterChassisPing", PollStatus=1, AgentName="ncp_m_timedstitcher", AgentKey="CHASSISPING", StitcherName="DefaultPing", Frequency=30, Scope = "EntityType = 1 AND IsActive = 1 AND ExtraInfo->m_BaseName like rtr.* ", StitcherInfo={152 Migrating to Netcool/Precision for IP Networks
  • 168. EventName=NmosPingFail, Severity=3, TimeOut=3000, Retries=2 } },3. We created a chassisPing for everything else using a frequency of 5 minutes, as in Example 6-16. Tip: When finished, examine each poll definition to make sure you have not inadvertently matched some devices in multiple ping definitions. Example 6-16 Chassis ping for non-routers { //------------------------------------- // // defaultChassisPing pollDef // // Use a timed stitcher agent (key CHASSISPING) // to perform the pinging // //------------------------------------- PollName="nonrouterChassisPing", PollStatus=0, AgentName="ncp_m_timedstitcher", AgentKey="CHASSISPING", StitcherName="DefaultPing", Frequency=300, Scope = "EntityType = 1 AND IsActive = 1 AND ExtraInfo->m_BaseName not like rtr.* ", StitcherInfo={ EventName=NmosPingFail, Severity=3, TimeOut=3000, Retries=2 } }, Chapter 6. Migrating NetView and Switch Analyzer 153
  • 169. 6.5.4 Configure SNMP link polling We used the SNMP link polling for the switches. To identify devices supporting layer 2 ports, we test that ExtraInfo->m_Dot1dBasePort is not NULL, as in Example 6-17. An alternative if we only wanted to poll the layer 2 switches would be to copy this Poll description to the CiscoNonRoutingSwitch class and any other layer 2 switch class. Using the class definitions for selection is more efficient. However, we chose this method so that both non-routing and routing switches would be included. Example 6-17 SNMP link polling { PollName = switchesSnmpLinkState2, AgentName = ncp_m_timedstitcher, AgentKey = DevicePolls, StitcherName = AocDefinedThreshold, // // control parameters // PollStatus = 1, Frequency = 300, Scope = "EntityType = 1 and Contains is not null and IsActive = 1 AND ExtraInfo->m_BaseBridgeAddress is not NULL ",6.5.5 Configure SNMP threshold polling A large number of useful threshold definitions are included in the AOC files out of the box. We show two examples, one predefined for avgBusy5 and the other for bandwidth utilization, to show how to add a new threshold expression. AvgBusy5 We set it to poll every 5 minutes as in Example 6-18. To adjust the targets, we could limit the scope using any attribute, including the classname. Example 6-18 Threshold definition for avgBusy5 { PollName="cpuBusyPoll", PollStatus=1, AgentName="ncp_m_timedstitcher", AgentKey="CiscoPolls",154 Migrating to Netcool/Precision for IP Networks
  • 170. Frequency=300, StitcherName="AocDefinedThreshold", Scope="EntityType = 1 and IsActive = 1", StitcherInfo={ EventName=resourceCPU, Severity=4, Varbinds= [ "avgBusy5" ], Threshold= ( eval(int,"&SNMP.VALUE.avgBusy5") >= 80 ), Description=CPU usage high (avgBusy5= eval(int,"&SNMP.VALUE.avgBusy5")), ClearThreshold= ( eval(int,"&SNMP.VALUE.avgBusy5") < 80 ), ClearDescription=CPU usage normal (avgBusy5= eval(int,"&SNMP.VALUE.avgBusy5")) } }, Bandwidth utilization This is a common metric to monitor and can be added to the AOC files. For convenience an implementation of bandwidth utilization for the threshold polling is shown in Appendix B.2.4, “Sample of threshold polling definition to be put into *.aoc file” on page 275. There are two threshold definitions included, one for inbound traffic and the other for outbound. We added these definitions to the Cisco.aoc file so that all Cisco devices could be polled. We could adjust the Scope if we only wanted to poll a subset of Cisco devices. The Frequency is set to 5 minutes, and the threshold value is set to 40%.6.5.6 Activating the changes Typically you would make changes to the AOC files during a regular maintenance period and the changes would become active on restart of the product. If you make changes to the AOC files while the product is running, you must restart the following processes in this order: 1. ncp_class Stop this process, letting ncp_ctrl automatically restart it. Make sure it comes up and stays up, indicating there are no syntax problems with the AOC files. 2. ncp_monitor Stop this process after ncp_class has restarted. Let ncp_ctrl automatically restart it. Chapter 6. Migrating NetView and Switch Analyzer 155
  • 171. 6.5.7 Passive monitoring If you configure the network devices to send SNMP traps to the OMNIbus server, mttrapd probe will receive the traps and issue events to the ObjectServer. We enabled the Netcool Knowledge Libraries during installation (see 5.4.3, “Install and verify Netcool Knowledge Library” on page 91), which has over a thousand traps predefined. These events are recognized by topology-based RCA and therefore contribute an instant notification of a network failure.6.5.8 Understanding how interfaces are managed Tivoli NetView users should understand the default behavior of how interfaces are automatically managed and unmanaged with respect to monitoring and how to change this behavior if desired. The default behavior is designed to prevent critical events being generated for uninteresting interfaces. Interfaces down at discovery will not be polled During a full discovery, interfaces that are down or dormant (operStatus is 2 or 6) are automatically unmanaged by setting the IsActive flag to 0. The monitoring policies, by default, will not poll interfaces with the IsActive flag set to zero, as seen in the previous examples. After the interface is fixed, Netcool/Precision monitoring will not detect this until the next full or partial discovery, when the IsActive flag will be set to 1 again. To change this and prevent the discovery from setting the IsActive flag to 0, simply rename the following stitcher: cd $NCHOME/precision/disco/stitchers mv CheckInterfaceStatus.stch CheckInterfaceStatus.disabled Interfaces automatically unmanaged at discovery The file $NCHOME/precision/disco/stitchers/TagManagedEntities.stch lists the criteria for setting an interface to the unmanaged state as shown in Example 6-19. Example 6-19 Criteria to automatically unmanage interfaces ExecuteOQL(" UPDATE scratchTopology.entityByName SET m_ExtraInfo->m_IsManaged = 0 WHERE ( m_ExtraInfo->m_IfDescr like Dialer OR m_ExtraInfo->m_IfDescr like Async OR m_ExtraInfo->m_IfDescr like Virtual OR m_ExtraInfo->m_IfDescr like Null OR156 Migrating to Netcool/Precision for IP Networks
  • 172. m_ExtraInfo->m_IfDescr like NULL OR m_ExtraInfo->m_IfDescr like Vlan OR m_ExtraInfo->m_IfDescr like VLAN OR m_ExtraInfo->m_IfAlias like NoMon ); "); To change this behavior, edit this file and change the criteria.6.5.9 Enabling new node events By default, Netcool/Precision does not issue events for new nodes that are discovered. NetView users might be accustomed to monitoring new nodes through the Node Added events. To enable this feature in Netcool/Precision, edit the file $NCHOME/etc/precision/ModelSchema.cfg and change the value inserted for Entity Creation Event to 1 as in Example 6-20. Example 6-20 Enable new node events create table model.config ( LingerTime int not null primary key, // default value 3 (discoveries) ClassTimeOut int not null, // default value 5 (seconds) EntityCreationEvent int type boolean not null, // default value 0 ( off ) unique(LingerTime) ); insert into model.config values (3, 5, 1); Figure 6-24 shows an example of new entity events with the sysObjectId by default.Figure 6-24 New entity events Chapter 6. Migrating NetView and Switch Analyzer 157
  • 173. 6.5.10 Examples of the monitoring events Figure 6-25 shows examples of some of the events from our monitoring policies.Figure 6-25 Examples of monitoring events6.6 Netcool OMNIbus automations The Netcool solution can be configured to perform automations on events. To demonstrate this ability, we decided to modify the built-in Netcool/OMNIbus example named “mail_on_critical.”158 Migrating to Netcool/Precision for IP Networks
  • 174. 6.6.1 Mail on critical automation Automations in Netcool OMNIbus can be created, deleted, and modified using the Netcool OMNIbus Administrator tool. We launched this tool with the command shown in Example 6-21. Example 6-21 Launching the Netcool/OMNIbus Administrator GUI $cd $OMNIHOME/bin $./nco_config Note: Access to the Netcool/OMNIbus Administrator from a UNIX server requires the ability to launch native UNIX GUIs. This can be done by working at the server console, using an X-Windows server on a remote machine, or using a tool such as VNC. The Netcool/OMNIbus Administrator tool can be run on either Windows® or UNIX systems. You can use the tool from one operating system even if the ObjectServer is installed on a different operating system. Note: The administrator tool, nco_config has its own properties file, /opt/netcool/omnibus/etc/nco_config.props, which has its own reference to the license server. If this is misconfigured, you will be unable to launch nco_config even though everything else is working. Using the Netcool/OMNIbus Administrator, we navigated to the Automation Triggers screen as shown in Figure 6-26 on page 160. Chapter 6. Migrating NetView and Switch Analyzer 159
  • 175. Figure 6-26 The Automations → Triggers page We chose mail_on_critical, which opened up the screen shown in Figure 6-27 on page 161.160 Migrating to Netcool/Precision for IP Networks
  • 176. Figure 6-27 Enabling mail on critical From the Evaluate tab we can see that the default settings for the mail_on_critical trigger look for critical events which are 30 minutes old and have not been acknowledged, as shown in Figure 6-28 on page 162. Chapter 6. Migrating NetView and Switch Analyzer 161
  • 177. Figure 6-28 Default settings for mail on critical trigger In the Action tab we made two changes. First, we changed the mail recipient parameter to netcool@localhost. We then modified the host parameter from localhost to precision2. In this tab, we can also see that this trigger performs one external function as well as one internal function. In addition to running an external script by passing arguments to the send_email() procedure, the internal alerts.status table is updated for the event and the flag, Grade, is set to 2. This prevents the trigger from sending an e-mail for this event each time the trigger is run. Note the order of the parameters passed to the external function send_email in Figure 6-29 on page 163.162 Migrating to Netcool/Precision for IP Networks
  • 178. Figure 6-29 Temporal Triggers Use the checkmark tool to verify the sql at each phase. It will validate the correctness of the total statement, as well as the parameter datatype matching between the trigger and the procedure. The final change that we made to configure and enable our mail_on_critical automation was to edit the procedure using the Netcool/OMNIbus Administrator tool, nco_config. Figure 6-30 on page 164 shows that we set the Host to our server: precision2. Chapter 6. Migrating NetView and Switch Analyzer 163
  • 179. Figure 6-30 External Procedure Details screen Attention: External procedures require that PA be functioning. Check the log file /opt/netcool/omnibus/log/<NCO_PA>.log for problems.164 Migrating to Netcool/Precision for IP Networks
  • 180. This took effect as soon as the trigger was enabled and saved. The mail came in looking like Example 6-22. Example 6-22 E-mail sent by event automation [netcool@precision2 utils]$ mail Mail version 8.1 6/6/93. Type ? for help. "/var/spool/mail/netcool": 5 messages 5 new >N 1 netcool@precision2.i Fri Nov 17 17:20 22/1004 "Netcool Email" N 2 netcool@precision2.i Fri Nov 17 17:20 22/1001 "Netcool Email" N 3 netcool@precision2.i Fri Nov 17 17:20 22/1003 "Netcool Email" N 4 netcool@precision2.i Fri Nov 17 17:20 22/1001 "Netcool Email" N 5 netcool@precision2.i Fri Nov 17 17:20 22/1003 "Netcool Email" & 5 Message 5: From Fri Nov 17 17:20:34 2006 Date: Fri, 17 Nov 2006 17:20:34 -0600 From: To: Subject: Netcool Email This message refers to node which has the following problem; Ping fail for ICMP Host Unreachable: host unreachable from gateway The Severity is 5 Sent by the Netcool/OMNIbus Automation system Once you have verified that you can trigger e-mails to root@localhost, you can move on to verifying that SMTP mail is correctly configured on the server just as you would with similar automation done from NetView. Then you can send your automatic e-mails elsewhere. The destination address can be calculated in the trigger based on other attributes of the event, such as the sysContact for the device in question.6.6.2 Event enrichment One powerful feature of Netcool/Precision is its ability to enrich events using any of the information that it retrieved during the discovery process, for instance: System location System contact Chapter 6. Migrating NetView and Switch Analyzer 165
  • 181. Interface description Interface alias While there are several ways to perform event enrichment using Netcool/Precision, the most common method is to use the Precision gateway. In our lab we decided to enrich our events using the name of the device (BaseName), the sysLocation (Location) and sysContact (Contact). The first step that we took was to create new fields in Netcool/OMNIbus for this information. To do this, we used the Netcool/OMNIbus Administrator tool. After selecting the alerts.status table, we used the Add Column tool as shown in Figure 6-31 on page 167.166 Migrating to Netcool/Precision for IP Networks
  • 182. Figure 6-31 Adding a new column to Netcool/OMNIbus’ alert.status table Chapter 6. Migrating NetView and Switch Analyzer 167
  • 183. The menu option for adding a new column produces a new window that asks for the column definition. The information we used for our new Contact field is shown in Figure 6-32. Figure 6-32 Entering details for new Contact field We created new fields named Contact, Location and BaseName. We created each of these as VarChar data types with a length of 64. Note: Adding new columns to the alerts.status table is dynamic, but all probes and gateways should be stopped and restarted in order for them to continue to operate properly after the new fields have been created in the ObjectServer. Once the new fields were created in our ObjectServer, we modified our $PRECISION_HOME/etc/NcoGateSchema.REDBOOK_P.cfg file to map the fields from Netcool/Precision to Netcool/OMNIbus. Important: It is not guaranteed that all interface events will be enriched with topology chassis object attributes using the NcoGateSchema approach alone. To make sure the chassis object attributes are available to enrich all interface events in all circumstances you should additionally use the stitchers described in 7.6, “Enriching interface events with chassis object attributes” on page 226 to copy these chassis attributes to the topology interface objects. Example 6-23 is an example of the ncp2nco section of our gateway configuration file using the new mappings.168 Migrating to Netcool/Precision for IP Networks
  • 184. Example 6-23 NcoGateSchema.REDBOOK_P.cfg insert into config.ncp2nco ( EventFilter, EventFieldMap ) values ( "ActionType <> 2", { Severity = "eval(int, &Severity)", NmosObjInst = "eval(int, &ExtraInfo->NmosObjInst)", NmosSerial = "eval(text, &ExtraInfo->NmosSerial)", Location = "eval(text, &&ExtraInfo->m_SysLocation)", Contact = "eval(text, &&ExtraInfo->m_SysContact)", BaseName = "eval(text, &&ExtraInfo->m_BaseName)", NmosCauseType = "eval(int, &CauseType)" } ); After we modified our NcoGateSchema.REDBOOK_P.cfg file, we stopped and restarted the gateway. The current events as well as all new events in the ObjectServer were then enriched with the data from Precision’s topology as shown in Figure 6-33.Figure 6-33 Enriched events in the ObjectServer Chapter 6. Migrating NetView and Switch Analyzer 169
  • 185. 6.7 Creating users for Netcool components Users in the Netcool GUI Foundation can be configured to authenticate either internally using the Netcool Security Manager or externally using one of the three methods allowed by Netcool Security Manager. During our installation of Netcool Security Manager, we chose to have the Netcool ObjectServer as our authentication source (Figure 6-34). Figure 6-34 Selecting ObjectServer as Security Manager authentication source To interact with the full set of tools and events in the Web tools, a user must be configured in Netcool/OMNIbus. If the user is set to authenticate internally to the NGF using Security Manager, the user must also be created separately in Netcool/OMNIbus. Without a Netcool/OMNIbus logon, the NGF users can view the Active Event Lists and Topology Views; however, they will not be able to fully interact with the events. An example of the menu items that a user without OMNIbus permissions will be able to access is shown in Figure 6-35 on page 171.170 Migrating to Netcool/Precision for IP Networks
  • 186. Figure 6-35 NGF user without OMNIbus user permissions Users with Netcool OMNIbus permissions will have many more options to interact with events in the event lists, as shown in Figure 6-36 on page 172. Chapter 6. Migrating NetView and Switch Analyzer 171
  • 187. Figure 6-36 NGF user with OMNIbus user permissions From the preceeding figures, you can see that a user without OMNIbus user permission has a very limited ability to perform actions such as changing an event severity or deleting events. To make user management easier to maintain, we used the following steps to create our Netcool/OMNIbus users.6.7.1 User creation in Netcool/OMNIbus Using the Netcool OMNIbus Administration GUI, we chose to add a new user, as shown in Figure 6-37 on page 173.172 Migrating to Netcool/Precision for IP Networks
  • 188. Figure 6-37 Adding OMNIbus user We then gave the user a username, user ID, and Full Name, and checked the box to have a conversion created. The conversion allows the event lists to display the Full Name instead of the User ID. Since we wanted the new user named “TeamLeader” to have full OMNIbus privileges, we assigned the user to all available groups (Figure 6-38 on page 174). Chapter 6. Migrating NetView and Switch Analyzer 173
  • 189. Figure 6-38 Assigning groups to OMNIbus user Since we will have the NGF authenticate against the ObjectServer, we created the password for the new user inside of OMNIbus and not within the NGF. The last step was checking the Enable box to activate the user (Figure 6-39 on page 175).174 Migrating to Netcool/Precision for IP Networks
  • 190. Figure 6-39 Entering password and enabling userOnce completed, we could see our new user in the Netcool OMNIbusAdministration GUI (Figure 6-40 on page 176). Chapter 6. Migrating NetView and Switch Analyzer 175
  • 191. Figure 6-40 New user added and enabled in OMNIbus ObjectServer6.7.2 Creating user in NGF with admin permissions After our user was configured in OMNIbus, we created a user with the same name in the Security tab of the NGF, as shown in Figure 6-41 on page 177.176 Migrating to Netcool/Precision for IP Networks
  • 192. Figure 6-41 New NGF user showing external authentication For our new user in the NGF, we selected having the user authenticate externally. This will cause the login for the user to pass through the Security Manager and authenticate against the ObjectServer. The password for the user will be the one entered when we created the user in OMNIbus and future password changes will be made using the Netcool/OMNIbus Administration GUI.6.7.3 Assign user roles and groups For the user TeamLeader we assigned all roles since this user will have full administrative access to the NGF, Webtop, and Precision components, as shown in Figure 6-42 on page 178. Chapter 6. Migrating NetView and Switch Analyzer 177
  • 193. Figure 6-42 Assigning roles This user was then added to all groups other than Restricted and TestAdmin (Figure 6-43).Figure 6-43 Assigning groups178 Migrating to Netcool/Precision for IP Networks
  • 194. 6.7.4 Creating a user with operator access Next we created a user with operator-level access. This would be the typical user of the tool. The user needs access to interact with the events in Netcool/OMNIbus as well as within the NGF; however, they do not need administrative permissions for any of the products. Since the user needs to interact with the events in Netcool/OMNIbus, we first created the user using the Netcool OMNIbus Administrator (Figure 6-44). Figure 6-44 Creating TeamMember in Netcool OMNIbus This user was only assigned to the following groups: Normal ISQLWrite ISQL Chapter 6. Migrating NetView and Switch Analyzer 179
  • 195. After entering the User Name and assigning groups for the TeamMember user, we used the Settings tab to assign a password, create a conversion, and enable the user (Figure 6-45). Figure 6-45 Assigning password to TeamMember user6.7.5 Creating the operator user in the NGF Once the operator-level user TeamMember was created in Netcool/OMNIbus, we needed to use the NGF to create the user and assign roles. Figure 6-46 on page 181 shows the first page used to create the TeamMember user within the NGF. Notice that we selected the check box next to Authenticate Externally for this user, just as we did when we created the TeamLeader user. Both users will authenticate against Netcool OMNIbus so that they can interact with the events.180 Migrating to Netcool/Precision for IP Networks
  • 196. Figure 6-46 First step of TeamMember creation in NGF In the User Roles tab for the TeamMember user, we assigned the following roles: GUI Foundation read write Precision IP OQL Workbench User Precision IP Network View - Administer Views for user Webtop User Precision IP MIB Browser User GUI Foundation user GUI Foundation read only Precision IP Hop View User This is shown in Figure 6-47 on page 182. Chapter 6. Migrating NetView and Switch Analyzer 181
  • 197. Figure 6-47 Assigning roles to TeamMember user The final step to creating the TeamMember user is the assignment of groups. For this user, we selected the following groups: System Desktop GUI Foundation Views Precision Desktop This is shown in Figure 6-48 on page 183.182 Migrating to Netcool/Precision for IP Networks
  • 198. Figure 6-48 Assigning groups to TeamMember user6.7.6 Creating a limited access executive view in the NGF Executive views are often useful as a way to display information to many people who just need a high-level view of the current network status. We created an executive user in the Netcool NGF for this purpose. This user was created with the following criteria: It authenticates within the NGF and has no ObjectServer permissions to interact with the events. It has ability to see general NGF status maps and charts, however; does not have the ability to access any Topology maps. It cannot access administrative portions of the NGF. From the Users menu in the Security tab, we selected Add User. We named the new user “Executive” and gave the user a password (Figure 6-49 on page 184). Chapter 6. Migrating NetView and Switch Analyzer 183
  • 199. Figure 6-49 Creating Executive user Note: We did not select the check box next to Authenticate Externally for the Executive user. This means that this user will only authenticate against the Security Manager and not against the ObjectServer as our previous user ‘djhart’ does. Since the Executive user authenticates locally and does not have a corresponding Netcool/OMNIbus user, the user will be unable to interact with the events in the event lists. After assigning a username, password, and authentication method, we selected the User Roles for the new Executive user. The roles that we selected for this user were: Webtop User184 Migrating to Netcool/Precision for IP Networks
  • 200. GUI Foundation read only user This is shown in Figure 6-50.Figure 6-50 Assigning roles to Executive user The final step in creating the Executive user is assigning the user to a group. For our Executive user, we only added the user to the GUI Foundation Views group, as shown in Figure 6-51 on page 186. Chapter 6. Migrating NetView and Switch Analyzer 185
  • 201. Figure 6-51 Assigning groups Executive user This user can now be used for viewing read-only event lists, charts, graphs, and other high-level status views within the NGF.6.7.7 Summary of new Netcool/OMNIbus and NGF users To summarize, for the Redbook, we created 3 users: TeamLeader TeamMember Executive All 3 users were created in the NGF, as shown in Figure 6-52 on page 187.186 Migrating to Netcool/Precision for IP Networks
  • 202. Figure 6-52 Showing all 3 new users within the NGF Only the TeamMember and TeamLeader users were created in Netcool/OMNIbus, as shown in Figure 6-53 on page 188. Chapter 6. Migrating NetView and Switch Analyzer 187
  • 203. Figure 6-53 Showing new users in Netcool OMNIbus6.8 Adding tools to the user interface There are tools menus in a number of places in the Netcool suite of products, as well as multiple methods of configuring and extending them. The main areas are: The menus on the Motif interface to the Event Lists in Omnibus The menus on the Web interface to the ActiveEvent Lists in NGF The right-click menus on the Web interface to the network map views in NGF188 Migrating to Netcool/Precision for IP Networks
  • 204. There is a summary guide to all of these menus, their administration, and the product documentation for them in 7.5, “The menus in Omnibus and Netcool/Precision” on page 220. This section focuses on three tasks: 1. Ensuring that the ping tools on the Web interface, similar to Netview’s Test → Ping, work 2. Adding MIB applications similar to Netview’s Monitor → MIB Values → Interface Information to both the event and map menus of the Web interface 3. Adding a Web tool to launch a browser at a device’s management interface, similar to Netview’s Tools → Web Device Management → Homepage These working examples provide the foundation for adding other functions as required.6.8.1 The Ping tool On the Active Events List in the NGF there is a Tools menu. That menu provides two sub-menus. One is for Local Tools and one is for CGI Tools. This is a convention. The functions on the Local Tools menu execute on the local workstation. It contains a Ping, a Telnet, and a Tracepath by default. The functions on the CGI Tools menu execute on the NGF server. It contains only a Ping by default. These two Ping functions ping the device named in the Node field of the selected event. That value may be a DNS name or it may be an address. On the map views there is a right-click, or context, menu. By default, that menu includes some functions to launch things like an events list, the MIB browser, and the node details display; they do so for the device that the mouse is near. It also includes a Ping function. Then there is a section of functions that come bundled together in Precision 3.6 as the “Netcool WebTools.” Included here are an Advanced Ping and a number of Cisco-specific functions. Most of these can only be run as root user, and many require logon access to the device. However, the plain Ping in that menu should work for all users. It is the exact same tool as the Ping in the CGI Tools menu on the Active Events List. Both, in the end, execute $NCHOME/etc/webtop/cgi-bin/nco_ping.cgi. Tip: If you find that cgi scripts do not execute properly, check your server for the correct location of the perl executable. The .cgi scripts in $NCHOME/etc/webtop/cgi-bin expect to find perl in #!/usr/local/bin/perl or #!/bin/perl. We changed all of them, to match our Redhat system, to #!/usr/bin/perl. Chapter 6. Migrating NetView and Switch Analyzer 189
  • 205. Tip: If you find that cgi-bin scripts do no launch properly, check your server for the correct definition of the $PERLLIB variable. Ours, set in /etc/profile, is just what was in the online help, substituting i686-linux for sun4-solaris. PERLLIB=$PRECISION_HOME/perl/lib/5.6.1: $PRECISION_HOME/perl/lib/site_perl:$PRECISION_HOME/perl/lib/site_perl/ 5.6.1:PRECISION_HOME/perl/lib/site_perl/5.6.1/i686-linux:$PRECISION_H OME/perl/lib/5.6.1/i686-linux Getting the CGI Ping to work from both places—the Active Events List and the map right-click menu—will verify that everything is set up correctly for further additions to the menus.6.8.2 Adding a MIB application This function will launch the MIB Browser tool pre-configured to fetch the MIB II Interface Table for the desired node, found at OID This is the equivalent of Netviews MIB Application Monitor → MIB Values → Interface Info. At a high level, the procedure is as follows: 1. Create a column containing the node IP address in all events. 2. Put a custom tool that retrieves the MIB data on the Active Events List tools menu (under CGI Tools) 3. Add that tool to the right-click menu on the map views. The detailed steps follow. Step 1 The MIB Browser utility at current code levels requires an IP Address, not just a DNS name. We want tools like this to work when launched for just about any event, so we need a column in events that reliably contains an IP address. Rather than re-using an existing field in the alerts.status table, we created a new one called “NodeAddress” with a datatype of VARCHAR and a length of 64. To populate this event field with the main address of the node from Netcool/Precision, we mapped it in the nco2ncp section of the gateway configuration file $NCHOME/etc/precision/NcoGateSchema.cfg. Adding the field and mapping it involved the same steps we followed to add the Location and Contact fields, as described in 6.6.2, “Event enrichment” on page 165. The mapping now looks like Example 6-24.190 Migrating to Netcool/Precision for IP Networks
  • 206. Example 6-24 Adding NodeAddress to the gateway map NmosSerial = "eval(text, &ExtraInfo->NmosSerial)", Location = "eval(text, &&ExtraInfo->m_SysLocation)", Contact = "eval(text, &&ExtraInfo->m_SysContact)", NodeAddress = "eval(text, &&Address(2))", BaseName = "eval(text, &&ExtraInfo->m_BaseName)", NmosCauseType = "eval(int, &CauseType)", Stop and start the ncp_ncogate daemon by killing it, and verify that the NodeAddress field is populated with an IP address in the Active Events List by clicking Alerts → Information, or revising the view to include the field. Step 2 Make a script $NCHOME/etc/webtop/cig-bin/ifTableDisplay.cgi. This will be executed both by the tool on the map and by the tool on the events list. Its job is to parse parameters and construct the URL. The entire text of the script we used is in the appendix, in “ifTableDisplay.cgi” on page 287. The parameter string passed to it differs depending on whether the tool is launched from the Events List menu or from the network view menu. This is due to our need for an IP address. The differences are shown in Example 6-25.Example 6-25 Constructing the relative URL# Build a URL using passed valuesif (defined $FORM{Node}) {# If this came from a map right-click menu, the parameter will be passed as Node $URL ="/ncp_mibbrowser/$FORM{Node}";} else {# If this came from the events list menu, the parameter will be NodeAddress $URL ="/ncp_mibbrowser/$FORM{NodeAddress}";} Step 3 In the Webtop Admin page, under CGI Registry, register a new entry with these values: CGI Name: ifTableDisplay.cgi File Name: ifTableDisplay.cgi This registration will be used for the tools on both menus. Chapter 6. Migrating NetView and Switch Analyzer 191
  • 207. Step 4 In the Webtop Admin page, under Tools, create a new tool launcher with these values: Tool Name: ifTableDisplay Groups: * Execute for each row: Check it URL: $(SERVER)cgi-bin/ifTableDisplay.cgi Fields: NodeAddress Method: GET Open In: New Window Window for each row: Check it This tool launcher will only be used for the tool on the Events List menu. Step 5 In the Webtop Admin page, under Menus, create a new menu entry. Select the CGI_Tools menu, select Modify, select the ifTableDisplay item, and click Add to move it to the Current Items list. Step 6 This takes effect automatically after a few seconds. On an Active Events List, test it by selecting an event, then Tools → CGI Tools → ifTableDisplay, as shown in Figure 6-54 on page 193.192 Migrating to Netcool/Precision for IP Networks
  • 208. Figure 6-54 Addition to the AEL CGI Tools menu The tool will launch another instance of your browser displaying the MIB II Interface Table for that device, provided it is reachable and capable of responding. Sample output is shown in Figure 6-55 on page 194. Chapter 6. Migrating NetView and Switch Analyzer 193
  • 209. Figure 6-55 Sample output of ifTableDisplay.cgi Step 7 Now we want to add this tool to the right-click menu on the network map. In the directory $NCHOME/etc/precision/menus, make a menu file for custom functions on the map right-click menu, called mytools_menu.xml. The file name is not important, but the menu ID inside it is important. Ours looks like Example 6-26. Example 6-26 Making a custom submenu for the map [netcool@precision2 menus]$ cat mytools_menu.xml <ncp_menu id="mytools_menu" label="My Tools..."> <definition> <tool id="My_ifacestatus"/> </definition> </ncp_menu> It only has one tool in it for now.194 Migrating to Netcool/Precision for IP Networks
  • 210. Step 8Hook this in as a submenu on the existing menu file, ncp_webtools.xml,referencing the menu ID for our custom tools. The changed ncp_webtools.xmlnow looks like Example 6-27.Example 6-27 Integrating a custom submenu for the map[netcool@precision2 menus]$ cat ncp_webtools.xml<ncp_menu id="ncp_webtools" label="WebTools"> <definition> <tool id="ncp_wt_help"/> <tool id="ncp_wt_gui"/> <tool id="ncp_wt_ping"/> <tool id="ncp_wt_traceroute"/> <tool id="ncp_wt_whois"/> <tool id="ncp_wt_dns"/> <separator/> <menu id="ncp_wt_cisco"/> <separator/> <menu id="ncp_wt_juniper"/> <separator/> <menu id="mytools_menu"/> </definition></ncp_menu>Make sure the menu ID field matches the ncp_menu id field in the file you made.Step 9Ensure that the ncp_webtools.xml menu is hooked into as thetop menu for right-click tools on the maps. Again, it is the menu ID that issignificant, not the file name. See Example 6-28.Example 6-28 Integrating the main menu with topoviz[netcool@precision2 menus]$ tail -5$NCHOME/etc/precision/topoviz.propertiesn# Modified by Netcool/Precision WebTool installer# The following property defines which menu Topoviz should use.# Only one menuid property is supported.# If more than one menuid property is specified, only the last entrywill be used by Topoviz.ntopoviz.device.menuid=ncp_webtoolsThis will already be done if the WebTools package is installed. If it is notinstalled, specify your own menu file here instead. Chapter 6. Migrating NetView and Switch Analyzer 195
  • 211. Step 10 In $NCHOME/etc/precision/tools, make a tool launcher for the tool you already made, in a file called My_ifacestatus.xml. The name is not important, but the tool ID inside it is. Ours looks like Example 6-29. Example 6-29 Tool to call the ifTableDisplay function from the map My_ifacestatus.xml: [netcool@precision2 tools]$ cat My_ifacestatus.xml <ncp_tool id="My_ifacestatus" label="ifTableDisplay" type="url"> <url value="/webtop/cgi-bin/ifTableDisplay.cgi" target="_blank"/> </ncp_tool> The ncp_tool ID must match the tool ID field in the menu file created in Step 7 or it will not work. Notice that the label used here matches the Tool Name we specified in Step 4. This ensures that the item appears with the same label on both menus. The name of the cgi script is also the same as that created in Step 2. The target value here means the results will be displayed in a new window. Step 11 These menus are reloaded automatically every few seconds. Check the log file $NCHOME/log/guifoundation/ngf.out for any errors parsing the menu files or tools files created for the map right-click tools. Refresh the browser, and you should see the menu additions. Test it against a node in a network view and you should get the same results as before. Tip: We followed these same steps to add another MIB application to display the IP Address table. The OID to use for that is and the cgi script is in the appendix, in “ipAddrTableDisplay.cgi” on page 285. You can skip Step 1 when you add more MIB applications.6.8.3 Adding an http management tool Adding a tool that launches a browser instance to point to the selected device is like adding the MIB Browser tools already described, but with this difference: the URL that you are constructing must include an explicit path rather than a relative path. Assuming the operator’s browser has access to the same name resolution as the server does, you can work with just the Node field rather than the NodeAddress field, and use the same URL for invocations from either the Events List or the network views. The format of the URL needed is shown in196 Migrating to Netcool/Precision for IP Networks
  • 212. Example 6-30. The sample CGI script and menus are included in the appendix,in “mgmtURL.cgi” on page 289 and “My_deviceURL.xml” on page 292.Example 6-30 An explicit URL in a tool$URL = "http://$FORM{Node}:8080"; Chapter 6. Migrating NetView and Switch Analyzer 197
  • 213. 198 Migrating to Netcool/Precision for IP Networks
  • 214. 7 Chapter 7. Migration topics This chapter covers in greater detail a number of topics that were mentioned briefly in earlier chapters. Scheduled discovery Schedule full discovery at regular intervals or a specific time of day Provisioning Netcool/Precision Add and remove nodes from the topology Problem determination How to recognize some common problems Populating the user interface by roles The steps to set up views on the UI for a operator account The menus in Omnibus and Netcool/Precision A detailed look at the menu mechanisms for the OMNIbus X11 application, NGF/Webtop, and NGF/Topoviz Enriching interface events with chassis object attributes How to use stitchers to copy chassis entity attributes to the interface entity object for use in event enrichment© Copyright IBM Corp. 2007. All rights reserved. 199
  • 215. 7.1 Scheduled discovery Theoretically, discovery in Netcool/Precision is continuous. If you look at the Discovery Status GUI, on the Discovery Details page you will see the Discovery Phase at the top. Between discovery passes the Phase sits at Idle/Standby (0). If anything were running that would trigger discovery, such as a ping finder, then a partial discovery would kick off on its own. But the final phases of discovery, where all of the stitchers operate, is a bigger operation in Netcool/Precision than it is in NetView, and adding a few devices can sometimes lead to quite a bit of rediscovery when adjacent nodes are taken into consideration. Therefore you might consider establishing a change control process that includes a scheduled full discovery. Scheduling is done in $NCHOME/precision/disco/stitchers/FullDiscovery.stch. We made a copy called FullDiscovery.REDBOOK_P.stch and uncommented the line in the trigger section that schedules a full discovery at 11 PM every night as shown in Example 7-1. Example 7-1 Scheduling discovery StitcherTrigger { // This is called from the FinalPhase stitcher, during // rediscovery. // An ActOnTimedTrigger trigger can also be inserted // but should not be included until a complete discovery // has been made. // // Activate at 11pm each day. // ActOnTimedTrigger(( m_TimeOfDay ) values ( 2300 ) ; ); // // Activate on sixth day of week since Sunday ( Saturday ) at 11pm. // Sun - 0, Mon - 1, Tue - 2, Wed - 3, Thur - 4, Fri - 5, Sat - 6 // ActOnTimedTrigger(( m_DayOfWeek , m_TimeOfDay ) // values ( 6 , 2300 ) ; ) ; // // Activate on 13th of each month at 2pm. // ActOnTimedTrigger(( m_DayOfMonth , m_TimeOfDay ) // values ( 13 , 1400 ) ; ); // // Activate on intervals of 13 hours. // ActOnTimedTrigger(( m_Interval ) values ( 13 ) ; ); ActOnTimedTrigger(( m_Interval ) values ( 1 ) ; );200 Migrating to Netcool/Precision for IP Networks
  • 216. There are other options for triggering discovery. If you are running Netcool/Impact, you can configure discovery to notice certain database table changes. This would be useful if you have an external database used by your provisioning or change control process for deploying new network devices or reconfiguring current ones.7.2 Provisioning Netcool/Precision This section discusses administrative techniques to handle adding or removing devices as management needs change. For instance, when an additional branch site comes on line, or when an MPLS backbone replaces an existing ATM or Frame Relay WAN, you will need to add this to Netcool/Precision. This section also covers what to do if you change the community names on your network devices. Adding new nodes You can add new nodes using the Discovery configuration GUI and run a partial rediscovery that will discover the new devices and any surrounding devices where connections have changed. Note: A partial rediscovery could lead to a full discovery if the algorithm determines that more than a certain percentage of the topology must be rediscovered. An alternative method uses the AddNode utility that is included with Netcool/Precision. This method simply inserts the objects into the topology for monitoring purposes. The new nodes will appear in the maps, but no inter-device connections will be discovered. This may be useful in cases where you do not want to do partial discoveries and will wait until the next full discovery is scheduled. With either method, you should also review: Discovery scope and filters Community names in the discovery configuration Monitoring policies, to ensure the new devices will be covered, making changes to the AOC files as necessary Adding new nodes via the GUI Chapter 7. Migration topics 201
  • 217. Perform the following step to add a new node: 1. Click the Partial Rediscovery toolbar button (Figure 7-1).Figure 7-1 Partial rediscovery 2. Click the New button to get the screen shown in Figure 7-2. Enter the new IP address or new subnet and click OK. Figure 7-2 Enter new IP address or subnet 3. The screen in Figure 7-3 is displayed. Click the Scope button to access the Scope page of the discovery. Verify that the new nodes are in scope; add the new entries if they are not.202 Migrating to Netcool/Precision for IP Networks
  • 218. Figure 7-3 New nodes and subnetsAdding new nodes via the AddNode utilityThis alternative method does not require a partial rediscovery, so if this is anissue you can use the AddNode utility. This is similar in idea to the loadhostsutility in Tivoli NetView. It lets you monitor and see the devices on the map,although the inter-device connections will not be created until after the next fulldiscovery. See the box for instructions to set up the AddNode utility. Attention: The AddNode utility is shipped in the $NCHOME/precision/contrib/AddNode directory. Read the document AddNodeUsage.pdf included in the directory for details on using it. Note that the installation instructions apply only to v3.5, so follow these steps instead. It is always advisable to back up any files you modify.Installation steps for the AddNode utility in Netcool/Precision v3.61. Change directory. cd $NCHOME/precision/contrib/AddNode/code Chapter 7. Migration topics 203
  • 219. 2. Copy cp $NCHOME/precision/perl/lib/site_perl/5.6.1/RIV 3. Copy the Perl script, ncp_addnode, and set execute permission. cp ncp_addnode $NCHOME/precision/scripts/perl/scripts chmod 554 $NCHOME/precision/scripts/perl/scripts/ncp_addnode 4. Copy NewNode.aoc and edit it. cp NewNode.aoc $NCHOME/precision/aoc Edit the file, changing: visual_icon = NewNode; to: visual_icon = Router; This will prevent the icons from disappearing when “Hide end nodes” is used. 5. Invoke the utility using the recommended syntax for Perl scripts in v3.6: ncp_perl $NCHOME/precision/scripts/perl/scripts/ncp_addnode -help Removing nodes There is no way to delete devices from the Netcool/Precision GUI as there is in NetView. As with adding nodes, adjust the Discovery configuration if any of the devices you want to remove are still active in the network, to ensure they will not be rediscovered. If you cannot run a full discovery immediately, you may want to prevent monitoring events related to the removed nodes. Remove node on next full discovery Devices removed from the network will automatically disappear when the LingerTime decrements to 0 (after 3 discoveries by default). To preempt this and force the next discovery to delete the device from the topology, change the LingerTime to 0. The recommended way to do this for several devices in a production network is to collect all the ObjectIds for each of the devices and then make one OQL update call to set the LingerTime for all the ObjectIds. This results in only one update to MODEL, which then pushes all the changes out to other subscribers. This reduces the overhead of pushing the changes out. Example 7-2 shows the two steps using OQL to set the LingerTime to 0 for the devices C1700-1 and Example 7-2 Set LingerTime for multiple devices |[netcool@precision2 bin]$ ncp_oql -domain REDBOOK_P -service Model -username admin -password "" ncp_oql ( Netcool/Precision OQL Interface )204 Migrating to Netcool/Precision for IP Networks
  • 220. Copyright (C) Micromuse Ltd., 1997-2006. All Rights Reserved.Netcool/Precision Version 3.6 (Build 13) created by lfredric at18:11:46 Tue Aug 15 BST 2006|precision2:1.> select EntityName,ObjectId from master.entityByNamewhere|precision2:2.> EntityName like karla or|precision2:3.> EntityName like C1700-1;|precision2:4.> go.{ EntityName=C1700-1[ Nu0 ]; ObjectId=509;}{ EntityName=C1700-1[ Se1 ]; ObjectId=494;}{ EntityName=C1700-1[ Lo1 ]; ObjectId=496;}{ EntityName=C1700-1[ Fa0 ]; ObjectId=492;}{ EntityName=C1700-1[ Lo0 ]; ObjectId=495;}{ EntityName=C1700-1[ Se0 ]; ObjectId=493;}{ EntityName=C1700-1; ObjectId=524;}{ EntityName=SUBNET_HOOK / / / NULL /C1700-1[ Se1 ]; ObjectId=159;}{ Chapter 7. Migration topics 205
  • 221. EntityName=SUBNET_HOOK / / / NULL / C1700-1[ Lo0 ]; ObjectId=152; } {; ObjectId=587; } ( 10 record(s) : Transaction complete ) |precision2:1.> update master.entityByName |precision2:2.> set LingerTime = 0 |precision2:3.> where ObjectId in (509,494,496,492,495,493,524,587); |precision2:4.> go . ( 0 record(s) : Transaction complete ) |> quit ncp_oql is dead. Unmanaging the removed nodes If a full discovery will not be run immediately, you may want to unmanage the removed nodes. Unmanage the devices by adding them to the polls.suspended table in the Monitor service. To do this, you will need the entity name of the device and its classname. Create entries for each of the interfaces as well as the device itself in order to suspend polls for both the interface (ICMP) and device (SNMP). See Example 7-3 for the two steps to get the classnames and to insert the entries in the table. Example 7-3 Unmanage a device First, get the ClassName from the Model service: [netcool@precision2 bin]$ ncp_oql -domain REDBOOK_P -service Model -username admin -password "" ncp_oql ( Netcool/Precision OQL Interface ) Copyright (C) Micromuse Ltd., 1997-2006. All Rights Reserved. Netcool/Precision Version 3.6 (Build 13) created by lfredric at 18:11:46 Tue Aug 15 BST 2006 |precision2:1.> select EntityName, ClassName from master.entityByName |precision2:2.> where EntityName like "C1700-1"; |precision2:3.> go .206 Migrating to Netcool/Precision for IP Networks
  • 222. { EntityName=C1700-1[ Se0 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1[ Fa0 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1[ Lo0 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1[ Se1 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1[ Lo1 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1[ Nu0 ]; ClassName=Cisco17xx;}{ EntityName=C1700-1; ClassName=Cisco17xx;}{ EntityName=SUBNET_HOOK / / / NULL /C1700-1[ Se1 ]; ClassName=Device;}{ EntityName=SUBNET_HOOK / / / NULL /C1700-1[ Lo0 ]; ClassName=Device;}{ EntityName=SUBNET_HOOK / / / NULL /C1700-1[ Fa0 ]; ClassName=Device;}( 10 record(s) : Transaction complete )|precision2:1.> quit Chapter 7. Migration topics 207
  • 223. ncp_oql is dead. Now, insert a record into the Monitor service for each Entity, .e.g.,: [netcool@precision2 bin]$ ncp_oql -domain REDBOOK_P -service Monitor -username admin -password "" ncp_oql ( Netcool/Precision OQL Interface ) Copyright (C) Micromuse Ltd., 1997-2006. All Rights Reserved. Netcool/Precision Version 3.6 (Build 13) created by lfredric at 18:11:46 Tue Aug 15 BST 2006 |precision2:1.> insert into polls.suspended |precision2:2.> (EntityName, ClassName, PollName) |precision2:3.> values |precision2:4.> ("C1700-1","Cisco17xx","*"); |precision2:5.> go . ( 0 record(s) : Transaction complete ) To make this process easier, a utility,, has been included with the migration tools associated with this Redbook. It is shown in the appendix in B.2.3, “Perl script to handle unmanaged nodes or interfaces” on page 270. SNMP community name changes If you change the SNMP community names on devices in the network, you should update the discovery configuration with the new community names and delete the cache file: $NCHOME/precision/cache/SnmpStack.Cache.snmpStack.verSecurityCache.R EDBOOK_P7.3 Problem determination This section covers some housekeeping tasks to maintain the product and some known problems and workarounds. Netcool GUI Foundation There is a known problem with a memory leak, which may cause the CPU usage to climb. If this becomes a noticeable problem with browser response times, stop the NGF as in Example 7-4 using kill to complete the shutdown.208 Migrating to Netcool/Precision for IP Networks
  • 224. Example 7-4 Stopping the NGF server[netcool@precision2 precision]$ ngf_server stopStopping Netcool GUI Foundation ...Adaptive Server Anywhere Stop Engine Utility Version Server: Running PID: 15019ASA Database: Not Running PID: n/a[netcool@precision2 precision]$ kill 15019[netcool@precision2 precision]$ ngf_server statusNGF Server: Not Running PID: n/aASA Database: Not Running PID: n/aRestart the ngf_server with the command:ngf_server startEvent enrichmentIf you notice that your events are missing the “enriched” added fields, checkwhether the ncp_ncogate process is running. If it is not, check the$NCHOME/precision/log/ncp_ncogate.<DOMAIN>.log file for reports of syntaxor other errors in $NCHOME/precision/etc/NcoGateSchema.cfg andNcoGateInserts.cfg. Since the ncp_ncogate process depends on thencp_f_amos process, check that it is running. When the ncp_ncogate processrestarts it will re-correlate all the events automatically.Root cause analysisIf there are many critical events that you would expect to be symptomatic events,check that the process ncp_f_amos is running. Check the$NCHOME/precision/log/ncp_f_amos.<DOMAIN>.log for errors. Also check forthe existence of error log files. When the ncp_f_amos restarts, it will re-correlateall the events for root cause automatically.Log filesIf a process experiences a fatal problem, it will generate a message in an errorfile in the log directory, $NCHOME/precision/log. Example 7-5 shows a typicalerror message. Normally the process is restarted automatically, so occasionalinstances is not a concern. However, if it becomes a problem, such as failingrepeatedly, it will reach a limit and stop restarting. Call Tivoli Support to reportthese types of problems. Chapter 7. Migration topics 209
  • 225. Example 7-5 Fatal error [netcool@precision2 log]$ more ncp_f_amos_7591_error.log Fatal Error: Undefined object (nil) invalid operation found in file at line 113 CAmosClassCore::ACCAddClass Cache files The cache files persist the topology data to disk. Topology database corruption If the ncp_model_to_mysql process stops unexpectedly while moving data from MODEL to mySQL database it can corrupt the data, resulting in problems in the topology maps. To fix this, kill the ncp_model_to_mysql process. It will restart automatically and retry the operation of moving data from MODEL to the mySQL database. If the data is still corrupt, delete the following cache file (in our case the domain is REDBOOK_P), and run a full discovery: $NCHOME/precision/cache/Store.Cache.kernel.activeModel.REDBOOK_P7.4 Populating the user interface by roles This section describes the steps to set up a typical set of views and functionality for a network operator. These views can easily be added to and modified later using the same Webtop GUI to create a customized interface to meet the needs of each type of user. We want to set up a tabbed page for the operators that consists of three views: the Active Event list, the Network topology views, and the MIB browser. We followed these steps: Create the network operators group. Create the tabbed view for the operators view. Create the network topology view. Build the operators tabbed view: – Network Topology consists of the navigation map tree view and the map view. – Active Event List viewpoint.210 Migrating to Netcool/Precision for IP Networks
  • 226. – MIB Browser viewpoint. Add the operators view to the user’s home page. Figure 7-4 shows the finished tabbed views we want for the Operators group. We assigned the user “Mat” to the Operators group so that this is one of the pages available to him.Figure 7-4 Finished tabbed views for Operators group7.4.1 Create the network operators group First we created a new group. We selected the Administration page from the drop-down, then selected the Security tab, and then Groups from the left menu. We clicked Add group and filled in the group properties as shown in Figure 7-5, including the assignment of operators. (Additional operators can be added to the group later.) Chapter 7. Migration topics 211
  • 227. Figure 7-5 Create the Network Operators group We clicked the Group Roles tab to assign the roles for this group (Figure 7-6).212 Migrating to Netcool/Precision for IP Networks
  • 228. Figure 7-6 Assign roles to the group7.4.2 Create the tabbed page for the operators view Now that we have a security group of NETOPERATOR, we can create the Operators page and assign it to this group. Then by assigning any operator to this group they will automatically have access to this view. To do this, we selected the Layout tab and Pages from the left menu. Then we clicked the Clone icon at the far right of one of the existing pages, as shown in Figure 7-7 on page 214 (this illustration shows the list of pages after we added the cloned Operators page). Chapter 7. Migration topics 213
  • 229. Figure 7-7 Pages available by group Clicking the Clone icon took us to the screen shown in Figure 7-8 on page 215, where we created the initial Operators page.214 Migrating to Netcool/Precision for IP Networks
  • 230. Figure 7-8 Create the initial view for operators7.4.3 Create the network topology view We now have the blank Operators view. The next step is to add the views and viewpoints. Before we do this we have to create another view for the Network Maps since this view will consist of two viewpoints, the Network Map tree and the Network Map view, as seen in Figure 7-4 on page 211. Similar to the Operators view in Figure 7-8, we clone another view for the Network Topology, as shown in Figure 7-9 on page 216. Chapter 7. Migration topics 215
  • 231. Figure 7-9 Create the initial view for the Network Maps Now that we have the Network Topology view, we can add the two viewpoints. From the screen showing the list of pages (Figure 7-7 on page 214), we clicked the Edit icon at the far right of the Network Topology View. This took us to the screen shown in Figure 7-10. We chose the Two column (25/75) layout and then clicked the Add Viewpoint™ button to select the Network Map Tree and Network Map View viewpoints.216 Migrating to Netcool/Precision for IP Networks
  • 232. Figure 7-10 Viewpoints added to Network Map view7.4.4 Build the Operators tabbed view Now that we have a new view for the Network Topology, we are ready to build all the tabbed views for the Operator view. Starting back at the Pages screen, (Figure 7-7 on page 214), we clicked the Edit icon at the far right of the new Operators row. From here, we clicked the Add View button, which took us to the screen shown in Figure 7-11, where we selected the Network Topology View. Chapter 7. Migration topics 217
  • 233. Figure 7-11 Add Network Topology view to the Operators view We repeated this process by clicking Add Viewpoint and, as shown in Figure 7-12, selected the MIB Browser and the Active Event List viewpoints.218 Migrating to Netcool/Precision for IP Networks
  • 234. Figure 7-12 Add the MIB Browser and Event list viewpoints to the Operators view We now see the completed setup for the Operators View with the new View and the two Viewpoints in Figure 7-13 on page 220. Chapter 7. Migration topics 219
  • 235. Figure 7-13 Setup for the Operators View complete7.5 The menus in Omnibus and Netcool/Precision This section summarizes the customizable menus that are in Omnibus and Precision IP. It includes their organization, and pointers to the documentation for them and the files that are involved.7.5.1 Omnibus X11 menus Since we used the nco_config command to launch the administrative tool for Omnibus, and used it to add fields and event automation tools, you may be confused by the functions in that utility for creating menus and tools. This section describes the menus that are administered using this tool, but only for220 Migrating to Netcool/Precision for IP Networks
  • 236. completeness. Those menus and tools do not appear on the NGF interface thatwe worked on in 6.8, “Adding tools to the user interface” on page 188.There is a full-fledged X11 interface to the event lists that we did not use in thisproject, and which some customers may never use unless their NGF serverbecomes unavailable. Tools menus appear on: – The conductor interface, raised by the nco command. The Tools menu is empty by default. – The Eventlist interface, raised by double-clicking the default on the conductor interface. The Tools menu is empty by default. – The SubEventlist interface, raised by clicking view on one of the lists in the Eventlist interface. • The Alerts menu contains functions mainly related to event disposition. • The Alerts → Tools menu contains Ping and URL (associated with selected events). • The Tools menu contains Ping and Telnet (prompted). The menus and tools are configured by the nco configuration interface, raised by the nco_config command. The fields available to be passed to the tools are event fields from the objectserver. This work is documented in the manual Netcool/Omnibus® Administration Guide V7.1, section 2.5 “Configuring menus, tools, and prompts.” The menus and tools configured by this are stored in the objectserver. The tools execute on the server where the objectserver is installed.The stored tools, such as the Ping tool, include functions as shown inExample 7-6.Example 7-6 Omnibus ping tool. $(OMNIHOME)/utils/nco_functionsnco_get_PATH$PING $NodeThe file $NCHOME/omnibus/utils/nco_functions includes functions likenco_get_PATH, as well as the definitions of things like $PING, mapping them tospecific operating system commands based on the platform. This allows us tocorrect for implementation differences in the paths, or adjust permissions on the Chapter 7. Migration topics 221
  • 237. executables as needed. There, for instance, we can see that for Linux, from the nco, PING means /bin/ping. We can edit it if necessary, or adjust the permissions on that executable.7.5.2 NGF/Webtop menus This section is about the menus on the Active Events Lists on the NGF interface, which is sometimes referred to in the manuals as the Webtop – as opposed to Topoviz, which relates to Netcool/Precision on the NGF, and is described separately. The functions in the Alerts and Tools menus here are intended to give the exact same capabilities as those on the nco menus, however they are not shared. They are repeated by different means, so any tools that you add to one interface are not available on the other interface unless you recreate them using the appropriate configuration method for that interface. This is worth considering if you plan to use the nco events interface as your fallback. Tools menus appear on the Active Events Lists. – The Alerts menu contains functions mainly related to event disposition. – The Tools menu contains submenus: CGI Tools, and Local Tools. • The Tools → CGI Tools submenu, for things that execute on the Webtop server, contains Ping by default. • The Tools → Local Tools submenu, for things that execute on the local PC, contains Ping, Telnet, and TracePath by default. These menus are configured by the Webtop Admin page of the NGF:P – Menus Editor – Tools Browser – CGI Registry The fields available to be passed to the tools are event fields from the obectserver as well as fields mapped by the Precision Gateway. This work is documented in the manual Netcool/Webtop Administration Guide V2.0, Chapter 10. “Tools and menu administration.” The menus and the tool launchers configured by this are stored in files $NCHOME/etc/webtop/configstore/ncwTools/*.nova. Tool launchers are of one of three types: sql used for event manipulation cgiurl used for cgi or url launches cmdline used for local tools The CGI Tools execute on the server where the NGF server is installed.222 Migrating to Netcool/Precision for IP Networks
  • 238. The Local Tools execute on the local system, and execute a platform-appropriate function. The CGI Tool Ping, as defined in Ping.nova, launches the URL at $(SERVER)cgi-bin/nco_ping.cgi, which is $NCHOME/etc/webtop/cgi-bin/nco_ping.cgi. That example, nco_ping.cgi, includes the lines in Example 7-7. Example 7-7 snippet from AEL cgi ping tool launcher #!/usr/bin/perl ..... $nodeName = $FORM{"$selected_rows.Node"}.$domainName; print "<p class="systemMsg">Pinging host ".$nodeName."</p>n"; ... open(PING,"/bin/ping -c 10 ".$nodeName."|"); In comparison, the Local Tool Ping, as defined in LocalPing.nova, runs locally. It includes the lines in Example 7-8. Example 7-8 snippet from AEL local ping tool launcher command(platform="Windows",windowforeach="true",enabled="true",foreach= "false") { path { text(data="start cmd /k %WINDIR%SYSTEM32PING.EXE ") field(list="false",convert="false",name="Node") } }7.5.3 NGF/Topoviz menus This section is about the right-click menu on the network views that come with Netcool/Precision on the NGF, sometimes referred to in the manuals as the Topoviz interface. When you right-click near a device on the map, a menu appears. That menu is extensible. Tools menus appear on the right-click menu and include, by default: – Functions to launch an Event List, the MIB browser, the node details display, a Hop View, and so forth, in context. – Individual tools such as a Ping. Chapter 7. Migration topics 223
  • 239. – A set of Web Tools that: • Launch a GUI that includes a variety of tools • Launch a number of those same tools directly • Launch the documentation for those tools – A submenu of Cisco-specific Web tools that: • Launch a GUI that includes a variety of cisco telnet tools • Launch a number of those same tools directly The standard functions on that menu are listed in $NCHOME/etc/precision/ for disable/enable purposes. See Example 7-9. Example 7-9 Standard ncp_wt menu items # Specifies which standard tools are displayed. topoviz.client.stdtools.recenter=true topoviz.client.stdtools.eventlist=true topoviz.client.stdtools.mibbrowser=true topoviz.client.stdtools.showdetails=true topoviz.client.stdtools.findinhopview=true topoviz.client.stdtools.findinnetworkview=true topoviz.client.stdtools.showmplscore=true The additional set of Web Tools is a special batch that is new in 3.6, and brings a GUI with it. These menus and tool launchers are configured by editing xml files in $NCHOME/etc/precision/menus and $NCHOME/etc/precision/tools. The tools executables can be the same cgi tools created and registered for use with events as described for the Webtop interface. The fields available to be passed to the tools are node attribute fields from the mysql database or fields shared with the objectserver via the gateway. The usage of the tools in the ncp_webtools package is documented online on the server where the NGF server is installed, under, which is in /opt/netcool/guifoundation/webapps/ncp_webtools. The administration of the Webtop Tools group is not really documented at all. They are intended to be used as-is. The creation of tools usable from either the NGF AEL or the Topoviz map views is documented in the manual Netcool/Precision IP Topology Visualization Guide 3.6, especially for launching the MIB browser. In some224 Migrating to Netcool/Precision for IP Networks
  • 240. respects it is a repeat of the documentation in the Netcool/WebtopAdministration Guide.The menu and submenu files are in $NCHOME/etc/precision/menus. Themain menu that comes with the Web Tools package is shown inExample 7-10.Example 7-10 Main menu file for the topoviz webtools package<ncp_menu id="ncp_webtools" label="WebTools"> <definition> <tool id="ncp_wt_help"/> <tool id="ncp_wt_gui"/> <tool id="ncp_wt_ping"/> <tool id="ncp_wt_traceroute"/> <tool id="ncp_wt_whois"/> <tool id="ncp_wt_dns"/> <separator/> <menu id="ncp_wt_cisco"/> <separator/> <menu id="ncp_wt_juniper"/> </definition></ncp_menu>The tool launcher files are in $NCHOME/etc/precision/tools. They are xmlfiles, and the only type currently supported is url. A sample is shown inExample 7-11.Example 7-11 Sample tool launcher for a topoviz tool<ncp_tool id="ncp_wt_dns" label="DNS Lookup" type="url"> <urlvalue="/ncp_webtools/cgi-bin/ncp_webtool.cgi?toolid=DNSLookup&amp;assign_srNode_to=query&amp;" target="_blank"/></ncp_tool>The CGI Tools and URL tools execute where they are sent, usually the serverwhere the NGF server is installed. The cgi scripts must reside in$NCHOME/etc/webtop/cgi-bin. If they are registered with the Webtop, theycan be used by the tools launchers there as well. Chapter 7. Migration topics 225
  • 241. 7.6 Enriching interface events with chassis object attributes Events are enriched by the ncp_ncogate process. This gateway reads events from the object server and creates internal representations based on the config.nco2ncp entries mapping data from event and topology sources. The gateway decides whether to send them to the root cause engine, AMOS, which may create new representations. The gateway then sends events back to the object server based on the entries in the config.ncp2nco table mapping data from the event and topology sources. Some interface events are built against the chassis topology entity object, and others against the interface topology object. Therefore, if you want to enrich interface events with chassis topology entity data, we recommend you use the stitchers described here to copy the specific attributes to the interface topology object during the discovery so it will always be available regardless of how the gateway constructs the interface events. This example shows how to use the two stitchers in the Additional Materials to copy the sysContact and sysLocation attributes to the topology interface object from its chassis parent so there is no problem with event enrichment for interface events. See 6.6.2, “Event enrichment” on page 165 for more details. Step 1 Install the stitchers Copy the AddInfo.stch and AddInterfaceInfo.stch files to: $NCHOME/precision/disco/stitchers/ Step 2 Enable the stitchers Run the stitcher at the end of discovery from the PostScratchProcessing stitcher by editing the file: $NCHOME/precision/disco/stitchers/PostScratchProcessing.stch Add a line in the StitcherRules block: ExecuteStitcher(AddInterfaceInfo); Step 3 Map the chassis and interface field names To copy the ExtraInfo->m_SysContact and ExtraInfo->m_SysLocation fields edit $NCHOME/precision/disco/stitchers/AddInfo.stch and add the calls to the StitcherRules block: ExecuteStitcher(AddInfo,m_SysContact); ExecuteStitcher(AddInfo,m_SysLocation);226 Migrating to Netcool/Precision for IP Networks
  • 242. A Appendix A. Useful information for Netcool installation and maintenance This appendix is intended as a reference when checking an installation.© Copyright IBM Corp. 2007. All rights reserved. 227
  • 243. A.1 Environment settings The settings in Example A-1 should be put into /etc/profile. Example A-1 /etc/profile NCHOME=/opt/netcool OMNIHOME=$NCHOME/omnibus PRECISION_HOME=$NCHOME/precision NCSM_HOME=$NCHOME/security NC_RULES_HOME=$NCHOME/rules NCLICENSE=$NCHOME/license NETCOOL_LICENSE_FILE=$NCLICENSE/etc #local license server #NETCOOL_LICENSE_FILE="27000@host.domain” #remote license server #NETCOOL_LICENSE_FILE="27000@host1:2700@host2” #multiple license servers PATH=$NCHOME/bin:$PATH PATH=$OMNIHOME/bin:$PATH PATH=$PRECISION_HOME/bin:$PATH PATH=$NCHOME/platform/linux2x86/mysql4.1/bin:$PATH PATH=$NCLICENSE/bin:$PATH PATH=$NCSM_HOME/bin:$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PRECISION_HOME/platform/linux2x86/lib / LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$NCHOME/platform/linux2x86/lib/ LANG=C PERLLIB=$PRECISION_HOME/perl/lib/5.6.1/i686-linux:$PRECISION_HOME/perl/ lib/5.6.1:$PRECISION_HOME/perl/lib/site_perl/5.6.1/i686-linux:$PRECISIO N_HOME/perl/lib/site_perl/5.6.1:$PRECISION_HOME/precision/perl/lib/site _perl export NCHOME OMNIHOME NCSM_HOME PRECISION_HOME export NETCOOL_LICENSE_FILE NCLICENSE NC_RULES_HOME export PATH LD_LIBRARY_PATH PERLLIB export LANG NOTE: When using Xwindows the LANG variable might get overwritten, so you have to make sure it is set correctly. A good idea is to put this statement in a file and source it.228 Migrating to Netcool/Precision for IP Networks
  • 244. A.2 License Server The following can help you control the License Server: Configuration files: – $NCLICENSE/etc/ – SERVER host ANY 27000 <-put local hostname here Process that runs: – lmgrd Control commands: – nc_start_license, nc_stop_license, nc_read_license – nc_print_license, nc_hostid Log files and debug: – $NCLICENSE/log/license.log Implementing change: – stop/start the license serverA.3 ObjectServer The following can help you control the ObjectServer: Environment variables: – OLOG=/opt/netcool/omnibus/log – OPROP=/opt/netcool/omnibus/etc – OPROBE=/opt/netcool/omnibus/probes/linux2x86 – OGATE=/opt/netcool/omnibus/gates Configuration files: – $OPROP/pam_omnibus_os.conf – $OPROP/<NCOMS_P>.props – $OPROP/<NCOMS_B>.props – $OPROP/nco_config.props (See Menus etc below) – $OPROP/ nco_pa.conf (See PA below) – etc/services for IDUC port if firewalls (or in .props files) Appendix A. Useful information for Netcool installation and maintenance 229
  • 245. Control Commands:nco_xigen, nco_igen (-notrunc) – nco_dbinit -server <NCOMS> – nco_objserv -help – nco_objserv -name <NCOMS> & – nco_confpack -list -server <NCOMS> -file mylist – nco_confpack -export -file mylist -package mypackage – nco_confpack -import -server <NCOMS> -package mypackage – nco_config – nco_patch, nco_id Log files and debug: – $OLOG Implementing Change:Without PA: – nco_sql -server <NCOMS> -user root – alter system shutdown; go – Restart with nco_objservA.4 OMNIbus probes The following can help you control the probes: Configuration files: – $OPROBE/simnet.props, rules – $OPROBE/syslog.props, rules Control commands: – nco_p_<probename> -server <NCOMS_V> & (or no parm if set in .props) – nco_p_<orobename> -help – nco_p_<probename> -dumpprops – Also controllable via PA Log files and debug: – $OLOG/<probename>.log – MessageLevel and MessageLog in props file or as startup parameter230 Migrating to Netcool/Precision for IP Networks
  • 246. Implementing change: – Without PA, kill and restartA.5 OMNIbus gateways The following can help you control the OMNIbus gateway: Configuration files: – Templates in: $OGATE/<gwtype>/ (objserv_bi or objserv_uni) – BI_GATE stuff is in $OGATE/objserv_bi/ – <BI_GATE>.map – <BI_GATE>.props – objserv_bi.<BI_GATE>.startup.cmd – objserv_bi.<NCOMS_P>.reader.tblrep.def – objserv_bi.<NCOMS_B>.reader.tblrep.def Control commands: – nco_g_objserv_uni -propsfile <propsfile> (dedicated) – nco_g_objserv_bi -propsfile <propsfile> (dedicated) – Also controllable via PA Log files and debug: – $OLOG Implementing change: – Without PA, kill and restartA.6 Process control (PA) The following can help you control the process control process: Configuration files: – $OPROP/nco_pa.conf (defines what is to be started under control of PA) Control commands: – nco_pad -name <HOST_PA> -authenticate PAM (starts PA at command line, forks) – /etc/init.d/nco start, stop, restart, reload (starts PA at boot, then starts nco procs) Appendix A. Useful information for Netcool installation and maintenance 231
  • 247. – nco_pa_status -server <HOST_PA> -user root -password xxxx – nco_pa_start -server <HOST_PA> -user root -password xxxx -service Core – nco_pa_stop -server <HOST_PA> -user root -password xxxx -service Core Log files and debug: – $OLOG/NCO_PA.log (hard-coded?); no debug levels? – nco_pad -name <HOST_PA> shows settings Implementing change: – nco_pa_stop….will stop daemons under control; kill nco_pad; restartA.7 Menus, tools, and prompts The following can help you control menus and prompts: Configuration files: – Configure in nco_config : Menus : Tools – Create the prompt before referencing it. – To reference a selected event, put it under the Alerts menu, for example, Alerts.Tools. – Tools can run scripts, or take update events. – Stored in the objectserver; uses ObjectServer SQL. Invocation: – Various Conductor menus - Alerts..Tools, Tools Log files and debug: – $OLOG/nco_config_audit.log Implementing change: – On the Monitor Box, File…Resynch ..All; or restart conductor or the event list.232 Migrating to Netcool/Precision for IP Networks
  • 248. A.8 Procedures The following can help you control automated procedures: Configuration files: – Configure in nco_config .Automations : Procedures. – Can be SQL procedures or external procedures. – Stored in the Objectserver; uses ObjectServer SQL. Invocation: – sql, automation triggers, or tools (EXECUTE PROCEDURE) Log files and debug: – $OLOG/nco_config_audit.logA.9 Automation triggers The following can help you control automation triggers: Configuration files: – Configure in nco_config :Automations : Triggers. – Action can be SQL action, SQL procedure, external procedure (if target is under PA). – Stored in the Objectserver; uses ObjectServer SQL. – Shipped: Generic Clear Automation, Deduplication Automation. Invocation: – Database triggers fire when a defined condition occurs in the objectserver db. – Temporal triggers fire on a time basis. – Signal triggers fire when system/user signals happen, like daemons fail. Log files and debug: – $OLOG/nco_config_audit.log Appendix A. Useful information for Netcool installation and maintenance 233
  • 249. A.10 Security Manager 1.3 The following can help you control the security manager: Configuration files: – $NCHOME/etc/sm/server.props configured at install time – $NCHOME/security/etc/SM*.props configured at install time Process that runs: – ncsm_server Control commands: – ncsm-config (to do over) – nohup ncsm_server & – ncsm_shutdown – ncsm_status Log files and debug: – $NCHOME/security/log/SM*.log Implementing change: – stop/startA.11 Webtop The following can help you control Webtop: Configuration files: – /opt/netcool/guifoundation/conf/*.properties configured at install time Process that runs: – NGF Server: java – ASA Database: dbsrv9 – grep for guifoundation, should get 2 processes.) Control commands: – ngf_server stop (and kill the leftover java process) – ngf_server start – ngf_server status234 Migrating to Netcool/Precision for IP Networks
  • 250. Log files and debug: – /opt/netcool/log/guifoundation/ – configured in /opt/netcool/guifoundation/common/classes/ – # tomcat.log – # autoconfig.log – # loginmodule.log – # ncsmMapping.log – # sdtout.log – # dev.log Implementing change: – ngf_server stop (kill leftover java process), ngf_server startA.12 Precision The following are environment variables needed by Netcool/Precision: PLOG=/opt/netcool/log/precision PPROP=/opt/netcool/etc/precision PPROBE=/opt/netcool/probes/linux2x86 AOC=/opt/netcool/precision/aoc PDISCO=/opt/netcool/precision/disco PMONITOR=/opt/netcool/precision/monitorA.12.1 Precision server The following can help you control the Netcool/Precision server: Configuration files: – $PPROP/CtrlServices.PRECISION_P.cfg – $PPROP/CtrlServices.PRECISION_B.cfg Process that runs: – $ncp_ctrl one for each domain, and all its children like ncp_disco Control commands: – nohup ncp_ctrl -domain <PRECISION_P> -logdir $NCHOME/log/precision >/tmp/ctrl,out 2>&1 & – kill <pid of ncp_ctrl> and it shuts down children or use Appendix A. Useful information for Netcool installation and maintenance 235
  • 251. Log files and debug: – $NCHOME/log/precision/*.<PRECISION_P/B>.log plus redirected ctrl.,out – kill -USR2 <pid of process> to increase debug level 0,1,2,3,4,0 Implementing change: – Use to stop and start, or restart. Take care not to start multiples.A.12.2 Precision monitors The following can help you control the Netcool/Precision monitors: Configuration files: – $AOC/*.<PRECISION_P/B>.aoc , Device.<>.aoc Process that runs: – ncp_monitor, 1 ncp_m_timedstitcher per defined poll Control commands: – table inserts, or use Log files and debug: – $PLOG/ncp_monitor.<PRECISION_X>.log ; kill -USR2 <pid> to raise debug level Implementing change: – restart ncp_class, restart ncp_monitor using NCPCtrl.shA.12.3 Precision monitor probe The following can help you control the Netcool/Precision monitor probes: Configuration files: – $PPROBE/nco_p_ncpmonitor.NCOMS_V.props – $PPROBE/ – $PPROBE/nco_p_ncpmonitor.NCOMS_V.rules – Set the PollServer and NetworkTimeout values for failback of objectserver. Process that runs: – nco_p_ncpmonitor Control commands: – table inserts, or use NCPCtrl.sh236 Migrating to Netcool/Precision for IP Networks
  • 252. Log files and debug: – $PLOG/ncp_p_monitor.<PRECISION_X>.log ; kill -USR2 <pid> to raise debug level Implementing change: – restart ncp_p_monitor using NCPCtrl.shA.12.4 Precision discovery The following can help you control the Netcool/Precision discovery: Configuration files: – $PDISCO/agents/*.PRECISION_P.agnt – $PDISCO/stitchers/*.PRECISION_P.stch – $AOC/*.<PRECISION_P/B>.aoc , Device.<>.aoc Process that runs: – ncp_disco, ncp_agent (many), stitchers during discovery Control commands: – table inserts, or use Log files and debug: – $PLOG/ncp_disco.<>.log; kill -USR2 <pid> to raise debug level Implementing change: – restart ncp_class, ncp_disco, ncp_model using NCPCtrl.shA.12.5 Precision bidirectional gateway The following can help you control the Netcool/Precision gateway: Configuration files: – $PPROP/NcoGateSchema.PRECISION_P.cfg – $PPROP/NcoGateSchema.PRECISION_B.cfg Process that runs: – ncp_ncogate (two, one for _P and one for _B as backup; talk to objectserver) Control commands: – table inserts, or use Appendix A. Useful information for Netcool installation and maintenance 237