Large scale implementation of ibm tivoli composite application manager for web sphere and response time tracking redp4162
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,276
On Slideshare
2,276
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Front coverLarge-Scale Implementation ofIBM Tivoli Composite ApplicationManager for WebSphere andResponse Time TrackingPlanning for performance ofmanagement infrastructureImplementing with multipleserversPerforming mass updateof agents Budi Darmawan Aleem Subhedar Celena Tan Howard Anglin Huang Chuan Rohit Dhallibm.com/redbooks Redpaper
  • 2. International Technical Support OrganizationLarge-Scale Implementation of IBM TivoliComposite Application Manager for WebSphereand Response Time TrackingDecember 2007
  • 3. Note: Before using this information and the product it supports, read the information in “Notices” on page vii.Second Edition (December 2007)This edition applies to Version V6.0 of ITCAM for Response Time Tracking (product number5698-A75) and Version 6.0 of ITCAM for WebSphere (product number 5698-A71).© Copyright International Business Machines Corporation 2006, 2007. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp.
  • 4. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix The team that wrote this Redpaper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Application management with IBM Tivoli. . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 IBM Tivoli systems management portfolio . . . . . . . . . . . . . . . . . . . . . 3 1.1.2 IBM Tivoli Composite Application Manager solution . . . . . . . . . . . . . . 5 1.2 Scope of and concerns relating to large-scale implementation . . . . . . . . . . 6 1.2.1 Defining large-scale implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.2 Concerns and considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Overview of IBM Tivoli Composite Application Manager. . . . . . . . . . . . . . . 8 1.3.1 Understanding ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.2 Understanding IBM Tivoli Composite Application Manager for Response Time Tracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4 Document organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Chapter 2. Planning for ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . . . 17 2.1 Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2 Product architecture of ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . . . 18 2.3 Deciding on the size of the servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.3.1 Sizing parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.3.2 Sizing estimation for ITCAM for WebSphere managing server. . . . . 22 2.3.3 Data collector overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.4 Implementation options for ITCAM for WebSphere. . . . . . . . . . . . . . . . . . 25 2.4.1 Designing the managing server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.4.2 Deploying a large number of data collectors . . . . . . . . . . . . . . . . . . . 28 2.5 Communication and security considerations . . . . . . . . . . . . . . . . . . . . . . . 29 2.5.1 Communication security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5.2 Firewall and port consideration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.6 Reliability and high availability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6.1 Failover and fault tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6.2 Disaster recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32© Copyright IBM Corp. 2006, 2007. All rights reserved. iii
  • 5. Chapter 3. Installing ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . . . . . . 33 3.1 Installing ITCAM for WebSphere managing server . . . . . . . . . . . . . . . . . . 34 3.1.1 Installation configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.1.2 Database installation and configuration . . . . . . . . . . . . . . . . . . . . . . 35 3.1.3 WebSphere Application Server considerations . . . . . . . . . . . . . . . . . 41 3.1.4 Configuring the split server of the managing server . . . . . . . . . . . . . 42 3.1.5 Installation and setup of split server . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.1.6 Verifying the installed components in a split environment . . . . . . . . 47 3.1.7 Adding additional publish servers and archive agents . . . . . . . . . . . 48 3.2 Deploying data collectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2.1 Setting up the silent installation process . . . . . . . . . . . . . . . . . . . . . . 50 3.2.2 Installing the data collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.2.3 Installing and configuring the data collector together . . . . . . . . . . . . 54 3.2.4 Configuring data collectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.2.5 Automatically discovering the installation parameters . . . . . . . . . . . 61 3.3 Configuring and setting up SSL communication . . . . . . . . . . . . . . . . . . . . 64 3.3.1 Managing server Secure Socket Layer setup . . . . . . . . . . . . . . . . . . 64 3.3.2 Data collector Secure Socket Layer setup . . . . . . . . . . . . . . . . . . . . 65 3.3.3 Working with custom certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Chapter 4. Maintenance of ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . 75 4.1 Operating ITCAM for WebSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.2 Performance and availability of the managing server . . . . . . . . . . . . . . . . 76 4.2.1 Performance of the WebSphere Application Server . . . . . . . . . . . . . 76 4.2.2 Database maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.2.3 Data trimming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.3 Backup and recovery configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.3.1 ITCAM for WebSphere backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.3.2 WebSphere configuration backup . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.3.3 Database backup and restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.4 Log files and configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.4.1 Managing log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.4.2 Managing the configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.5 Performing product maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.5.1 Getting software updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.5.2 Updating ITCAM for WebSphere managing server. . . . . . . . . . . . . . 87 4.5.3 Updating ITCAM for WebSphere data collectors . . . . . . . . . . . . . . . 90 Chapter 5. Planning for ITCAM for Response Time Tracking . . . . . . . . . . 93 5.1 Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.2 Product architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.3 Sizing the servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.3.1 Sizing parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96iv Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 6. 5.3.2 Sizing estimation for ITCAM for Response Time Tracking . . . . . . . . 975.4 Deployment of ITCAM for Response Time Tracking . . . . . . . . . . . . . . . . . 98 5.4.1 Designing the management server . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.4.2 Deploying the management agent . . . . . . . . . . . . . . . . . . . . . . . . . . 995.5 Communication and security considerations . . . . . . . . . . . . . . . . . . . . . . 100 5.5.1 Communication security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.5.2 Firewall and port considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015.6 Reliability and high availability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.6.1 Failover and fault tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.6.2 Disaster recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102Chapter 6. Installing ITCAM for Response Time Tracking. . . . . . . . . . . . 1056.1 Clustering the management server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.1.1 Preparing the operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.1.2 Installing the database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6.1.3 Installing WebSphere Application Server . . . . . . . . . . . . . . . . . . . . 109 6.1.4 Installing WebSphere Load Balancer . . . . . . . . . . . . . . . . . . . . . . . 118 6.1.5 Installing the management server . . . . . . . . . . . . . . . . . . . . . . . . . . 124 6.1.6 Checking the configuration of the RTT cluster application . . . . . . . 1286.2 Deploying the management resources . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.2.1 Silent installation of the management agent . . . . . . . . . . . . . . . . . . 129 6.2.2 Command-line interface for management components . . . . . . . . . 130 6.2.3 Defining management resources . . . . . . . . . . . . . . . . . . . . . . . . . . 1316.3 Setting up Secure Sockets Layer certificates . . . . . . . . . . . . . . . . . . . . . 133 6.3.1 Secure Sockets Layer for ITCAM for Response Time Tracking . . . 133 6.3.2 Working with custom certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . 134Chapter 7. Maintenance of ITCAM for Response Time Tracking . . . . . . 1377.1 Operational issues pertaining to a large environment . . . . . . . . . . . . . . . 1387.2 Performance and availability of management server . . . . . . . . . . . . . . . 138 7.2.1 Performance of WebSphere Application Server . . . . . . . . . . . . . . . 138 7.2.2 Database maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1397.3 ITCAM for Response Time Tracking files . . . . . . . . . . . . . . . . . . . . . . . . 140 7.3.1 Backup and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 7.3.2 Managing the log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1417.4 Performing product maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.4.1 Getting software updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.4.2 Updating ITCAM for Response Time Tracking management server142 7.4.3 Updating ITCAM for Response Time Tracking management agents143Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Contents v
  • 7. Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151vi Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 8. NoticesThis information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the usersresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send licenseinquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.The following paragraph does not apply to the United Kingdom or any other country where such provisionsare inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDESTHIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM maymake improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirmthe accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrates programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment to IBM for the purposes ofdeveloping, using, marketing, or distributing application programs conforming to IBM applicationprogramming interfaces.© Copyright IBM Corp. 2006, 2007. All rights reserved. vii
  • 9. TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both: Redbooks (logo) ® ETEWatch® OMEGAMON® pSeries® IBM® OS/400® z/OS® IMS™ Rational® AIX® Lotus Notes® Redbooks® CICS® Lotus® Tivoli® Database 2™ Monitoring On Demand® WebSphere® DB2 Universal Database™ MVS™ Workplace™ DB2® Notes® ETE™ Operating System/400®The following terms are trademarks of other companies:Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporationand/or its affiliates.Snapshot, and the Network Appliance logo are trademarks or registered trademarks of Network Appliance,Inc. in the U.S. and other countries.ITIL is a registered trademark, and a registered community trademark of the Office of GovernmentCommerce, and is registered in the U.S. Patent and Trademark Office.Enterprise JavaBeans, EJB, Java, JavaBeans, JDBC, JMX, JNI, JRE, JVM, J2EE, Solaris, and allJava-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, orboth.Microsoft, Outlook, Windows, and the Windows logo are trademarks of Microsoft Corporation in the UnitedStates, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.viii Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 10. Preface This IBM® Redpaper discusses large-scale implementation of IBM Tivoli® Composite Application Manager for WebSphere® and IBM Tivoli Composite Application Manager for Response Time Tracking. Large-scale implementation is typically characterized by the number of monitoring agents deployed and the number of transactions load-managed. A typical large-scale implementation of a monitoring product contains the following challenges: Keeping up the performance of the monitoring tools to accommodate the processing load from the agents. Automation of installation, update, and maintenance of monitoring agents based on silent installation and automated update. Specific day-to-day maintenance actions to ensure performance and availability of the monitoring solution. This IBM Redpaper addresses these issues with regard to the implementation of ITCAM for WebSphere and ITCAM for Response Time Tracking on distributed platforms. The discussion is divided into planning issues, implementation guides, and maintenance considerations.The team that wrote this Redpaper This Redpaper was produced by a team of specialists from around the world working at the IBM International Technical Support Organization (ITSO), Austin Center. Budi Darmawan is a Consulting IT Specialist at the IBM ITSO, Austin Center. He writes extensively and teaches IBM classes worldwide on all areas of Tivoli systems management products. Before joining the ITSO in 1999, Budi worked as Solution Architect and Implementer in Integrated Technical Services, IBM Indonesia. His current interests include availability management, z/OS® systems management, and Java™ programming. Aleem Subhedar is a Staff Software Engineer with India Software Labs in Pune, India. He has seven years of experience in AIX® and Middleware System Administration. He holds a degree in Chemistry from Pune University. His areas of expertise include AIX, pSeries®, and related system technologies. He is an IBM Certified System Expert. His areas of interest include pSeries virtualization and high availability.© Copyright IBM Corp. 2006, 2007. All rights reserved. ix
  • 11. Celena Tan is a Managing Consultant with IBM Software Group Services in Australia. She has 14 years of experience in the IT field. She holds a Masters of Technology from National University of Singapore and a Bachelor of Electrical Engineering (Hons) from the University of Tasmania. Her areas of expertise include ITCAM family products and rational testing, and change and configuration management products. Howard Anglin is a Deployment Expert for ITCAM for WebSphere, Response Time Tracking, IBM Tivoli Monitoring in the United States. He has worked with various large customers, and in his role as an IT Specialist he has resolved deployment, integration, and performance issues. He has nine years of experience in the software test and development field with emphasis on the WebSphere Application Server. He holds a Bachelor of Science in Electrical Engineering from Manhattan College, Riverdale, New York. Howard began his career at IBM in the pSeries Hardware Group as a Test Engineer developing automation solutions for the production line. He then transferred to the software group. Huang Chuan is a Senior Test Lead of IBM China CSDL lab. He has five years of experience in software developing and over six years of experience in software product testing. He has led the ITCAM for Response Time Tracking test project for several releases. He holds a degree in Computer Science from the University of Electronic Science and Technology of China. Rohit Dhall is an IT Architect with GBS, IBM India. He has 10 years of IT experience in technologies like client-server computing, Web-based transactional systems, data warehousing, and data mining. His major expertise is in designing, implementing, and tuning large-scale Internet banking, eMortgage, and anti-money laundering solutions for the banking and financial sector. He is EXIN ITIL® certified and also holds certification in Java and EJB™ from Brainbench. His current interests include SOA and IBM Virtualization offerings. Thanks to the following people for their contributions to this project: Donna Martin, Noel Lewis, Tony Williams, Marco De Gregorio, Sushanto Pandit IBM Software Group, Tivoli Software John Horton Author of the first edition of Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time TrackingLarge-Scale Implementation of IBM Tivoli Composite Application Manager, REDP-4162 Julie Czubik International Technical Support Organization, Poughkeepsie Centerx Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 12. Become a published author Join us for a two-week to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will team with IBM technical professionals, Business Partners, or customers. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.htmlComments welcome Your comments are important to us! We want our papers to be as helpful as possible. Send us your comments about this Redpaper or other Redbooks® in one of the following ways: Use the online Contact us review book form found at: ibm.com/redbooks Send your comments in an e-mail to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 Preface xi
  • 13. xii Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 14. 1 Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation This chapter provides an overview of the large-scale implementation issues for IBM Tivoli Composite Application Manager. This chapter covers the following topics: 1.1, “Application management with IBM Tivoli” on page 2 1.2, “Scope of and concerns relating to large-scale implementation” on page 6 1.3, “Overview of IBM Tivoli Composite Application Manager” on page 8 1.4, “Document organization” on page 15© Copyright IBM Corp. 2006, 2007. All rights reserved. 1
  • 15. 1.1 Application management with IBM Tivoli Computer-based applications are the lifeblood of modern enterprises. Most business processes are driven by the so-called computer application that promotes productivity, automates processing, and minimizes human errors. These applications enable business persons to focus on what must be done, instead of how to do it. However, as business processes rely more on these applications, the applications become critical to the business. The applications must be available for the execution of the business processes. Most applications evolved from centralized applications typically managed by the information technology (IT) department or mainframe-based applications, where all the application layers are maintained from the central mainframe. Today, applications tend to have multiple layers, often distributed across different servers, different platforms, and different components. These applications are called composite applications. This complicates the management of applications on matters such as operational settings, problem determination, and performance management. Applications as a business-critical entity must be available with adequate response time for users to perform their tasks. With application components spread throughout the enterprise, problem determination and performance management are typically complicated. There is no clear path for finding which component faces the problem. Is it the database? A network problem? The application server experiencing a bottleneck? A user machine stall? Sometimes, these components even belong to different organizations. Figure 1-1 shows a typical composite application. This is used by multiple users through the Internet and intranet. It consists of multiple application layers, each with its own abstraction level. Some of the applications have the original back end in the mainframe transactions.Figure 1-1 Composite application2 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 16. Composite applications are regarded as the ultimate application management challenge, as they span different application servers that communicate with each other. This architecture enables modular application development, where changes in a layer may not affect other layers, but introduces the complexity of multiple components. This paper demonstrates how to implement the IBM Tivoli Composite Application Manager family of products in a large-scale environment. This chapter introduces IBM Tivoli product portfolio and how IBM Tivoli Composite Application Manager product fits.1.1.1 IBM Tivoli systems management portfolio IBM Tivoli product solutions are aligned towards an overall IBM IT Service Management approach. Figure 1-2 shows the IBM IT Service Management portfolio structure. IT CRM & Service Service Information Business Business Delivery Deployment Management Resilience Management & Support IT Process Management Products IT Service Change and Configuration Management Platform Management Database IT Operational Management Products Best Practices Business Server, Network Storage Security Application & Device Management Management Management ManagementFigure 1-2 IBM IT Service Management This approach provides Information Technology Infrastructure Library-aligned automation work flows. Future offerings will provide an open standard-based and configuration management database-based solution, as well as a workflow engine. Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 3
  • 17. The operational management pillar shown in Figure 1-2 on page 3 is divided into software families. The availability solution addressed in business application management and server, network, and device management can be viewed as an integrated offering, as shown in Figure 1-3. Business Service Management Orchestration and Provisioning Security Event Correlation and Automation Storage Composite Application Management Resource Monitoring Figure 1-3 IBM Tivoli software portfolio As shown in Figure 1-3, the Tivoli software portfolio is divided into the following components: Resource monitoring Measures and manages IT resource performance, including servers, databases, and middleware. Composite application management Monitors and manages an application and its components, and understands applications from the availability standpoint. Event correlation and automation Correlates and automates events or faults that are generated by resource monitoring, application monitoring, or both to provide a concise root-cause analysis of failure in the environment. Orchestration and provisioning Provides the ability to deploy or redeploy servers or components as requested on demand to fulfill processing requirements, if the necessity arises as indicated by the correlation engine.4 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 18. Business service management Provides a high-level view of business status as reflected by its underlying monitoring components. The view is either in real time or based on a service-level agreement.1.1.2 IBM Tivoli Composite Application Manager solution The IBM Tivoli Composite Application Manager family resides in the application management pillar of the Tivoli software portfolio. The current application management portfolio consists of the following products: ITCAM for Response Time Tracking V6.1 ITCAM for Response Time V6.2 IBM Tivoli Composite Application Manager for service-oriented architecture (SOA) V6.1 ITCAM for WebSphere V6.1 ITCAM for J2EE™ V6.1 ITCAM for Web Resources V6.2 ITCAM for CICS Transactions V6.1 ITCAM for IMS Transactions V6.1 IBM Tivoli OMEGAMON® XE for Messaging V6.0 Figure 1-4 shows the scope of composite application management. Response Time WebSphere CICS/IMS Tracking performance transaction Web Services calls WBI messagingFigure 1-4 Composite application management Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 5
  • 19. Manage the overall composite application from the following sides: Get the user side of response time and availability with ITCAM for Response Time Tracking. Get IBM WebSphere middleware performance and analyze in-depth resource usage through ITCAM for WebSphere. Manage messaging from IBM WebSphere Business Integration MQ Series using IBM Tivoli OMEGAMON XE for IBM WebSphere Business Integration. For more details, refer to Implementing IBM Tivoli OMEGAMON XE for WebSphere Business Integration V1.1, SG24-6768. Manage message flow in an SOA environment and collect metrics for Web service calls using IBM Tivoli Composite Application Manager for service-oriented architecture (SOA). Provide the integration view with a mainframe-based, back-end application such as Information Management System (IMS™) or Customer Information Control System (CICS®) using ITCAM for IMS Transactions or ITCAM for CICS Transactions.1.2 Scope of and concerns relating to large-scale implementation This paper discusses large-scale implementation of IBM Tivoli Composite Application Manager. It specifically provides information about the implementation of ITCAM for WebSphere and ITCAM for Response Time Tracking in large-scale environments. The discussion is about large-scale implementation in distributed and mainframe environments, and includes the following topics: 1.2.1, “Defining large-scale implementation” on page 6 1.2.2, “Concerns and considerations” on page 71.2.1 Defining large-scale implementation There are several indications relating to large-scale implementation. These indications are based on the following factors: The number of application servers to be monitored Each application server must have an agent installed to be monitored and managed. With the number of application servers ranging from hundreds to thousands, additional care must be taken to manage the deployment, maintenance, and processing of the managing server.6 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 20. The transaction rates on application servers The transaction rates contribute to the overhead of the monitoring system. A balance of data collection and system health must be achieved. A large number of transactions potentially require larger management server processing. The number of network sites The number of network sites typically corresponds to the potential bottlenecks between the sites. The bottlenecks may be from production data, monitoring data, or a security requirement such as a firewall. The requirement for high availability or fail over This additional requirement, although not directly related to the scale, is typically a must for a large-scale implementation. The existence of multiple managed spaces that a site must handle Managed space is defined as a group of environments with a single management database and a set of management server processes. Different managed spaces are usually used to separate the production and development environments. They are also used to prepare and test the changes to the management environment.1.2.2 Concerns and considerations Following is a list of concerns and considerations that are specific to a large-scale environment: Server size As this is a large-scale implementation, sizing the servers to manage the environments is critical. The placement, configuration, and specification of a single server or multiple servers must be predetermined in order to avoid bottlenecks in processing. This sizing must also take into consideration special processing requirements such as debugging and troubleshooting and data collection and recovery. Deploying agents The number of agents that must be deployed are enormous and prohibitive to being performed manually. Automated efforts must be included in the ability to deploy and implement the agents automatically with minimal manual intervention. This must cover initial deployment, fix pack implementation, and maintenance action. Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 7
  • 21. Security This includes confidentiality support and firewall support. – Confidentiality support secures information transfer between the agents and the servers. – Firewall support allows the sites to be secured, with management action still flowing through in order to effectively manage the environment. Reliability Fail over and fault tolerance are critical to maintain while monitoring business-critical applications. The reliability factor must be promptly addressed and ensured. Maintenance Changes do happen, as with deployment. These changes must be applied to both the servers and the agents. Special consideration must be provided for a large-scale implementation with changes on both the servers and the agents. While server consideration applies to preserving, monitoring, and data collection with minimal downtime, agent consideration relates to automating the deployment process with minimal manual intervention and outage. This paper deals with and addresses these concerns for ITCAM for Response Time Tracking and ITCAM for WebSphere implementations.1.3 Overview of IBM Tivoli Composite Application Manager This section explains the following topics: 1.3.1, “Understanding ITCAM for WebSphere” on page 8 1.3.2, “Understanding IBM Tivoli Composite Application Manager for Response Time Tracking” on page 111.3.1 Understanding ITCAM for WebSphere This section provides an overview of ITCAM for WebSphere. The discussion includes the following topics: “Features and functions” on page 9 “Components” on page 9 “Platforms supported” on page 108 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 22. For more information about ITCAM for WebSphere, visit the following Web site:http://www.ibm.com/software/tivoli/products/composite-application-mgr-websphere/Features and functionsITCAM for WebSphere helps increase the performance and availability ofbusiness-critical applications by providing facilities for real-time problemdetection, analysis, and repair. Correlation spanning Java 2 Platform, EnterpriseEdition (J2EE), Customer Information Control System, and InformationManagement System, and diagnostics at the method level pinpoint codeproblems to help resolve problems quickly and reduce support and operationscosts.Today’s business processes often depend on a number of complex applications.Although most businesses have traditional monitoring tools to manage individualresources at a high level, many lack an integrated solution to automaticallymonitor, analyze, and resolve problems at the service, transaction, application,and resource levels. As a result, operations and development may take a longtime to identify, isolate, and fix composite application problems.ITCAM for WebSphere is an application management tool that helps maintainthe availability and performance of on demand applications. It helps you toquickly pinpoint, in real time, the source of bottlenecks in application code, serverresources, and external system dependencies. This product also providesdetailed reports that you can use to enhance the performance of yourapplications. ITCAM for WebSphere provides in-depth, WebSphere-basedapplication performance analysis and a tracing facility.ITCAM for WebSphere enables multiple levels of analysis to get a complete viewof the application, depending on the requirement. From production-levelmonitoring to detailed heap and method debugging, it digs into Structured QueryLanguage (SQL) performance analysis without the need for database monitors. Itprovides SQL information and information about calls that were made throughJava Database Connectivity (JDBC™). ITCAM for WebSphere provides acomposite status correlation for transactions that use Customer InformationControl System and Information Management System as the back end.ComponentsITCAM for WebSphere contains the following components: Managing server This acts as the central component that manages and administers the data collectors. It stores that data in a relational database repository. A Web-based Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 9
  • 23. application is provided to show monitoring results. This interface is also called the visualization engine. Data collector This runs on the application server and collects performance information for the managing server. Tivoli Enterprise Monitoring Agent This collects information that shows the status of the WebSphere Application Server and sends it to the Tivoli Enterprise Monitoring Server for display on the Tivoli Enterprise Portal. The Tivoli Enterprise Monitoring Agent is installed on individual machines where data collectors reside. This component is moved to IBM Tivoli Composite Application Manager for Web Resources in Version 6.2. Platforms supported For a complete platform coverage list, refer to the following Web site: http://publib.boulder.ibm.com/tividd/td/ITCAMWAS/prereq60/en_US/HTML/itc am6.html Table 1-1 provides an overview of the platforms supported for ITCAM for WebSphere V6.Table 1-1 Platforms supported for ITCAM for WebSphere Component Software Managing server operating IBM AIX V5.2 and V5.3 system Solaris™ 8 and Solaris 9 (SPARC) Hewlett-Packard UNIX® (HP-UX) 11i 1 Windows® 200 Server or Advanced Server with Service Pack 4 (SP4) Windows 2003 Server Standard Edition/Enterprise Edition (SE/EE) Red Hat Enterprise Linux® (RHEL) 3.0 and 4.0 SUSE Linux Enterprise Server (SLES) 8 and 9 Managing server database IBM DB2® V8.1 Fix Pack 6 (FP6) or IBM DB2 V8.2 Oracle® 8i SE R3 8.1.7, Oracle 9i SE R2 9.2, Oracle 10g Managing server WebSphere WebSphere Application Server V5.1.x or WebSphere Application Server V6.x10 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 24. Component SoftwareData collector platform AIX V5.2 and V5.3 Solaris 8 and 9 SPARC HP-UX 11i 1 Windows 200 Server or Advanced Server with SP4 Windows 2003 Server SE/EE RHEL 3.0 and 4.0 SLES 8 and 9 Red Flag Advanced Server (RFAS) 4.0 and 4.1(xLinux) IBM Operating System/400® (OS/400®) V5.2 and V5.3 IBM z/OS V1.4, V1.5, or V1.6Customer Information Control V2.2, V2.3, and V3.1SystemInformation Management V7.1, V8.1, and V9.1System1.3.2 Understanding IBM Tivoli Composite Application Manager for Response Time Tracking This section provides an overview of ITCAM for Response Time Tracking. It discusses the following topics: “Features and functions” on page 9 “Components” on page 12 “Platforms supported” on page 14 For more information about ITCAM for Response Time Tracking, visit the following Web site: http://www.ibm.com/software/tivoli/products/composite-application-mgr-rtt/ Features and functions ITCAM for Response Time Tracking proactively recognizes, isolates, and resolves transaction performance problems by using robotic and real-time techniques. It is an end-to-end transaction management solution that monitors user response time and helps you to visualize the transaction’s path through your application systems, including the response time contributions of each step. ITCAM for Response Time Tracking uses Application Response Measurement (ARM) technology to track the response time of a distributed application. Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 11
  • 25. Today’s business processes often depend on composite applications that span Web servers, J2EE application servers, integration middleware, and mainframe systems. Although most businesses have traditional monitoring tools to manage individual resources, many lack an integrated solution to automatically monitor, analyze, and resolve user response time problems. As a result, it may take a long time to identify, isolate, and fix distributed transaction performance problems. ITCAM for Response Time Tracking enables you to follow the path of a user transaction end-to-end across your business infrastructure. You can drill down to each step the transaction takes as it travels across multiple systems, and measure how each component of a transaction contributes to the overall response time. The entire transaction analysis process is transparent to customers and application developers. It collects transaction performance through robot and browser simulation, in-depth J2EE server instrumentation, and feedback from Customer Information Control System and Information Management System. ITCAM for Response Time Tracking feeds the Tivoli Enterprise Monitoring Server to provide a comprehensive performance management solution on Tivoli Enterprise Portal. This enables the development of custom monitoring workspaces for managing enterprise applications. Components ITCAM for Response Time Tracking consists of the following components: Management server This acts as the central point of contact for ITCAM for Response Time Tracking. It consists of a WebSphere-based J2EE application that performs the management and administrative functions. The management server stores data in a central database repository. Store and Forward Agent This relays traffic to and from the management agents. Typically, the Store and Forward agent is used in a firewall environment. It consolidates the port requirements for the connectivity. Management agent This performs the monitoring function. Typically, it investigates the performance of the distributed application, depending on the management components deployed on it. The components that you can deploy are: – Generic Windows workstation This allows deployment of IBM Rational® Robot to measure transaction performance.12 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 26. – Client Application Tracker This uses IBM ETEWatch® scripts to collect performance information. Default monitoring is available for measuring IBM Lotus® Notes® and Microsoft® Outlook® performance. – Synthetic Transaction Investigator (STI) This performs Web-based transactions and measures the resulting response time. – Quality of Service monitoring agent This collects information about user performance by acting as reverse proxy between the user and the Web server. – JavaTM 2 Platform, Enterprise Edition (J2EE) monitoring agent This instruments and collects performance information about J2EE-based application servers such as WebSphere or WebLogic. – Web Response Monitor component – Rational Performance Tester – Tomcat and JBoss monitoring component – Generic Application Response Measurement (ARM) agent This collects ARM events from a custom-instrumented application. Tivoli Enterprise Monitoring Agent for Tivoli Enterprise Monitoring Server This feeds data from the ITCAM for Response Time Tracking server to display on the Tivoli Enterprise Portal.Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 13
  • 27. Platforms supported For a complete platform coverage list, visit the following Web site: http://publib.boulder.ibm.com/tividd/td/ITCAMRTT/prereq60/en_US/HTML/ Version60.html Table 1-2 provides an overview of the platforms supported for ITCAM for Response Time Tracking V6.0.Table 1-2 Platforms supported for ITCAM for Response Time Tracking Component Software level Management server operating Microsoft Windows 2000 Server with SP4 system Windows 2000 Advanced with SP4 Windows 2003 Server SE or EE IBM AIX V5.2 or V5.3 Solaris 9 or 10 HP-UX 11i 1 RHEL 3.0 or 4.0 SLES 8 or 9 Management server database Oracle 9i SE 9.2 IBM DB2 V8.1 ESE with FP3+ (required for WebSphere Application Server V5.1.x) IBM DB2 V8.1 ESE with FP6a+ (required for WebSphere Application Server V6.x) IBM DB2 V8.2 Management server WebSphere WebSphere Application Server V5.1.x or later versions WebSphere Application Server V6.0.1.x or later versions Management agent platform Windows 2000 Professional, Server or Advanced Server with SP4 Windows 2003 Server SE or EE Windows XP Professional with SP1 IBM AIX V5.2 or V5.3 Solaris 9 or 10 HP-UX 11i RHEL 3.0 or 4.0 SLES 8 or 9 RFAS 4.0 or 4.1 (xLinux) z/OS V1.4, V1.5, or V1.6 OS/400 V5.2 or V5.314 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 28. 1.4 Document organization This paper discusses the following topics: Before the implementation Chapter 2, “Planning for ITCAM for WebSphere” on page 17, and Chapter 5, “Planning for ITCAM for Response Time Tracking” on page 93, discuss the planning and sizing considerations. The implementation Chapter 3, “Installing ITCAM for WebSphere” on page 33, and Chapter 6, “Installing ITCAM for Response Time Tracking” on page 105, discuss additional steps that are required, such as reliability and automation considerations. After the implementation Chapter 4, “Maintenance of ITCAM for WebSphere” on page 75, and Chapter 7, “Maintenance of ITCAM for Response Time Tracking” on page 137, discuss maintenance considerations and operational concerns relating to a large-scale implementation. Chapter 1. Overview of IBM Tivoli Composite Application Manager implementation 15
  • 29. 16 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 30. 2 Chapter 2. Planning for ITCAM for WebSphere This chapter provides information about areas that must be considered during the planning phase of IBM Tivoli Composite Application Manager implementation in a large environment. This chapter discusses the following topics: 2.1, “Planning considerations” on page 18 2.2, “Product architecture of ITCAM for WebSphere” on page 18 2.3, “Deciding on the size of the servers” on page 21 2.4, “Implementation options for ITCAM for WebSphere” on page 25 2.5, “Communication and security considerations” on page 29 2.6, “Reliability and high availability” on page 32© Copyright IBM Corp. 2006, 2007. All rights reserved. 17
  • 31. 2.1 Planning considerations This section discusses the following aspects pertaining to large-scale implementations (see also 1.2.2, “Concerns and considerations” on page 7): Understanding the product architecture This allows you to make the correct decisions. Section 2.2, “Product architecture of ITCAM for WebSphere” on page 18, describes the architecture for ITCAM for WebSphere. Sizing the servers This is important to correctly acquire adequate servers and choose a sound software configuration option. Section 2.3, “Deciding on the size of the servers” on page 21, describes one approach. Understanding the servers’ configuration options and agent deployment This is discussed for ITCAM for WebSphere in 2.4, “Implementation options for ITCAM for WebSphere” on page 25. Planning for communication security This is a mandatory step for an enterprise with business-critical and sensitive information in a transaction environment. Section 2.5, “Communication and security considerations” on page 29, discusses confidentiality and firewall requirements. Discussing reliability, failover, and disaster recovery issues These are the other mandatory aspects pertaining to a critical business process on a large enterprise. Section 2.6, “Reliability and high availability” on page 32 discusses this.2.2 Product architecture of ITCAM for WebSphere This section discusses the product architecture of ITCAM for WebSphere. This understanding is critical to plan and decide about the server configuration and other implementation issues. See also IBM Tivoli Composite Application Manager V6.1 Family Installation, Configuration, and Basic Usage, SG24-7151. ITCAM for WebSphere V6.0 evolved from WebSphere Studio Application Monitor (WSAM) and IBM Tivoli OMEGAMON XE for WebSphere. ITCAM for WebSphere observes and reports on the health of Java 2 Platform, Enterprise Edition-based applications. It tracks the progress of applications as they traverse through Java 2 Platform, Enterprise Edition (J2EE) application servers,18 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 32. middleware adapters and transports, and database calls, to back-end systemssuch as Customer Information Control System (CICS) or InformationManagement System (IMS) to extract business data or to invoke mainframebusiness processes.The tracking of applications produces request traces, where the events in arequest’s life are recorded and stored in a monitoring repository database.ITCAM for WebSphere captures the CPU and the elapsed internal times whenevents are called and exited, measuring as far down as the CPU consumed andthe elapsed internal times charged to individual methods in J2EE classes. Themethods or events taking the most time are marked as an application’s parts thatdeserve attention for runtime improvement studies and code optimizations.ITCAM for WebSphere does not require modification of any J2EE or mainframeapplication code. Java Virtual Machine Tool Interface (JVMTI) interfaces andprimitives, along with WebSphere Performance Management Interface (PMI) andz/OS System Measurement Facility (SMF) 120 records, are ITCAM forWebSphere’s principal data sources. The monitoring data is collected andanalyzed to offer a wealth of information about the health of J2EE applicationsand their servers.Many system-level performance metrics are collected and reported about J2EEapplication servers. The status of the servers and their resources, particularly atvital checkpoints such as CPU utilization, memory usage, and the status ofinternal components such as database connection pools, Java Virtual Machine(JVM™) thread pools, Enterprise JavaBeans™ (EJB) usage, and requestprocessing statistics, are very important in locating real-time problems with J2EEapplications. ITCAM for WebSphere brings attention to these critical indicatorswith real-time, graphical displays of their values and their trends over a span oftime.ITCAM for WebSphere is a distributed performance monitoring application forapplication servers. Its components are connected through IP networkcommunication. The central component of ITCAM for WebSphere, the managingserver, is its heart and brain. It collects and displays various performanceinformation from application servers.The application servers run a component of ITCAM for WebSphere called datacollector, which is a collecting agent that runs in the application server andsends monitoring information to the management server. These data collectorsoperate independently of each other. Chapter 2. Planning for ITCAM for WebSphere 19
  • 33. Figure 2-1 shows the overall architecture of ITCAM for WebSphere. Browser interface ITCAM for WebSphere Managing Server I Web Server Tivoli Enterprise Monitoring Server and Application servers with Tivoli Enterprise ITCAM for WebSphere Portal Server Data collectors Figure 2-1 ITCAM for WebSphere architecture The application monitor comprises the following main parts: Managing server A managing server comprises several Java-based components that provide the environment to collect and present management data. Data collector agent A data collector agent runs on each monitored application server, whether J2EE, Customer Information Control System (CICS), or Information Management System (IMS), and communicates essential operational data to the managing server. Unique sampling algorithms maintain low CPU and network overhead, while providing application-specific performance information.20 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 34. 2.3 Deciding on the size of the servers The scale of the implementation must decide the size of the servers to be used. Sizing determines the hardware configuration and implementation consideration of the servers. This section discusses the following topics: 2.3.1, “Sizing parameters” on page 21 2.3.2, “Sizing estimation for ITCAM for WebSphere managing server” on page 222.3.1 Sizing parameters The following parameters must be considered before deciding on the size of the servers: The number of data collectors for ITCAM for WebSphere This value assumes that the application servers run a similar load profile. If the application servers have several load profiles, consider them in different groups. The transaction rate for application servers The number of transactions executed for each minute, when multiplied with the number of data collectors or monitoring agents, gives the total amount of transaction information captured for a given period. The complexity of a transaction It is not easy to understand the complexity of a transaction. This requires a more subjective approach than transaction rate counting, which can be retrieved from the transaction data or the application log. The relative complexity of transactions is determined by the number of method calls per transaction. Typically, the number of methods a complex transaction invokes is around four to six times that of a simple transaction. There are some product-specific parameters that affect sizing considerations. These parameters are built to filter out unimportant or insignificant information from the data that is collected. These parameters are: – Data collection filter – Sampling rate – Monitoring level – Listening policy mask – Instrumentation level Chapter 2. Planning for ITCAM for WebSphere 21
  • 35. 2.3.2 Sizing estimation for ITCAM for WebSphere managing server Specific to ITCAM for WebSphere, consider the following parameters for sizing: Communication bandwidth Memory size Processing requirement Database size Important: Sizing estimation for ITCAM for WebSphere managing server must be estimated for a worst-case scenario, that is, in the state that level 3 monitoring is run for the highest number of data collectors concurrently. Communication bandwidth Several communication traffic flows exist between the managing server and the data collector. The communication traffic flows are: Initial communication with the kernel to collect configuration information This only happens in the initial connection when the data collector is started. This configuration information consists of sending the configuration and managing server Java archives. Management information to modify data collection level, sampling interval, or logging level from the kernel This happens by request or when scheduled by Monitoring On Demand®. The size of this communication is small and negligible. Visualization engine requests for current active transactions The impact of these requests depends on the following factors: – The transaction rate and the average transaction response time that make up the average number of in-flight transactions – The number of concurrent Web console users who may request the in-flight transaction information Transaction information is streamed to the publish server as it happens This is the largest contributor to network load. It uses up the largest amount of network bandwidth. The formula is as follows: – Monitoring in level 1: transaction rate x 353 – Monitoring in level 3: (transaction rate x 353) + (transaction rate x method call x 172)22 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 36. As an illustration, we use a sample environment with a transaction rate of 3,000 requests per minute on level 1 and 300 requests per minute on level 3 monitoring. The average method calls is 500 methods per requests. The transaction bandwidth required is: – Level 1 transaction load: (3000 transaction / 60 seconds) x 353 = 17,650 bytes/sec – Level 3 transaction load: (30 transaction / 60 seconds) x 353 + (30 / 60) x 5,000 x 172 = 430,176 bytes/sec As shown in this example, the majority of network usage is spent on level 3 analysis. In a real production environment, for the majority of time, ITCAM for WebSphere runs on level 1. Therefore, the communication requirement is low. However, prepare an installation to occasionally increase monitoring in level 3 for problem determination purposes.Memory sizeMemory requirement is typically important for the following components: Kernel The memory size of the kernel is directly related to the number of data collectors. The typical size of 64 MB in the setenv.sh may have to be increased for more than 50 data collectors. Publish server The memory size is related to the number of transactions the publish server has to process, with some consideration to the transaction complexity factor, that is, the number of methods invoked. The publish server’s memory must be adequate to handle the data size between garbage collector intervals. For garbage collection per minute, you must accommodate a minute’s worth of data. In the example provided in “Communication bandwidth” on page 22, the total size of publish server memory for processing the load must be around 4.3 x 60 x (1.5) = 387 MB. Note that the base publish server was already using around 100 MB of storage. Archive agent This requires memory as a subset to the publish server and is masked by the sampling percentage from the publish server. The archive agent uses more memory than the sampling rate percentage, as it performs Java Database Connectivity (JDBC) database calls. Chapter 2. Planning for ITCAM for WebSphere 23
  • 37. Visualization engine memory size This depends on the number of users who are connected and the activities that they perform. Users are categorized into the following groups: – Users monitoring the availability screens – Users collecting performance reports – Users monitoring in-flight threads Modify the visualization engine’s memory size by using the WebSphere Application Server administration console. Memory sizes for ITCAM for WebSphere components are defined in the setenv.sh file that is sourced by all overseer components. Processing requirement The processor requirement for ITCAM for WebSphere is directly related to the transaction rate. The largest processor usage is for the following components: Publish server: to process transaction data Database engine: for interface to the database Archive agent: to perform SQL calls WebSphere Application Server: to process user requests Database size The typical database size requirement depends on: The number of application server statistics The transaction volume to be stored The complexity of transaction The duration to keep the data Database table information that increases in size during ITCAM for WebSphere execution is: requests: number of requests x 353 bytes methods: number of methods x # requests in L3 x 172 bytes pmidata: number of data collectors x (3600/polling interval) x 73 bytes serverstats: number of data collectors x (3600/polling interval) x 107 bytes volumestats: number of data collectors x (3600/polling interval) x 74 bytes memorydata: number of data collectors x (3600/polling interval) x 115 bytes gcdata: number of data collectors x (3600/garbage collection interval) x104 bytes24 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 38. 2.3.3 Data collector overhead Monitoring with ITCAM for WebSphere has overhead related to data collectors running on a production WebSphere Application Server. The overhead is minimal for data collectors running on level 1 monitoring. This is typically around a 2–3% increase of CPU time with no notable memory or disk input/output (I/O) requirement. When the monitoring level is increased, the processing overhead of ITCAM for WebSphere data collectors also increases. This increase is due to the fact that ITCAM for WebSphere collects more data from more sources. A typical level 2 monitoring generates around a 10% increase in processing usage, while a level 3 monitoring generates around 25–30% overhead. This means that level 2 or level 3 monitoring must be used sparingly in your production environment. To change the monitoring level for purposes of problem determination, schedule it to start and then step back to level 1 automatically in order to reduce the impact on users.2.4 Implementation options for ITCAM for WebSphere Depending on the size of your implementation, there are some considerations for implementing ITCAM for WebSphere. This section discusses the following topics: 2.4.1, “Designing the managing server” on page 25 2.4.2, “Deploying a large number of data collectors” on page 282.4.1 Designing the managing server The ITCAM for WebSphere managing server consists of the following products: IBM DB2 Universal Database™ Enterprise Server or Oracle database server WebSphere Application Server ITCAM for WebSphere managing server Chapter 2. Planning for ITCAM for WebSphere 25
  • 39. Figure 2-2 shows the conceptual relationship between the components. Snapshot traffic Publish traffic Global Publish Server (SAM) Publish Server (PS) Kernel (KL) Visualization Engine Message Dispatcher Provide services on: Provide services on: (MD) - Lookup -Administration - Registration -Availability - Recovery -Problem Determination Archive Agent (AA) - Configuration -Performance Management Polling Agent (PA) OCTIGATE databaseFigure 2-2 ITCAM for WebSphere components The following ITCAM for WebSphere components are displayed in Figure 2-2: Kernels These control the managing server. There are always two copies of kernels running on an ITCAM for WebSphere managing server for redundancy and failover. The kernels register components as they join the managing server, periodically renew connections and registrations with components and data collectors, and collect server and component availability information. Publish servers These receive application and system event data from the data collectors, gather and compute request-level information about performance metrics such as response times, and implement the trap monitoring and alerts features. Archive agents These receive monitoring data from the publish servers and store the monitoring data in ITCAM for WebSphere’s repository. Global publishing server This collects information from the publish servers and correlates all parts and pieces of multi-server requests, such as requests from J2EE servers to execute Customer Information Control System (CICS) or Information Management System (IMS) programs.26 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 40. Message dispatcher This is a conduit for messages from ITCAM for WebSphere using e-mail and Simple Network Management Protocol (SNMP) facilities. Polling agent This collects data from Web servers for Apache 2.0 and later versions. Visualization engine This is a Web-based graphical user interface (GUI) with access to graphics, ITCAM for WebSphere performance reports, real-time views of different slices of monitoring data, ITCAM for WebSphere internal commands, and event-driven functions. The visualization engine runs on a J2EE server such as WebSphere Application Server.Although ITCAM for WebSphere provides the facility to install all the componentsin a single wizard, which is called embedded installation, individually installingeach component allows more flexibility in terms of verifying each component andconfiguring them to suit your requirements. The considerations that you mustkeep in mind when installing the components are: Database You can install the database locally on the managing server or on a separate database server. ITCAM for WebSphere provides database configuration scripts to assist with the configuration of a remote database. Utilizing a remote database, regardless of whether it is a DB2 Universal Database or an Oracle database, relieves the processing load on the managing server. An environment with hundreds of data collectors generates a large amount of data flowing into the database. This amount increases considerably if the data collectors are set to run monitoring at level 2 or level 3, even for a short period of time. A remote database allows database query processing and recording to be processed using dedicated hardware, instead of sharing with the main managing server that is already busy with processing the transaction information. WebSphere Application Server The visualization engine of the managing server acts as the administration console for ITCAM for WebSphere. The visualization engine is deployed on a WebSphere Application Server JVM that resides in a standalone application server or an application server that is a part of a network deployment environment. We recommend that you install the visualization engine on a separate application server JVM that is not monitored by ITCAM for WebSphere data collectors, especially in a network deployment environment. This reduces any Chapter 2. Planning for ITCAM for WebSphere 27
  • 41. possible conflicts that may arise with respect to ITCAM for WebSphere. In addition, if an issue does arise, problem determination will be somewhat easier due to the separation. ITCAM for WebSphere components Configure the managing server to handle large amounts of data by adding additional components, such as the publish servers and the archive agents. When adding the publish servers and the archive agents, the distribution of data is handled by the managing server. The amount of data being written to the database is handled more efficiently as well. Another major consideration for the managing server is the split server installation. This option provides the managing server with the overseer processes that exist on separate machines, including the kernel, which provides load balancing and failover capabilities. There are benefits to this type of configuration when there are hundreds of data collectors providing data to the managing server. This type of setup not only allows the managing server to handle more memory and disk space usage, but also provides a failover capability. For more information about split server installation, refer to 3.1, “Installing ITCAM for WebSphere managing server” on page 34.2.4.2 Deploying a large number of data collectors Installation of a small amount of ITCAM for WebSphere data collectors is performed by using the graphical-based installation and configuration wizard provided by the product. When presented with the task of deploying hundreds of data collectors into an environment, the graphical interface is no longer a good option. This non-interactive automated installation method is commonly known as silent installation. The use of silent installation provides a means to deploy a larger number of data collectors in a more efficient manner. When performing the silent installation, information about the WebSphere environment must be known ahead of time. A response file will be used during the installation, and if incorrect information is used, may result in a failed install. When performing the silent installation of data collectors, the WebSphere Application Server version must be taken into account, as V6 introduced the usage of profiles. The response files for silent installation are different for various versions of the WebSphere Application Server. In some cases, when two versions of WebSphere Application Server are present, it is better to have two separate master response files.28 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 42. Installing the Tivoli Enterprise Monitoring Agent (TEMA) is also an option of the silent installation. Although Tivoli Enterprise Monitoring Agent can be installed using silent installation, more configuration must be performed to connect to IBM Tivoli Monitoring V6.1 Tivoli Enterprise Monitoring Server. Mass automated installation is also possible by using a software distribution or provisioning solution such as the IBM Tivoli Configuration Manager. There is an additional consideration for deploying data collectors on a machine that has multiple application servers installed. Consider installing a separate data collector directory set for each application server, because applying a fix pack for data collectors requires you to stop the application server. You have a more flexible scheduling option with separate data collector installation for each application server.2.5 Communication and security considerations Communication and security issues are vital to the inter-networked world that we live in. Applications and their management infrastructure must be secured in order to protect resources from unauthorized sources. This section discusses the following planning considerations: 2.5.1, “Communication security” on page 29 2.5.2, “Firewall and port consideration” on page 302.5.1 Communication security Communication security relates to the confidentiality of the information transmitted over a network. Management information that is used by IBM Tivoli Composite Application Manager products may contain details about application processing internals. This requires the content of the management information to be secured from being accessed by unauthorized sources. WebSphere security WebSphere security plays a significant role in a large-scale implementation. In some cases, WebSphere security is not enabled during the test phase of an implementation, but in a production environment. This requires certain additional considerations. The WebSphere user must have the appropriate permissions to, for instance, issue a wsadmin command. The configuration of data collectors involves the use of Java Command Language (JACL) scripts, and can fail when there is a permission problem. Chapter 2. Planning for ITCAM for WebSphere 29
  • 43. If any of the application servers on which the data collector is installed has WebSphere security enabled on it, the entire ITCAM for WebSphere environment must have it enabled as well. This includes WebSphere security being enabled on the ITCAM for WebSphere managing server. Secure Sockets Layer communication Secure communication between the managing server and the data collector is a viable option if there is a requirement for data to be encrypted during transmission. Using Secure Sockets Layer (SSL) provides secure data transmission from the data collector to the managing server and must appease corporate security requirements, if necessary. Additional configuration must take place on the managing server and the data collector when enabling SSL. A certificate key generator is included with the product. This key generator provides the facility to use custom-generated keys. A best practice is to complete the default installation of the managing server and the data collector and then enable SSL for both. This isolates problems (that is, whether the problem is caused by the basic installation or the SSL configuration).2.5.2 Firewall and port consideration Firewall and port issues arise when the data collectors are on a different site, location, or subnet from the managing server. Problems such as name resolution occur if the Domain Name System (DNS) is not set up correctly on either the managing server or the data collectors. Routing problems occur if the Internet Protocol (IP) addresses used belong to different subnets. The entire network environment must be looked into in order to determine where a firewall, router, or bridge may exist.30 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 44. Figure 2-3 shows the communication port requirement for ITCAM forWebSphere. DC Command Agent KL1 DC Event Agent DC Command Agent KL2 Port Consolidator DC PS1 Event Agent DC Command Agent PS2 DC Event AgentFigure 2-3 Communication port requirementsThe managing server requires open ports for each kernel and publish server.The data collector requires open ports for the command agent and the eventagent. The port consolidator requires a port to communicate to the managingserver. Use a single port consolidator to consolidate communication frommultiple data collectors.A port consolidator is useful to limit the number of ports required forcommunication between the data collector and the managing server. Portconsolidation is a viable option if there is a limit to the number of ports that canbe opened on the firewall. Additional configuration must be carried out on thedata collector, including the configuration of the data collector to go through theport consolidator, and starting the port consolidator process. Chapter 2. Planning for ITCAM for WebSphere 31
  • 45. 2.6 Reliability and high availability This section discusses reliability issues that relate to failover and disaster recovery.2.6.1 Failover and fault tolerance Split server configuration for ITCAM for WebSphere or the clustering server for ITCAM for Response Time Tracking consists of having two or more management servers running on separate physical machines. Hardware or software errors do occur on a machine and cause the server to cease functioning. Using the separate server configuration, the secondary server can handle the entire load until the failing machine is recovered. The switchover to the secondary managing server is not automatic. Manual intervention must take place for the failover to be successful. There are specific ITCAM for WebSphere components that can only be run on one managing server. They must therefore be started on a secondary server, such as the global publish server or the message dispatcher, if the primary server goes down.2.6.2 Disaster recovery There are three areas where a backup is necessary for disaster recovery with respect to the IBM Tivoli Composite Application Manager server: Database Perform database backup regularly to collect the most up-to-date information. Use the database utility function to perform the backup function. WebSphere Application Server configuration for the server and agent or data collectors Perform a backup for these for them to be restored in a disaster recovery scenario. IBM Tivoli Composite Application Manager servers These servers must be physically considered for recovery in the disaster recovery site.32 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 46. 3 Chapter 3. Installing ITCAM for WebSphere This chapter discusses the installation and deployment procedure for ITCAM for WebSphere. This chapter provides information about the following topics: 3.1, “Installing ITCAM for WebSphere managing server” on page 34 3.2, “Deploying data collectors” on page 50 3.3, “Configuring and setting up SSL communication” on page 64© Copyright IBM Corp. 2006, 2007. All rights reserved. 33
  • 47. 3.1 Installing ITCAM for WebSphere managing server This section discusses some of the considerations for installing the ITCAM for WebSphere managing server and provides information pertaining to the following topics: 3.1.1, “Installation configuration” on page 34 3.1.2, “Database installation and configuration” on page 35 3.1.3, “WebSphere Application Server considerations” on page 41 3.1.4, “Configuring the split server of the managing server” on page 42 3.1.5, “Installation and setup of split server” on page 43 3.1.7, “Adding additional publish servers and archive agents” on page 483.1.1 Installation configuration The installation configuration for IBM Tivoli Composite Application Manager for WebSphere managing server in a large-scale environment is shown in Figure 3-1. OCTIGATE Managing Server machines Database Server Figure 3-1 Installation configuration for ITCAM for WebSphere managing server Depending on the scale of your implementation, choose to implement one or more of the following structures: Remote database, so that the database processing overhead does not hinder the processing of the managing server, although the database server must be connected through a local network to the managing server to minimize the networking overhead for storing the data over the network.34 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 48. The managing server split-server configuration facilitates the load to be spread into several machines, besides allowing fault tolerance of the managing server components. The configuration that is discussed here covers the database, application server, and the managing server overseer components.3.1.2 Database installation and configuration As mentioned in 2.4, “Implementation options for ITCAM for WebSphere” on page 25, utilizing a remote database for the managing server helps reduce memory consumption and processing load in a large-scale implementation. This section discusses how to deploy the database on a remote server using IBM Database 2™ (DB2) Universal Database (UDB), Enterprise Server Edition. Instructions about how to set up and configure a remote database are available in Chapter 13 of IBM Tivoli Composite Application Manager for WebSphere Installation and Customization Guide, GC32-9506. Implementing ITCAM for WebSphere with a remote database involves the following steps: 1. Installing the DB2 UDB, Enterprise Server Edition on the database server machine. 2. Installing DB2 Client Application Enabler on the managing server machines and visualization engine machines. These clients are necessary for communicating with the remote database server. 3. Cataloging database information about the DB2 clients. 4. Defining the OCTIGATE database structure on the remote database server machine. The software installation is not discussed here, as this must be performed in accordance with the database documentation. DB2 Universal Database server considerations The DB2 UDB installation includes defining the distribution files and file systems for the database itself. In a large environment where performance is critical, follow these recommendations: Store the database files in a separate physical disk rather than the operating system paging and DB2 UDB system files. This allows you to minimize disk contention for accessing the database files. Chapter 3. Installing ITCAM for WebSphere 35
  • 49. Allocate adequate memory for DB2 UDB application pools to ensure performance. The most important pool is the DB2 bufferpool, which is used to buffer the database data for quick and efficient access. Use a faster disk for database storage, possibly separating the file systems for recovery log and data. Database installation scripts ITCAM for WebSphere provides remote database creation scripts in the tar file supplied in the managing installation image: <ms-installation-image-directory>/base/scripts/db2-remote-scripts.tar The scripts in the tar file are aimed at a UNIX-based environment installation or a Windows-based environment with Microsoft Windows Services for UNIX (SFU) installed. On untarring the db2-remote-scripts.tar file, you will be presented with various installation scripts in the bin directory that is created. The primary executable is db2install. Depending on the platform, different syntax exist for executing the db2install command. The following example uses amuser as the owner ID and admin as the default user from the visualization engine. For Windows, the syntax is: db2install.bat jdbc_user jdbc_password adminVEuser instance_user instance_password For example: db2install.bat amuser ampasswd admin db2inst1 dbpasswd For UNIX, the syntax is: db2install.sh db2username db_user absolute_path_of_install_directory adminVEUser For example: ./db2install.sh db2inst1 amuser /tmp/db2/ admin The script uses the su command several times to switch the user as the DB2 instance owner. We recommend that you log in as the root user, so that you are not prompted for the DB2 user password multiple times. If you log in as the root user, add the root user to the DB2 administrator group, typically called db2grp1. You must also define the installation owner user in the operating system of the database machine. This user is typically called amuser and requires administrative access.36 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 50. Cataloging the database on the DB2 Universal Database clientThe DB2 client must be able to connect to the actual DB2 OCTIGATE database.This connection is defined using the following definitions: Node definition This indicates where the DB2 UDB server resides. Database definition This indicates the database selection in the node.Start the DB2 UDB command-line interface using the db2cmd command in aWindows-based platform or by sourcing the db2profile on a UNIX-basedplatform.To catalog the node, issue the following command:db2 catalog tcpip node <ipnode> remote <ipaddr> server <ipport>Following is a description of the parameters in this command: ipnode: the database node name. ipaddr: the TCP/IP host name or address the client addresses the server by. ipport: the port number on which the DB2 server listens. The default port number is 50000.To catalog the database, issue the following command:db2 catalog database OCTIGATE at node <ipnode>In this command, ipnode is the database node name as defined by the catalogtcpip node command. Chapter 3. Installing ITCAM for WebSphere 37
  • 51. The actual OCTIGATE database must exist to connect to the database. The connection is performed using the db2 connect command to OCTIGATE. Figure 3-2 shows a successful connection. # . /home/db2inst1/db2profile # db2 connect to OCTIGATE user db2inst1 using db2passw Database Connection Information Database server = DB2/AIX 8.2.0 SQL authorization ID = DB2INST1 Local database alias = OCTIGATE Figure 3-2 Successful connection to OCTIGATE We recommend that you create the database first and then catalog it. This is to help you to test the connection to the database from the DB2 client immediately. Although you can catalog the database without testing the connection, testing it ensures connectivity. Changing database name from OCTIGATE Note: Although we do not recommend changing the database name of the ITCAM for WebSphere managing server from OCTIGATE, some installations may have strict naming conventions for the database name. To change the database name OCTIGATE to another name, there are some modifications required on: Installation and configuration scripts Runtime database connection Example 3-1 illustrates the changes required in db2configuration.sh to modify the database name from OCTIGATE to CAMFWAS. Example 3-1 Modification to db2configuration.sh # Drop the octigate database first echo "$DBUSER> db2 drop database camfwas" db2 drop database camfwas #Fix SQL1005N problem. #One of the causes is that octigate database may be created by other DB2 instance #so it still exists in system catalog. echo "$DBUSER> db2 catalog database camfwas"38 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 52. db2 catalog database camfwasecho "$DBUSER> db2 drop database camfwas"db2 drop database camfwasecho "$DBUSER> db2 uncatalog database camfwas"db2 uncatalog database camfwas# Create the octigate databaseecho "$DBUSER> db2 create database camfwas using codeset UTF-8 territory US"db2 create database camfwas using codeset UTF-8 territory USstatus="$?"echo "$DBUSER> db2empfa camfwas"db2empfa camfwas# Configure DB2 and the Octigate Databaseecho "$DBUSER> db2 -tf $2"db2 -tf $2# echo "Change IBMDEFAULTBP size to -1 so BUFFPAGE setting will be used"db2 connect to camfwas...Example 3-2 illustrates the changes required in db2createschema.sh to modifythe database name to CAMFWAS.Example 3-2 Modification to db2createschema.sh# Connect to DB2echo "$DBUSER> db2 connect to camfwas"echo "$DBUSER> db2 -tf $DBSCRIPT"su - $DBUSER -c "db2 connect to camfwas; db2 -tf $DBSCRIPT; db2 terminate"#Change the database table to update the user "admin"echo "$DBUSER> db2 connect to camfwas"status="$?"echo "$DBUSER> db2 update users set username = $ADMINUSER, ext_user =$ADMINUSER where username = admin and ext_user = admin"su - $DBUSER -c "db2 connect to camfwas; db2 "update users set username =$ADMINUSER, ext_user = $ADMINUSER where username = admin and ext_user =admin" ; db2 terminate"Example 3-3 illustrates a successful database connection and creation. Start theDB2 UDB command-line interface using the db2cmd command. Then connect tothe database as the schema user (for example, amuser) and list the table.Example 3-3 Checking databasedb2 => connect to camfwas user amuser using ampasswdDatabase Connection InformationDatabase server = DB2/LINUX 8.2.4SQL authorization ID = AMUSERLocal database alias = CAMFWASdb2 => list tables Chapter 3. Installing ITCAM for WebSphere 39
  • 53. Table/View Schema Type Creation time ------------------------------- --------------- ----- -------------- CONFIG AMUSER T 2007-06-07-11.25.24.284983 CTG_SR_GATEWAY AMUSER T 2007-06-07-11.25.25.150959 ... 2007-06-07-11.25.22.927237 VOLUMESTAT AMUSER T 2007-06-07-11.25.23.838550 WBI_BO_VIEW AMUSER V 2007-06-07-11.25.25.555333 WBI_REQUEST_OVERVIEW AMUSER T 2007-06-07-11.25.25.435186 WEBSERVERCHARTDATA AMUSER T 2007-06-07-11.25.24.772717 65 record(s) selected. Two further modifications are required in the managing server to reference the database: $ITCAM_HOME/bin/setenv.sh To change the database name from OCTIGATE to CAMFWAS, change the definition of JDBC_DRIVER_URL to point to CAMFWAS. For example: JDBC_DRIVER_URL=jdbc:db2://lima.itsc.austin.ibm.com:50000/camfwas Set the JDBC properties in the WebSphere Application Server where visualization engine is running: a. Log on to the WebSphere Application Server administrative console. b. Go to Resources → JDBC → Data Sources. c. Select ITCAMDataSource.40 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 54. d. Modify the database name in the DB2 Universal Database Property to reflect the name of the ITCAM Database. For example, Figure 3-3 illustrates the change to the database CAMFWAS. Figure 3-3 Configuration for ITCAMDataSource3.1.3 WebSphere Application Server considerations ITCAM for WebSphere requires the WebSphere Application Server V5 or V6 to run the visualization engine. The WebSphere Application Server runs the visualization engine Web application that acts as the ITCAM for WebSphere Web console. The WebSphere Application Server can be a standalone server or a part of a network deployment environment. Considerations for operating the Web console In case of a visualization engine error, minimize the load of the non-managing server application and ensure access to the WebSphere administration console. This is not a problem for a network deployment environment because the administration console runs independently of the application server. For a Chapter 3. Installing ITCAM for WebSphere 41
  • 55. standalone server with WebSphere Application Server V5, do not use the default server1, as it runs the administration console application. The reasons why you must not do this are: There will be some overhead from the administration console. If ITCAM for WebSphere stops functioning, you are left without an administration console and have to update the eXtensible Markup Language (XML) files manually.3.1.4 Configuring the split server of the managing server The split server configuration of the managing server consists of running two kernels on separate physical machines. Implementing the ITCAM for WebSphere managing server in this manner provides failover capabilities for reporting data collectors. Figure 3-4 shows the split-server configuration in a two-system environment. Note that this figure shows only the managing server components. Kernel 1 Kernel 2 Publish Server 1 Publish Server 3 Publish Server 2 Publish Server 4 Archive Agent 1 Archive Agent 3 Archive Agent 2 Archive Agent 4 Visualization Engine Message Dispatcher Global Publish Server Managing server - 1 Managing server - 2 Figure 3-4 Split server configuration Note: Currently, the alerts and traps setting requires the publish server to contact a local message dispatcher process. We recommend that you have a message dispatcher on server1.42 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 56. As Figure 3-4 on page 42 shows, there are two managing servers: Managing server 1 This acts as the main server. In a split server configuration for ITCAM for WebSphere, the following components can reside only on one of the managing servers, particular the primary server: – Visualization engine – Message dispatcher – Global publish server Managing server 2 This acts as a backup server and offloads the processing of the primary server.3.1.5 Installation and setup of split server Both of the managing servers have the following requirements: For a Windows-based server, install Microsoft Services for UNIX. Access to the OCTIGATE database, either locally or using the database client software. Our example sets the DB2 client similarly for both of the managing server machines. Java environment, which is typically acquired from the WebSphere Application Server installation on the machine. Even though you do not install WebSphere Application Server, you still have to adjust JAVA_HOME to a suitable Java Runtime Environment (JRE™). The WebSphere’s j2ee.jar must also be present. Copy $WAS_HOME/java and $WAS_HOME/lib/j2ee.jar from the primary managing server to the secondary server. For ease of maintenance, we recommend that both servers run the same level of software. One approach is to tar the primary managing server directory and restore it on the secondary managing server. Alternatively, you can also install the managing server software on both the machines. Chapter 3. Installing ITCAM for WebSphere 43
  • 57. Perform the following changes to the configuration files on one or both the servers: $ITCAM_HOME/bin/setenv.sh This file determines the configuration information for the managing server. a. To allow a split server, change the definition for KERNEL_HOST01 and KERNEL_HOST02 to point to each managing server, assuming that you use the same port definitions for both the servers. Each kernel requires three ports for operation. Perform this activity on both the servers, as shown in Example 3-4. Example 3-4 Modification of setenv.sh KERNEL_HOST01=srv152.itsc.austin.ibm.com PORT_KERNEL_CODEBASE01=9122 PORT_KERNEL_RFS01=9120 PORT_KERNEL_RMI01=9178 KERNEL_HOST02=srv153.itsc.austin.ibm.com PORT_KERNEL_CODEBASE02=9123 PORT_KERNEL_RFS02=9121 PORT_KERNEL_RMI02=9119 b. Edit the $ITCAM/etc/ms.properties file and set the kernel.hosts property, as shown in Example 3-5. Example 3-5 Example of ms.properties kernel.hosts=srv152.itsc.austin.ibm.com:9122:9120:9178:9178,srv15 3.itsc.austin.ibm.com:9123:9121:9119 c. Adjust the setting for JAVA_HOME accordingly. This must refer to the same Java Virtual Machine (JVM) environment that is similar to the WebSphere Application Server. To check the Java version, use the java -version command to make sure that you are using the same JVM as your WebSphere Application Server.44 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 58. $ITCAM_HOME/etc/ve.propertiesThis is the configuration for the visualization engine. Reflect the correct kernelcodebase and rfs addresses that are stored in the value of kernel.codebaseand kernel.rfs.address, as shown in Example 3-6. Perform this activity forboth the managing servers.Example 3-6 Modification of ve.propertieskernel.codebase=http://srv152.itsc.austin.ibm.com:9122/kernel.core.jar http://srv153.itsc.austin.ibm.com:9123/kernel.core.jarkernel.rfs.address=srv152.itsc.austin.ibm.com:9120 srv153.itsc.austin.ibm.com:9121For all the components that have been started in Managing server 2, edit thecorresponding control files for these components. Make sure that the entriespointing to kernel codebase and ports like RMI and RFS are pointing to thecorrect location. We edited the aactl.sh and psctl.sh files. These were thecomponents running on managing server 2, as shown in Figure 3-4 onpage 42.Example 3-7 Modification of aactl.shCODEBASE_ADDRESS02=$KERNEL_HOST02:$PORT_KERNEL_CODEBASE02KERNEL_CODEBASE=”http://$CODEBASE_ADDRESS02/kernel.core.jar”RFS_ADDRESS=”KERNEL_HOST02:$PORT_KERNEL_RFS02”RMI_CODEBASE=”http://$CODEBASE_ADDRESS02/archiveagent-intf.jarhttp://$CODEBASE_ADDRESS02/archiveagent.jar” Chapter 3. Installing ITCAM for WebSphere 45
  • 59. $ITCAM_HOME/bin/am-start.sh This is the startup script for ITCAM for WebSphere. Modify this script to indicate which components must be started on which machine. Refer to Figure 3-4 on page 42 to comment out the appropriate component. The secondary managing server must suppress the kernel 1 and kernel 1 watch dog. The primary managing server must suppress the kernel 2 and kernel 2 watch dog. Similarly, you have to add appropriate commands to start the individual components on each server. Our sample am-start.sh for managing server 1 is shown in Example 3-8. You must also shorten the wait time for the watch dog, as there is only one kernel to start in each machine. Example 3-8 Sample content of am-start.sh for managing server 1 ${CYANEA_HOME}/bin/amctl.sh wd1 start ${CYANEA_HOME}/bin/amctl.sh aa1 start ${CYANEA_HOME}/bin/amctl.sh aa2 start ${CYANEA_HOME}/bin/amctl.sh ps1 start ${CYANEA_HOME}/bin/amctl.sh ps2 start ${CYANEA_HOME}/bin/amctl.sh md start ${CYANEA_HOME}/bin/amctl.sh sam1 start $ITCAM_HOME/bin/am-stop.sh Perform the similar modification in the am-stop.sh script as well, in order to eliminate errors when stopping the components. Note: For am-start.sh and am-stop.sh, start the primary managing server first. Use an automation mechanism to coordinate this process. One such automation mechanism is the use of a remote execution shell.46 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 60. 3.1.6 Verifying the installed components in a split environment Start ITCAM for WebSphere on both the managing servers using the am-start.sh command. Once all the components are started on both the machines, verify the split server configuration using the Web console. Select Administration → Managing server → Self Diagnosis. In the component window, the running components and their Internet Protocol (IP) addresses are displayed, as shown in Figure 3-5.Figure 3-5 Self-diagnostic window that shows the managing server components Note: The polling agent has been removed from this version of ITCAM for WebSphere. You cannot add or configure the polling agent, even though an empty node for the polling agent is displayed in the application monitor component tree in the Web console. Chapter 3. Installing ITCAM for WebSphere 47
  • 61. 3.1.7 Adding additional publish servers and archive agents The basic configuration has only two publish servers and two archive agents. To further distribute the load, add publish servers or archive agents. Typically, in a split-server environment (shown in Figure 3-4 on page 42), two publish servers and two archive agents per machine are defined. Restriction: You can add only publish servers and archive agents. There are only two instances of kernels and a single instance of global publish server, message dispatcher. The instructions for adding additional publish servers and archive agents are derived from IBM Tivoli Composite Application Manager for WebSphere Installation and Customization Guide, GC32-9506. Defining new components includes the following modifications: 1. Run the following command from the $ITCAM_HOME/bin/ folder: ./add-ps.sh publish_server_port_number This adds a new publish server. The output of the command is shown in Example 3-9 Example 3-9 Output of executing add-ps.sh [root@srv153 bin]# ./add-ps.sh 9105 Current number of ps = 2 New number of ps = 3 Creating a new /opt/IBM/itcam/WebSphere/MS/etc/ps3.properties, using ps1.properties as a template testing UUID=f0039d70-0f1b-dc01-c4dc-000c293bdac6 Use UUID=f0039d70-0f1b-dc01-c4dc-000c293bdac6 Creating a new /opt/IBM/itcam/WebSphere/MS/etc/log-ps3.properties by copying log-ps1.properties Done modifying setenv.sh Done modifying am-start.sh Done modifying am-stop.sh Done modifying am-forcestop.sh 2. Run the following command from the $ITCAM_HOME/bin/ folder: ./add-aa.sh archive_agent_port_number48 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 62. This adds a new archive agent. The output of the command is shown in Example 3-10. Example 3-10 Output of executing add-aa.sh [root@srv153 bin]# ./add-as.sh 9031 Current number of AA = 2 New number of AA = 3 Creating a new /opt/IBM/itcam/WebSphere/MS/etc/aa3.properties, using aa1.properties as a template testing UUID=d078edc1-0e1b-dc01-2da7-000c293bdac6 Use UUID=d078edc1-0e1b-dc01-2da7-000c293bdac6 Creating a new /opt/IBM/itcam/WebSphere/MS/etc/log-aa3.properties by copying log-aa1.properties Done modifying setenv.sh Done modifying am-start.sh Done modifying am-stop.sh Done modifying am-forcestop.sh3. Modify the am-start.sh and am-stop.sh scripts to add the new components for both servers: a. For am-start.sh, if you have added two more archive agent aa3 and aa4, respectively, and published server ps3 and ps4, the following entries must be made (Example 3-11). Example 3-11 Addition to am-start.sh MS_home/bin/amctl.sh ps3 start MS_home/bin/amctl.sh ps4 start MS_home/bin/amctl.sh aa3 start MS_home/bin/amctl.sh aa4 start b. For am-stop.sh, use the commands shown in Example 3-12. Example 3-12 Addition to am-stop.sh MS_home/bin/amctl.sh ps3 stop MS_home/bin/amctl.sh ps4 stop MS_home/bin/amctl.sh aa3 stop MS_home/bin/amctl.sh aa4 stop Chapter 3. Installing ITCAM for WebSphere 49
  • 63. 3.2 Deploying data collectors This section lists the configuration options for installing the data collector. You modify the configuration options either in the response file or at the command line to run the silent installation. You have the option to only install the data collector, both install and configure the data collector, or only configure the data collector using this procedure. If you only want to configure the data collector, perform this the configuration after the data collector has been installed. Data collector deployment for a large installation requires a custom-automated silent installation and configuration of ITCAM for WebSphere data collectors on the target application servers. This section discusses the following topics: 3.2.1, “Setting up the silent installation process” on page 50 3.2.2, “Installing the data collector” on page 53 3.2.3, “Installing and configuring the data collector together” on page 54 3.2.4, “Configuring data collectors” on page 56 3.2.5, “Automatically discovering the installation parameters” on page 613.2.1 Setting up the silent installation process The silent installation process is supported by the install-DC command, with the following syntax: install-DC -silent -options inputfile The input file can either be modified from a supplied sample or generated using the installation wizard. In Version 6.1, the installation wizard and the configuration tool wizard both have the ability to generate the silent installation option file. Table 3-1 lists the available sample option files for the wizard. These options files are supplied in the silent sub-directory of the installation image. Table 3-1 Options files list Application server type Sample option file name WebSphere Application Server V6 DC6.opt WebSphere Application Server V5 DC.opt WebSphere Application Server DC_was.opt WebSphere Portal Server DC_portal.opt50 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 64. Application server type Sample option file name WebSphere Process Server DC_ps.opt WebSphere Enterprise Service Bus DC_esb.opt DC_esb.opt Workplace™ Collaboration Service Mail Server DC_wcsmail.opt Option for upgrading WebSphere V5 or V6 data collector migrate_DC.opt V6.0 into V6.1 The install settings in the option files are commented using leading number characters (###). If you enable an option, you must supply a value. A blank or a null value is not acceptable. The silent installation option requires advance knowledge of the target machine, including the target WebSphere environment. A prefilled response file may not be efficient in a large environment. An external mechanism is required to collect the information and populate the silent installation control files beforehand. Some of this information requires the use of naming conventions. See 3.2.5, “Automatically discovering the installation parameters” on page 61. Table 3-2 lists the parameters for installation.Table 3-2 Installation parameters Option string Description -V disableOSPrereqChecking=true | false Whether to check the operating system level before the installation. -V LICENSE_REJECT_BUTTON=true | false Must be false. -P installLocation=value Specifies the path of the installation directory for the data collector. -V LOG_DIR=value This specifies the writable directory to which the installation and configuration programs will write log files. -W LogSetting.consoleOut=true | false Specifies whether to display messages issued by the installation and configuration program on the console. -V DC_WD_JAVAHOME=value Path of the Java home directory. -V DC_CC_TEMA=true | false Whether to install Tivoli Enterprise Monitoring Agent collector interface. -V LAUNCH_CONFIG=true | false Specifies whether to launch or defer the data collector Configuration Tool after installation. -V DEFER_CONFIG=true | false Chapter 3. Installing ITCAM for WebSphere 51
  • 65. -V GC_LOG_PATH=value If Tivoli Enterprise Monitoring Agent is true, specifies the path for the garbage collector log. -W LogSetting.logLevel=level Level: FATAL, ERROR, WARNING, INFO, DEBUG_MIN, DEBUG_MID, DEBUG_MAX, ALL. Table 3-3 lists the parameters for configuration.Table 3-3 Configuration parameters Option String Description -V LOG_DIR=value Writable directory to which the installation and configuration programs write log files. -V LAUNCH_CONFIG=true | false Specifies whether to launch or defer the data collector configuration tool after installation. -V DEFER_CONFIG=true | false -V DC_CCUC_CONFIG=true | false Specifies whether to configure or unconfigure a data collector. -V DC_CCUC_UNCONFIG=true | false -V DC_RECONFIG_ALLOW=true | false Indicates whether reconfiguring the data collector is allowed. -V DC_MSKS_SERVERNAME=value Fully qualified host name of the managing server. -V DC_MSKS_CODEBASEPORT=value Remote file system port for the managing server. -V DC_MS_AMHOME=value Installation directory of the managing server. -V DC_CC_ITCAMFWAS=true | false Whether the application monitor interface support will be installed. -V DC_CAS_WAS=true | false Specifies which type of data collector to configure, WebSphere Application Server, Enterprise Service Bus, -V DC_CAS_ESB=true | false Process Server, or WebSphere Portal Server. -V DC_CAS_PS=true | false -V DC_CAS_WPS=true |false -V APP_SERVER_NAMES=value Path to the application server. -V WS_NODE_NAME=value Application server node name. -V DC_WD_JAVAHOME=value Location of the Java home directory. -V DC_WD_PROFILENAME=value Applies only to Version 6 application servers. -V DC_WD_WASBASEDIR=value Location of the application server base directory. -V DC_WD_WASVER=value Version of the application server.52 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 66. -V DC_ASL_HOSTNAME=value Administrative application host name where the data collector will be installed. -V DC_ASL_SOAPPORT=value SOAP interface port for the administrative application. -V WAS_BASEDIR=value Only for ESB, portal server, and process server, indicates path and version of the base WebSphere -V WAS_BASE_VERSION=value Application Server. -V DC_CC_TEMA=true | false Whether to run secondary collector for Tivoli Enterprise Monitoring Agent. -V GC_LOG_PATH=value If Tivoli Enterprise Monitoring Agent is true, specifies path for the Garbage Collector log. -V TEMA_OFFLINE_ALLOW=true | false Allows Tivoli Enterprise Monitoring Agent to be off line. -V FIREWALL_ENABLED=true | false Whether the data collector machine is behind a firewall. -V PROBE_RMI_PORT=value If the firewall is enabled, define RMI port for probe agent. -V PROBE_CONTROLLER_RMI_PORT=value If the firewall is enabled, define RMI port for controller. -W LogSetting.logLevel=level Level: FATAL, ERROR, WARNING, INFO, DEBUG_MIN, DEBUG_MID, DEBUG_MAX, ALL.3.2.2 Installing the data collector A sample option file for our WebSphere Application Server V6.1 is shown in Example 3-13. This installation is on the Linux-based server. Example 3-13 Sample option file for WebSphere Application Server V6.1 # Log Parameters -V LOG_DIR="/var/ibm/tivoli/common" -V disableOSPrereqChecking="true" -W LogSetting.logLevel="ALL" -W LogSetting.consoleOut="false" -P installLocation="/opt/IBM/itcam/WebSphere/DC" -V DC_WD_JAVAHOME=/opt/IBM/WebSphere/AppServer/java -V LICENSE_ACCEPT_BUTTON="true" -V LICENSE_REJECT_BUTTON="false" -V LAUNCH_CONFIG="false" -V DEFER_CONFIG="true" -V DC_CC_TEMA="true" -V GC_LOG_PATH="/var/ibm/tivoli/common/CYN/logs/gc.log" Chapter 3. Installing ITCAM for WebSphere 53
  • 67. Run the following silent installation command to silently install the data collector: ./setup_DC_<platform>.bin -silent -options <PATH>/DC6.opt The option file in Example 3-13 on page 53 does not launch the configuration tool. You can check the installation result in the /var/ibm/tivoli/common/CYN/logs/trace-install.log file. When the installation is successful, the end of trace-install.log contains the message in Example 3-14. Example 3-14 Sample trace-install.log for a successful data collector installation <LogText><![CDATA[Exit, return value = The InstallShield Wizard has successfully installed ITCAM for WebSphere Data Collector 6.1. Choose Finish to exit the wizard.]]></LogText>3.2.3 Installing and configuring the data collector together A sample option file for our WebSphere Application Server V6.1 is shown in Example 3-15. This installation is on the Linux-based network deployment server called srv107 with the managing server on srv152. Notice that the option -V LAUNCH_CONFIG="true" since we are both installing and configuring the data collector. Example 3-15 Sample option file for WebSphere Application Server V6.1 # Install Parameters -V LOG_DIR="/var/ibm/tivoli/common" -V disableOSPrereqChecking="true" -W LogSetting.logLevel="ALL" -P installLocation="/opt/IBM/itcam/WebSphere/DC" -V DC_WD_JAVAHOME=/opt/IBM/WebSphere/AppServer/java -V LICENSE_ACCEPT_BUTTON="true" -V LICENSE_REJECT_BUTTON="false" -V LAUNCH_CONFIG="true" -V DEFER_CONFIG="true" -V DC_CC_TEMA="true" -V GC_LOG_PATH="/var/ibm/tivoli/common/CYN/logs/gc.log" # Configuration Parameters -V DC_CCUC_CONFIG="true" -V DC_CCUC_UNCONFIG="false" -V DC_RECONFIG_ALLOW="true" -V DC_OFFLINE_ALLOW="false" -V DC_MSKS_SERVERNAME="srv152.itsc.austin.ibm.com" -V DC_MSKS_CODEBASEPORT="9122"54 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 68. -V DC_MS_AMHOME="/opt/IBM/itcam/WebSphere/MS"-V DC_CC_ITCAMFWAS="true"-V DC_CC_TEMA="true"-V DC_CAS_WAS="true"-V DC_CAS_ESB="false"-V DC_CAS_PS="false"-V DC_CAS_WPS="false"-V DC_CAS_WCSMAIL="false"-VAPP_SERVER_NAMES="cells/srv107Node01Cell/nodes/srv107Node02/servers/server1"-V WS_NODE_NAME="cells/srv107Node01Cell/nodes/srv107Node02"-V DC_WD_PROFILEHOME="/opt/IBM/WebSphere/AppServer/profiles/ProcSrv01"-V DC_WD_PROFILENAME="ProcSrv01"-V DC_WD_WASBASEDIR="/opt/IBM/WebSphere/AppServer"-V DC_WD_WASVER="60"-V DC_ASL_HOSTNAME="srv107.itsc.austin.ibm.com"-V DC_ASL_SOAPPORT="8880"-V DC_ASL_USERNAME="NULL"-V DC_ASL_PASSWD="NULL"-V WAS_BASEDIR="/opt/IBM/WebSphere/AppServer"### TEMA Parameters##-V DC_CC_TEMA="true"-V GC_LOG_PATH="/var/ibm/tivoli/common/CYN/logs/GC"-V TEMA_OFFLINE_ALLOW="false"### DC Host Parameters##-V AM_SOCKET_BINDIP="srv107.itsc.austin.ibm.com"-V FIREWALL_ENABLED="false"For additional flexibility and the reuse of response files, you can manually typesome configuration options on the command line, instead of including them in theresponse file. This is useful for providing: Unique options: to reuse the response file in multiple installations and configurations, but some of the options are unique to each. Password protection: to safeguard the password by manually entering it during each installation and configuration. If you record the password in the option (.opt) file, the password is unencrypted and visible to anyone who opens the file. Chapter 3. Installing ITCAM for WebSphere 55
  • 69. The following command sets administrative server passwords on the command line and relies on a response file to provide all other configuration details: ./setup_DC_lin.bin -silent -V DC_ASL_PASSWD="tivoli" -options /<PATH>/DC6.opt Note: Configuration options specified in the response file take precedence over those entered in the command line.3.2.4 Configuring data collectors As a machine may have more than one application server, we can now configure additional application servers on that machine. This is typically performed using the config_DC utility that is installed in the ITCAM_HOME/config_DC directory. You may also need to run the configuration if you defer launching the configuration tool, as in 3.2.2, “Installing the data collector” on page 53. There are two ways of configuring the data collectors without interaction: Using the config_dc -silent utility and modifying the dcInpouts.txt The drawback of this is that all the error messages require a graphical user interface (Windows or X11) to be available and responded upon. Using the Java Command Language (JACL) script configDataCollector.jacl Running config_dc Example 3-16 shows the options file that allows multiple data collector configurations to be performed in a single run. Example 3-16 Sample properties for configuring two servers in the same node # Configuration Parameters -V DC_CCUC_CONFIG="true" -V DC_CCUC_UNCONFIG="false" -V DC_RECONFIG_ALLOW="true" -V DC_OFFLINE_ALLOW="false" -V DC_MSKS_SERVERNAME="srv152.itsc.austin.ibm.com" -V DC_MSKS_CODEBASEPORT="9122" -V DC_MS_AMHOME="/opt/IBM/itcam/WebSphere/MS" -V DC_CC_ITCAMFWAS="true" -V DC_CC_TEMA="false" -V DC_CAS_WAS="true" -V DC_CAS_ESB="false" -V DC_CAS_PS="false" -V DC_CAS_WPS="false"56 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 70. -V DC_CAS_WCSMAIL="false"-VAPP_SERVER_NAMES="cells/khartoumCell01/nodes/khartoumNode01/servers/ClientSvc"-V WS_NODE_NAME="cells/khartoumCell01/nodes/khartoumNode01"-V DC_WD_PROFILEHOME="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"-V DC_WD_PROFILENAME="AppSrv01"-V DC_WD_WASBASEDIR="/opt/IBM/WebSphere/AppServer"-V DC_WD_WASVER="61"-V DC_ASL_HOSTNAME="khartoum.itsc.austin.ibm.com"-V DC_ASL_SOAPPORT="8879"-V DC_ASL_USERNAME="NULL"-V DC_ASL_PASSWD="NULL"-V WAS_BASEDIR="/opt/IBM/WebSphere/AppServer"### TEMA Parameters##-V DC_CC_TEMA="false"-V GC_LOG_PATH="NULL"-V TEMA_OFFLINE_ALLOW="false"### DC Host Parameters##-V AM_SOCKET_BINDIP="khartoum.itsc.austin.ibm.com"-V FIREWALL_ENABLED="false"### Configuration Parameters##-V DC_CCUC_CONFIG="true"-V DC_CCUC_UNCONFIG="false"-V DC_RECONFIG_ALLOW="true"-V DC_OFFLINE_ALLOW="false"-V DC_MSKS_SERVERNAME="srv152.itsc.austin.ibm.com"-V DC_MSKS_CODEBASEPORT="9122"-V DC_MS_AMHOME="/opt/IBM/itcam/WebSphere/MS"-V DC_CC_ITCAMFWAS="true"-V DC_CC_TEMA="false"-V DC_CAS_WAS="true"-V DC_CAS_ESB="false"-V DC_CAS_PS="false"-V DC_CAS_WPS="false"-V DC_CAS_WCSMAIL="false"-VAPP_SERVER_NAMES="cells/khartoumCell01/nodes/khartoumNode01/servers/ServerSvc" Chapter 3. Installing ITCAM for WebSphere 57
  • 71. -V WS_NODE_NAME="cells/khartoumCell01/nodes/khartoumNode01" -V DC_WD_PROFILEHOME="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01" -V DC_WD_PROFILENAME="AppSrv01" -V DC_WD_WASBASEDIR="/opt/IBM/WebSphere/AppServer" -V DC_WD_WASVER="NULL" -V DC_ASL_HOSTNAME="khartoum.itsc.austin.ibm.com" -V DC_ASL_SOAPPORT="8879" -V DC_ASL_USERNAME="NULL" -V DC_ASL_PASSWD="NULL" -V WAS_BASEDIR="/opt/IBM/WebSphere/AppServer" ## # TEMA Parameters ## -V DC_CC_TEMA="false" -V GC_LOG_PATH="NULL" -V TEMA_OFFLINE_ALLOW="false" ## # DC Host Parameters ## -V AM_SOCKET_BINDIP="khartoum.itsc.austin.ibm.com" -V FIREWALL_ENABLED="false" Run the following silent configuration command: /opt/IBM/itcam/WebSphere/DC/config_dc/config_dc.sh -silent -options itcam_dc_config.opt The file dcInputs.txt gets created during the install process of the data collector and is later used by the data collecter configuration wizard, along with an options file (.opt). During the configuration process the dcInputs.txt file (see Example 3-17) gets further updated and is used by the remainder of the configuration process. Example 3-17 Sample dcInputs.txt file TEMP_DIR=/tmp/ibm_am_installer_dc/ TOOLKITHOME=/opt/IBM/itcam/WebSphere/DC/toolkit JAVA_HOME=/opt/IBM/itcam/WebSphere/DC/_jvm/jre AM_HOME_CONFIG_DC=/opt/IBM/itcam/WebSphere/DC/config_dc OS=linux FILESEP=/ DCHOME=/opt/IBM/itcam/WebSphere/DC/itcamdc ITCAM61HOME=/opt/IBM/itcam/WebSphere/DC UNCONFIGURE_SUCCESSFUL=true58 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 72. DMGR_RMI=TIVOLI_COMMON_DIR=/var/ibm/tivoli/commonOS_VERSION=2.6.9-42.ELsmpCONNECTION=SOAPJDK_VERSION=WS_RMI_PORT=CONFIGURE=trueJDK_TYPE=COBRAND=ITCAMREPLACE_JAVA_SECURITY_POLICY=trueLOG_COMMON_DIR=/var/ibm/tivoli/commonOS_NAME=RedHatEnterpriseLinuxAM_HOME=/opt/IBM/itcam/WebSphere/DC/itcamdcLICENSE=true Note: The <PATH>/DC/dc_config/dcInputs.txt file must exist in order for the data collector configuration to succeed.Example 3-18 shows a successful silent configuration entry in trace-install.log.Example 3-18 Sample trace-install.log for a successful data collector configuration<LogText><![CDATA[DATA: Configure Finish: Successfullyconfigured the application servercells/khartoumCell01/nodes/khartoumNode01/servers/ClientSvccells/khartoumCell01/nodes/khartoumNode01/servers/ServerSvc.]]></LogText>Running configDataCollector.jaclUse a JACL script called configDataCollector.jacl that exists in the config_DCdirectory. The input file for this script is found in the temporary directory that iscreated from the installation.To install the data collector into a secondary application server in the nodelaredoNode02 with profile vbdpro01, we use the parameter file shown inExample 3-19.Example 3-19 Sample properties for configuring data collectorserver=server1(cells/laredoNode02Cell/nodes/laredoNode02/servers/server1|server.xml#Server_1144783086875)server.genericJvmArguments=-Xbootclasspath/p:${AM_HOME}/lib/bcm-bootstrap.jar;${AM_HOME}/lib/ppe.probe-bootstrap.jar -Xrunam Chapter 3. Installing ITCAM for WebSphere 59
  • 73. -Djlog.propertyFileDir.CYN=${AM_HOME}/etc -Djlog.qualDir=laredoNode02.server1 server.systemproperty0=com.ibm.websphere.classloader.plugin=com.cyanea. bcm.websphere.BcmPlugin server.systemproperty1=java.rmi.server.codebase=file://${MS_AM_HOME}/li b6.0/ppe.probe.jar file://${MS_AM_HOME}/lib/ppe.probe-intf.jar file://${MS_AM_HOME}/lib6.0/ppe.was_6.jar file://${MS_AM_HOME}/lib6.0/ppe.probe-bootstrap.jar server.systemproperty2=appserver.platform=was6010 server.systemproperty3=am.appserver=server1 server.systemproperty4=java.security.policy=C:ITCAMDC/etc/datacolle ctor.policy server.systemproperty5=deploymentmgr.rmi.connection= server.systemproperty6=am.profilename=vbdpro01 server.systemproperty7=am.nodename=laredoNode02 server.systemproperty8=am.home=C:ITCAMDC server.environment0=PATH=C:ITCAMDC/lib; server.environment1=QUALDIR=laredoNode02.server1 server.environment2=_JVM_THREAD_DUMP_BUFFER_SIZE=25000000 server.customservice0.displayName=am server.customservice0.classpath= server.customservice0.enable=true server.customservice0.description=Custom Service for ITCAMfWAS DC server.customservice0.classname=com.cyanea.ws6.ProbeService server.pmiservice.initialSpecLevel=connectionPoolModule=H:wsgwModule=N: orbPerfModule=H:cacheModule=N:webAppModule=H:threadPoolModule=H:wlmModu le=M:webServicesModule=N:beanModule=X:jvmRuntimeModule=H:systemModule=M :servletSessionsModule=H:transactionModule=M:j2cModule=H server.am.install.root=C:ITCAMDC server.am.ms.home=/C:/IBM/itcam/WebSphere/MS Apart from the apparent configuration values, the first line indicates the server ID. Retrieve this from the appropriate server.xml file by finding the xmi:id=”Server_ string. The command we use to activate this property is: . $WAS_HOME/bin/setupCmdLine.bat $WAS_HOME/bin/wsadmin.bat -conntype SOAP -hostname laredo -port 8881 -f $ITCAM_HOME/config_DC/configDataCollector.jacl attr.properties60 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 74. 3.2.5 Automatically discovering the installation parameters To analyze the installed WebSphere software in a machine, use the vpd.properties file. Depending on your platform, the vpd.properties file resides on the following directories: On AIX: /usr/lib/objrepos/vpd.properties On Windows: %WINDIR%vpd.properties or %HOMEPATH%WINDOWS On Linux, Solaris, HP-UX: /vpd.properties or /root/vpd.properties The entries in the vpd.properties file are delimited by a long | character field similar to that shown in Example 3-20. Example 3-20 vpd.properties file entries WSBAA60|6|0|0|0|6.0.0.0|1=WebSphere Application Server|IBM WebSphere Application Server| |IBM| |6.0.0.0|C:IBMWebSphereAppServer|0|1|WSBAA60CoreRuntime|6|0|0|0|6.0. 0.0|1|0|2|ref_33390|1|WSBAA60|6|0|0|0|6.0.0.0|1|0|false|"_uninst" "uninstall.jar" "uninstall.dat" ""|true|3|WSBAA60|6|0|0|0|6.0.0.0|1 The following list is aimed at helping you identify the information: The first identifier is the product ID. WSB* identifies the base WebSphere Application Server. The second field provides the version number. The twelfth field helps retrieve the WebSphere Application Server installation directory. Chapter 3. Installing ITCAM for WebSphere 61
  • 75. Retrieve the WebSphere Application Server instances by analyzing the directory structure of the installation. The profiles for WebSphere Application Server V6 are in the profiles directory. From the WebSphere home path, get the application server from the config subdirectory. Figure 3-6 shows a sample directory structure. Figure 3-6 Configuration directory structure In the cell level, the application server type is seen. In Figure 3-6, the file is profiles/default/config/cells/laredoNode01Cell/cell.xml. Example 3-21 shows our sample cell.xml. Example 3-21 Sample cell.xml <?xml version="1.0" encoding="UTF-8"?> <xmi:XMI xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:topology.cell="http://www.ibm.com/websphere/appserver/schemas/5.0 /topology.cell.xmi" xmlns:ipc="http://www.ibm.com/websphere/appserver/schemas/5.0/ipc.xmi"> <xmi:Documentation> <contact>WebSphere Application Server v5.0 Default Configuration Files v1.7 7/31/02</contact> </xmi:Documentation> <topology.cell:Cell xmi:id="Cell_1" name="laredoNode02Cell" cellDiscoveryProtocol="TCP" cellType="STANDALONE"> </topology.cell:Cell> </xmi:XMI>62 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 76. In Example 3-21 on page 62, the bold type indicates the cell type — aSTANDALONE cell or a DISTRIBUTED network deployment-based cell. For a standalone cell, retrieve the port information from serverindex.xml under the node name folder, for example, config/cells/laredoNode02Cell/nodes/laredoNode02/serverindex.xml. Example 3-22 shows parts of the server index. Example 3-22 Sample serverindex.xml <?xml version="1.0" encoding="UTF-8"?> <serverindex:ServerIndex xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:serverindex="http://www.ibm.com/websphere/appserver/schemas/5. 0/serverindex.xmi" xmi:id="ServerIndex_1" hostName="laredo.itsc.austin.ibm.com"> . . . <specialEndpoints xmi:id="NamedEndPoint_1144783085594" endPointName="SOAP_CONNECTOR_ADDRESS"> <endPoint xmi:id="EndPoint_1144783085594" host="laredo.itsc.austin.ibm.com" port="8881"/> </specialEndpoints> . . . </serverindex:ServerIndex> For a distributed cell, first select the core group information to retrieve the node name of the deployment manager. The core group information is stored in a sub-directory under the cell called coregroupsDefaultCoreGroupcoregroup.xml. Get the association between the node and the dmgr server, as shown in Example 3-23. Example 3-23 Excerpt from coregroup.xml . . . <coreGroupServers xmi:id="CoreGroupServer_1130943100094" nodeName="laredoCellManager01" serverName="dmgr"/> . . .The serverindex.xml is found under the appropriate node, for example,cells/laredoCell01/nodes/laredoCellManager01. TheSOAP_CONNECTOR_ADDRESS is similar to that shown in Example 3-22. Chapter 3. Installing ITCAM for WebSphere 63
  • 77. Based on the above consideration, a shell script can be written to retrieve all the available application servers on a machine and build the response files individually to perform the installation. Another option is to use an automated software distribution solution such as IBM Tivoli Configuration Manager or IBM Tivoli Provisioning Manager.3.3 Configuring and setting up SSL communication ITCAM for WebSphere provides the facility to communicate with data collectors securely over TCP/IP networks using Secure Socket Layer (SSL). Default certificates and keystores are set up during initial installation. SSL is not enabled by default. Therefore, manual configuration is necessary in order to have the managing server and data collectors communicate over a secure connection. This section discusses the following topics: 3.3.1, “Managing server Secure Socket Layer setup” on page 64 3.3.2, “Data collector Secure Socket Layer setup” on page 65 3.3.3, “Working with custom certificates” on page 673.3.1 Managing server Secure Socket Layer setup Enabling SSL and using the default certificate information to verify SSL communication works must be your first step. This is to rule out any possible errors while attempting to generate and use your own certificates and keystores. 1. Go to the ITCAM _HOME/etc directory. The kl1.properties and kl2.properties kernel configuration files contain the following parameters that must be activated for Secure Sockets Layer: security.enabled=true codebase.security.enabled=true 2. Set the system properties on the WebSphere Application Server where the visualization engine is running. Add the system properties using the WebSphere administration console. Where you set the properties depends on the version of the WebSphere Application Server. – certificate.path: MS_home/etc/mgmttomgmt.cer – keystore.location: MS_home/etc/CyaneaMgmtStore – keystore.storepass: cyanea94612 – keystore.keypass: cyanea94612 – nodeauth.userid: cyaneamgmt64 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 78. Figure 3-7 shows these settings in the administration console. Figure 3-7 New SSL-related settings 3. Restart the kernel components and the visualization engine application server to activate the new settings.3.3.2 Data collector Secure Socket Layer setup The SSL setup is stored in the datacollector.properties file. This file is located in the etc subdirectory of the data collector installation directory. Activate the definition shown in Example 3-24. Example 3-24 SSL settings for datacollector.properties file security.enabled=true certificate.path=/opt/IBM/itcam/WebSphere/DC/etc/dctomgmt.cer;/opt/IBM/ itcam/WebSphere/DC/etc/dctoproxy.cer keystore.location=/opt/IBM/itcam/WebSphere/DC/etc/CyaneaDCStore keystore.storepass=oakland94612 keystore.keypass=oakland94612 nodeauth.userid=cyaneadc comm.use.ssl.dc=true Chapter 3. Installing ITCAM for WebSphere 65
  • 79. Remove the system-generated datacollector.properties file and restart the data collector. The system-generated property file name is similar to <nodename>.<servername>.datacollector.properties. On restarting the data collector, the message shown in Example 3-25 is generated in msg-dc.log, which is located in the Tivoli common log directory. Once you receive the message, communication between the managing server and the data collector is encrypted. Example 3-25 SSL active information <Message Id="CYND5237I" Severity="INFO"> <Time Millis="1143583253402"> 2006-03-28 17:00:53.402-05:00</Time> <Server Format="IP">wsamaix1.tivlab.raleigh.ibm.com</Server> <ProductId>CYN</ProductId> <Component>CYN.msg.datacollector</Component> <ProductInstance>6</ProductInstance> <LogText><![CDATA[CYND5237I Using SSL server socket for component: CommandAgent. All clients connecting to this component MUST connect via SSL, else the client socket may hang till the timeout expires. No user action is required.]]></LogText> <Source FileName="com.cyanea.command.CommandAgent" Method="start"/> <TranslationInfo Type="JAVA" Catalog="com.ibm.tivoli.probe.resource.LogMessages" MsgKey="CYND5237I"></TranslationInfo> <Principal>wsamaix1.tivlab.raleigh.ibm.com/9.42.153.102</Principal> </Message> Note: Only the CommandAgent port uses SSL. Other communications opened by the data collector, such as the ProbeController port and the data collector - Publish Server port, do not use SSL. Therefore, when SSL is enabled, only the data in the channels connected to the CommandAgent port is encrypted.66 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 80. All the data processed in the CommandAgent channel is encrypted when SSL is enabled. Table 3-4 shows the data classification.Table 3-4 Data content classification Classification Data Command and control data Configuring and unconfiguring data collector User actions related to threads Starting and stopping JVM threads Changing thread priorities Getting thread priorities and thread status Requesting drill information to see cookies, and so on Generating thread dumps Getting thread stack traces System information Application server information Operating system platform information JVM information Application information All the applications installed on the monitored application server Application binaries and location information Thread pool information related to Java Message Service (JMS), Java 2 Enterprise Edition (J2EE) Connector Architecture (JCA), Java Transaction API (JTA), Servlet, Enterprise JavaBeans (EJB), and so on Data source information Performance data All performance monitor interface (PMI) data Transport data Object Request Broker (ORB) data SOAP ports Memory information Obtaining JVM Heap Snapshot™ data Performing memory leak analysis Performing heap dump3.3.3 Working with custom certificates This section discusses one approach to using custom certificates. There are different ways of configuring the certificates. The approach that we use categorizes the machines into three types: the managing server, the port consolidator, and the data collector. Each type can communicate with the other type and understand one another. Chapter 3. Installing ITCAM for WebSphere 67
  • 81. Three keystores must be generated for custom certificates. The three keystores for a distributed environment implementation are: MSStore This contains the following managing server certificates: – mgmttomgmt.cer (cn=cyaneamgmt) – dctomgmt.cer (cn=cyaneadc) – proxytomgmt.cer (cn=cyaneaproxy) DCStore This contains the following data collector certificates: – proxytodc.cer (cn=cyaneaproxy) – mgmttodc.cer (cyaneamgmt) ProxyStore This contains the following port consolidator certificates: – mgmttoproxy.cer (cn=cyaneamgmt) – dctoproxy.cer (cn=cyaneadc) The keystore files are generated using the keytool command, which comes with Java Runtime Environment (JRE). In order to run the keytool command, you must be in the JAVA_HOME/bin directory or have it specified in the PATH. Perform a quick check by entering the keytool command. A help list showing the syntax must be visible on the console. In order to better manage the keystores, we create a directory called keyfiles in ITCAM_HOME/etc and run the keytool command from this directory.68 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 82. Creating MSStoreThe first keystore that must be created is for the managing server. The name ofthe keystore can be customized to fit your environment. In the examplesprovided here, NEWMSStore is used as the keystore name. Because threealiases must be added to this keystore, the command must be run three times,once for each alias. For communication between the managing servers, create the alias mgmttomgmt for the keystore NEWMSStore with the command shown in Example 3-26. Example 3-26 Command to create the alias mgmttomgmt keytool -genkey -alias mgmttomgmt -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 -dname "cn=cyaneamgmt, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" This command creates the NEWMSStore file and the mgmttomgmt key in it. The password, keypass, and storepass can be anything you want them to be. For communication from the managing server to the data collector, create the alias dctomgmt using the command shown in Example 3-27. Example 3-27 Command to create the alias dctomgmt keytool -genkey -alias dctomgmt -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 -dname "cn=cyaneamgmt, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" This command adds the dctomgmt alias into the NEWMSStore file. For communication from the managing server to the port consolidator, create the alias proxytomgmt to the keystore using the command shown in Example 3-28. Example 3-28 Command to create the alias proxytomgmt keytool -genkey -alias proxytomgmt -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 -dname "cn=cyaneamgmt, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" Chapter 3. Installing ITCAM for WebSphere 69
  • 83. Creating DCStore The second set of keystores that must be created is for the data collector. As with the managing server keystore, customize the data collector keystore name to your environment. In this example, we use NEWDCStore. Create the alias proxytodc for keystore NEWDCStore using the command shown in Example 3-29. Example 3-29 Command to create the alias proxytodc keytool -genkey -alias proxytodc -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 -dname "cn=cyaneaproxy, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" Create the alias mgmttodc for the NEWDCStore file using the command shown in Example 3-30. Example 3-30 Command to create the alias mgmttodc keytool -genkey -alias mgmttodc -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 -dname "cn=cyaneaproxy, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" Creating ProxyStore ProxyStore is the last keystore that must be created before extracting the certificates. This is used by the port consolidator in a distributed environment. We use the file NEWProxyStore for this purpose. Create the alias mgmttoproxy for the keystore NEWProxyStore using the command shown in Example 3-31. Example 3-31 Command to create the alias mgmttoproxy keytool -genkey -alias mgmttoproxy -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2 -dname "cn=cyaneaproxy, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US"70 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 84. Create the alias dctoproxy using the command shown in Example 3-32. Example 3-32 Command to create the alias dctoproxy keytool -genkey -alias dctoproxy -keyalg RSA -keysize 1024 -sigalg MD5withRSA -validity 2000 -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2 -dname "cn=cyaneaproxy, OU=Tivoli, O=IBM, L=Raleigh, ST=NC, C=US" Extracting certificates from the keystores Once all the three keystores are created, extract the certificates from each keystore and give them to their communication partners. These certificates are used to validate the SSL connection between the components. Figure 3-8 shows the keystores and certificate exchanges. Managing Port Data Server Consolidator Collector MSStore ProxyStore DCStore mgmttomgmt mgmttoproxy mgmttodc proxytomgmt dctoproxy proxytodc key dctomgmt mgmttomgmt mgmttoproxy proxytodc dctoproxy certificates mgmttodc proxytomgmt dctomgmtFigure 3-8 Keystores and certificate exchanges Chapter 3. Installing ITCAM for WebSphere 71
  • 85. Do the following: Extract all the certificates from the managing server keystore NEWMSStore. The export commands are shown in Example 3-33. Example 3-33 Commands to extract certificates from the managing server keystore keytool -export -alias mgmttomgmt -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 -file mgmttomgmt.cer keytool -export -alias dctomgmt -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh 2 -file dctomgmt.cer keytool -export -alias proxytomgmt -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 -file proxytomgmt.cer Extract all the certificates from NEWDCStore by using the commands shown in Example 3-34. Example 3-34 Commands to extract all the certificates from NEWDCStore keytool -export -alias proxytodc -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 -file proxytodc.cer keytool -export -alias mgmttodc -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 -file mgmttodc.cer Extract all the certificates from NEWProxyStore by using the commands shown in Example 3-35. Example 3-35 Commands to extract all the certificates from NEWProxyStore keytool -export -alias mgmttoproxy -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2 -file mgmttoproxy.cer keytool -export -alias dctoproxy -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2 -file dctoproxy.cer After completing the extraction of all the certificates, the directory must have seven .cer files. Import these certificates to their appropriate partners using these commands: For NEWMSStore, use the commands shown in Example 3-36. Example 3-36 Commands to import certificates: NEWMSStore keytool -import -alias mgmttomgmt -file mgmttomgmt.cer -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 keytool -import -alias mgmttoproxy -file mgmttoproxy.cer -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh2 keytool -import -alias mgmttodc -file mgmttodc.cer -keypass raleigh1 -keystore ./NEWMSStore -storepass raleigh272 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 86. For NEWDCStore, use the commands shown in Example 3-37. Example 3-37 Commands to import certificates: NEWDCStore keytool -import -alias dctomgmt -file dctomgmt.cer -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 keytool -import -alias dctoproxy -file dctoproxy.cer -keypass raleigh1 -keystore ./NEWDCStore -storepass raleigh2 For NEWProxyStore, use the commands shown in Example 3-38. Example 3-38 Commands to import certificates: NEWProxyStore keytool -import -alias proxytomgmt -file proxytomgmt.cer -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2 keytool -import -alias proxytodc -file proxytodc.cer -keypass raleigh1 -keystore ./NEWProxyStore -storepass raleigh2Deploy these newly generated keystores to the appropriate machines. StoreNEWMSStore on the managing servers, NEWProxyStore on the portconsolidator machines, and NEWDCStore on the data collector machines. Copythe certificate files to the respective machines, too.Update the appropriate configuration entries specified in 3.3.1, “Managing serverSecure Socket Layer setup” on page 64, and 3.3.2, “Data collector SecureSocket Layer setup” on page 65. Restart all the components to activate the SSLwith the custom certificates. Chapter 3. Installing ITCAM for WebSphere 73
  • 87. 74 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 88. 4 Chapter 4. Maintenance of ITCAM for WebSphere This chapter discusses the issues relating to maintenance of ITCAM for WebSphere in a large-scale environment. This chapter discusses the following topics: 4.2, “Performance and availability of the managing server” on page 76 4.2, “Performance and availability of the managing server” on page 76 4.3, “Backup and recovery configuration” on page 83 4.4, “Log files and configuration files” on page 85 4.5, “Performing product maintenance” on page 87© Copyright IBM Corp. 2006, 2007. All rights reserved. 75
  • 89. 4.1 Operating ITCAM for WebSphere This chapter discusses some operational issues regarding a large ITCAM for WebSphere environment. The focus is on the issues that arise largely because of environment size. Following is a list of these issues along with details about the sections that describe them: Section 4.2, “Performance and availability of the managing server” on page 76, shows that as the size of the environment grows, even a slight inefficiency can cause a major problem. A large-scale environment requires the performance and availability of the managing server to be soundly maintained. Section 4.3, “Backup and recovery configuration” on page 83, shows that backup and recovery of configuration is a necessity. The ability to quickly identify and recover system configuration is critical. The large size of the environment prohibits ad hoc backup and recovery. Section 4.4, “Log files and configuration files” on page 85, shows that log and configuration files must be maintained. There are two aspects of file maintenance: keeping the size of the log files under control and ensuring that the correct property files are used. Section 4.5, “Performing product maintenance” on page 87, discusses implementation of WebSphere, Database 2 (DB2), and ITCAM for WebSphere patches. Typically, with the number of data collectors involved, this can be a huge task.4.2 Performance and availability of the managing server Performance and availability of the ITCAM for WebSphere managing server are mainly related to the IBM WebSphere Application Server that hosts the visualization engine and the OCTIGATE database. This section discusses the following topics: 4.2.1, “Performance of the WebSphere Application Server” on page 76 4.2.2, “Database maintenance” on page 77 4.2.3, “Data trimming” on page 784.2.1 Performance of the WebSphere Application Server There are several critical tuning factors for the visualization engine. The sole use of the visualization engine is to provide users with information regarding the enterprise activity, not collecting and analyzing the activity. This eases the role of the visualization engine to not being completely critical.76 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 90. Tuning of the visualization engine is mainly to ensure that the WebSphere Application Server has adequate memory. This also relates to the fact that the visualization engine has to run on one of the managing servers that runs other Java-based kernel components. Each logged-in user and the activity of each user requires additional storage to be acquired by the visualization engine. The Java garbage collection process also plays an integral part in providing adequate memory resource.4.2.2 Database maintenance Database performance is largely related to how the data is stored physically and how well the database manager understands the nature of the data. This section primarily discusses the use of DB2 as the database manager. The primary tools to ensure this are the REORG and RUNSTAT tools. You can run both of these utilities from the DB2 utilities from the DB2 command line. The REORGCHK utility reads the table information and analyzes the necessity of running reorg. REORGCHK uses the current statistics or generates its own statistics. The REORG utility reorganizes the database structure to ensure that data is allocated coherently and in a typical access sequence requested by most queries. The syntax for running REORG is: DB2 REORG TABLE creator.table_name INDEX primary_index_name The RUNSTATS utility collects the statistics of the tables and indexes to ensure that the database manager chooses the most optimized method to get the data requested by an Structured Query Language (SQL) request. The syntax for running RUNSTATS is: DB2 RUNSTATS ON TABLE creator.table_name WITH <runstat_options> Typically, you must run REORGCHK for frequently modified tables and then determine whether you want to run REORG on those tables. After all the modifications have been performed, run RUNSTATS to ensure that the statistics are current and that DB2 obtains the most optimal path to the data an Structured Query Language (SQL) requested. Chapter 4. Maintenance of ITCAM for WebSphere 77
  • 91. ITCAM for WebSphere supplies a script to run the RUNSTATS utility. This script is called run-stat-cmds and resides in the bin directory. This command selects specific tables that are necessary to perform RUNSTATS on. The syntax of this command depends on the platform you execute. It can be either one of the following: run-stat-cmds.bat OCTIGATE amuser export JDBC_USER=amuser export JDBC_PASSWORD=ampasswd run-stat-cmds.sh OCTIGATE The table that ITCAM for WebSphere constantly modifies is the REQUEST table. This table must be checked frequently. The run-stat-cmds automatically checks this table and stores its output in the logs/reorg.log file. If the cluster ratio of the REQUEST table is lower than 90, run REORG on this table and rerun RUNSTATS on the REQUEST table. This REORG command is as follows: db2 "REORG TABLE AMUSER.REQUEST INDEX R_TIME " We recommend a regular, maybe monthly, maintenance window to REORG all the tables for ITCAM for WebSphere and perform a complete RUNSTATS.4.2.3 Data trimming Another aspect of database performance is processing the data size. Because monitoring information is accumulated, access to current information may be degraded, as the database has to process more information than it must. Old data must be trimmed to allow more efficient processing. This process is application-specific, as the database manager does not know the relationship between the data. This also minimizes the disk space usage for ITCAM for WebSphere. There are two methods of pruning the tables. Run both of them, as they address different sets of tables. The pruning methods are: Automatic pruning by the archive agent This prunes data every two hours and keeps data for 48 hours. See “Archive agent trimming” on page 79. Manual pruning using the datatrim.sh command We recommend that you run this regularly. A daily run to minimize the amount of data to be pruned may be appropriate. See “Running datatrim.sh” on page 80.78 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 92. Note: You may see a datatrimmer.sh script in the <MS_HOME>/bin directory. This script is not recommended and is not discussed in this paper.Archive agent trimmingOnce enabled, the managing server archive agent automatically prunes thefollowing tables: MEMORY_DATA GC_DATA MQI_QUEUEMGR_SR_OVERVIEW MQI_QUEUE_SR_OVERVIEW CTG_SR_OVERVIEW WEBSERVERCHARTDATA IMSTHREADS VOLUMESTAT PORTALSTATSWhen data trimming is enabled, pruning occurs every two hours, keeping 48hours worth of data in each table. These tables, and the criteria for trimming thetables, are hard-coded in ITCAM for WebSphere, and cannot be changed.Automatic data trimming using the archive agent is enabled by setting theENABLE_DATATRIMMER property to true in the etc/aa1.properties file or theetc/aa2.properties file, or both. If both the properties files have data trimmingenabled, there is a built-in delay for the second archive agent in order to preventthe occurrence of database deadlocks. Note: If you are using more than two archive agents, do not enable the data trimmer for the other archive agents. Only aa1 and aa2 must have the ENABLE_DATATRIMMER=true property.Stop and start the archive agents to pick up the changes made to theENABLE_DATATRIMMER property using the following commands:<MS_HOME>/bin/aactl.sh stop aa1<MS_HOME>/bin/aactl.sh start aa2 Chapter 4. Maintenance of ITCAM for WebSphere 79
  • 93. Running datatrim.sh The datatrim.sh script is a command-line utility that can be scheduled or run manually to trim data that is no longer required from the database. As it is shipped with ITCAM for WebSphere, this script prunes data from the following tables: REQUEST METHOD SERVERSTATS PMISTATS VOLUMESTAT APAR PK17797 allows additional tables to be trimmed, including the following: MEMORY_DATA IMSEVENTS The datatrim.sh script carries out two major processes, marking and trimming: Mark deleting process. This process updates the rows for the REQUEST and METHOD tables that will be deleted in the data trim process. This method is controlled by the etc/deleterelatedtables.xml control file shown in Example 4-1. Example 4-1 Deleterelatedtables.xml control file <?xml version="1.0" encoding="UTF-8"?> <tableinfo> <!-- elements work for all tables --> <commitcount>2000</commitcount> <useoracle>false</useoracle> <table> <name>request</name> <daystokeep>7</daystokeep> <!-- timestamp column name --> <columnname>end_time</columnname> <!-- The format of startdate and enddate is mm/dd/yy hh:mm:ss. --> <startdate></startdate> <enddate></enddate> <!-- If a user would like to trim data based on appserver, then use adminserver and appserver elements --> <adminserver></adminserver> <appserver></appserver> <idcolumn>request_id</idcolumn> <delayinterval>15</delayinterval> <relatedtables> <name>method</name>80 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 94. <columnname>start_time</columnname> </relatedtables> </table></tableinfo> Data trim process This process deletes the data marked in the REQUEST and METHOD tables as a result of the marking process. This process is controlled by the etc/deletesingletable.xml control file shown in Example 4-2.Example 4-2 Deletesingletable.xml control file<?xml version="1.0" encoding="UTF-8"?><tableinfo> <commitcount>2000</commitcount> <useoracle>true</useoracle> <table> <name>serverstats</name> <daystokeep>7</daystokeep> <!-- timestamp column name --> <columnname>current_time</columnname> <!-- The format of startdate and enddate is mm/dd/yy hh:mm:ss. --> <startdate></startdate> <enddate></enddate> <!-- If a user would like to trim data based on appserver, then useadminserver and appserver elements --> <adminserver></adminserver> <appserver></appserver> <serverid>probe_id</serverid><delayinterval>5</delayinterval> </table> <table> <name>pmistats</name> <daystokeep>7</daystokeep> <!-- timestamp column name --> <columnname>current_time</columnname> <!-- The format of startdate and enddate is mm/dd/yy hh:mm:ss. --> <startdate></startdate> <enddate></enddate> <!-- If a user would like to trim data based on appserver, then useadminserver and appserver elements --> Chapter 4. Maintenance of ITCAM for WebSphere 81
  • 95. <adminserver></adminserver> <appserver></appserver> <serverid>probe_id</serverid> <delayinterval>5</delayinterval> </table> </tableinfo> For both the eXtensible Markup Language (XML) files in Example 4-1 on page 80 and Example 4-2 on page 81, the following apply: The parameter commitcount controls the number of records to be committed to a database in a transaction. Change the commitcount setting from a default value of 2000 to something higher if you have large numbers of records to be deleted. This speeds up the trimming process. The commitcount number applies to all the tables specified in the XML files. The useoracle setting is no longer used. Instead, the datatrim.sh script uses the Java Database Connectivity (JDBC) driver to determine the type of database (DB2 or Oracle) being used. For each individual table, set either daystokeep or the startdate and enddate pair settings. If daystokeep is set, it overrides the startdate and enddate values. To invoke datatrim.sh, log in using the ITCAM for WebSphere owner, typically called amuser. Run the utility from the managing server. It will write its output in the log/datatrim.log file. The syntax of the command is: $MS_HOME/bin/datatrim.sh octigate amuser ampasswd The result and status of the datatrim.sh command can be analyzed in the logs/datatrim.log file. The datatrim.sh command prunes the tables, but does not optimize or reclaim free space. Perform these actions by using the REORG utility in DB2 or by exporting and reimporting the tables in Oracle. We recommend that you change the daystokeep value to the number of days of data that you want to keep, and then run the datatrim.sh script every night. Running the script every night facilitates the removal of fewer records from the database more often. Running the command less frequently can result in the script running for extended periods of time.82 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 96. While it is not necessary to stop the archive agents (<MS_HOME>/bin/aactl.sh stop) prior to running the datatrim.sh script, stop the archive agents to avoid database deadlocks. Note: Stopping the archive agents leads to loss of performance data.4.3 Backup and recovery configuration The backup and recovery procedure covers both WebSphere backup and database backup. The aim of a backup policy is to quickly restore the operation. Another objective is to minimize the backup’s impact on the operation. This section discusses the following topics: 4.3.1, “ITCAM for WebSphere backup” on page 83 4.3.2, “WebSphere configuration backup” on page 83 4.3.3, “Database backup and restore” on page 844.3.1 ITCAM for WebSphere backup Back up the installation directory structure of ITCAM for WebSphere using any backup utility. The restore of the file system of ITCAM for WebSphere must be owned by the ITCAM for WebSphere user ID, which is typically amuser.4.3.2 WebSphere configuration backup WebSphere configuration backup must be considered for both the managing server and the data collector. Depending on whether you want to reinstall or restore the backup, initiate a backup. As WebSphere configuration is not a dynamic entity, taking a backup of the initial implementation and a backup procedure to be implemented before and after a configuration change must be sufficient. To dump the WebSphere Application Server configuration, use the backupConfig command. The syntax for the backupConfig command is: backupConfig <file> -nostop -logfile <filename> -user <name> -password <pswd> To restore the WebSphere configuration, use the restoreConfig command. The syntax for restoreConfig command is: restoreConfig <file> -nostop -logfile <filename> -user <name> -password <pswd> Chapter 4. Maintenance of ITCAM for WebSphere 83
  • 97. 4.3.3 Database backup and restore For backing up the DB2 UDB, use the DB2 UDB backup utility that is launched by using the DB2 command-line processor. The syntax for the DB2 backup utility is: db2 backup database <dbname> user <username> using <password> to <file> Note: A directory path for the target file must exist. Manually create the path before executing the backup command. If not, you will get an exception similar to: SQL2036N. The path for the file or device "/tmp/octigate" is not valid. A successful backup returns a message that is similar to: Backup successful. The timestamp for this backup image is : 20041116104855 You must either keep the timestamp or rename the file according to the timestamp. The timestamp is required to restore the database. To restore the database, execute the following command: db2 restore database <dbname> user <user> using <password> from <file> taken at <timestamp> <option> An example of running the restore with an option to prohibit rolling forward is shown in Example 4-3. Rolling forward is a feature that allows the post-backup changes to the database to be applied from the transaction redo log. A typical distributed DB2 installation does not activate this feature. Example 4-3 Restoring DB2 database > db2 restore database octigate user itcamusr using passwd from /tmp/octigate taken at 20041116104855 without rolling forward SQL2539W Warning! Restoring to an existing database that is the same as the backup image database. The database files will be deleted. Do you want to continue ? (y/n) y DB20000I The RESTORE DATABASE command completed successfully.84 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 98. 4.4 Log files and configuration files Perform the file maintenance process regularly to reduce the amount of disk space used and to manage monitoring configuration. Log files maintenance applies to both managing servers and data collectors, with configuration file maintenance applying more to data collectors. This section discusses the following topics: 4.4.1, “Managing log files” on page 85 4.4.2, “Managing the configuration files” on page 864.4.1 Managing log files Log files in ITCAM for WebSphere are used for problem determination of the product’s working mechanism. There are several locations for log files, depending on the components that generate them. These locations are: Tivoli common directory This is where most of the log files are stored. Some typical directory names are: – %ProgramFiles%ibmtivolicommon (Windows) – /var/ibm/tivoli/common (UNIX) The directory under the common directory is typically identified by a three-character product code such as CYN for ITCAM for WebSphere. In the product directory, there are sub-directories: – First-failure data capture (FFDC) for first-failure data capture application The FFDC directory contains diagnostic information pertaining to the program exceptions that are captured. – Logs subdirectory of Tivoli common directory The logs subdirectory is typically filled with log files and must be maintained. The logs subdirectory of the installation path These are the logs that are typically initiated either before the product is active, such as standard output and standard error redirection, or from a leftover log file from a previous release. Some utility commands such as datatrim.sh also use this directory to store their output. Because the log files in the common directory have a maximum size and predefined number of generations, they cannot exceed a certain size. This path must be maintained and checked for ad hoc logging that may use up the space. Chapter 4. Maintenance of ITCAM for WebSphere 85
  • 99. We recommend that you use a separate file system to contain the common directory. The contents of the logs subdirectory of the installation path must be regularly cleaned up manually to prune the content.4.4.2 Managing the configuration files Figure 4-1 shows the configuration files for data collectors inter-relation. In Version 6.1, there is a significant change in the structure of the configuration files. This is due to the changes of the byte code instrumentation mechanism. $DC_home custom_requests.xml lock_analysis.xml memory_leak_diagnosis.xml method_entry_exit.xml toolkit_config_ctg.xml itcamdc/etc toolkit_config_ims.xml toolkit_config_jdo.xml toolkit_config_kwj.xml runtime toolkit_config_mqi.xml toolkit_config_ra.xml <wasver>.<wasnode>,<server> custom/toolkit_custom.properties cyn-cclog.properties cynlogging.properties jiti.properties *.datacollector.properties *.datacollector.policy *.kwjdc.properties *.toolkit.properties *.toolkit.xml Figure 4-1 Configuration file structure From time to time, you may have to modify some XML configuration files to enable certain product features. There is currently no mechanism in the product to make this switch. Employ an external automation mechanism to switch these configurations back and forth. Configuration change is typically performed by changing the toolkit_custom.properties and several related xml files under the itcamdc/etc directory. This is to add or remove the classes to be used for testing with memory allocation probe or locking probe.86 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 100. 4.5 Performing product maintenance Various considerations must be taken into account when dealing with the application of product maintenance, fixes, and fix packs. In a large environment this must be performed remotely and automated with minimal impact to the production system. This section discusses the following topics: 4.5.1, “Getting software updates” on page 87 4.5.2, “Updating ITCAM for WebSphere managing server” on page 87 4.5.3, “Updating ITCAM for WebSphere data collectors” on page 904.5.1 Getting software updates Software updates are available at the following IBM Web sites: For ITCAM for WebSphere: http://www-306.ibm.com/software/sysmgmt/products/support/IBMTivoli CompositeApplicationManagerforWebSphere.html For DB2 Universal Database: http://www-1.ibm.com/support/docview.wss?rs=71&uid=swg27007053 For WebSphere Application Server: http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg27006899 Typically, you do not have to get updates for DB2 or WebSphere on the managing server. ITCAM for WebSphere only supports a certain level of these products. However, you may sometimes have to fix an impending problem with a certain software level.4.5.2 Updating ITCAM for WebSphere managing server The ITCAM for WebSphere managing server updates must be performed as a scheduled maintenance. This means that: You must have a backup of the management server system. You must perform an orderly shutdown of the management server. Users must be notified of the system’s unavailability. The managing server updates for ITCAM for WebSphere software are typically performed using a silent installer. The ITCAM for WebSphere fix packs consist of two sets of files that are distributed as tar files: 6.1.0-TIV-ITCAMfWAS_SVR-FP000n.tar ITCAMfWAS_V6_UpdateInstaller.tar Chapter 4. Maintenance of ITCAM for WebSphere 87
  • 101. Fix packs and interim fixes are distributed as tar files, which are installed using the silent command-line update installer. Multiple fixes may be applied by combing all *.update files found in the <fix-dir>/update directory to a common update directory. The procedure to apply the updates is derived from the readme for the update installer V6.1.0.pdf. 1. Modify the silentUpdate.properties file to reflect your environment. Example 4-4 illustrates the changes required for our managing server, srv152. Example 4-4 The silentUpdate.properties # product location product.location=/opt/IBM/itcam/WebSphere/MS # updates location updates.location=./updates ####################################################################### # Managing Server properties # These properties apply only for the MS update installation, ####################################################################### # change to false if database should not be updated updateDb=true # change to false if VisualEngine should not be updated updateVe=true # TODO: uncomment this property and specify correct WAS_HOME updateVe.wasHome=/opt/IBM/WebSphere/AppServer/profiles/AppSrv01 updateVe.was.soap.host=srv152.itsc.austin.ibm.com updateVe.was.soap.port=8880 updateVe.was.node=srv152Node01 updateVe.was.server=server1 updateVe.was.user=root updateVe.was.password=rpasswd #ms.oracle.driver.jar=${ORACLE_JDBC_DRIVER_PATH}/ojdbc14.jar #ms.oracle.driver.path= # Set this property to reconfigure MS processes to bind to the specified address. #ms.bind.host.name=172.17.11.82 ####################################################################### # Uninstall properties ####################################################################### # the updates to be uninstalled uninstall.updates=last88 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 102. 2. Ensure that the following environment is set prior to running the silent updater: export JAVA_HOME=/opt/IBM/WebSphere/AppServer/java export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=JAVA_HOME:$CLASSPATH3. Prepare the installation by executing: ./silentUpdate.sh -prepareInstall4. Do not proceed with the installation. Clean the results of the previous command by executing: ./silentUpdate.sh -cleanPrepared5. Install by executing: ./silentUpdate.sh -install6. If the install failed, rollback is achieved by: ./silentUpdate.sh -rollback7. Display updates that have been installed by executing: ./silentUpdate.sh -displayInstalledUpdatesExample 4-5 shows an example of the output after running the command.Example 4-5 Output from silentUpdate.sh -displayInstalledUpdates# ./silentUpdate.sh -displayInstalledUpdatesUpdate installer version 6.1.0, build 20070513145421Logging details into/opt/IBM/itcam/WebSphere/MS/logs/update/update_20070616101145.logAction: display installed updatesUpdates installed 5/31/07 2:51 PM 6.1.0.1.1 Interim Fix 1 after Fix Pack 1Updates installed 5/31/07 2:09 PM 6.1.0.1.5 Interim Fix 5 after Fix Pack 1Updates installed 5/31/07 2:08 PM 6.1.0.1.4 Interim Fix 4 after Fix Pack 1Updates installed 5/31/07 12:20 PM 6.1.0.1.3 Interim Fix 3Updates installed 5/31/07 11:56 AM 6.1.0.1 Fix Pack 1Finished successfully Chapter 4. Maintenance of ITCAM for WebSphere 89
  • 103. 4.5.3 Updating ITCAM for WebSphere data collectors Data collectors are typically run in a production environment. Updating these machines requires strict change control. Because data collectors run as an integral part of the WebSphere Application Server, updating the data collectors typically involves restarting the application server. When dealing with a clustered environment, alternately restarting the members of a cluster does not affect application availability, while a standalone application server means that the application is unavailable. The primary prerequisite of this fix pack is generally to have the managing server updated before installing the fix pack to the data collectors. Installation of data collectors can be performed in stages. All the data collectors do not have to be updated together. As with the managing server updates, the data collector updates, too, come as two tar files: 6.1.0-TIV-ITCAMfWAS_MP-FP00nn.tar. ITCAMfWAS_V6_UpdateInstaller.tar The following discussion is derived from our experience in installing fix pack 1 of ITCAM for WebSphere V6.1. This fix pack is distributed as 6.1.0-TIV-ITCAMfWAS_MP-FP0001. Note: Use the following steps following the DataCollector Fixpack1 update or after multiple fix updates that includes Fixpack 1 - 6.1.0.1-TIV-ITCAMfWAS_MP-IF0001. 1. Download and untar the update installer tar file ITCAM_V61_UpdateInstaller.tar. 2. Download and untar the fixes tar files. You may untar all available updates to the same updates directory. Both fix packs and interem fixes are may be applied together. Example 4-6 lists the fixes that we apply. Example 4-6 A list of update files to be applied 6.1.0-TIV-ITCAMfWAS_MP-FP0001.update 6.1.0.0-TIV-ITCAMfWAS_MP-IF0003.update 6.1.0.0-TIV-ITCAMfWAS_MP-IF0004.update 6.1.0.0-TIV-ITCAMfWAS_MP-IF0005.update 6.1.0.0-TIV-ITCAMfWAS_MP-IF0006.update 6.1.0.1-TIV-ITCAMfWAS_MP-IF0001.update90 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 104. 3. The update installer requires the Java Runtime Environment Version 1.3.1 or later. Either include the java command into PATH or define the JAVA_HOME environment variable to point to the Java Runtime Environment home directory. In UNIX or Linux, run: export JAVA_HOME=/opt/WebSphere/AppServer/java4. The update installer is a non-interactive silent installer that works based on the response file. The default name of the response file is silentUpdate.properties, and it is supplied with the update installer. Update the response file, as in Example 4-7. Example 4-7 The silentUpdate.properties for applying fixes to the data collector product.location=/opt/IBM/itcam/WebSphere/DC updates.location=./updates updates.gclog=/tmp/gclog.log uninstall.updates=last5. Stop the WebSphere Application Servers that have the data collector installed.6. In order to prepare for installation, change to the directory where the update installer scripts are stored. The preparation is recommended for checking the environment. Run the command ./silentUpdate.sh -prepareInstall. A typical output of the prepareInstall is shown in Example 4-8. Example 4-8 The prepare for installation # ./silentUpdate.sh -prepareInstall Update installer version 6.1.0, build 20070513145421 Logging details into /opt/IBM/itcam/WebSphere/DC/logs/update/update_20070608170706.log Action: prepare install Finished successfully7. To update the data collector with all the update files in the updates sub-directory, run the command ./silentUpdate.sh -install. A typical output of the install command is shown in Example 4-9. Example 4-9 Running the patch installation # ./silentUpdate.sh -install Update installer version 6.1.0, build 20070513145421 Logging details into /opt/IBM/itcam/WebSphere/DC/logs/update/update_20070608171205.log . . . Chapter 4. Maintenance of ITCAM for WebSphere 91
  • 105. 8. When the installation has completed, you can verify the data collector version using the command. ./silentUpdate.sh -displayInstalledUpdates. The status in our environment is shown in Example 4-10. Example 4-10 Patch installation result # ./silentUpdate.sh -displayInstalledUpdates Update installer version 6.1.0, build 20070513145421 Logging details into /opt/IBM/itcam/WebSphere/DC/logs/update/update_20070608172553.log Action: display installed updates Updates installed 6/8/07 5:13 PM 6.1.0.1.1 Interim Fix 1 6.1.0.1 Fix Pack 1 6.1.0.0.6 Interim Fix 6 6.1.0.0.5 Interim Fix 5 6.1.0.0.4 High CPU usage for TEMA PMI collection 6.1.0.0.3 JITI support Finished successfully Note: At least one server must first be configured before updating the version level of the data collecter. 9. In a WebSphere Application Server V6.1 environment, activate the OSGI configuration initialization. From the WebSphere /profiles/<profile-name>/bin directory, run the osgiCfgInit.sh script. You should expect no messages from this script. 10.Restart the application server to activate the data collectors. Some patches may have to be updated manually through the WebSphere administration console. This is typically to add certain options on WebSphere. To perform the updates silently, build a Java Command Language (JACL) script.92 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 106. 5 Chapter 5. Planning for ITCAM for Response Time Tracking This chapter discusses areas that must be considered during the planning phase of IBM Tivoli Composite Application Manager implementation in a large environment. The topics discussed in this chapter are: 5.1, “Planning considerations” on page 94 5.2, “Product architecture” on page 94 5.3, “Sizing the servers” on page 96 5.4, “Deployment of ITCAM for Response Time Tracking” on page 98 5.5, “Communication and security considerations” on page 100 5.6, “Reliability and high availability” on page 102© Copyright IBM Corp. 2006, 2007. All rights reserved. 93
  • 107. 5.1 Planning considerations As discussed in 1.2.2, “Concerns and considerations” on page 7, the following aspects of a large-scale implementation are discussed in this section, too: Understanding the product architecture This allows you to make the correct choices. Section 5.2, “Product architecture” on page 94, describes the architecture for ITCAM for Response Time Tracking. Sizing the servers This is important to correctly acquire adequate servers and choose a sound software configuration option. Section 5.3, “Sizing the servers” on page 96, describes an approach that can be used. Understanding the servers’ configuration options and agent deployment This is discussed for ITCAM for Response Time Tracking in 5.4, “Deployment of ITCAM for Response Time Tracking” on page 98. Planning for communication security This is a mandatory step for an enterprise with business-critical and sensitive information in a transaction environment. Section 5.5, “Communication and security considerations” on page 100, discusses confidentiality and firewall requirements. Reliability discussion, failover, and disaster recovery These, too, are mandatory aspects of a business-critical process in a large enterprise. Section 5.6, “Reliability and high availability” on page 102, discusses this issue.5.2 Product architecture This section discusses the product architecture of ITCAM for Response Time Tracking. This understanding is critical for planning and deciding about server configuration and other implementation issues. See also IBM Tivoli Composite Application Manager V6.1 Family Installation, Configuration, and Basic Usage, SG24-7151. ITCAM for Response Time Tracking V6.0 has evolved from IBM Tivoli Monitoring for Transaction Performance V5.3 and inherits the major components and functions of IBM Tivoli Monitoring for Transaction Performance V5.3.94 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 108. Figure 5-1 shows the components of ITCAM for Response Time Tracking. RTT RTT RTT RTT Management Store and Store and management server forward agent forward agent agent FIREWALL FIREWALL RTT RTT RTT RTT Tivoli management management management management Enterprise agent agent agent agent Monitoring AgentFigure 5-1 Components of ITCAM for Response Time Tracking Basically, ITCAM for Response Time Tracking is controlled from the management server, which provides a centralized repository of policy, configuration, and data for the ITCAM for Response Time Tracking environment. The rest of ITCAM for Response Time Tracking is the management agent. The management agent performs response time data collection and performance functions on behalf of the management server. The management agent is a single agent that can have different components deployed on it to perform different functions. The management server and management agent typically operate in an unrestricted port environment. When there is a firewall between them, they restrict the port usage to communicate. The firewall typically requests that they use a single communication port to talk back and forth. This is where the Store and Forward agent comes in. It bundles the communication between the management server and the management agent to use a single port to pass through the firewall. Because the Store and Forward agent can be cascaded, in this sense, there can be a chain of Store and Forward agents passing through multiple layers of firewalls. Chapter 5. Planning for ITCAM for Response Time Tracking 95
  • 109. A special management agent resides on z/OS machines. The management agent on z/OS machines has the transaction server component activated to receive performance information from the Customer Information Control System (CICS) and Information Management System (IMS) data collector. In ITCAM for Response Time Tracking V6.0, information from the management server can be forwarded to the Tivoli Enterprise Monitoring Server for display in the Tivoli Enterprise Portal. This is achieved using the Tivoli Enterprise Monitoring Agent for ITCAM for Response Time Tracking.5.3 Sizing the servers Due to the large scale of the implementation, the size of the servers assumes importance. Sizing determines the hardware configuration and implementation consideration of the servers. This section discusses the following topics: 5.3.1, “Sizing parameters” on page 96 5.3.2, “Sizing estimation for ITCAM for Response Time Tracking” on page 975.3.1 Sizing parameters To evaluate the size of the management server, you must understand the data collection mechanism for ITCAM for Response Time Tracking. Overall, the load for the management server in a steady state environment mainly relates to the collection of response time information and serving the Web console for operators. Additional work may be required to perform the following functions: Running a discovery policy Performing administrative tasks from the Web console or the command-line interface (CLI) Filing the uploading activities from the Synthetic Transaction Investigator or the Rational Robot Fulfilling an ad hoc request to reload data from the Web console The other additional tasks are not addressed in this Redpaper as these are not significant in a steady-state production environment. Reporting load parameters Reporting load is mainly local access to Database 2 (DB2) Universal Database. This does not have a significant network overhead. The processing overhead is the processor requirement to serve the users and DB2 buffering concerns.96 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 110. This load is typically proportional to the number of operators with direct access to ITCAM for Response Time Tracking. However, in ITCAM for Response Time Tracking V6.0, the majority of view-only access for response time information can be consolidated into the Tivoli Enterprise Portal, which allows a single Tivoli Enterprise Monitoring Agent to collect information from the ITCAM for Response Time Tracking management server. With the Tivoli Enterprise Portal solution in place, typical Web console users must be limited to administrative staff, because they can be easily contained. Response time measurement The response time information is recorded by the Application Response Measurement (ARM) agent process from various components that perform monitoring policies, such as listening and playback. This information is uploaded to the management server using the eXtensible Markup Language (XML) data structure in a predefined and scheduled interval. The size of the data transferred is dependent on the number of response time data collected: For playback policies, you have a fixed amount of response time information based on the playback occurences on the time interval. For listening policies, the response time depends on the number of transactions actually invoked, the sampling rate, and the number of monitored components invoked by the transactions.5.3.2 Sizing estimation for ITCAM for Response Time Tracking Specific to ITCAM for Response Time Tracking, consider the following parameters for sizing: “Communication bandwidth” on page 97 “Management server processing” on page 98 “Database size” on page 98 Communication bandwidth Communication between the management server and the management agent primarily consists of: Initial loading of policies and schedules This happens when the management agent is started. The size depends on how many policies are deployed to the management agent. Data retrieval for response time data Data retrieval is typically performed every hour. You can also request an ad hoc retrieval from the Web console when data is retrieved. Chapter 5. Planning for ITCAM for Response Time Tracking 97
  • 111. Additional actions such as deploying a monitoring component or assigning a new monitoring policy to an agent do not contribute to a large workload in a production environment. Management server processing The management server is constructed in a single, large enterprise application. The processing of the management server application is affected by the following parameters: The number of connected management agents The size of data retrieved for each interval The number of connected Web console users Database size The size of the database depends on the amount of response time information loaded from the management agents. The response time information is collected from policies such as: Discovery policies Listening policies Playback policies5.4 Deployment of ITCAM for Response Time Tracking The considerations for installing and configuring ITCAM for Response Time Tracking are covered in this section. The topics discussed here are: 5.4.1, “Designing the management server” on page 98 5.4.2, “Deploying the management agent” on page 995.4.1 Designing the management server The ITCAM for Response Time Tracking management server consists of the following products: DB2 Universal Database Enterprise Server or Oracle database server WebSphere Application Server ITCAM for Response Time Tracking management server application The ITCAM for Response Time Tracking management server is a single contact point for all the management agents and the user Web console. This in turn leads to the ITCAM for Response Time Tracking management server being a single point of failure for response time monitoring.98 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 112. In a large environment, we recommend that you separate the management server into a multiple-server implementation in a clustered environment. Figure 5-2 shows a clustered environment. WebSphere Node Deployment Cluster WebSphere Application Server WebSphere Edge Server DB2 WebSphere Application Server Figure 5-2 Clustered environment The environment shown in Figure 5-2 introduces the following: Load balancing using the WebSphere Edge Server for access to the management servers Separation of load for database access, as the database server is now a separate, shared machine Better reliability because if one of the management servers fails, the other must still be able to process management request5.4.2 Deploying the management agent Management agent deployment for a large environment requires the following planning: Naming convention Each agent must be easily identifiable by using a program to understand its role. Silent installation Automated, non-interactive installation is a better option to deploy the agent to hundreds or thousands of agents. Chapter 5. Planning for ITCAM for Response Time Tracking 99
  • 113. Communication requirement Each agent must communicate to the management server. If there is a firewall, create a Store and Forward agent to minimize the holes in the firewall rule. Mass automated installation Perform this by using a software distribution or provisioning solution such as IBM Tivoli Configuration Manager. Perform management agent configuration and component deployment by using a command-line interface. Scripting the deployment facilitates efficient, automated, consistent, and a less error-prone deployment.5.5 Communication and security considerations Communication and security issues are vital in the inter-networked world we live in. Applications and their management infrastructure must be secured to protect resources from unauthorized sources. This section discusses some of the planning considerations pertinent to this area: 5.5.1, “Communication security” on page 100 5.5.2, “Firewall and port considerations” on page 1015.5.1 Communication security Communication security relates to the confidentiality of the information transmitted over the network. Management information that is used by IBM Tivoli Composite Application Manager products may contain details about the application processing internals. This requires that the content of the management information be secured from unauthorized access. WebSphere security WebSphere security plays a significant role in a large-scale implementation. In some cases, WebSphere security is not enabled during the test phase of an implementation, but during the production environment. This requires additional considerations such as the WebSphere user having the appropriate permissions to, for example, issue wsadmin commands. The configuration of data collectors involves the use of Java Command Language (JACL) scripts, and can fail when there is a permission or rights problem.100 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 114. If any of the application servers on which the data collectors are installed has WebSphere security enabled, the entire ITCAM for WebSphere environment must have it enabled as well. This includes WebSphere security being enabled on the ITCAM for WebSphere managing server. Secure Sockets Layer communication Secure communication between the managing server and the data collector is a viable option if there is a requirement for data to be encrypted during transmission. Using Secure Sockets Layer (SSL) facilitates secure data transmission from the data collector to the managing server, and must appease corporate security requirements, if necessary. Additional configuration must be carried out on the managing server and the data collector when enabling the SSL setup. A certificate key generator is included with the product. This key generator provides the facility to use custom-generated keys. A best practice is to complete the default installation of the managing server and the data collector and then enable SSL for both. This isolates problems (that is, whether the problem is caused by the basic installation or the SSL configuration).5.5.2 Firewall and port considerations Firewall and port issues can arise when the data collectors are on a different site, location, or subnet from the managing server. Problems such as name resolution can occur if the Domain Name System (DNS) is not set up correctly on either the managing server or the data collector. Routing problems may occur if the IP addresses used belong to different subnets. The entire network environment must be looked at to determine where a firewall, router, or bridge can exist. Figure 5-3 shows the communication port requirements for ITCAM for Response Time Tracking. Store and Store and Management Management HTTP port Forward SnF port Forward SnF port server agent Agent Agent Figure 5-3 Communication port requirements The management server running WebSphere Application Server listens at the listening port for a Hypertext Transfer Protocol (HTTP) request and for a Simple Object Access Protocol (SOAP) request. The management agent talks either directly to these ports or in a consolidated manner through the Store and Forward agent. The Store and Forward agent acts as the proxy agent to Chapter 5. Planning for ITCAM for Response Time Tracking 101
  • 115. consolidate communication from the management agent to the management server. The Store and Forward agent can be cascaded to facilitate communication through multiple firewalls. This facility is provided, as some agents may be located across the Internet to monitor the response time from a remote agent.5.6 Reliability and high availability This section discusses reliability issues such as those relating to failover and disaster recovery.5.6.1 Failover and fault tolerance Clustering implementation for the ITCAM for Response Time Tracking management server facilitates a more reliable management of the management servers to allow one of the servers to take care of the entire management load when the other server fails. The clustering implementation uses WebSphere Edge Server load balancing to ensure that both the management servers are used. The WebSphere Edge Server component automatically redirects traffic to the available machine. In the ITCAM for Response Time Tracking management server, there is no indication of the primary server and the secondary server. They all perform an equal role in a WebSphere cluster.5.6.2 Disaster recovery There are three areas where backup is necessary for disaster recovery, with regard to the IBM Tivoli Composite Application Manager server: Database Database backup must be performed regularly in order to collect the most up-to-date information. Use the database utility function to perform the backup function. WebSphere Application Server configuration for the server and the agent or the data collector This must be backed up so that it can be restored in a disaster recovery scenario.102 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 116. IBM Tivoli Composite Application Manager serversThese must also be physically considered to facilitate recovery at the disasterrecovery site. Chapter 5. Planning for ITCAM for Response Time Tracking 103
  • 117. 104 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 118. 6 Chapter 6. Installing ITCAM for Response Time Tracking This chapter provides information about the installation and deployment procedure for ITCAM for Response Time Tracking. This chapter discusses the following topics: 6.1, “Clustering the management server” on page 106 6.2, “Deploying the management resources” on page 129 6.3, “Setting up Secure Sockets Layer certificates” on page 133© Copyright IBM Corp. 2006, 2007. All rights reserved. 105
  • 119. 6.1 Clustering the management server The clustering action of installing the ITCAM for Response Time Tracking management server is performed using a WebSphere Network Deployment cluster. The database must be located in a remote machine to overload the processing and eliminate dependency on either of the machines. This database may also run on a database 2 (DB2) cluster solution. The overall installation requirement is described in Figure 6-1. Edge server Management server Database server WebSphere Edge WebSphere Application DB2 Universal Database Component Server Node Deployment server Load balancer ITCAM for RTT clustering server Figure 6-1 Installation requirement Communication from the management agents and the Web console must take place through the WebSphere Edge Server, which distributes the load for all the management servers. This section discusses the following topics: 6.1.1, “Preparing the operating system” on page 106 6.1.2, “Installing the database” on page 107 6.1.3, “Installing WebSphere Application Server” on page 109 6.1.4, “Installing WebSphere Load Balancer” on page 118 6.1.5, “Installing the management server” on page 1246.1.1 Preparing the operating system Operating system preparation includes the following tasks: Creating a user ID to own the ITCAM for Response Time Tracking server. We create a user called itcamusr. Modifying the following operating system (OS) parameters: – Kernel parameters for Hewlett-Packard UNIX (HP-UX) or Solaris See IBM Tivoli Composite Application Manager for Response Time Tracking Installation and Configuration Guide, GC32-9482. – User right assignment for Windows The ITCAM for Response Time Tracking user requires Act as part of the operating system and Log on as service rights.106 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 120. Allocating the file systems that are required for: – DB2 product /usr/lpp/db2 or C:Program FilesIBMSQLLIB – DB2 instance /home/db2inst1 or C:DB2 – WebSphere Application Server: /opt/IBM/WebSphere or C:Program FilesIBMWebSphere – Tivoli shared path /var/IBM/tivoli/common or C:Program FilesIBMtivolicommon – Product path /opt/IBM/itcam or C:Program FilesIBMitcam6.1.2 Installing the database This section discusses the preparations for using a DB2 database. We use IBM DB2 Universal Database Enterprise Server Edition V8.2 with Fix Pack Level 5. The DB2 database takes up space on the current installed drive. Make that sure you have enough space to accommodate the ITCAM for Response Time Tracking database. Chapter 6. Installing ITCAM for Response Time Tracking 107
  • 121. The databse installation steps are: 1. Launch the DB2 installation launch pad and select Install Product. In the DB2 Setup Wizard window that appears (Figure 6-2), we install the base DB2 server without data warehousing by selecting Typical. Figure 6-2 DB2 options 2. We install with mostly default options. The instance user ID that we create is db2admin. 3. ITCAM for Response Time Tracking comes with a script to create the database schema. Use this script to preinstall the database. Optionally, you can also have the wizard create the database. This script is in the installation compact-disc read-only memory (CD-ROM) in the disk1/utilities/<dbversion> directory. The DB2 creation script is called crtdb_db2. The format for running the script is as follows: crtdb_db2 version dbname dbuser dbpasswd country_code create_db was_user In this syntax: – version specifies the DB2 version, such as 8.1 or 8.2. – dbname is the name of the database being created, such as ITCAMRTT.108 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 122. Note: DB2 supports only eight characters for the database name. – dbuser specifies the user name for the DB2 instance owner. – dbpasswd specifies the password for dbuser. – country_code is the locale name as understood by DB2. – create_db indicates whether to create a new database, Y or N. – was_user is the user that the WebSphere Application Server is running under. In our environment, we issue the following command: crtdb_db2 8.2 ITCAMRTT db2admin xxxxxxxx US Y itcamusr Important: You must have the proper DB2 environment to issue this command. On Windows, run this under the DB2 command window. On UNIX platforms, source the db2profile under the DB2 instance owner home directory. 4. ITCAM for Response Time Tracking management server connects directly to the database using Java Database Connectivity (JDBC) type 4. Cataloging the database locally on the ITCAM for Response Time Tracking management server is not required if the database is installed remotely.6.1.3 Installing WebSphere Application Server We use WebSphere Application Server V6.0.2 and install the network deployment configuration. The installation first creates the network deployment manager, dmgr instance. Individual application servers are then installed using the application server profile. We use the default AppSrv01 profile to connect to the deployment manager. Any of the machines can be the deployment manager, which is used mainly to administer the cluster and not an operational runtime tool. Chapter 6. Installing ITCAM for Response Time Tracking 109
  • 123. The WebSphere Application Server installation process is: 1. Launch the WebSphere Application Server Version 6.0 Network Deployment edition installation wizard, which installs the base WebSphere code. We install WebSphere Application Server in C:IBMWebSphereAppServer (Figure 6-3). Figure 6-3 Installation path for WebSphere Application Server110 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 124. 2. As can be seen in Figure 6-4, we do not install the sample applications. The Javadocs are optional because they do not create additional processing requirements. Click Next.Figure 6-4 WebSphere Application Server components Chapter 6. Installing ITCAM for Response Time Tracking 111
  • 125. 3. The WebSphere Application Server installation wizard shown in Figure 6-4 on page 111 facilitates the launch of the profile creation wizard. Select Create an Application Server profile, as shown in Figure 6-5, and click Next. Note: We use default values for ports and other options. Figure 6-5 Profile creation Note: The deployment manager profile must only be created in the deployment manager node. 4. After the installation concludes, launch the profile creation wizard again. On Windows, start this by using the following command: $WAS_HOMEbinProfileCreatorpctWindows Alternately, use the wasprofile command to create a deployment manager profile to host the deployment manager.112 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 126. 5. We create a standard deployment manager profile Dmgr01 from the Profile creation wizard (Figure 6-6).Figure 6-6 Application server profile6. Install any necessary WebSphere Application Server fix packs. In our environment, we install Refresh Pack 2 for WebSphere Application Server V6.0.2. Copy the updateinstaller directory into the WebSphere directory. The refresh pack must be uncompressed into the WebSphere home directory and launched from there. Note: Download the WebSphere fix packs or refresh packs from the following URL: ftp://ftp.software.ibm.com/software/websphere/appserv/support/fix packs/<was_version>/<refreshpack or fixpack>/<Platform> Our code is downloaded from the following URL: ftp://ftp.software.ibm.com/software/websphere/appserv/support/fix packs/was60/refreshpack2/Windows When applying WebSphere fix packs on Windows servers, Java may not be found even after sourcing the WebSphere Application Server environment with setupCmdLine.bat. In this case, Java Virtual Machine (JVM) must be indicated manually using the following command: update -is:javahome %JAVA_HOME% Chapter 6. Installing ITCAM for Response Time Tracking 113
  • 127. You may have to enclose JAVA_HOME within quotes if you have a space in the installation path, such as program files (Figure 6-7). C:Program FilesibmWebSphereAppServerupdateinstaller>echo %JAVA_HOME% C:Program FilesibmWebSphereAppServerjava C:Program FilesibmWebSphereAppServerupdateinstaller>update -is:javahome "%JAVA_HOME%" Figure 6-7 Updating WebSphere Application Server 7. Start federating the application servers to the deployment manager cluster. Perform this from the deployment manager administrative console: http://dmgr_host:9061/ibm/console 8. In the administrative console, select System Administration → Nodes. Click Add Node, as shown in Figure 6-8.Figure 6-8 Federating application server114 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 128. 9. The add node wizard guides you to connect to the appropriate node, as shown in Figure 6-9. The node wizard connects to the administration server to collect information and create the node agent instance. The node agent talks with the deployment manager to perform administrative tasks for the deployment manager. It also includes the server1 instance into the deployment manager cluster.Figure 6-9 Add node wizard Chapter 6. Installing ITCAM for Response Time Tracking 115
  • 129. 10.Federate all the application server instances that you want to include in the ITCAM for Response Time Tracking clustering server. After the nodes have been federated, create a cluster. Outage from a single application server results in the load being forwarded to other members of the cluster. A cluster is created from the Administrative Console → System Administration → Servers → Cluster menu. Click New, as shown in Figure 6-10. Figure 6-10 Cluster list116 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 130. 11.Create the cluster, as shown in Figure 6-11, for defining the name and its members. At the completion of each step, click Next. Click Finish when done. Save the WebSphere configuration.Figure 6-11 Cluster creation 12.The WebSphere cluster must be authenticated uniformly. To perform this action, activate global security and use an external Lightweight Directory Access Protocol (LDAP) server for authentication. This paper does not discuss this, as implementation of different LDAP servers differs considerably. Note: LDAP is required, as local OS authentication cannot span a server for a UNIX or a Linux machine. If you are using a Windows domain user authentication, you can use the Windows domain users for the cluster security. Chapter 6. Installing ITCAM for Response Time Tracking 117
  • 131. 13.Install the IBM Hypertext Transfer Protocol (HTTP) server. Install this server from the installation media of the WebSphere Application Server in the IHS subdirectory. Install the IBM HTTP Server using the default options in the C:IHS directory. You must also install the IBM HTTP Server plug-in to the IBM HTTP Server system. 14.Generate the IBM HTTP Server plug-in. The plug-in generates the configuration file from the connected WebSphere server. To recreate the plug-in configuration file, run the configurewebserver1 command from the bin subdirectory of the plug-in installation directory. 15.Modify the IBM HTTP Server configuration file (httpd.conf) for additional ports to receive communication from the management agent, typically 9082. Add the following directive to the httpd.conf file: Listen 9082 The WebSphere Application Server is ready.6.1.4 Installing WebSphere Load Balancer The WebSphere Load Balancer is a component of the WebSphere Edge Component. The WebSphere Load Balancer is installed using the WebSphere Edge Component installation media. This is the same media that you used for installing the WebSphere Caching Proxy for the Store and Forward agent. We use the dispatcher component of the Load Balancer. Conceptually, the dispatcher component works as shown in Figure 6-12. request srv150 srv105 (non forwarding) rtt srv151 Figure 6-12 Load Balancer diagram The Load Balancer requires two Internet Protocol (IP) addresses. The physical address, called the non-forwarding address, is used to access the box directly.118 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 132. However, the Load Balancer uses a second address that is an alias to thephysical IP address. Any request to this address is forwarded to either of thebackend servers.Perform the following tasks to install the WebSphere Load Balancer:1. The WebSphere Load Balancer requires a loopback address to be defined as a forwarding address. – In AIX, issue the following command: ifconfig lo0 alias address netmask mask – In the Windows environment, add the Microsoft loopback adapter device and assign the cluster IP address.2. On the welcome panel (Figure 6-13), click Next.Figure 6-13 Welcome dialog Chapter 6. Installing ITCAM for Response Time Tracking 119
  • 133. 3. In the license agreement dialog (Figure 6-14), accept the license agreement by clicking Yes. Figure 6-14 License agreement 4. The installer then checks the prerequisites. If the check is successful, it then displays the language selection panel (Figure 6-15). Figure 6-15 Language selection120 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 134. 5. Figure 6-16 shows type of installation. We use a custom installation and click Next. Note that we use content-based routing.Figure 6-16 Select the installation components Chapter 6. Installing ITCAM for Response Time Tracking 121
  • 135. 6. The next panel (Figure 6-17) shows the summary of selections made and starts the installation upon clicking Next. Figure 6-17 Installation summary dialog 7. Finally, the installation result is shown Figure 6-18. Click Finish to complete the installation and exit the installation wizard. Figure 6-18 Installation summary and result122 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 136. Note: WebSphere Load Balancer assumes that you have the Java Runtime Environment (JRE) javaw executable in the system path. If you do not have the javaw in the system path, add this before proceeding.8. Start our content-based routing server. We are running a Linux server. Run the cbrserver command to start the server.9. The interaction for defining the load balancer is using a command line interface. This is performed using the cbrcontrol command. a. Start the executor: cbrcontrol executor start b. Define the cluster: cbrcontrol cluster add rtt.itsc.austin.ibm.com c. Define the ports that the cluster listens to: cbrcontrol port add rtt.itsc.austin.ibm.com:9081 d. Select the backend servers that the Load Balancer will try to balance. Add all the WebSphere cluster members into this definition: cbrcontrol server add rtt.itsc.austin.ibm.com:9081:srv150.itsc.austin.ibm.com cbrcontrol server add rtt.itsc.austin.ibm.com:9081:srv150.itsc.austin.ibm.com e. Add the rule based on which content base routing would route/load balance the requests. Right-click the port name and select Add Rule Content: cbrcontrol rule add rtt.itsc.austin.ibm.com:9081:defRule type content pattern uri=*/tmtpUI/* Note: You can choose the type of rule depending on your need. For this illustration we use content type as the type of rule. f. Associate the servers in the cluster to the rule defined in previous step e: cbrcontrol rule useserver rtt.itsc.austin.ibm.com:9081:defRule srv150.itsc.austin.ibm.com cbrcontrol rule useserver rtt.itsc.austin.ibm.com:9081:defRule srv151.itsc.austin.ibm.com g. Start the manager: cbrcontrol manager start Chapter 6. Installing ITCAM for Response Time Tracking 123
  • 137. h. Start the advisor on the port configured in step c on page 123: cbrcontrol advisor start http 9081 Note: The above-mentioned steps can also be performed by GUI. Run lbadmin to start the content-based routing GUI. For more information about Load Balancer and various configuration commands refer to Load Balancer Administration Guide, GC31-6858. 10.The result of the configuration is illustrated in Example 6-1. Example 6-1 Checking result of configuration [root@srv105 logs]# cbrcontrol server report rtt.itsc.austin.ibm.com:9081 Cluster: rtt.itsc.austin.ibm.com Port: 9081 ------------------------------------------------------------- | Server | Map port | Total | CPS | ------------------------------------------------------------- | srv151.itsc.austin.ibm.com | | | 9081 | 0 | 0 | | srv150.itsc.austin.ibm.com | | | 9081 | 0 | 0 | ------------------------------------------------------------- Your WebSphere Load Balancer is now ready.6.1.5 Installing the management server In a clustering environment, the management server is installed using a command-line script. In WebSphere, the installation process requires access to the DB2 Java Database Connectivity (JDBC) driver. Copy the Java subdirectory of the DB2 installation or install the DB2 client locally for this. Before starting the installation process, get all the following installation parameters: WebSphere Application Server information: – Installation path – Administrator user and password – Profile name, cell name, node name, and server name – Admin console port – Simple Object Access Protocol (SOAP) connector port124 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 138. – (Optional) Lightweight Directory Access Protocol (LDAP) user name and password – (Optional) Secure Sockets Layer (SSL) information • Client Key file name and password • Client Trust file name and password Database information: – Database server host name – Database server port number – Database name for DB2 or system ID for Oracle – Database user ID and password – (DB2 only) JDBC driver pathThe installation of the ITCAM for Response Time Tracking clusteringmanagement server uses a special CD-ROM image. It does not include theinstallation wizard. Installation is performed interactively using an installationscript. The installation must have both the cluster CD-ROM and the commonCD-ROM images copied to the same installation directory. Example 6-2 shows asample copy command for UNIX platforms.Example 6-2 Sample copy command for UNIX platformscp -r RTTMSclustr/shareddir/* <shared dir>cp -r RTTMScommon/disk2/shareddir/* <shared dir>Example 6-3 shows a sample copy command for Windows.Example 6-3 Sample copy command for Windowsxcopy /S/I RTTMSclustrshareddir <shared dir>xcopy /S/I RTTMScommondisk2shareddir <shared dir>The installation script is a Java Command Language (JACL) script thatconfigures and installs tmtpCluster.ear into your cluster. Invoke the JACL scriptusing the wsadmin command from the network deployment server. Theinstallation command is:wsadmin –username <userid> -password <password> –f itcamfrtt.jacl allThe existing options for itcamfrtt.jacl are: all: This performs all the installation procedures such as install and configure. configure: This configures the ear application only. install: This installs the ear application only. unconfigure: This unconfigures the ear application. uninstall: This uninstalls the ear application. Chapter 6. Installing ITCAM for Response Time Tracking 125
  • 139. removeall: This removes ear application and configuration. Detailed instructions for running the JACL script are provided in ITCAM for RTT: Installing a Management Server in a WebSphere Cluster Environment, SC32-1804. Example 6-4 shows our installation result. Example 6-4 Running itcamfrtt.jacl ------------------------------------------------ ITCAMfRTT Install/Configuration Script Operation: all Silent: false ------------------------------------------------ Have the JVM heap settings been verified? (yes|no) [yes]: Is global security enabled? (yes|no) [yes]: Have all nodes been federated and network connectivity verified? (yes|no) [yes] Please enter the cluster name [ITCAMfRTTCluster]: Available Nodes: srv150Node01 srv151Node01 Select the desired node [srv150Node01]: Please enter the cluster member name [ITCAMfRTTServer1]: Please enter the path to the shared directory [/mnt/clustershare]:/rttshare Is this a Windows node (yes|no) [no]: Current Cluster Nodes and Members: srv150Node01 - ITCAMfRTTServer1 Add more cluster members (yes|no) [yes]: Available Nodes: srv150Node01 srv151Node01 Select the desired node [srv150Node01]: srv151Node01 Please enter the cluster member name [ITCAMfRTTServer2]: Please enter the path to the shared directory [/rttshare]: Is this a Windows node (yes|no) [no]: Current Cluster Nodes and Members: srv150Node01 - ITCAMfRTTServer1 srv151Node01 - ITCAMfRTTServer2 Add more cluster members (yes|no) [yes]: no -------------------------------------------------- Cluster information obtained... -------------------------------------------------- Please enter the WebServer/LoadBalancer port [81]:126 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 140. Please enter the HTTP transport port for ITCAMfRTTServer1 [9081]:Please enter the HTTP transport port for ITCAMfRTTServer2 [9081]:Collecting Database/Datasource Information------------------------------------------------Select the backend database type (db2|oracle|oracle10G) [db2]:Please enter the database driver path for ITCAMfRTTServer1 [c:/sqllib/java]:/home/db2inst1/sqllib/javaPlease enter the database driver path for ITCAMfRTTServer2[/home/db2inst1/sqllib/java]:Please enter the database name (location) [ITCAMRTT]: tmtpdbPlease enter the DB2 database hostname [localhost]: lima.itsc.austin.ibm.comPlease enter the DB2 database port number [50000]:Please enter the database username [db2admin]: db2inst1Please enter the database password [password]: XXXXXXXX------------------------------------------------- Collecting Security Information for JMS-------------------------------------------------Please enter the system username [LocalOSUserID]: wasadminPlease enter the system password [password]: XXXXXXXX------------------------------------------------- Collecting Information for Install-------------------------------------------------Please enter the full path name of the EAR file [tmtpCluster.ear]:/rttclust/RTTMSclustr/ear/DB2UDB_V81/tmtpCluster.ear------------------------------------------------ Configuring Cluster and Cluster Members Scope: srv150Cell01(cells/srv150Cell01|cell.xml#Cell_1)------------------------------------------------Creating Cluster ITCAMfRTTCluster... ITCAMfRTTCluster created successfully!Creating Cluster Member ITCAMfRTTServer1... ITCAMfRTTServer1 createdsuccessfully!Enabling SIB Service on ITCAMfRTTServer1... SIB Service enabled successfully!Creating Cluster Member ITCAMfRTTServer2... ITCAMfRTTServer2 createdsuccessfully!Enabling SIB Service on ITCAMfRTTServer2... SIB Service enabled successfully!------------------------------------------------ Cluster Configuration Completed!!!------------------------------------------------------------------------------------------------ Installing ITMTP-Cluster------------------------------------------------- Chapter 6. Installing ITCAM for Response Time Tracking 127
  • 141. Installing application {ITMTP-Cluster}... Application Name: ITMTP-Cluster Ear file: /rttclust/RTTMSclustr/ear/DB2UDB_V81/tmtpCluster.ear Target Cluster: ITCAMfRTTCluster Authentication Alias: TMTPDataSourceAuthData Starting application install... ------------------------------------------------ ITMTP-Cluster Installation Completed!!! ------------------------------------------------ Do you wish to restart the cluster (yes|no) [yes]: Starting ITCAMfRTTCluster... Ripple start of ITCAMfRTTCluster initiated.6.1.6 Checking the configuration of the RTT cluster application After the installation completes, log on to the WebSphere server administration console and check the Map virtual hosts for Web modules setting of RTT cluster application. See whether they have been mapped to the itcamfrtt_host correctly. Change the virtual host to itcamfrtt_host if they are not currently mapped to it. Select OK and restart the application. See Figure 6-19. Figure 6-19 Virtual hosts definition of RTT cluster application128 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 142. 6.2 Deploying the management resources The management agent in ITCAM for Response Time Tracking is a neutral component. Management components are deployed after the installation is complete, using either the Web interface or the command-line interface (CLI). This section discusses the following topics: 6.2.1, “Silent installation of the management agent” on page 129 6.2.2, “Command-line interface for management components” on page 1306.2.1 Silent installation of the management agent Installing the management agent by using the silent installation method allows unattended installation of a large number of agents to be performed. The silent installation of the ITCAM for Response Time Tracking management agent is performed using the following command: setup_MA -silent -options <responsefile> The response file sample is provided in the responsefiles subdirectory of the installation media. The options in the response file are divided into: License agreement setting for management agents -G licenseAccepted=true Installation directory for management agents -P ma.installLocation="C:IBMitcamRTTMA” Installation settings -W msConnection.offline="true" -W msConnection.hostName="fully_qualified_host_name" -W msConnection.userName="tmtpuser" -W msConnection.password="tmtppassword" -W msConnection.sslValue="[true | false]" -W msConnection.portNumber="9446" -W msConnection.maPort="1976" (This is the default value.) -W msConnection.epKeyStore="c:/tmtp/agent.jks" -W msConnection.epKeyPass="changeit" -W serviceUser.user="TMTPAgent" (Windows only) -W serviceUser.password="tivoli" (Windows only) Global response settings -G replaceExistingResponse="yesToAll | yes | noToAll | no" -G replaceNewerResponse="yesToAll | yes | noToAll | no" -G removeExistingResponse="yesToAll | yes | noToAll | no" Chapter 6. Installing ITCAM for Response Time Tracking 129
  • 143. -G removeModifiedResponse="yesToAll | yes | noToAll | no" Log settings [-W logSettings.level="trace_level"] [-W logSettings.consoleOut="true | false"] A sample response file MA.opt is under the management agent installation file directory. You can use it to proceed the silent installation by modifying the default setting in the sample response file.6.2.2 Command-line interface for management components To deploy the management agent, use the CLI. The CLI for ITCAM for Response Time Tracking is provided in the MS_Home/downloads/cli directory. This directory, which can be copied to any machine, provides the functionality from a command line. The CLI is useful for providing a scripted deployment of ITCAM for Response Time Tracking. Copy the entire directory from MS_Home/downloads/cli. If you are running a Windows system, copy w32util.dll to %WINSYSDIR%w32util.dll. Modify tmtpcli.bat or tmtpcli.sh to define CLI_HOME to the path where you put the directory. In our environment, we run the CLI from the management server, which is installed in C:IBMitcamRTTMS. Therefore, CLI_HOME is set to: CLI_HOME=C:IBMitcamRTTMSdownloadscli You must have JAVA_HOME defined to the JRE 1.4 environment. With the WebSphere Application Server, you can run setupCmdLine.bat or setupCmdLine.sh to establish your environment. Initially, generate the tmtpcli.properties file using the CreateCliProperties command. Depending on whether you activate global security, you may have to use Hypertext Transfer Protocol Secure (HTTPS) protocol and specify the KeyStore file and its password. The Universal Resource Locator (URL) for the management server is the primary URL for the application server, and not the Web console URL. In our environment we issue the following command: tmtpcli -CreateCliProperties –MsUrl http://khartoum:9081/ –UserName itcamrtt –Password xxxxxxx See IBM Tivoli Composite Application Manager for Transaction Tracking Reference Guide, SC32-9485.130 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 144. Deploy the monitoring components using the CLI. Each management component type requires different invocation parameters. Deploying quality of service component: tmtpcli -DeployQoS –AgentName laredo –ProxyHostname laredo.itsc.austin.ibm.com –ProxyPort 7081 –OriginHostname laredo.itsc.austin.ibm.com –OriginPort 9081 -Wait Deploying synthetic transaction investigator component: tmtpcli -DeploySti –AgentName geneva –Wait Deploying generic Windows component: tmtpcli –DeployGenWin –AgentName ser1 –ProjectName trader001 –ProjectPassword pwd –ProjectUserId Administrator –Wait Deploying Application Resource Measurement (ARM) custom application component: tmtpcli –DeployCustomApp –AgentName laredo –ArmAppName Trader Deploying Java 2 Platform, Enterprise Edition (J2EE) component: tmtpcli –DeployJ2ee –AgentName laredo –ServerType WebSphere –Version 6.0 –ServerHome C:IBMWebSphereAppServer –ServerName laredo1 –ProfileName AppSrv01 –AutoRestart –NetworkDeploy –DeployManagerName laredo.itsc.austin.ibm.com –AdminPort 9060 –Wait6.2.3 Defining management resources Management resource definition for a large-scale implementation is cumbersome if performed using the Web console. Define these management resources, such as agent groups, policy groups, operators, and schedules, by using the command-line, too: Defining policy groups: tmtpcli -CreatePolicyGroup <policy group name> Defining agent groups: tmtpcli -CreateAgentGroup <agent group name> -Agents <comma separated agent list> Defining users: tmtpcli -CreateUser <user> Defining schedules: tmtpcli -CreateSchedule <xml filename> Chapter 6. Installing ITCAM for Response Time Tracking 131
  • 145. The schedule itself must be defined in an eXtensible Markup Language (XML) file. Example 6-5 shows the schedule XML file definition. Example 6-5 Schedule XML file <?xml version="1.0" encoding="UTF-8"?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Body> <ns1:ScheduleData xmlns:ns1="urn:TMTP"> <arg0 href="#every5min"/> </ns1:ScheduleData> <multiRef id="every5min" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns2:com.ibm.tivoli.transperf.core.ejb.common.ScheduleData" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns2="typeNS"> <created xsi:type="xsd:dateTime">1970-01-01T00:00:00.000Z</created> <creator xsi:type="xsd:string"></creator> <deleted xsi:type="xsd:dateTime">1970-01-01T00:00:00.000Z</deleted> <deletor xsi:type="xsd:string"></deletor> <description xsi:type="xsd:string">Changed temporally by Danilo</description> <end xsi:type="xsd:int">0</end> <endDateTime xsi:type="xsd:dateTime">1970-01-01T00:00:00.000Z</endDateTime> <endTime xsi:type="xsd:string">00:00</endTime> <isDeleted xsi:type="xsd:boolean">false</isDeleted> <iteration xsi:type="xsd:int">5</iteration> <iterationType xsi:type="xsd:int">0</iterationType> <iterationUnit xsi:type="xsd:string">MINUTES</iterationUnit> <lastUpdated xsi:type="xsd:dateTime">2005-10-14T15:37:07.672Z</lastUpdated> <lastUpdator xsi:type="xsd:string"></lastUpdator> <name xsi:type="xsd:string">Every_5_5min</name> <objectVersion xsi:type="xsd:int">1</objectVersion> <run xsi:type="xsd:int">1</run> <runFri xsi:type="xsd:boolean">false</runFri> <runMon xsi:type="xsd:boolean">false</runMon> <runSat xsi:type="xsd:boolean">false</runSat> <runSun xsi:type="xsd:boolean">false</runSun> <runThu xsi:type="xsd:boolean">false</runThu> <runTues xsi:type="xsd:boolean">false</runTues>132 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 146. <runWed xsi:type="xsd:boolean">false</runWed> <start xsi:type="xsd:int">0</start> <startDateTime xsi:type="xsd:dateTime">2005-10-14T21:36:00.656Z</startDateTime> <startTime xsi:type="xsd:string">00:00</startTime> <timeZone xsi:type="xsd:string">US/Central</timeZone> <type xsi:type="xsd:string">PLAYBACK_SCHEDULE</type> <useDST xsi:type="xsd:boolean">true</useDST> <uuid xsi:type="xsd:int">221</uuid> </multiRef>6.3 Setting up Secure Sockets Layer certificates To secure communication between various components of ITCAM for Response Time Tracking, activate SSL. ITCAM for Response Time Tracking provides a default set of certificates to let you implement SSL encryption using the default setting. The default setting has the communication encrypted, and provides the quickest way to implement SSL. It also lets you test the function with the SSL enabled, but without the risk of potential authentication problems with custom certificates. Anybody with the installation image can extract the certificate and unscramble the information. This section discusses the following topics: 6.3.1, “Secure Sockets Layer for ITCAM for Response Time Tracking” on page 133 6.3.2, “Working with custom certificates” on page 1346.3.1 Secure Sockets Layer for ITCAM for Response Time Tracking Secure Sockets Layer in ITCAM for Response Time Tracking is handled using the Java Secure Socket Extension (JSSE). In this configuration, SSL certificates and keys are stored in a jks (Java key store) file. Management of these jks files is performed using the IBM GSKIT. Note: The IBM GSKIT is different from the keytool command that is supplied in the Java environment. Chapter 6. Installing ITCAM for Response Time Tracking 133
  • 147. Perform Secure Sockets Layer activation directly with the installation wizard or the silent installation options when you install ITCAM for Response Time Tracking. The default port assignments for SSL communication are: Management server port: 9446 Store and Forward Agent port: 443 for the management agent or downstream Store and Forward Agent, and 1976 for the management server or upstream Store and Forward Agent To activate SSL communication after you have ITCAM for Response Time Tracking running without SSL, perform the following tasks: 1. Activate SSL on the management server. The SSL setting on the management server is stored in the WebSphere Application Server settings. Changing to SSL requires modification in the following paths: – Select Security → SSL and modify host/TMTPSettings_Without_ClientAuth and host/TMTPSettings_With_ClientAuth to specify the jks file and its password. – Select Environment → Manage WebSphere Variables and modify the TMTP_KEYSTORE and TMTP_KEYPASS variables. The TMTP_KEYPASS password contains an encrypted password that can be created by using pass_util.jar. 2. Activate SSL on the Store and Forward agent. The Store and Forward agent uses WebSphere Caching Proxy. SSL modification requires changes in ITCAM for Response Time Tracking and WebSphere Caching Proxy. – Modify the SnF/config/endpoint.properties file and the change endpoint.keystore and endpoint.keypass parameters. – Modify etc/cp/en_US/ibmProxy.conf and the KeyRing and KeyRingStash variables. 3. Activate SSL on the management agent. Modify the config/endpoint.properties file and change the endpoint.keystore and endpoint.keypass parameters.6.3.2 Working with custom certificates You may want to implement your own set of custom certificates. The basic operation of certificates lets you define your identity certificate to encrypt the134 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 148. information that you send and lets everyone you trust have a public key todecrypt your information.Although it is possible to define a separate certificate identity for every machinethat communicates, the sheer number of machines and operation changesinvolved in managing them are prohibitive, especially in a large environment. Weimplemented the following set of certificates: Management server certificate Store and Forward agent certificate Management agent certificate that communicates directly to the management server Management agent certificate that communicates through the Store and Forward agentConfiguration involves the following steps:1. The jks files are built and managed using the iKeyMan command. The command resides in the WebSphere Application Server’s bin directory. Create the following jks files: – For the management server: jks file for the management server. We call this file prodms.jks. – For the Store and Forward agent: jks file for the Store and Forward agent. We call this file prodsnf.jks. – For the management agent: jks file for the management agent. We call this file prodma.jks. Besides these files, the iKeyMan tool also generates a password key file with the same name and extension of arm.2. Run the gsk7ikm command from the WebSphere Application Server’s gskit7install directory. This is used to create a key database (kdb file) for the Store and Forward agent, and possibly the quality of service proxy. The quality of service SSL is not discussed in this paper as it relates to the application requirement. This task produces a kdb file, a stash file (sth) (where the kdb password is stored), and an arm file.3. The jks and kdb file sets must be installed and activated on the ITCAM for Response Time Tracking, as discussed in 6.3.1, “Secure Sockets Layer for ITCAM for Response Time Tracking” on page 133. Chapter 6. Installing ITCAM for Response Time Tracking 135
  • 149. 136 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 150. 7 Chapter 7. Maintenance of ITCAM for Response Time Tracking This chapter discusses the issues relating to the maintenance of ITCAM for Response Time Tracking in a large-scale environment. This chapter discusses the following topics: 7.1, “Operational issues pertaining to a large environment” on page 138 7.2, “Performance and availability of management server” on page 138 7.3, “ITCAM for Response Time Tracking files” on page 140 7.4, “Performing product maintenance” on page 142© Copyright IBM Corp. 2006, 2007. All rights reserved. 137
  • 151. 7.1 Operational issues pertaining to a large environment This chapter discusses operational issues regarding a large-scale ITCAM for Response Time Tracking environment, and focuses on the issues that arise largely because of the environment size. The topics discussed in this chapter are: 7.2, “Performance and availability of management server” on page 138 As the size of the environment grows, even a slight inefficiency can cause a major problem. A large-scale environment requires the performance and availability of the managing server to be soundly maintained. 7.3, “ITCAM for Response Time Tracking files” on page 140 This is a necessity. This section discusses specific backup and recovery considerations and maintenance of files. 7.4, “Performing product maintenance” on page 142 This discusses the implementation of WebSphere, Database 2 (DB2), and ITCAM for Response Time Tracking patches. Typically, with the number of data collectors involved, this can be a huge task.7.2 Performance and availability of management server Performance and availability of the ITCAM for Response Time Tracking management server is related mainly to the IBM WebSphere Application Server that hosts it and the availability of its database. This section discusses the following topics: 7.2.1, “Performance of WebSphere Application Server” on page 138 7.2.2, “Database maintenance” on page 1397.2.1 Performance of WebSphere Application Server In ITCAM for Response Time Tracking, the management server runs entirely on the WebSphere Application Server process. The performance of the management server and the Web interface are greatly dependent on the performance of the WebSphere Application Server. This makes the performance tuning of the WebSphere Application Server critical.138 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 152. The WebSphere Application Server basically runs on a Java Virtual Machine (JVM). The typical performance indicators for a JVM are: Adequate processing power from the original machine The clustering environment ensures that you can distribute the processing power across enough server machines. These machines must be dedicated and must not be running any unnecessary tasks. Class loading disk performance WebSphere boot classes and other jar files from ITCAM for Response Time Tracking must be stored in a contiguous and fast disk. If possible, separate the system paging area with the jar files. Adequate memory size for processing As JVMs use a lot of memory, provide adequate memory for the server. You may want to specify the minimum JVM size from the WebSphere Application Server configuration. You must be able to find the size after running the server for a while. Allocating memory on the fly is expensive. Another aspect of JVM memory tuning is the garbage collection. The frequency, as well as the garbage collection options, must be tuned to not interfere with the processing and waste critical memory space. Network connection to the database engine and from the WebSphere Edge Server Primary network connection considerations are request and uploading data to the management server, and data to be put into the database. If possible, the cluster and the WebSphere Edge Server must be running, using a separate local network to minimize packet collision with other applications on the Ethernet bus.7.2.2 Database maintenance The performance of the management server is also related to database performance, which is in turn related to how the data is stored physically and how well the database manager understands the nature of data. This section is primarily aimed at discussing the use of DB2 as the database manager. The primary tools to ensure this are the REORG and RUNSTAT tools. Run both of these utilities from the DB2 utilities from the DB2 command line. Following is a description of these tools: The REORGCHK utility reads the table information and analyzes the necessity of running reorg. REORGCHK used current statistics or generates its own statistics. Chapter 7. Maintenance of ITCAM for Response Time Tracking 139
  • 153. The REORG utility reorganizes the database structure to ensure that data is allocated coherently and in a typical access sequence requested by most queries. The syntax for running REORG is: DB2 REORG TABLE creator.table_name INDEX primary_index_name The RUNSTATS utility collects the statistics of the tables and indexes to ensure that the database manager chooses the most optimized method to get the data requested by a Structured Query Language (SQL) request. The syntax for running RUNSTATS is: DB2 RUNSTATS ON TABLE creator.table_name WITH <runstat_options> Typically, you should run REORGCHK for frequently modified tables, and then determine whether you want to run REORG on those tables. After all the modifications are performed, run RUNSTATS to ensure that the statistics are current and the DB2 obtains the most optimal path to the data that an SQL requested. We also recommend a regular, maybe monthly, maintenance window to REORG all the tables for ITCAM for Response Time Tracking and perform a complete RUNSTATS. Database performance also relates to the size of the data itself. Regular pruning of the collected data must be performed to ensure database performance. This pruning action must be carried out before any REORG or RUNSTATS utilities are performed.7.3 ITCAM for Response Time Tracking files This section discusses the file considerations for ITCAM for Response Time Tracking and provides information about the following topics: 7.3.1, “Backup and recovery” on page 140 7.3.2, “Managing the log files” on page 1417.3.1 Backup and recovery The backup and recovery procedure for ITCAM for Response Time Tracking is similar to the backup and recovery procedure for ITCAM for WebSphere discussed in 4.3, “Backup and recovery configuration” on page 83. It consists of server backup, WebSphere Application Server configuration, and DB2 database backup.140 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 154. Note: WebSphere Application Server backup must be performed on the network deployment engine instead of on individual application servers. An additional requirement for backup may arise from the WebSphere Edge Server. Typical backup of the product installation directories must be adequate.7.3.2 Managing the log files Log files in ITCAM for Response Time Tracking are used for problem determination of the product’s working mechanism. Several locations exist for the log files, depending on the components that generate them. The locations are: Tivoli common directory This is where most of the log files are stored. The typical directory names are: – %ProgramFiles%ibmtivolicommon (Windows) – /var/ibm/tivoli/common (UNIX) The directory under the common directory is typically identified by a three-character product code, such as BWM for ITCAM for Response Time Tracking. In the product directory, the following sub-directories exist: – First-failure data capture (FFDC) for first failure data capture application The FFDC directory contains diagnostic information pertaining to the program exceptions that are captured. – The logs subdirectory of Tivoli common directory This is typically filled with log files and must be maintained. WebSphere log files WebSphere log files are located on each WebSphere Application Server instance under the path $WAS_HOME/profiles/<profile>/logs/<server>. As these log files have a maximum size and a predefined number of generations, they cannot exceed a certain size. This path must be maintained and checked for ad hoc logging that may use up the space. We recommend that you use a separate file system to contain the common directory. Chapter 7. Maintenance of ITCAM for Response Time Tracking 141
  • 155. 7.4 Performing product maintenance Various considerations must be taken into account when dealing with the application of product maintenance, fixes, and fix packs in a large environment, as this must be performed remotely, and automated with minimal impact to the production system. This section discusses the following topics: 7.4.1, “Getting software updates” on page 142 7.4.2, “Updating ITCAM for Response Time Tracking management server” on page 142 7.4.3, “Updating ITCAM for Response Time Tracking management agents” on page 1437.4.1 Getting software updates Obtain the software updates from the following IBM Web sites: For ITCAM for Response Time Tracking: http://www-306.ibm.com/software/sysmgmt/products/support/IBMTivoliCo mpositeApplicationManagerforResponseTimeTracking.html For DB2 Universal Database: http://www-1.ibm.com/support/docview.wss?rs=71&uid=swg27007053 For WebSphere Application Server: http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg27006899 Typically, you do not have to obtain updates for DB2 or WebSphere on the managing server. ITCAM for Response Time Tracking only supports a certain level of these products. However, you may sometimes require a certain software level to fix a problem.7.4.2 Updating ITCAM for Response Time Tracking management server The ITCAM for Response Time Tracking management server updates must be performed as a scheduled maintenance. This means that: You must have a backup of the management server system. You must perform an orderly shutdown of the management server. Users must be notified of the system’s unavailability. The patch for ITCAM for Response Time Tracking is distributed as a self-extracting executable called 6.0.0.0-TIV-RTT-FP000n_<platform> with the142 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 156. bin or exe extensions for each supported management server platform. It also provides zip files for cluster-based installation. Non-cluster installation is performed by launching the executable and following the prompt, while cluster-based implementation is performed by extracting the zip file and running the supplied Java Command Language (JACL) scripts with the wsadmin command. You may have to update the ITCAM for Response Time Tracking database using the supplied script programs. See the corresponding readme files for more information. Restart the WebSphere Application Server after applying the update.7.4.3 Updating ITCAM for Response Time Tracking management agents Perform management agent updates in ITCAM for Response Time Tracking directly from the management server process. Use the automatic agent through the Web console. Select System Administration → Agent Updates. Install the agents manually using the following command: <MA_BASEDIR>jre142jrebinjava -Dtmtp.user.dir=<MA_BASEDIR> -jar ma_fixpack.jar -silent Restriction: Manually install the management agent for z/OS and OS/400 using a platform interface such as System Modification Program/Extended (SMP/E). Reinstall the Java 2, Enterprise Edition (J2EE) component after updating the agent. Use the Web interface or the command line for this. For a large installation, perform both the steps using the command-line interface, as shown in Example 7-1. Example 7-1 Updating J2EE components using the command line tmtpcli -RemoveComponent -AgentName <agent> -ComponentType J2EE_WAS tmtpcli -DeployJ2ee –AgentName <agent> –ServerType WebSphere –Version 6.0 –ServerHome serverHome –ServerName serverName –ProfileName profileName [–AutoRestart] [–IsSecure –AdminUserName <adminUserName> –AdminPassword <adminPassword>] [–NetworkDeploy –DeployManagerName <deployManagerName> –AdminPort <adminPort>] [–Wait] Chapter 7. Maintenance of ITCAM for Response Time Tracking 143
  • 157. 144 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 158. Abbreviations and acronymsAIX Advanced Interactive JMX™ Java Management eXecutive ExtensionsAPI Application programming JNI™ Java Native Interface interface JRE Java Runtime EnvironmentARM Application Response JVM Java virtual machine Measurement JVMPI Java Virtual Machine ProfilerBCM Byte Code Modification InterfaceCD-ROM Compact-disc read-only JVMTI Java Virtual Machine Tool memory InterfaceCLI Command-line interface LDAP Lightweight Directory AccessCPU Central processing unit ProtocolDB2 Database 2 MSC Multiple Systems CouplingEJB Enterprise JavaBeans MVS™ Multiple Virtual StorageETE™ End-to-end PCB Program control blockFFDC First-failure data capture PDF Portable Document FormatFMID Function modification PMI Performance monitor identifier interfaceGUI Graphical user interface RMI Remote Method InvocationHTML Hypertext Markup Language RTE runtime environmentHTTP Hypertext Transfer Protocol SNMP Simple Network ManagementHTTPS HTTP Secure ProtocolIBM International Business SOAP Simple Object Access Machine Corp. ProtocolITIL Information Technology SQL Structured Query Language Infrastructure Library SSH Secure SHellITM IBM Tivoli Monitoring SSL Secure Sockets LayerITSO International Technical STI Synthetic Transaction Support Organization InvestigatorJ2C Java2 Connectivity TCP/IP Transmission ControlJ2EE Java 2 Platform, Enterprise Protocol/Internet Protocol Edition TEMA Tivoli Enterprise MonitoringJAR Java archive AgentJDBC Java Database Connectivity TEMS Tivoli Enterprise Monitoring Server TEP Tivoli Enterprise Portal© Copyright IBM Corp. 2006, 2007. All rights reserved. 145
  • 159. TEPS Tivoli Enterprise Portal ServerUDB Universal DatabaseUML Universal Markup LanguageURI Universal Resource IdentifierURL Universal Resource LocatorWSDL Web Services Definition LanguageXML eXtensible Markup Language146 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 160. Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this Redpaper.IBM Redbooks For information about ordering these publications, see “How to get IBM Redbooks” on page 149. Note that some of the documents referenced here may be available in softcopy only. Installing WebSphere Studio Application Monitor V3.1, SG24-6491 Unveil Your e-business Transaction Performance with IBM TMTP 5.1, SG24-6912 WebSphere Studio Application Monitor V3.2 Advanced Usage Guide, SG24-6764Other publications These publications are also relevant as further information sources: ITCAM for Response Time Tracking publications – IBM Tivoli Composite Application Manager for Response Time Tracking Administrator’s Guide, SC32-9483 – IBM Tivoli Composite Application Manager for Response Time Tracking Checking Performance and Availability, SC32-9484 – IBM Tivoli Composite Application Manager for Response Time Tracking Installation and Configuration Guide, GC32-9482 – IBM Tivoli Composite Application Manager for Response Time Tracking: Installing a Management Server in a WebSphere Cluster Environment, SC32-1804 – IBM Tivoli Composite Application Manager for Response Time Tracking Prerequisites, SC32-9486 – IBM Tivoli Composite Application Manager for Response Time Tracking Problem Determination Guide, SC32-9513© Copyright IBM Corp. 2006, 2007. All rights reserved. 147
  • 161. – IBM Tivoli Composite Application Manager for Response Time Tracking Reference Guide, SC32-9485 – IBM Tivoli Composite Application Manager for Response Time Tracking V6.0 Program Directory, GI11-4099 ITCAM for WebSphere publications – IBM Tivoli Composite Application Manager for WebSphere Installation and Customization Guide, GC32-9506 – IBM Tivoli Composite Application Manager for WebSphere: Installing and Configuring the Tivoli Enterprise Monitoring Agent, SC32-1801 – IBM Tivoli Composite Application Manager for WebSphere Operator’s Guide, SC32-9508 – IBM Tivoli Composite Application Manager for WebSphere Problem Determination Guide, SC32-9509 – IBM Tivoli Composite Application Manager for WebSphere: Tivoli Enterprise Monitoring Agent Problem Determination Guide, SC32-1800 – IBM Tivoli Composite Application Manager for WebSphere User’s Guide, SC32-9507Online resources These Web sites and URLs are also relevant as further information sources: Microsoft link for InstallShield error http://support.microsoft.com/default.aspx?scid=kb;en-us;295278 Microsoft Windows Services for UNIX http://www.microsoft.com/windows/sfu/default.asp Open group Web site for Application Response Time Management http://www.opengroup.org/arm Product prerequisites information http://publib.boulder.ibm.com/tividd/td/ITCAMRTT/prereq60/en_US/HTML /Version60.html http://publib.boulder.ibm.com/tividd/td/ITCAMWAS/prereq60/en_US/HTML /itcam6.html148 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 162. Product Web pages http://www-306.ibm.com/software/tivoli/products/composite-applicatio n-mgr-rtt/ http://www-306.ibm.com/software/tivoli/products/composite-applicatio n-mgr-websphere/ WebSphere fix pack links ftp://ftp.software.ibm.com/software/websphere/appserv/support/fixpac ks/was60/refreshpack1/Windows ftp://ftp.software.ibm.com/software/websphere/appserv/support/fixpac ks/was60/refreshpack1/LinuxHow to get IBM Redbooks You can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooksHelp from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services Related publications 149
  • 163. 150 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 164. Index gsk7ikm 135A iKeyMan 135application management 2 install-DC 50 pillar 5 java 44application monitor 20 keytool 68 data collector agent 20 restoreConfig 83 managing server 20 run-stat-cmds 78Application Response Measurement SetupCmdLine.bat 113 ARM 97 setupCmdLine.bat 130Application Response Measurement, See ARM su 36application server 19 tmtpcli 130archive agent 23, 26, 48 update 113 trimming 79 wasprofile 112ARM 11 wsadmin 29 Application Response Measurement 97 communication port requirement 31B security 100backup communication security 29 database 84 composite application 2 ITCAM for WebSphere 83 confidentiality support 8backupConfig command 83 config_DC command 56 configuration files 44, 86 configuration management database, See CMDBC configurewebserver1 command 118certificate key generator 30 CreateCliProperties command 130CICS 6 crtdb_db2 command 108 Customer Information Control System 96 custom certificate 67CLI custom certificates 134 command-line interface 96 Customer Information Control SystemClient Application Tracker 13 CICS 96cluster 106 Customer Information Control System, See CICS environment 99CMDB 3command-line interface 130 D CLI 96 data collection mechanism 96commands data collector 19, 96 backupConfig 83 additional 56 config_DC 56 deploying 28 configurewebserver1 118 deployment 50 CreateCliProperties 130 data retrieval 97 crtdb_db2 108 data trimming 78 datatrim.sh 80 database db2cmd 37, 39 cataloging 37 db2install 36 connection pools 19© Copyright IBM Corp. 2006, 2007. All rights reserved. 151
  • 165. engine 139 sponse Time Tracking installation 107 backup, recovery 140 installation scripts 36 components 12 maintenance 139 management agent 12 table information 24 management server 12 gcdata 24 Store and Forward Agent 12 memorydata 24 deployment 98 methods 24 features 11 pmidata 24 file systems 107 requests 24 functions 11 volumestats 24 installation, deployment 105database maintenance 77 maintenance 137datatrim.sh command 80 management agents, update 143DB2 management server, processing 98 Client Application Enabler 35 management server, update 142 database 107 overview 11 Universal Database platforms supported 14 Enterprise Server 25, 98 server sizing installation 35 communication bandwidth 97db2cmd command 37, 39 IBM Tivoli Composite Application Manager for Web-db2install command 36 Sphere 19designing managing server 25 architecture 19–20disaster recovery 32, 102 availability 6 backup, recovery configuration 83 componentE Tivoli Enterprise Monitoring Agent 10EJB 19 components 9 usage 19 archive agent 26, 28embedded installation 27 data collector 10Enterprise JavaBeans, See EJB global publishing server 26ETEWatch 13 kernels 26 managing server 9F message dispatcher 27failover 102 polling agent 27fault tolerance 102 publish server 26, 28file system 107 visualization engine 27firewall 101 data sources 19 considerations 30 features 9 support 8 functions 9 implementation 25 managing server 20, 25Gglobal publish server 43 communication bandwidth 22gsk7ikm command 135 database size 24 sizing 22 overview 8I platforms supported 10IBM GSKIT 133 product architecture 18IBM IT Service Management approach 3 IBM Tivoli Composite Application Manager for Web-IBM Tivoli Composite Application Manager for Re- Sphere managing server152 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 166. processing requirement 24 JIBM Tivoli Composite Application Manager portfolio J2EE5 application server 19 for IBM CICS Transactions 5 monitoring agent 13 for IBM IMS Transactions 5 JACL scripts 29 for Response Time Tracking 5 java command 44 for SOA 5 Java Database Connectivity, See JDBC for WebSphere 5 Java Runtime Environment, See JRE IBM Tivoli OMEGAMON® XE for IBM Web- Java Virtual Machine Tool Interface, See JVMTI Sphere Business Integration 5 JDBC 9IBM Tivoli Composite Application Manager Re- JRE 43sponse Time Tracking JVM thread pools 19 designing management server 98 JVMTI 19IBM Tivoli software portfolio 4 business service management 5 composite application management 4 K kernel 23, 26 event correlation and automation 4 parameters 106 orchestration and provisioning 4 keystore 68 resource monitoring 4 extraction of certificates 71iKeyMan command 135 keytool command 68IMS 6, 19 Information Management System 96in-depth resource usage 6 LInformation Management System large-scale environment 3 IMS 96 definition 6Information Management System, See IMS large-scale implementationinstallation parameters 61 concerns, considerations 7install-DC command 50 deploying agents 7IT Infrastructure Library, See ITIL maintenance 8ITCAM for Response Time Tracking reliability 8 planning considerations 94 security 8 product architecture 94 sizing servers 7 sizing parameters 96 listening policies 97 sizing servers 96 load balancing 99ITCAM for WebSphere log files 85, 141 components 26 database installation, configuration 35 features 9 M management agent 95, 135 installation of managing server 34 deployment 99 configuration 34 management resources maintenance 75 definition 131 operational issues 76 deployment 129ITCAM for WebSphere managing server management server 95 sizing installation 124 memory size 23 parameters 124ITIL 3 performance, availability of 138 managing server 19 global publishing server 26 Index 153
  • 167. managing server split server 35 reliability 102mass automated installation 29, 100 remote database 34message dispatcher 27, 43 reporting load 96Microsoft Windows Services for UNIX, See SFU request table 78modular application development 3 response time 6monitoring components 131 data 97Monitoring On Demand 22 response time measurement 97monitoring policies 97 restoreConfig command 83 run-stat-cmds command 78Nname resolution 30 Snaming convention 99 Secure Sockets Layer communication 30 Secure Sockets Layer, See SSL securityO communication, firewall, port usage 29OCTIGATE database 35 service level agreement 5OMEGAMON XE 18 setenv.sh 23operating system preparation 106 SetupCmdLine.bat 113operational management pillar 4 setupCmdLine.bat 130Oracle database server 25, 98 SFU 36overhead silent installation 28, 50, 99, 129 data collector 25 Simple Network Management Protocol, See SNMPoverseer components 24 sizing servers 21 parameters 21P complexity of transaction 21Performance Management Interface, See PMI data collection filter 21planning considerations 18 instrumentation level 21playback policies 97 listening policy mask 21PMI 19 monitoring level 21policy number of data collectors 21 discovery 98 sampling rate 21 listening 98 transaction rate 21 playback 98 SNMP 27polling agent 27, 43 software updates 87, 142port 101 split server consolidation 31 configuration 42 issues 30 installation, setup 43product maintenance 87, 142 SSL 30, 64pruning tables 78 certificates 133publish server 23, 26, 48 communication 101 data collector setup 65 managing server setup 64QQuality of Service monitoring agent 13 SSL communication 30 STI 13 Store and Forward Agent 95, 135R su command 36Rational Robot 12, 96 Synthetic Transaction Investigator 96Redbooks Web site 149 Synthetic Transaction Investigator, See STI Contact us xi154 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 168. TTivoli common directory 141Tivoli Enterprise Monitoring Agent 13Tivoli Enterprise Monitoring Server 96Tivoli Enterprise Portal 10, 96tmtpcli command 130tmtpcli.properties 130Uupdate managing server 87update command 113user right assignment 106utilities REORGCHK 139 RUNSTATS 140utility REORGCHK 77 RUNSTATS 77Vvisualization engine 24, 27, 41, 43, 76Wwasprofile command 112WebSphere log files 141 security 100WebSphere Application Server 25, 98, 109 installation 109 performance 76, 138WebSphere Edge Component installation 118WebSphere Edge Server 106WebSphere security 29WebSphere Studio Application Monitor 18WebSphere Studio Application Monitor, See WSAMWindows workstation 12wsadmin command 29WSAM 18 Index 155
  • 169. 156 Large-Scale Implementation of IBM Tivoli Composite Application Manager for WebSphere and Response Time Tracking
  • 170. Back cover ®Large-Scale Implementation ofIBM Tivoli Composite ApplicationManager for WebSphere andResponse Time Tracking RedpaperPlanning for This IBM® Redpaper discusses large-scale implementationperformance of of IBM Tivoli® Composite Application Manager for INTERNATIONALmanagement WebSphere® and IBM Tivoli Composite Application Manager TECHNICALinfrastructure for Response Time Tracking. Large-scale implementation is SUPPORT typically characterized by the number of monitoring agents ORGANIZATION deployed and the number of transactions load-managed. AImplementing with typical large-scale implementation of a monitoring productmultiple servers contains the following challenges: Keeping up the performance of the monitoring tools BUILDING TECHNICALPerforming mass to accommodate the processing load from the INFORMATION BASED ONupdate of agents agents. PRACTICAL EXPERIENCE Automation of installation, update, and maintenance of monitoring agents based on silent installation and IBM Redbooks are developed by automated update. the IBM International Technical Specific day-to-day maintenance actions to ensure Support Organization. Experts performance and availability of the monitoring from IBM, Customers and solution. Partners from around the world This IBM Redpaper addresses these issues with regard to the create timely technical implementation of ITCAM for WebSphere and ITCAM for information based on realistic Response Time Tracking on distributed platforms. The scenarios. Specific recommendations are provided discussion is divided into planning issues, implementation to help you implement IT guides, and maintenance considerations. solutions more effectively in your environment. For more information: ibm.com/redbooks