• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Monitoring system performance and health of i CEC 2012
 

Monitoring system performance and health of i CEC 2012

on

  • 578 views

COMMON Europe Congress 2012 - Vienna

COMMON Europe Congress 2012 - Vienna

Statistics

Views

Total Views
578
Views on SlideShare
578
Embed Views
0

Actions

Likes
0
Downloads
18
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Monitoring system performance and health of i CEC 2012 Monitoring system performance and health of i CEC 2012 Presentation Transcript

    • IBM Power SystemsMonitoring System Performance and Health of IBM iDawn May - dmmay@us.ibm.com © 2012 IBM Corporation
    • IBM Power SystemsGoals  Do you need to understand system performance in real-time?  Do you need to know who is consuming the system CPU?  Do you want to notify an operator when a partition is not performing as expected?  Do you want to know immediately when an inquiry message is sent or when a critical job ends?  This session will show you how to automate monitoring so you can focus on other aspects of your job.  By the end of this session, you will be able to: – Create and use management central monitors – Understand monitoring with IBM Systems Director – Understand Health Indicators with IBM Systems Director Navigator2 © 2012 IBM Corporation
    • IBM Power SystemsAgenda Three Toolsets Monitoring with Management Central Monitoring with IBM Systems Director Performance Health Indicators3 © 2012 IBM Corporation
    • IBM Power SystemsMonitoring Interfaces Three different management tools – System i Navigator • Windows client application  “iSeries Navigator”, “Operations Navigator” • Management Central Monitors – IBM Systems Director • Cross-platform systems management solution • Platform management, HW alerts • Real-time performance monitoring – IBM Systems Director Navigator • Browser-based interface to manage a single IBM i partition • IBM i Performance Tasks - Performance Data Investigator • Health Indicators4 © 2012 IBM Corporation
    • IBM Power Systems Notes: Monitoring Interfaces - where to get them?System i Navigator, is included with IBM iTM at no additional cost. The IBM i function is integrated into the base of the operating system. The client function is shipped as part of System i Access for Windows.Management Central is a technology integrated into System i Navigator and is not directly installed. When installing System i Access for Windows, choose Custom Install. Expand the System i Navigator option tree and select the appropriate components such as Monitors, Commands.....The general rule for Management Central functions for connectivity is that N-2 and N+2 releases are supported. However, for the best performance and most functions available it is strongly recommended that your IBM System i Navigator and your Central System be at the highest release you have available.Your endpoint systems can then be at a mix of previous releases.Also, note that the functions available to you are only as current as the client and Central System combination. (i.e. if System i Navigator is at 6.1 and the Central System is at 5.4, only 5.4 functions will be available). Also, in almost all cases, function new in a certain release can’t be run to endpoints at older releases.IBM Systems Director Navigator is the Web console available beginning with IBM i 6.1. It is included with IBM i; no installation is necessary, all you need in a Web browser.IBM Systems Director is a browser-based management tool. You need to download and install the management server (it doesnt run on i) as well as installing the agents required on each endpoint. For monitoring on i, you need to install the 5770- UME licensed program product and 5733-SC1, Option 1 (SSH) on each IBM i partition you want to monitor. IBM Systems Director can be downloaded from http://www-03.ibm.com/systems/software/director/downloads/index.html 5 © 2012 IBM Corporation
    • IBM Power Systems Management Central Monitors6 © 2012 IBM Corporation
    • IBM Power SystemsKnow your Tools Firewall and Internet Web application Endpoint Systems server Central System System Group IBM System i Navigator for WindowsIBM Systems Director -- monitoring -- historical trendingNavigator for i – Performance tasks7 © 2012 IBM Corporation
    • IBM Power Systems Notes - Terminology IBM System i Navigator provides a graphical user interface to IBM i® . It comes in 2 complimentary options: Windows and Web. System i Navigator for Windows is in installed on the PC so the user can have a rich graphical interface to interact with their systems. System i Navigator tasks on the Web perform a subset of Navigator tasks through an Internet Web browser. These are URL-addressable links only. IBM Systems Director Navigator for i is the browser-based console that has much of the function of System i Navigator; monitors however, are not available through this interface. The Performance tasks is only available through this Web console. IBM® i® integrated Web application server - The i integrated Web application server (5761-SS1) integrates an OSGi-based Web-servlet container with the i operating system. (5.4 and later) Central System helps to manager your other systems (called endpoints) and stores most management information (inventory, command, package, product, and user definitions, etc). Endpoints are the systems which your PC does not need to be in direct contact with in order to "manage". Source System is the system from which objects, files and information are sent using Management Centrals send tasks. The Source System is the source of the objects, files and information being sent. Model System has all and only desired fixes installed or has all system values set properly for the targets. Target System is where objects, files and information are sent within Management Centrals send tasks. The Target Systems are the destinations of the objects, files and information being sent. Target Systems (and more generally, endpoint systems) are often grouped into System Groups.8 © 2012 IBM Corporation
    • IBM Power Systems Real-time performance graphs with System Monitors9 © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors  Monitor system performance – Predefined metrics ● CPU Utilization, Disk Arm Utilization...  Enabled with... – Event logging – Trigger/Reset notification – CL Command Automation – System Actions ● Job actions (hold, release...)10 © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors Select New Monitor... and specify General properties11 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Select New Monitor... & specify General properties You need to name your monitor – specific names work well. You can select one or multiple metrics in a single monitor Question: How many metrics do you put in a single monitor? Answer: It depends 1. do you like to see all monitors on a single screen? 2. do you prefer to have more granular monitor notifications There is no limit on the number of endpoint systems that a monitor can be started on. However you do get into usability issues when displaying the graph, too many systems on a graph and it might get to difficult to view. General - The General page for New Monitor or Monitor Properties allows you to view and change general information about the monitor. The general information includes the name of the monitor, a brief description of the monitor. Name - The unique name of the monitor. You can change the name, using up to 64 characters for the new name. Do not use any of the following characters: asterisk (*), backslash (), colon (:), greater than (>), less than (<), question mark (?), quotation mark (“), slash (/), or vertical bar (|). Description - A brief description to help you identify this monitor in a list of monitors. You can change the description, using up to 64 characters for the new description.12 © 2012 IBM Corporation
    • IBM Power Systems System MonitorsSelect the Metrics to monitor‘, when done press OK to create WhatHow often Vertical axisHorizontal axis13 © 2012 IBM Corporation
    • IBM Power Systems Notes: Define A MonitorThe Metrics page for New Monitor or Monitor Properties allows you to select the metrics that you want to monitor. You can view and change information about the collection interval, the maximum graphing value, and the display time for each metric. You can also click Threshold 1 or Threshold 2 to specify information about the thresholds for each metric.Metrics is the piece of information to collect. Possible values are:CPU Utilization (Average) Communications IOP Utilization (Average)CPU Utilization (Interactive Jobs) Communications IOP Utilization (Maximum)CPU Utilization (Interactive Feature) Communications Line Utilization (Average)CPU Utilization Basic (Average) Communications Line Utilization (Maximum)CPU Utilization (Secondary Workloads) LAN Utilization (Average)CPU Utilization (Database Capability) LAN Utilization (Maximum)Interactive Response Time (Average) Machine Pool FaultsInteractive Response Time (Maximum) User Pool Faults (Average)Transaction Rate (Average) User Pool Faults (Maximum)Transaction Rate (Interactive) Disk Storage (Average)Batch Logical Database I/O Disk Storage (Maximum)Disk Arm Utilization (Average) Disk IOP Utilization (Average)Disk Arm Utilization (Maximum) Disk IOP Utilization (Maximum)Collection Interval is the time to wait in-between each collection of data.Maximum graphing value is the highest value to be displayed on the vertical axis of the graph.Display time is how many minutes you want displayed on the horizontal axis of the graph.14 © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors Setting Thresholds WhatCondition Automation15 © 2012 IBM Corporation
    • IBM Power Systems Notes: Setting Thresholds Threshold - A threshold is a setting for a metric that is being collected by a monitor. This setting allows you to specify actions to be taken when a specified value (called the trigger value) is reached. You can also specify actions to be taken when a second value (called the reset value) is reached. For example, you can specify a CL command that stops any new jobs from starting when CPU utilization reaches 90% and another command that allows new jobs to start when CPU utilization falls to less than 70%. You can also choose to add an event to the Event Log whenever the trigger value or the reset value is reached. You can set up to two thresholds for each metric that the monitor is collecting. Trigger - considered bad (usually high but can be low), reset - consider good (opposite of trigger) The two Thresholds tabs on the metrics page provide a place for you to specify whether or not you want to monitor this metric for a particular threshold. You must check the Enable threshold box before you can specify the conditions to trigger and to reset this threshold. You can also specify the action to be taken when the threshold is triggered and when it is reset. The action that you specify must be a CL command. When you click OK, this metric will be actively monitored for this threshold if the monitor is currently running. If the monitor is not currently running, this metric will be monitored for this threshold the next time the monitor is started You can specify the following conditions and commands for Threshold trigger and for Threshold reset: Value - Specifies the condition that must be met to trigger or to reset this threshold. Duration - Specifies the number of consecutive collection intervals that the value must meet the criterion to cause a threshold trigger or reset event. Specifying a higher number of collection intervals for Duration helps to avoid unnecessary threshold activity due to frequent spiking of values. i command - Specifies the command to be run on the i endpoint system when the threshold is triggered or reset on that endpoint. This command can be as simple as sending a message, or as complex as submitting or calling a program.16 © 2012 IBM Corporation
    • IBM Power Systems Notes: Threshold variablesSystem Monitor Replacement Variables:Parameter Passed Data&DATE The Date the monitor triggered or reset&INTVL Collection interval: How often the monitor collected data (in seconds)&MON The Monitor name&RDUR Reset duration: How many intervals does the reset value have to be met before the monitor resets.&RVAL Reset value: The value that the metric was monitoring for when the monitor reset&SEQ Sequence number: A unique, incrementing number assigned to each collection interval. Can be used in a program to compare when triggers happened and in what sequence.&TDUR Trigger duration: How many intervals does the trigger value have to be met before the monitor triggers&TIME The time the monitor triggered or reset&TVAL Trigger value: The value that the metric was monitoring for when the monitor triggered&VAL Current value: The actual value of the metric when the monitor triggered (2)Note: A couple of things to note about system monitor replacement parameters:- The dollar sign ($) that was available in previous releases is still supported, for example, $TIME.- The wording is a bit different on some metrics and values:- Batch I/O is shown as I/O operations rather than transactions per second.- Transaction rates are shown as transactions rather than transactions per second.- Interactive response times (both average and maximum) are shown in milliseconds rather than seconds.17 © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors Select the monitor, then the start button18 © 2012 IBM Corporation
    • IBM Power Systems Notes: Start A MonitorThe Start Monitor dialog allows you to select the endpoint systems and system groups on which you want to start the monitor (if no endpoint systems or system groups have been previously selected for this monitor).To add a system or group to the Selected systems and groups list, select it in the Available systems and groups list, and then click Add. If a monitor is started and then a system is added, the monitor will be started on that endpoint system automatically.To remove a system or group from the Selected systems and groups list, select it in the list, and then click Remove. If a monitor is started and then a system is removed, the monitor will be stopped on that endpoint system automatically.Available systems and groups - A list of endpoint systems and system groups from which you can select a system or group. Click the plus sign (+) next to any group to see the systems that are included in the group.Monitor data is collected and stored on the endpoint system. A minimum amount of data is actually sent back to the client when viewing the graph, The more specific, detailed data is only sent to the client when the graphs are openPC is not required to be connected once monitor is started. The graph window can also be minimized and the monitor will still be active.The data shown in the graph is obtained from Collection Services. Collection Services houses the data in management collection objects. This data is used by system monitors, job monitors and other performance tools.19 © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors View the status Overall status20 © 2012 IBM Corporation
    • IBM Power Systems Notes: View Status for a Monitor The Status dialog allows you to see the current status of each endpoint system and system group associated with a monitor. The status of each system and group is updated automatically as changes occur. You can expand any group in the System or Group list to see the status of individual systems in the group. By clicking the Restart button, you can restart the monitor on any systems on which it has failed. Overall status - The current status of the monitor. Possible values are: x thresholds triggered - The number of thresholds that are currently active for the monitor (that is, x represents the number of thresholds that have been triggered but have not been reset). Started on x of y systems - The monitor is collecting data on x of y endpoint systems, where x represents the number of systems where the monitor is running and y represents the number of systems where you requested to start the monitor. The monitor is in the process of starting on the remaining systems. Started - The monitor is collecting data on all endpoint systems where you requested to start the monitor. Starting - The monitor is in the process of starting. Stopping - The monitor is in the process of stopping. Stopped - The monitor is no longer collecting data. Failed - An attempt was made to start the monitor on the specified systems or groups, but the monitor was not started on any systems. The failure may have occurred because the systems were not running when you tried to start the monitor, or it may be because a connection was lost or a server was not started. Click Restart to try starting this monitor again. Failed on x of y systems - The monitor has failed to start or unexpectedly stopped working on x of y systems (where x is the number of systems on which work has stopped and y is the total number of systems on which the monitor is to be run). The monitor is starting or started on the remaining systems. The failure may have occurred because the systems were not running when you tried to start the monitor, or it may be because a connection was lost or a server was not started. Click Restart to try starting this monitor again. See the System or Group Status for a list of the endpoint systems and system21 groups associated with the monitor and the current status of each system and group. © 2012 IBM Corporation
    • IBM Power SystemsSystem Monitors Viewing the thresholds Threshold Indicators Drill down with Actions22 © 2012 IBM Corporation
    • IBM Power SystemsChanging Thresholds Properties Active Control Menu23 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Changing Thresholds You can change the thresholds several ways. Properties Active graphical control Menu items You can change thresholds while a monitor is started, e.g., you do not need to stop the monitor to change the thresholds. The general properties of the monitor can be accessed view the toolbar or menu items for making any changes or additions to the thresholds and values. To change the trigger value or the reset value for a threshold using the active graphical control, place the mouse pointer on the threshold indicator. When the ToolTip indicates Trigger, hold the mouse button down and move up or down to change the trigger value. The changing values are shown in the ToolTip. When the ToolTip indicates Reset, hold the mouse button down and move up or down to change the reset value . Click any collection point on a Monitor graph line to see Details of the data associated with the collection point. By accessing the menu items, you will taken directly to the thresholds page in properties to make any changes. There are several visual indicators when a threshold occurs: Status in the toolbar area. Upper Left corner icon will change. Line in the graph will change to red. Metric graph title will change to red with icon indicator24 © 2012 IBM Corporation
    • IBM Power SystemsThreshold Actions IBM i PC Client25 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Threshold Actions The Actions page for Monitor Properties allows you to specify the actions to occur when a threshold is triggered and when a threshold is reset which apply to all metrics. Log event - Adds an entry to the Event Log on the central system indicating that the threshold was triggered. The entry also includes the date and time the event occurred, the endpoint system being monitored, the metric being collected, and the monitor that logged the event. Open Event Log - Displays the Event Log, which is a list of threshold trigger and reset events that have occurred. Open monitor - Displays a graphical view of the metrics as they are being collected. Sound alarm - Sounds an alarm on the PC. Threshold commands will be run under the monitors owners user profile. When a threshold gets triggered/reset, your PC client does not need to be up and running to run the Operating System command. However, if it is not up the corresponding PC action will not happen.26 © 2012 IBM Corporation
    • IBM Power SystemsViewing Events From Thresholds27 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Event Logs The Event Log window displays a list of threshold trigger and reset events for all of your monitors. You can specify on the Properties page for each monitor whether or not you want events added to the Event Log. To see the Properties page for any monitor, select the monitor in the Monitors window and then select Properties from the File menu. The list of events is arranged in order by date and time by default, but you can change the order by clicking on any column heading. For example, to sort the list by the endpoint system where the event occurred, click on System. An icon to the left of each event indicates the type of event:: A red circle with white x - indicates that this event is a trigger event for which you did not specify a host command to be run when the threshold was triggered. A yellow circle with red x - Indicates that this event is a trigger event for which you specified a host command to be run when the threshold was triggered. A white check with a check mark -indicates that this event is a threshold reset event. You can customize the list of events to include only those that meet specific criteria by selecting Options from the menu bar and then selecting Include. You can have more than one Event Log window open at the same time, and you can work with other windows while the Event Log windows are open. Event Log windows are updated continuously as events occur.28 © 2012 IBM Corporation
    • IBM Power SystemsCustomize Event Log Information29 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Options – Include Options menu choices Click Options on the menu bar to display the actions you can perform to change what information is displayed. The possible choices are: Include... Displays the Include dialog, which allows you to specify which events you want to display in the list. Columns... Displays the Columns dialog, which allows you to specify which columns of information you want to display in the list. You can also specify the order in which you want the columns to be displayed.30 © 2012 IBM Corporation
    • IBM Power SystemsEvent Properties - Trigger (reset similar)31 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Event Properties The Trigger/Reset page for Event Properties allows you to view additional information about the event. This information includes the value, the duration, the Operating System command and the sequence number of the event. Trigger/Reset value - The value specified in the monitor properties. Actual value - The actual value that exceeded the trigger value and caused the trigger event. Duration - The number of collection intervals specified for the duration in the monitor properties. Operating System command - The command that was run on the endpoint system when the event occurred. The General page for Event Properties allows you to view general information about the event. The general information includes the type of event (trigger or reset), the date and time the event occurred, the endpoint system that the event occurred on, the metric that was being collected, and the name of the monitor that logged the event. For more information, select the following: Event type System Date Time Monitor Metric32 © 2012 IBM Corporation
    • IBM Power SystemsManagement Central Job Monitors  Monitor the system by selecting ... – Criteria to subset data ● Job criteria - subsystem, job name, job type or user ● Server name - web server, ftp server... – Predefined metrics ● Job Count, Thread count, CPU Utilization, …  Enabled with... – Event logging – Trigger/Reset notification – CL Command Automation – System Actions ● Job actions such hold, release...33 © 2012 IBM Corporation
    • IBM Power SystemsJob MonitorsSelect New Monitor... and specify General properties34 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors Select Metrics to monitor and thresholds then press OK to create Different Metrics  Types  Levels Problem Condition Automation35 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Define A Monitor The Metrics page for New Monitor or Monitor Properties allows you to select the metrics that you want to monitor. You can view and change information for each metric. You can also click Threshold 1 or Threshold 2 to specify information about the thresholds for each metric. Metrics are the pieces of information to collect. Possible values are: Job Count, Job Log Message and Job Status Job Numeric Values: CPU Percent Utilization Logical I/O Rate Disk I/O Rate Communications I/O Rate Transaction Rate Transaction Time Thread Count Page Fault Rate Summary Numeric Values (same a job level)36 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Threshold variables Job Monitor Replacement Variables: Parameter Passed Data &DATE The Date the monitor triggered or reset &INTVL Collection interval: How often the monitor collected data (in seconds) &MON The monitor name &TIME The time the monitor triggered or reset &ENDPOINT The endpoint system name &EVENTTYPE Event type: The type of trigger or reset that is happening, defined as follows: Triggered Event =1 Auto Reset Event =2 Manual Reset Event =3 &JOBNAME The job name of the job causing the trigger/reset &JOBNUMBER The job number of the job causing the trigger/reset &JOBSTATUS The job status causing a trigger/reset &JOBTYPE The job type of the job causing the trigger/reset &JOBUSER The job user of the job causing the trigger/reset37 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Threshold variables Job Monitor Replacement Variables (continued): Parameter Passed Data &METRICTYPE The category of the metric. For a Job monitor, the categories are as follows: Status Metric = 10010 Message Metric = 10020 Numeric Metric = 10030 &METRIC Metric that has triggered/reset, defined as follows: Job CPU Utilization = 1010 Summary Comm I/O = 2040 Job Logical I/O = 1020 Summary Trans. Rate = 2050 Job Disk I/ = 1030 Summary Trans. Time = 2060 Job Comm I/O = 1040 Summary Thread Cnt = 2070 Job Transaction Rate = 1050 Summary Page Faults = 2080 Job Transaction Time = 1060 Job Status = 3010 Job Thread Count = 1070 Job Log Messages = 3020 Job Page Faults = 1080 Summary Job Count = 4010 Summary CPU Utilization = 2010 Summary Logical I/O = 2020 Summary Disk I/O = 203038 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Threshold variables Job Monitor Replacement Variables (continued): Parameter Passed Data &NUMCURRENT Current numeric value &NUMRESET Threshold value to cause auto-reset of numeric metric &NUMTRIGGER Threshold value to cause trigger of a numeric metric &OWNER Monitor owner &RDUR Reset duration, in intervals, as set in the threshold &RESETTYPE Reset type and defined as follows: Manual reset = 1 Automatic reset = 2 &SBS Subsystem of the job causing the trigger/reset &SERVER Server type of the job causing the trigger/reset. Note: Not supported for summary metrics. &TDUR Trigger duration, in intervals, as set in the threshold &THRESHOLD Threshold number causing the trigger &MSGID Message ID causing the trigger/reset &MSGSEV Message severity causing the trigger/reset &MSGTYPE Message type causing the trigger/reset39 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Threshold variables Invalid Job Monitor Replacement Variable Combinations: Job Monitor substitution parameter notes: • If a monitor is triggered and the user performs a manual reset ("Reset with Commands" or "Reset Only") , there is no substitution value for the Parm &NUMRESET, &RDUR. It will only have a value if the reset is automated. • &MSGID, &MSGSEV, or &MSGTYPE you need to be monitoring the Job Log Message metric - otherwise there is no substitution value for these. Additionally, these are only valid in the trigger and reset commands of Job Log Messages thresholds. • &RESETTYPE only has a valid substitution value on a reset command. Constant values are used to determine whether the reset type is manual or automated. • &EVENTTYPE is valid for all substitution and has constant values that are used to determine the type of monitor event that occurred (automated trigger, automated reset, or manual reset). In a trigger command, the value is always the trigger constant; in a reset command, it can either be the automated reset or manual reset constant. • &TDUR, &NUMTRIGGER, and &NUMCURRENT only have valid substitution when a trigger occurs, in the trigger command. • &NUMTRIGGER, &NUMCURRENT, and &NUMRESET only have valid substitution when a "numeric" metric is being monitored, in the trigger and reset commands of numeric metric thresholds. • &JOBSTATUS only has valid substitution when the Job Status metric is monitored, in the trigger and reset commands of Job Status thresholds. • Job Count metric not valid with: &JOBNAME, &JOBUSER, &JOBNUMBER, &JOBTYPE, &SBS, &SERVER, &MSGID, &MSGSEV, &MSGTYPE, AND &JOBSTATUS • Job Log Message metric not valid with: &RDUR, &NUMRESET, &TDUR, &NUMTRIGGER, &NUMCURRENT, and &JOBSTATUS • Job Status metric not valid with: &NUMRESET, &NUMTRIGGER, &NUMCURRENT, &MSGID, &MSGSEV, AND &MSGTYPE • The Job Numeric Values metrics of CPU Percent Utilization, Logical I/) Rate, Disk I/O Rate, Communications I/) Rate, Transaction Rate, Transaction Time, Thread Count, and Page Fault Rate are not valid with: &MSGID, &MSGSEV, &MSGTYPE AND &JOBSTATUS • The Summary Numeric Values metrics of CPU Percent Utilization, Logical I/) Rate, Disk I/O Rate, Communications I/) Rate, Transaction Rate, Transaction Time, Thread Count, and Page Fault Rate are not valid with: &JOBNAME, &JOBUSER, &JOBNUMBER, &JOBTYPE, &SBS, &SERVER &MSGID, &MSGSEV, &MSGTYPE AND &JOBSTATUS40 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors Setting collection interval How often? 5,15, 30 minutes or 1 hour Tuning options Caution Uses system resources!41 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Job Monitors and System ResourcesJob monitors connect to a QZRCSRVS job for each job that is being monitored for the Job Log Messagesand the Job Status metrics.QZRCSRVS jobs are not Management Central jobs. They are IBM i TCP Remote Command Server jobsthat the Management Central Java server uses for calling commands and APIs. In order to process theAPI calls for the Job Log Messages and Job Status metrics in a timely fashion within the job monitor’sinterval length, the APIs are called for each job concurrently at interval time.When both metrics are specified on the same monitor, two QZRCSRVS jobs are started for each job. Forexample, if 5 jobs are monitored for Job Log Messages, 5 QZRCSRVS jobs are started to support themonitor. If 5 jobs are monitored for Job Log Messages and Job Status, then 10 QZRCSRVS jobs arestarted.Thus, it is recommended that for standard systems, when you are using the Job Log Message and JobStatus metrics, you limit the number of jobs monitored on a small system to 40 jobs or less.42 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors Actions Server PC Client Job, Message and File Monitors Data will collect without thresholds and actions43 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors Select the monitor, then the start button44 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors View the status Overall status Restart on failed systems45 © 2012 IBM Corporation
    • IBM Power SystemsJob Monitors Viewing the system and job informationStatus Detailed Information Actions46 © 2012 IBM Corporation
    • IBM Power SystemsMessage Monitors Message monitors can be used to view messages across systems that match monitor criteria. You can work with the messages listed in the monitor (display details, reply, and delete). Monitor the system by selecting ... – A single message queue – Message criteria ● ID, severity or type – Predefined metrics ● Count... Enabled with... – Event logging – Trigger/Reset notification – CL Command Automation – System Actions ● Message reply, delete...47 © 2012 IBM Corporation
    • IBM Power SystemsMessage Monitors Select New Monitor... and specify General properties48 © 2012 IBM Corporation
    • IBM Power SystemsMessage Monitors Select Metrics to monitor and thresholds then press OK to create n Messages d itio n Co atio n tom Au49 © 2012 IBM Corporation
    • IBM Power Systems Notes: Threshold variables Message Monitor Replacement Variables: Parameter Passed Data &DATE Date &MON Monitor name &INTVL Collection interval length in seconds &TIME Time &ENDPOINT Endpoint system name &EVENTTYPE Event type and defined as follows: Triggered Event = 1 Manual Reset Event = 3 &FRMJOBNUMBER Job number for the job causing the triggering message &FRMJOBNAME Job name for the job causing the triggering message &FRMUSER User owning the job causing the triggering message &FRMPROGRAM Name of the program causing the triggering message &MSGKEY 4-byte message key for the message causing the trigger (as a hex string) &MSGID Message ID causing the trigger &MSGSEV Message severity causing the trigger &MSGTYPE Message type causing the trigger &MSGCOUNT Current message count (that caused the trigger) &OWNER Monitor owner &THRESHOLD Threshold number causing the trigger &TOLIB Message queues library to which this message was sent (the library of the queue being monitored) &TOMSGQ Message queue name to which this message was sent (the queue being monitored)50 © 2012 IBM Corporation
    • IBM Power SystemsMessage Monitors Viewing the system and message information Status Actions51 © 2012 IBM Corporation
    • IBM Power SystemsMonitoring with Watches Watches can be use to automate the actions taken when the following occur: – Message – Licensed Internal Code Log (LIC Log) – Problem Activity Log Entry (PAL entry) Start Watch (STRWCH) command or API (QSCSWCH) End Watch (ENDWCH) command or API (QSCEWCH) When the condition being watched occurs, your program gets control and you can take any action you wanthttp://ibmsystemsmag.blogs.com/i_can/2010/01/i-can-automate-monitoring-with-watches.htmlhttp://publib.boulder.ibm.com/infocenter/iseries/v7r1m0/topic/rzahb/rzahb_eventfunction.htm52 © 2012 IBM Corporation
    • IBM Power SystemsWatches Low Overhead – Watches are an exit – Almost no overhead until the watched condition occurs – Your program gets control to determine what action to take – For message watches • Can watch for messages sent to any message queue, including  QSYSOPR, History Log • Can watch for messages sent to any job log  Can specify generic job name  Can specify *ALL to watch for a message to all job logs53 © 2012 IBM Corporation
    • IBM Power SystemsFile Monitors You can use a file monitor to notify you whenever a selected file has changed, reached a specified size, or for specified text strings.  Monitor the system by selecting ... – History Log (QHST) or specific files – File criteria ● File location – Predefined metrics ● Status, size and text  Enabled with... – Event logging – Trigger/Reset notification – CL Command Automation54 © 2012 IBM Corporation
    • IBM Power SystemsFile Monitors You can select to monitor all system log files or selected files.55 © 2012 IBM Corporation
    • IBM Power SystemsFile Monitors Metrics – Text – File Status – Size56 © 2012 IBM Corporation
    • IBM Power SystemsAutomatic Reset for Message and File Monitor Triggers Some metrics can be reset automatically after a trigger command runs – Only available for Message and File monitors57 © 2012 IBM Corporation
    • IBM Power SystemsSharing of Monitors Monitors can be shared Owner – User that created the monitor None – No one else can see it Read-only – Others can see it ● i.e. view properties & copy it Controlled – Other can perform actions ● i.e. start and stop58 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Sharing The owner has specified one of the following levels of sharing: None Other users cannot view this item. Read-Only Other users can view this item and use it (but can not start or stop it). Other users can create a new item based on this one and make changes to the new one as needed. However, other users cannot delete or change this item in any way. If you are the owner of a monitor and have specified actions (such as opening the event log window or sounding an alarm on the PC), these actions occur for all users of the monitor whenever a threshold is triggered or reset. The other users cannot change these actions. Controlled Other users can start and stop this item. Only the owner can change the level of sharing. Other users can also view this item and use it to create a new item based on this one. If you are the owner of a monitor and have specified actions (such as opening the event log window or sounding an alarm on the PC), these actions occur for all users of the monitor whenever a threshold is triggered or reset. The other users cannot change these actions. Actions are run under the profile of owner!59 © 2012 IBM Corporation
    • IBM Power Systems Monitoring with IBM Systems Director60 © 2012 IBM Corporation
    • AIX® and PowerVM™ Workshop Cross-platform Management Upward Integration modules supporting Tivoli, Computer Associates, Hewlett Packard, Microsoft Windows Linux Administrator Administrator AIX Administrator IBM i Administrator Microsoft Windows™ VM VM VM VMware ESX™ VM VM VIO VIO Managed Systems Common agent, Platform agent, No agent66 © 2012 IBM Corporation66
    • IBM Power Systems IBM i Prerequisites for IBM Systems Director 6.3  Agentless - SSH is required to discover IBM i – SSH, 5733-SC1 Option 1  Platform agent – IBM Universal Management Enablement – For CIM capabilities (for example, IBM i Monitors) – 5770-UME V1R3 – Operating System and UME fixes are required http://www-912.ibm.com/s_dir/slkbase.NSF/DocNumber/618224537  Common Agent (CAS) – Download from IBM Systems Director web site – Install manually on i or through the IBM Systems Director UIhttp://publib.boulder.ibm.com/infocenter/director/pubs/topic/com.ibm.director.tbs.helps.doc/fqm0_tbs_ibm_i_endpoints.html62 © 2011 IBM Corporation
    • IBM Power SystemsCollection Services Prerequisites for Monitoring  Collection Services must be started  Consider the collection interval for frequency of updates through System Director – Default is every 15 minutes  Data must be in Collection Services DB2 files to be displayed on IBM i Monitors in Systems Director – CFGPFRCOL – CRTDBF parameter must be *YES  “No Data Available” – Symptom of possible Collection Services problem when viewing IBM i Monitors63 © 2011 IBM Corporation
    • IBM Power SystemsPerformance Summary64 © 2012 IBM Corporation
    • IBM Power SystemsMonitor Your i65 © 2012 IBM Corporation
    • IBM Power SystemsCommon CIM Monitors66 © 2012 IBM Corporation
    • IBM Power SystemsIBM i Monitors67 © 2012 IBM Corporation
    • IBM Power SystemsIBM i Monitors  34 Metrics for common monitoring scenarios68 © 2011 IBM Corporation
    • IBM Power Systems69 © 2011 IBM Corporation
    • IBM Power SystemsCreate Your Own Monitor70 © 2011 IBM Corporation
    • IBM Power SystemsDirector Agent Monitors (requires CAS agent)71 © 2011 IBM Corporation
    • IBM Power Systems72 © 2011 IBM Corporation
    • IBM Power Systems73 © 2011 IBM Corporation
    • IBM Power SystemsCIM Monitors The are many metrics that you can monitor with CIM, but it can be difficult to figure out whats available.74 © 2011 IBM Corporation
    • IBM Power SystemsMonitor Your IOA Cache Batterieshttp://ibmsystemsmag.blogs.com/i_can/2012/04/monitoring-cache-battery-status-with-ibm-systems-director-63.html75 © 2011 IBM Corporation
    • IBM Power SystemsIOA Cache Battery Monitor Metrics76 © 2011 IBM Corporation
    • IBM Power SystemsManage Processes … aka, Work with Active Jobs (Requires CAS Agent)77 © 2011 IBM Corporation
    • IBM Power SystemsMonitoring IBM i Processes  Create a process monitor from the Manage Processes task  Once a process monitor is created additional metrics can be monitored for that process using the Create monitor task  Requires CAS agent78 © 2011 IBM Corporation
    • IBM Power SystemsMonitor Thresholds The process of setting threshold and enabling automation is the same regardless of the metric type For any metric that you want to monitor, you can set thresholds Once thresholds are set, you then can create an event filter for the event that occurs when the threshold setting is hit Automation plans complete the set up to enable automatic notification that the threshold was hit79 © 2012 IBM Corporation
    • IBM Power SystemsMonitor Thresholds80 © 2012 IBM Corporation
    • IBM Power SystemsMonitor Thresholds81 © 2012 IBM Corporation
    • IBM Power SystemsCreate Event Filters82 © 2012 IBM Corporation
    • IBM Power SystemsEvent Automation Plans83 © 2012 IBM Corporation
    • IBM Power Systems84 © 2012 IBM Corporation
    • IBM Power SystemsEvent Actions85 © 2012 IBM Corporation
    • IBM Power SystemsGraphs and Dashboard86 © 2012 IBM Corporation
    • IBM Power SystemsHealthSummary87 © 2012 IBM Corporation
    • IBM Power Systems Monitoring Messages with IBM Systems Director Events88 © 2012 IBM Corporation
    • IBM Power SystemsEvent Filters for QSYSOPR messages89 © 2012 IBM Corporation
    • IBM Power SystemsEvent Filters for QSYSOPR message90 © 2012 IBM Corporation
    • IBM Power SystemsEvent Filters for QSYSOPR message91 © 2012 IBM Corporation
    • IBM Power SystemsMonitoring QSYSOPR .... Event Automation Plan After you establish what you want to monitor by creating the event filter for IBM i messages ... You must also create an event automation plan for the action Director is to take when those events occurEven though these events are generated by their respective operating systems (or an optional layerthat is installed on the operating system), IBM Systems Director does not process these events unlessyou create an event automation plan to do so.92 © 2012 IBM Corporation
    • IBM Power Systems Monitoring System Performance with IBM Systems Director Navigator93 © 2012 IBM Corporation
    • IBM Power SystemsIBM Systems Director Navigator for i Performance Tasks94 © 2012 IBM Corporation
    • IBM Power SystemsHealth IndicatorsManually Monitor your System Performance95 © 2012 IBM Corporation
    • IBM Power SystemsHealth IndicatorsCustomize Health Indicator Thresholds96 © 2012 IBM Corporation
    • IBM Power SystemsContinuing Education A Few More Things....97 © 2012 IBM Corporation
    • IBM Power SystemsMonitor Restart Options98 © 2012 IBM Corporation
    • IBM Power SystemsNotes: Changing the Monitor Restart Options Restarting Monitors Monitor Restart was added to provide a way to automatically restart monitors when the Management Central servers have been interrupted. These interruptions could be as simple as the MC Central Server or MC Endpoint Server being restarted, or something more dramatic such as the temporary loss of communications between the Central Server and an Endpoint Server or a system being IPLed. If you select to have the system automatically attempt to restart your monitors, you may also specify how long you want the central system to keep trying to restart the monitors and how often you want the system to try during that time period. For example, if you want the system to try to restart monitors every five minutes for a period of 3 hours, you select Automatically restart monitors on failed systems and then specify 180 minutes for How long to attempt restart and 5 minutes for How often to attempt restart. A change to this setting takes effect the next time the Management Central servers are restarted. All Monitors support the restart option. Default behavior is OFF. Documentation on how to automatically restart Management Central Monitors: https://www-304.ibm.com/support/entdocview.wss?uid=nas16e5b0871315547a68625729e004737ce99 © 2012 IBM Corporation
    • IBM Power SystemsReferences IBM i Home Page – http://www-03.ibm.com/systems/power/software/i/ Information Center – http://publib.boulder.ibm.com/iseries/ IBM i Systems Management – www.ibm.com/systems/i/solutions/management/ – www.ibm.com/systems/i/software/navigator/index.html http://www.redbooks.ibm.com/redbooks/pdfs/sg246226.pdf100 © 2012 IBM Corporation
    • IBM Power SystemsSpecial notices This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area. Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied. All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a clients credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice. IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies. All prices shown are IBMs United States suggested list prices and are subject to change without notice; reseller prices may vary. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally- available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment. Revised September 26, 2006101 © 2012 IBM Corporation
    • IBM Power SystemsSpecial notices (cont.)IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/OS, i5/OS (logo), IBM BusinessPartner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pSeries, Rational, RISCSystem/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xSeries, z/OS, zSeries, Active Memory, Balanced Warehouse,CacheFlow, Cool Blue, IBM Systems Director VMControl, pureScale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000,EnergyScale, Enterprise Workload Manager, General Parallel File System, , GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager,iSeries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor,Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+,POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarksor registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms aremarked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM atthe time this information was published. Such trademarks may also be registered or common law trademarks in other countries.A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/orother countries.AltiVec is a trademark of Freescale Semiconductor, Inc.AMD Opteron is a trademark of Advanced Micro Devices, Inc.InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registeredtrademarks of Intel Corporation or its subsidiaries in the United States and other countries.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce.Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs aretrademarks of the Standard Performance Evaluation Corp (SPEC).The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).UNIX is a registered trademark of The Open Group in the United States, other countries or both.Other company, product and service names may be trademarks or service marks of others. Revised December 2, 2010 102 © 2012 IBM Corporation