Keeping track of your messaging infrastructure is essential to be able to rapidly respond to your changing business demands whilst complying with your constantly changing audit demands. This session will provide illustrations and insight on the techniques and capabilities available in IBM MQ to help you meet these monitoring and auditing demands.
1. 3452: Managing your applications
Tim McCormick
IBM MQ Service Architect
timmccor@uk.ibm.com
2. Who is this presentation aimed at?
• Application developers who want to understand how their application is
really interacting with MQ
– Does the reality match the design and expectation?
• MQ administrators who want to understand how applications are
interacting with their queue manager
– Is the application behaving as described?
– Has the application been coded to required standards?
– Is the application exhibiting best practice?
5. What are accounting messages?
• Information about the MQI operations performed by an application
• PCF messages
– Delivered to SYSTEM.ADMIN.ACCOUNTING.QUEUE
• When the application disconnects and…
• When the specified interval has been reached (ACCTINT)
– Not persistent
6. What are accounting messages?
• Two types
– MQI accounting
• MQI accounting messages contain information relating to the number of MQI calls
made using a connection to a queue manager.
– Queue accounting
• Queue accounting messages contain information relating to the number of MQI calls
made using connections to a queue manager, grouped by queue.
8. How do I read accounting messages?
• Read PCF messages from SYSTEM.ADMIN.ACCOUNTING.QUEUE
• amqsmon sample
– Some output is in cryptic list form, example:
• OpenCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
• This means 2 objects of type MQOT_Q (queue) were opened
• Demystify via documentation:
– Search ‘MQI accounting message data’
• Vendors
• Roll your own
9. amqsmon example output (MQI accounting)
MonitoringType: MQIAccounting
QueueManager: 'MQ80'
IntervalStartDate: '2016-02-16'
IntervalStartTime: '18.16.56'
IntervalEndDate: '2016-02-16'
IntervalEndTime: '18.17.00'
CommandLevel: 800
ConnectionId: x'41…02e01'
SeqNumber: 0
ApplicationName: 'C:WMQ8.0bin64amqsput.exe'
ApplicationPid: 35388
ApplicationTid: 1
UserId: 'timmccor'
ConnDate: '2016-02-16'
ConnTime: '18.16.56'
DiscDate: '2016-02-16'
DiscTime: '18.17.00'
DiscType: Normal
OpenCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
OpenFailCount: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
CloseCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
CloseFailCount: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
PutCount: [2, 0]
– Much more data is logged than appears
on this slide
– Interval start/end matches connection
life, means no intermediates generated
– See useful information about the
application connected
– Most interesting fields are
[-,Q,-,-,-,QMGR,-,-,Topic,-,-,-,-,-]
We opened 2 queues.
– PutCount: [Nonpersistent,Persistent].
We put 2 non-persistent messages
10. amqsmon example output (queue accounting)
MonitoringType: QueueAccounting
QueueManager: 'MQ80'
ApplicationName: 'C:WMQ8.0bin64amqsput.exe'
UserId: 'timmccor'
QueueAccounting: 0
QueueName: 'Q1'
CreateDate: '2015-02-05'
CreateTime: '13.45.58‘
QueueDefinitionType: Local
OpenCount: 1
OpenDate: '2016-02-18'
OpenTime: '14.04.20'
CloseCount: 1
CloseDate: '2016-02-18'
CloseTime: '14.04.22'
PutCount: [1, 0]
PutFailCount: 0
Put1Count: [0, 0]
Put1FailCount: 0
PutBytes: [4, 0]
PutMinBytes: [4, 0]
PutMaxBytes: [4, 0]
– Much more data is logged than appears
on this slide
– See useful information about the
application connected and when it
opened and closed the queue
– See useful information about how the
queue was defined
– PutCount: [Nonpersistent,Persistent].
We put 1 non-persistent messages
11. When should I use accounting messages?
• Low impact on system
• Scoped to an individual connection or queue
– Connection level information, won’t provide much detail on the objects accessed
– Queue level information may be too broad
13. What is API trace?
• API trace is a part of the diagnostic trace IBM support will request in
PMRs
– Cut down to show only the API calls and less of the internal working of the system
– Intended for IBM support, but can be useful for customers
14. How do I enable API trace?
• strmqtrc –t api –p myapplicationname
– Start API trace for applications with named ‘myapplicationname’
– Don’t forget the .exe on Windows
• endmqtrc -a
– Stop trace
15. How do I read API trace?
• MQ_DATA_DIR/trace/AMQPID.0.TRC
– Where PID is the process ID
• Trace is output in a binary non-readable format
– Run dspmqtrc to format the trace into human readable format
– Use dspmqtrc from the same installation which generated the trace
– Except on Windows where trace is output in human readable format
16. API trace example (header)
• Header at top of every file
• A 64-bit version of amqsput generated
this file
• Application was run on a Windows 7
system named MYHOST
• Application was at MQ version 8.0.0.4
• MQ install and data paths
• This install is MQ80, there are 3 others
on the system
Process : C:WMQ8.0bin64amqsput.exe (64-bit)
Arguments :
Host : MYHOST
Operating System : Windows 7 Professional x64 Edition, Build
7601: SP1
Product Long Name : WebSphere MQ for Windows (x64 platform)
Version : 8.0.0.4 Level : p800-004-151017
O/S Registered : 1
Data Path : C:WMQdata
Installation Path : C:WMQ8.0
Installation Name : MQ80 (3)
License Type : Production
UTC Date : 2016/02/18: Time : 11:52:44.759
LCL Date : 2016/02/18: Time : 11:52:44.759 GMT Standard Time
17. API trace example (trace records)
• The counter is a hexadecimal field which
increments by 1 for every trace record
• The timestamp is the time in the
timezone from the header
• PID.TID – Process ID and Thread ID
logging the trace record
• Ident provides additional identity
information depending on context
• Data is the ‘interesting’ data which was
traced
Counter TimeStamp PID.TID Ident Data
==================================================
000001DE 12:09:21.687697 25528.1 : !! - __________
000001DF 12:09:21.687713 25528.1 : !! - MQCONN >>
000001E9 12:09:21.702901 25528.1 CONN:525958 !! - MQCONN <<
20. When should I use API trace?
• Easy to enable
• Impact on disk I/O
• Difficult to read
– Intended as diagnostic information for IBM support teams
• Scoped to process
– All Java applications will be seen as ‘java’ as the JVM is the process
22. What is application activity trace?
• Similar to accounting messages, but…
– Much more detail on what the application was doing
– Can be controlled at the application level
– Could be used to audit application behavior
• PCF messages
– Delivered to SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE when…
– Application disconnects, or…
– ActivityCount or ActivityInterval thresholds are reached, or…
– When the size of the message which will be written reaches the maximum message length for the
queue
– Non-persistent messages
23. How do I enable application activity trace?
• Queue manager attribute
– ACTVTRC
• MQCONNX connection option
– ACTVCONO queue manager attribute must be enabled
– MQCNO_ACTIVITY_TRACE_DISABLED
– MQCNO_ACTIVITY_TRACE_ENABLED
• mqat.ini configuration file
– Found in the queue manager data directory
– Does not require queue manager restart, reconnect or
alter qmgr object in any way
– Can specify individual settings by application name Overrides
24. Application activity trace on the MQ Appliance
• Subscribe to application activity trace system topic
– $SYS/MQ/INFO/QMGR/my_qmgr/ActivityTrace/…
• mqat.ini configuration file
– Only global settings available on the MQ Appliance via setmqini
25. mqat.ini example, global settings
AllActivityTrace:
ActivityInterval=1
ActivityCount=100
TraceLevel=MEDIUM
TraceMessageData=0
StopOnGetTraceMsg=ON
– Application activity trace won’t be logged without at least one ApplicationTrace stanza, or the queue
manager attribute ACTVTRC set to on
– Log an application activity trace message every one second, or one hundred MQI or XA operations
– Log a medium amount of detail, documentation lists which operations are logged for each level
– Do not trace any message data for get or put operations
– Don’t generate application activity trace when handling application activity trace messages (to avoid
looping)
26. mqat.ini example, application settings
ApplicationTrace:
ApplClass=ALL
ApplName=amqs*
Trace=ON
TraceLevel=HIGH
– You need at least one ApplicationTrace stanza to log application activity trace, or ACTVTRC set to on
– User applications only or include the MCA application?
– Application name, can be wildcarded. By default the application name is the process name. From MQ
7.5 JMS clients can specify application name via the connection factory.
– Turn application activity trace on
– Override global settings as needed
27. How do I read application activity trace?
• Read PCF messages from
SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE
• amqsact
• MS0P
• Other vendors
• Roll your own
28. amqsact example
MonitoringType: MQI Activity Trace
QueueManager: 'MQ80'
Host Name: ‘MYHOST'
IntervalStartDate: '2016-02-17'
IntervalStartTime: '11:59:20'
IntervalEndDate: '2016-02-17'
IntervalEndTime: '11:59:22'
CommandLevel: 800
SeqNumber: 0
ApplicationName: 'C:WMQ8.0bin64amqsput.exe'
UserId: 'timmccor'
Trace Detail Level: 2
Platform: MQPL_WINDOWS_NT
==========================================================
Tid Date Time Operation CompCode MQRC HObj (ObjName)
001 2016-02-17 11:59:20 MQXF_CONNX MQCC_OK 0000 -
001 2016-02-17 11:59:20 MQXF_OPEN MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_PUT MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_CLOSE MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_DISC MQCC_OK 0000 -
==========================================================
– amqsput
– Running on Windows
– Connected to MQ80
– On MYHOST
– With user ID timmccor
– This activity trace was logged with
medium detail
• 1 = Low, 2 = Medium, 3 = High
– Summary of API calls made
29. amqsact verbose example (MQOPEN)
– 2nd MQI call
– MQOPEN
– Of a queue
– Named Q1
– Which resolved to a queue named Q1
on MQ80
– Opened on object handle 2
– With options 8208
• MQOO_FAIL_IF_QUIESCING &
MQOO_OUTPUT
– Successfully
– And the API call took 41ms to complete
on the queue manager
MQI Operation: 1
Operation Id: MQXF_OPEN
ApplicationTid: 1
OperationDate: '2016-02-17'
OperationTime: '16:31:42'
Object_type: MQOT_Q
Object_name: 'Q1'
Object_Q_mgr_name: ''
Hobj: 2
QMgr Operation Duration: 41
Completion Code: MQCC_OK
Reason Code: 0
Open_options: 8208
Resolved_Q_Name: 'Q1'
Resolved_Q_mgr: 'MQ80'
Resolved_local_Q_name: 'Q1'
Resolved_local_Q_mgr: 'MQ80'
Resolved_type: MQOT_Q
Dynamic_Q_name: 'AMQ.*'
30. amqsact verbose example (MQPUT extract)
– Much more data than can be displayed on this slide.
– 3rd MQI call
– MQPUT
– In addition to usual timings, you get the epoch value when the
operation started
– The object was Q1 which resolved to a local queue named Q1.
– The message was put under object handle 2
– With options 8260
• MQPMO_NO_SYNCPOINT & MQPMO_NEW_MSG_ID &
MQPMO_FAIL_IF_QUIESCING
– Successfully
– And the API call took 115ms to complete on the queue manager
– Did the application use best practices?
MQI Operation: 2
Operation Id: MQXF_PUT
High Res Time: 1455726703762959
QMgr Operation Duration: 115
Reason Code: 0
Hobj: 2
Put Options: 8260
Object_name: 'Q1‘
Resolved_Q_Name: 'Q1'
Resolved_local_Q_name: 'Q1‘
Resolved_type: MQOT_Q
Report Options: 0
Expiry: -1
Format_name: 'MQSTR'
Priority: -1
Persistence: 2
Msg_id: …
Correl_id: …
Reply_to_Q : ''
Reply_to_Q_Mgr: ''
Coded_char_set_id: 0
Encoding: 546
Put_date: '20160217'
Put_time: '16314375'
31. amqsact verbose example (MQGET extract)
– Much more data than can be displayed on this slide.
– 3rd MQI call
– MQGET
– In addition to usual timings, you get the epoch value when the
operation started
– The object was Q1 which resolved to a local queue named Q1.
– The MQGET was under object handle 2
– With options 16400
• MQGMO_BROWSE_FIRST & MQGMO_CONVERT
– But it failed, MQRC_NO_MSG_AVAILABLE
– And the API call took 17ms to complete on the queue manager
MQI Operation: 2
Operation Id: MQXF_GET
ApplicationTid: 1
OperationDate: '2016-02-18'
OperationTime: '11:18:50'
QMgr Operation Duration: 17
Completion Code: MQCC_FAILED
Reason Code: 2033
Hobj: 2
Get Options: 16400
Msg length: 0
High Res Time: 1455794330745687
Object_type: MQOT_Q
Object_name: ‘Q1'
Object_Q_mgr_name: ''
Resolved_Q_Name: ‘Q1'
Resolved_Q_mgr: 'MQ80'
Resolved_local_Q_name: ‘Q1'
Resolved_local_Q_mgr: 'MQ80'
Resolved_type: MQOT_Q
Buffer Length: 102400
32. Performance Impact
• Only trace the applications you’re interested in
• Set the TraceLevel at the lowest level which provides the information
you require
• Tune ActivityInterval and ActivityCount, experiment
– Less frequent messages will use more memory
– More frequent messages will use more IO
• Frequently consume messages from the queue to avoid queue full
33. When should I use application activity trace?
• Scoped to application
• Application audit trail
• Performance analysis
– How long are my API calls taking to complete on the queue manager?
• Resource usage
– Which queues or topics are actually being used
• Problem Determination
– Which Queue / Queue Manager is the application
actually opening
• Application Coding Standards
– Does everyone use the MQI in the recommended way
• And more …
38. Notices and Disclaimers Con’t.
38
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not
tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the
ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual
property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®,
FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG,
Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®,
PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®,
StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business
Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
39. Thank You
Your Feedback is Important!
Access the InterConnect 2016 Conference Attendee
Portal to complete your session surveys from your
smartphone,
laptop or conference kiosk.
40. Monday
10:30-11:30 3592 New MQ features
3452 Managing applications
12:00-13:00 2835 MQ on z/OS and Distributed
15:00-16:00 3470 Latest MQ z/OS features
2833 Where is my message?
3544 MQ Light in an MQ infrastructure
16:30-17:30 3573 Hybrid cloud messaging
2941 MQ Advanced
Tuesday
08:30-09:30 3540 The MQ Light API
12:00-13:00 3456 The IBM MQ Appliance
13:15-14:15 3499 Introducing Message Hub
3458 MQ Appliance administration
14:30-15:30 6432 MQ updates and futures (InnerCircle)
2849 Messaging feedback roundtable
16:00-17:00 3544 MQ Light in an MQ infrastructure
3513 MQ hands on lab
Wednesday
08:30-09:30 3602 Managing your MQ environment
12:00-13:00 3613 Designing MQ self service
6408 Hybrid messaging roadmap (InnerCircle)
13:15-14:00 3416 HA and DR with MQ
3433 Why secure your messaging?
15:45-16:30 3429 Securing MQ
2847 Meet the messaging experts
16:00-17:00 3508 MQ Light hands on lab
16:45-17:30 2275 Migrating to the IBM MQ Appliance
Thursday
08:30-09:15 3420 MQ Clustering
2931 Business agility with self service MQ
09:30-10:15 3479 MQ z/OS clusters and shared queue
3450 Optimising MQ applications
2849 Messaging feedback roundtable
10:30-11:15 3465 MQ Appliance high availability
3481 MQ z/OS messaging connectivity
11:30-12:15 3474 Active-active messaging
3537 Monitoring and managing MQ
3425 MQ publish/subscribe
Find us at the EXPO:
Hybrid Integration peds 65-68
Check out the Hybrid Messaging sub topic under the
Hybrid Integration topic for further customer and business
partner sessions
Hybrid Messaging from the IBM experts at InterConnect 2016
Sunday
14:30-15:30 6408 Hybrid messaging roadmap (InnerCircle)