Your SlideShare is downloading. ×
IMS Performance Tuning
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

IMS Performance Tuning

3,083
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,083
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
167
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. IMS Performance Tuning Update New York City IMS User Group Steve Nathan Senior Software Engineer IBM Corporation snathan@us.ibm.com © 2005 IBM Corporation
  • 2. IBM Software Group New York City IMS Users Group Important Disclaimer THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: • CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR • ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE. IMS Performance Tuning Update 2
  • 3. IBM Software Group New York City IMS Users Group Agenda What’s New in IMS Performance Tuning? IMS Connect Clients TCP/IP IMS Connect IMS Connect Extensions XCF OTMA Enhanced Performance Tools – IMSPA and IMSPI IMS Performance Tuning Update 3
  • 4. IBM Software Group New York City IMS Users Group What’s New in IMS Performance Tuning? Question: – What’s New in IMS Performance Tuning? Answer: – It’s a TCP/IP world out there and you have to tune IMS and its TCP/IP Interface Tuning IMS in a TCP/IP world is your next big challenge This presentation will show you the steps and tools involved IMS Performance Tuning Update 4
  • 5. IBM Software Group New York City IMS Users Group What’s New in IMS Performance Tuning? SERVER z/OS,WIN, AIX, SUN, … z/OS z/OS IMS Control Center Any Websphere TCP/IP ITRA SCI Websphere App IMS TM MWS Websphere Resource OM Adapter PC ITRA MFS Web XCF IMS Services PC ICON XCF IMS SOAP TCP/IP ICON Gateway RYO TCP/IP Client OTMA XCF TCP/IP Application IMS BRIDGE IMS BRIDGE MQSeries SNA TN3270 MQSeries MQSeries BTAM <V10 VTAM LU1 VTAM LU2 LU 6.1 LU 6.2 End User IMS Performance Tuning Update 5
  • 6. IBM Software Group New York City IMS Users Group IMS Connect Clients IMS Performance Tuning Update 6
  • 7. IBM Software Group New York City IMS Users Group IMS Connect Clients There are several things that can be tuned in the IMS Connect Client – Socket type – Using one Write per message – TCP/IP Parameters – Application Tracing IMS Performance Tuning Update 7
  • 8. IBM Software Group New York City IMS Users Group IMS Connect Clients There is a basic flow of TCP/IP verbs used by the ICON Client – Socket – Establish the environment – Connect – Establish the socket connection to the Host Port • The socket stays open until a Disconnect from the Client or ICON – Write – Send data (could be an ACK/NAK or Resume TPIPE) – Read – Retrieve data (could be an ACK/NAK or Deallocate) – Close - Disconnect IMS Performance Tuning Update 8
  • 9. IBM Software Group New York City IMS Users Group IMS Connect Clients There are three types of connections that determine how these verbs are used – Non-Persistent socket • Close (Disconnect) after each send to the ICON Client • This includes iterations of an IMS conversation • Never use this – except maybe for long running (all day) IMS conversations with little traffic per conversation – Transaction socket • Close (Disconnect) after each transaction or complete conversation • This is the default • This has Connect/Disconnect overhead in the ICON Client and ICON and TCP/IP for each message – Persistent socket • Connect once then multiple write/read iterations for multiple transactions before Close (Disconnect) • Only the FIRST Client ID is used for all messages • This is much more efficient • This will keep the socket open – make sure you have defined enough sockets IMS Performance Tuning Update 9
  • 10. IBM Software Group New York City IMS Users Group IMS Connect Clients Use one send for the entire message – Some customer ICON Clients used two (or more) TCP/IP Writes for one input message, e.g. • One for the LLLL • One for the prefix (IRM) • One for the IMS transaction data • One for the suffix (x’00040000’) – This is bad for performance – If you do not specify NODELAYACK on the Port statement in z/OS then TCP/IP on z/OS will wait for up to 300ms before ACKing each transmission • If you are using multiple Writes be sure to set the NODELAYACK option – Better yet – just use one Write IMS Performance Tuning Update 10
  • 11. IBM Software Group New York City IMS Users Group IMS Connect Clients IMS Connect Client Side TCP/IP Parameters – SO_KEEPALIVE • Activates KEEPALIVE for this socket – SO_LINGER • It is possible for a Socket Close issued by a Client to reach ICON before all the data from the previous Write – ICON gets very upset by this – Affected by TCP/IP “SO_LINGER=Y/N, VALUE=N” – Use SO_LINGER=Y,VALUE=10 > Return to Client Code when ACK is received from the host or after 10 seconds > Socket Close should not lose previously sent data – TCP_NODELAY=DISABLE • Optimizes write performance • Will buffer Writes while waiting for an ACK from the previous Write – These parameters are specified on the SETSOCKOPT statement • All of the other options should be reviewed to make sure they are optimized for the customer environment IMS Performance Tuning Update 11
  • 12. IBM Software Group New York City IMS Users Group IMS Connect Clients Application Tracing and Debugging – The BEST tracing and debugging is that which you do yourself – There is much that can be done • In application code • In exits – The end user should be able to turn on debugging at a message or system level – The debugging indicator should be able to be passed from one system to the next and back again – All debugging displays should also have a timestamp in the form hh:mm:ss.nnnnnn • This level of precision is needed to merge records from various systems • Unfortunately those systems not in the z/OS SYSPLEX will have a different timer IMS Performance Tuning Update 12
  • 13. IBM Software Group New York City IMS Users Group IMS Connect Clients IMS Connect Client – The IMS Connect Client should be able to get a debugging request from the end user or .INI file or some other method – The IMS Connect Client should issue debugging messages for the work that it does • Messages sent • Message received – The IMS Connect Client should pass the debug flag to IMS Connect • In the IRM or user prefix • In the OTMA User Data if the OTMA prefix is pre-built • This is difficult today with the IMS TM Resource Adapter IMS Performance Tuning Update 13
  • 14. IBM Software Group New York City IMS Users Group TCP/IP IMS Performance Tuning Update 14
  • 15. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Traces – For difficult problems IMS Level 2 may request TCP/IP traces • You can use them for your own debugging too – Get to know and be on friendly terms with your TCP/IP systems programmer • Chocolate helps – Set the trace on for only the IMS Connect ports – APAR II12014 gives detailed instructions IMS Performance Tuning Update 15
  • 16. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Traces – The output can be formatted with IPCS // EXEC PGM=IKJEFT01,DYNAMNBR=100,REGION=3500K //IPCSDDIR DD DSN=USER1.IPCS.DUMPDIR,DISP=SHR //SYSPROC DD DSN=SYS1.SBLSCLI0,DISP=SHR //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSTSIN DD * PROFILE MSGID IPCS NOPARM SETDEF DSN(‘USER1.PACKET.TRACE') LIST NOCONFIRM CTRACE COMP(SYSTCPDA) LOCAL FULL END /* IMS Performance Tuning Update 16
  • 17. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP flows – There are 3 basic flows that you need • Connect • Write • Disconnect – A complete list of flows and TCP/IP states is found in APAR II12449 IMS Performance Tuning Update 17
  • 18. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – Connect from IMS Connect Client • TCP/IP will send SYN • TCP/IP on the host will respond ACK SYN • Client TCP/IP will respond ACK – At this point there is a session established – but there is no Clientid • The Clientid is DELDUMMY IMS Performance Tuning Update 18
  • 19. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – Connect – SYN from IMS Connect Client TCP/IP 4930628 TSO1 PACKET 00000004 13:42:15.196771 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=60 Tod Clock : 2008/03/20 13:42:15.196769 Intfx: 4 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 60 ID Number: 9BF5 Fragment : DontFragment Offset: 0 TTL : 59 Protocol: TCP CheckSum: C3C8 FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 11111 () Destination Port: 22222 () Sequence Number : 1421204386 Ack Number: 0 Header Length : 40 Flags: Syn Window Size : 5840 CheckSum: B57D FFFF Urgent Data Pointer: 0000 Option : Max Seg Size Len: 4 MSS: 1460 Option : SACK Permitted Option : Timestamp Len: 10 Value: 25754057 Echo: 00000000 Option : NOP Option : Window Scale OPT Len: 3 Shift: 0 IP Header : 20 000000 4500003C 9BF54000 3B06C3C8 C751521E C68C0002 Protocol Header : 40 000000 888E7D08 54B5DBA2 00000000 A00216D0 B57D0000 020405B4 0402080A 25754057 000020 00000000 01030300 IMS Performance Tuning Update 19
  • 20. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – Connect - ACK SYN from IMS Connect TCP/IP 4930629 TSO1 PACKET 00000004 13:42:15.205221 Packet Trace To Interface : GIG2 Device: QDIO Ethernet Full=60 Tod Clock : 2008/03/20 13:42:15.205221 Intfx: 6 Sequence # : 0 Flags: Pkt Out IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 60 ID Number: A668 Fragment : Offset: 0 TTL : 64 Protocol: TCP CheckSum: 0000 0BAA Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 22222 () Destination Port: 11111 () Sequence Number : 539507775 Ack Number: 1421204387 Header Length : 40 Flags: Ack Syn Window Size : 65535 CheckSum: E02C B7B4 Urgent Data Pointer: 0000 Option : Max Seg Size Len: 4 MSS: 1452 Option : NOP Option : Window Scale OPT Len: 3 Shift: 1 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 7CC0CDA4 Echo: 25754057 IP Header : 20 000000 4500003C A6680000 40060000 C68C0002 C751521E Protocol Header : 40 000000 7D08888E 20283C3F 54B5DBA3 A012FFFF E02C0000 020405AC 01030301 0101080A 000020 7CC0CDA4 25754057 IMS Performance Tuning Update 20
  • 21. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – Connect – ACK from IMS Connect Client TCP/IP 4930630 TSO1 PACKET 00000004 13:42:15.206427 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.206427 Intfx: 4 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: 9BF6 Fragment : DontFragment Offset: 0 TTL : 59 Protocol: TCP CheckSum: C3CF FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 11111 () Destination Port: 22222 () Sequence Number : 1421204387 Ack Number: 539507776 Header Length : 32 Flags: Ack Window Size : 5840 CheckSum: 3D64 FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 25754058 Echo: 7CC0CDA4 IP Header : 20 000000 45000034 9BF64000 3B06C3CF C751521E C68C0002 Protocol Header : 32 000000 888E7D08 54B5DBA3 20283C40 801016D0 3D640000 0101080A 25754058 7CC0CDA4 IMS Performance Tuning Update 21
  • 22. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – VIEWHWS – The DELDUMMY is the IMS Connect Client that just connected – The Status is RECV • IMS Connect is waiting for the first message from the IMS Connect Client for this connection – IMS Connect has been waiting for the first message for 19 seconds – If the first message is not received by the Timeout value (180 seconds) the connection will be disconnected by IMS Connect – IMS Connect will not timeout a RECV session for an established Client 14.06.29 STC02699 R 22,VIEWHWS 14.06.29 STC02699 *28 HWSC0000I *IMS CONNECT READY* ICONA 14.06.29 STC02699 HWSC0001I HWS ID=ICONA Racf=N 14.06.29 STC02699 HWSC0001I Maxsoc=2000 Timeout=18000 14.06.29 STC02699 HWSC0001I Datastore=DS1 Status=ACTIVE 14.06.29 STC02699 HWSC0001I Group=GR1 Member=MB1 14.06.29 STC02699 HWSC0001I Target Member=IMSAOTMA 14.06.29 STC02699 HWSC0001I RACF APPL NAME= 14.06.29 STC02699 HWSC0001I NO ACTIVE IMSPLEX 14.06.29 STC02699 HWSC0001I Port=22222 Status=ACTIVE 14.06.29 Clientid Userid Trancode Status Second ClntPort IP-Address DELDUMMY RECV 19 0 111.111.111.111 CLIENT01 RECV 221 0 111.111.111.112 CLIENT02 CONN 12167 0 111.111.111.113 Total Clients=3 RECV=2 CONN=3 XMIT=0 OTHER=0 IMS Performance Tuning Update 22
  • 23. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – Now the IMS Connect Client sends in the message • LLLL • Prefix • Message • Suffix – This should all be sent with one WRITE – The IMS Connect Client TCP/IP sends ACK PSH • Push (PSH) forces data delivery without waiting for buffers to fill • This is used for interactive traffic • The data will also be delivered to the application on the receiving end with out buffering. – IMS Connect TCP/IP replies ACK • This could have been ACK PSH IMS Performance Tuning Update 23
  • 24. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – IMS Connect Client sends in message – The LLLL is for the whole message • LLLL • llzzIRM • llzzTrancode data • Suffix (x’00040000’) – It was sent with one WRITE – The x’3B’ at offset x’15’ is the timeout value = 20 seconds • IMS Connect will for for 20 seconds for an answer from IMS – The x’00’ at offset x’16’ says this is a Transaction Socket • The IMS Connect will disconnect after each transaction – The x’4001’ at offset x’21’ says this is a CM0 SL1 message IMS Performance Tuning Update 24
  • 25. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging 4930631 TSO1 PACKET 00000004 13:42:15.331775 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=167 Tod Clock : 2008/03/20 13:42:15.331767 Intfx: 4 Sequence # : 0 Flags: Pkt Adj IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 167 ID Number: 9BF7 Fragment : DontFragment Offset: 0 TTL : 59 Protocol: TCP CheckSum: C35B FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 11111 () Destination Port: 22222 () Sequence Number : 1421204387 Ack Number: 539507776 Header Length : 32 Flags: Ack Psh Window Size : 5840 CheckSum: 1C9E FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 25754065 Echo: 7CC0CDA4 IP Header : 20 000000 450000A7 9BF74000 3B06C35B C751521E C68C0002 Protocol Header : 32 000000 888E7D08 54B5DBA3 20283C40 801816D0 1C9E0000 0101080A 25754065 7CC0CDA4 Data : 115 Data Length: 115 000000 00000073 00540000 5CE2C1D4 D7D3F15C |........*SAMPL1* ...s.T........| 000010 00000000 403B0000 C3D3C9C5 D5E3F4F4 |.... ...CLIENT44 ....@;..@@@@@@@@| 000020 00400140 E3D9C1D5 F0F14040 C9D4E2C1 |... TRAN01 IMSA . .@......@@....| 000030 40404040 D3E2C5D9 D4F0F0F1 00000001 | LTERM001.... @@@@@@@@@@@@....| 000040 40404040 40404040 40404040 40404040 | @@@@@@@@@@@@@@@@| 000050 40404040 40404040 00170000 E3D9C1D5 | ....TRAN @@@@@@@@....@@@@| 000060 F0F140C4 C1E3C140 F1F2F3F4 F5F6F700 |01 DATA 1234567. ................| 000070 040000 |... ... | IMS Performance Tuning Update 25
  • 26. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – TCP/IP ACK’s the input message 4930632 TSO1 PACKET 00000004 13:42:15.331867 Packet Trace To Interface : GIG2 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.331867 Intfx: 6 Sequence # : 0 Flags: Pkt Out IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: A66D Fragment : Offset: 0 TTL : 64 Protocol: TCP CheckSum: 0000 0BA7 Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 22222 () Destination Port: 11111 () Sequence Number : 539507776 Ack Number: 1421204502 Header Length : 32 Flags: Ack Window Size : 65477 CheckSum: E024 8CB2 Urgent Data Pointer: 000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 7CC0CE20 Echo: 25754065 IP Header : 20 000000 45000034 A66D0000 40060000 C68C0002 C751521E Protocol Header : 32 000000 7D08888E 20283C40 54B5DC16 8010FFC5 E0240000 0101080A 7CC0CE20 25754065 IMS Performance Tuning Update 26
  • 27. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – VIEWHWS – The Clientid is CLIENT44 • This will also be the TPIPE name in OTMA – The Status is CONN • IMS Connect is waiting for the reply from IMS 14.06.29 STC02699 R 22,VIEWHWS 14.06.29 STC02699 *28 HWSC0000I *IMS CONNECT READY* ICONA 14.06.29 STC02699 HWSC0001I HWS ID=ICONA Racf=N 14.06.29 STC02699 HWSC0001I Maxsoc=2000 Timeout=18000 14.06.29 STC02699 HWSC0001I Datastore=DS1 Status=ACTIVE 14.06.29 STC02699 HWSC0001I Group=GR1 Member=MB1 14.06.29 STC02699 HWSC0001I Target Member=IMSAOTMA 14.06.29 STC02699 HWSC0001I RACF APPL NAME= 14.06.29 STC02699 HWSC0001I NO ACTIVE IMSPLEX 14.06.29 STC02699 HWSC0001I Port=22222 Status=ACTIVE 14.06.29 Clientid Userid Trancode Status Second ClntPort IP-Address CLIENT44 CONN 3 0 111.111.111.111 CLIENT01 RECV 225 0 111.111.111.112 CLIENT02 CONN 12171 0 111.111.111.113 Total Clients=3 RECV=1 CONN=2 XMIT=0 OTHER=0 IMS Performance Tuning Update 27
  • 28. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – IMS Connect returns the reply to the IMS Connect Client • If an OTMA ACK is required it will be indicated in the CSM if the IMS Connect sample exit it being used – The Z1 field in the LLZZ will have the x’20’ bit on • If a custom user exit is being used it will have to indicate that an ACK is required – The TCP/IP flags are ACK PSH IMS Performance Tuning Update 28
  • 29. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging 4930633 TSO1 PACKET 00000004 13:42:15.685386 Packet Trace To Interface : GIG2 Device: QDIO Ethernet Full=149 Tod Clock : 2008/03/20 13:42:15.685384 Intfx: 6 Sequence # : 0 Flags: Pkt Adj Out IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 149 ID Number: A672 Fragment : Offset: 0 TTL : 64 Protocol: TCP CheckSum: 0000 0C0D Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 22222 () Destination Port: 11111 () Sequence Number : 539507776 Ack Number: 1421204502 Header Length : 32 Flags: Ack Psh Window Size : 65477 CheckSum: E085 34D7 Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 7CC0CF79 Echo: 25754065 IP Header : 20 000000 45000095 A6720000 40060000 C68C0002 C751521E Protocol Header : 32 000000 7D08888E 20283C40 54B5DC16 8018FFC5 E0850000 0101080A 7CC0CF79 25754065 Data : 97 Data Length: 97 000000 00490300 E3D9C1D5 F0F14040 40F0F0F0 |....TRAN01 000 .I........@@@...| 000010 F0F0F0F0 F0F5F0F1 F2F3F4F5 F6F7F8F9 |0000050123456789 ................| 000020 F0F1F2F3 F4F5F6F7 F8F9F0F1 F2F3F4F5 |0123456789012345 ................| 000030 F6F7F8F9 F0F1F2F3 F4F5F6F7 F8F9F0F1 |6789012345678901 ................| 000040 F2F3F4F5 F6F7F8F9 F0001820 005CC3E2 |234567890....*CS ...............| 000050 D4D6D2E8 5C000000 01000000 00000000 |MOKY*........... ...............| 000060 00 |. . | IMS Performance Tuning Update 29
  • 30. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – The IMS Connect Client TCP/IP ACK’s the reply message – This is not the OTMA ACK • That will require a separate WRITE by the IMS Connect Client 4930634 TSO1 PACKET 00000004 13:42:15.686630 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.686629 Intfx: 4 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: 9BF8 Fragment : DontFragment Offset: 0 TTL : 59 Protocol: TCP CheckSum: C3CD FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 11111 () Destination Port: 22222 () Sequence Number : 1421204502 Ack Number: 539507873 Header Length : 32 Flags: Ack Window Size : 5840 CheckSum: 3A8B FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 25754088 Echo: 7CC0CF79 IP Header : 20 000000 45000034 9BF84000 3B06C3CD C751521E C68C0002 Protocol Header : 32 000000 888E7D08 54B5DC16 20283CA1 801016D0 3A8B0000 0101080A 25754088 7CC0CF79 All packets check summed correctly No packets required reassembly IMS Performance Tuning Update 30
  • 31. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – VIEWHWS – The Status for CLIENT44 is RECV • IMS Connect is waiting for input from the IMS Connect Client • There is no indication that IMS Connect is waiting for ACK/NAK • There is an indication on the TPIPE in OTMA (WAIT_A) • If the IMS Connect Client tries to send in anything other than ACK/NAK it will be rejected as a protocol error – message HWSP1495E 14.06.29 STC02699 R 22,VIEWHWS 14.06.29 STC02699 *28 HWSC0000I *IMS CONNECT READY* ICONA 14.06.29 STC02699 HWSC0001I HWS ID=ICONA Racf=N 14.06.29 STC02699 HWSC0001I Maxsoc=2000 Timeout=18000 14.06.29 STC02699 HWSC0001I Datastore=DS1 Status=ACTIVE 14.06.29 STC02699 HWSC0001I Group=GR1 Member=MB1 14.06.29 STC02699 HWSC0001I Target Member=IMSAOTMA 14.06.29 STC02699 HWSC0001I RACF APPL NAME= 14.06.29 STC02699 HWSC0001I NO ACTIVE IMSPLEX 14.06.29 STC02699 HWSC0001I Port=22222 Status=ACTIVE 14.06.29 Clientid Userid Trancode Status Second ClntPort IP-Address CLIENT44 RECV 2 0 111.111.111.111 CLIENT01 RECV 228 0 111.111.111.112 CLIENT02 CONN 12174 0 111.111.111.113 Total Clients=3 RECV=2 CONN=1 XMIT=0 OTHER=0 IMS Performance Tuning Update 31
  • 32. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – IMS Connect Client sends in the ACK – The LLLL is for the whole message • LLLL • llzzIRM • Suffix (x’00040000’) – There is no data in the message – just the prefix indicating this is an ACK (or NAK) – The timeout value was set to x’03’ seconds in case there is an error message from the ACK • The IMS Connect Client should then do a READ • It will receive a timeout message after 0.03 seconds • If there was another IOPCB message from the same instance of the transaction it would have been delivered – The flags are ACK PSH IMS Performance Tuning Update 32
  • 33. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging 4930635 TSO1 PACKET 00000004 13:42:15.331775 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=167 Tod Clock : 2008/03/20 13:42:15.687001 Intfx: 4 Sequence # : 0 Flags: Pkt Adj IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 167 ID Number: 9BF7 Fragment : DontFragment Offset: 0 TTL : 59 Protocol: TCP CheckSum: C35B FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 11111 () Destination Port: 22222 () Sequence Number : 1421204387 Ack Number: 539507776 Header Length : 32 Flags: Ack Psh Window Size : 5840 CheckSum: 1C9E FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 25754065 Echo: 7CC0CDA4 IP Header : 20 000000 450000A7 9BF74000 3B06C35B C751521E C68C0002 Protocol Header : 32 000000 888E7D08 54B5DBA3 20283C40 801816D0 1C9E0000 0101080A 25754065 7CC0CDA4 Data : 92 Data Length: 92 000000 0000005C 00540000 5CE2C1D4 D7D3F15C |........*SAMPL1* ...s.T........| 000010 00000000 40030000 C3D3C9C5 D5E3F4F4 |.... ...CLIENT44 ....@;..@@@@@@@@| 000020 004001C1 E3D9C1D5 F0F14040 C9D4E2C1 |...ATRAN01 IMSA . .@......@@....| 000030 40404040 D3E2C5D9 D4F0F0F1 00000001 | LTERM001.... @@@@@@@@@@@@....| 000040 40404040 40404040 40404040 40404040 | @@@@@@@@@@@@@@@@| 000050 40404040 40404040 00040000 | .... | IMS Performance Tuning Update 33
  • 34. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – TCP/IP ACK’s the OTMA ACK message 4930636 TSO1 PACKET 00000004 13:42:15.331867 Packet Trace To Interface : GIG2 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.687012 Intfx: 6 Sequence # : 0 Flags: Pkt Out IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: A66D Fragment : Offset: 0 TTL : 64 Protocol: TCP CheckSum: 0000 0BA7 Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 22222 () Destination Port: 11111 () Sequence Number : 539507776 Ack Number: 1421204502 Header Length : 32 Flags: Ack Window Size : 65477 CheckSum: E024 8CB2 Urgent Data Pointer: 000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 7CC0CE20 Echo: 25754065 IP Header : 20 000000 45000034 A66D0000 40060000 C68C0002 C751521E Protocol Header : 32 000000 7D08888E 20283C40 54B5DC16 8010FFC5 E0240000 0101080A 7CC0CE20 25754065 IMS Performance Tuning Update 34
  • 35. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – IMS Connect will now send the timeout for the ACK – If the IMS Connect Sample Exit is being used this is an “RSM” – Request Status Message • The eyecatcher is *REQSTS* – If a custom user exit is being used it will have to indicate the timeout – The flags are ACK PSH IMS Performance Tuning Update 35
  • 36. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – IMS Connect timeout message 4930637 TSO1 PACKET 00000004 13:42:15.331867 Packet Trace To Interface : GIG2 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.717012 Intfx: 6 Sequence # : 0 Flags: Pkt Out IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: A66D Fragment : Offset: 0 TTL : 64 Protocol: TCP CheckSum: 0000 0BA7 Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 22222 () Destination Port: 11111 () Sequence Number : 539507776 Ack Number: 1421204502 Header Length : 32 Flags: Ack Psh Window Size : 65477 CheckSum: E024 8CB2 Urgent Data Pointer: 000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 7CC0CE20 Echo: 25754065 IP Header : 20 000000 45000034 A66D0000 40060000 C68C0002 C751521E Protocol Header : 32 000000 7D08888E 20283C40 54B5DC16 8010FFC5 E0240000 0101080A 7CC0CE20 25754065 Data : 20 Data Length: 20 000000 00140000 2A524551 5354532A 00000020 |................ ....*REQSTS*...(| 000010 00000003 |.... ...d | IMS Performance Tuning Update 36
  • 37. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flows – The IMS Connect Client should now disconnect •This was a Transaction Socket – This results in 4 interactions between the IMS Connect Client TCP/IP and the IMS Connect TCP/IP • ACK PSH FIN • ACK PSH • ACK PSH FIN • ACK PSH IMS Performance Tuning Update 37
  • 38. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – Disconnect – ACK PSH FIN from IMS Connect Client TCP/IP 4930638 TSO1 PACKET 00000004 13:42:15.866075 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.866074 Intfx: 2 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: D151 Fragment : DontFragment Offset: 0 TTL : 64 Protocol: TCP CheckSum: 71C8 FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 1111 () Destination Port: 2222 () Sequence Number : 1067130745 Ack Number: 1067221926 Header Length : 32 Flags: Ack Psh Fin Window Size : 65349 CheckSum: CC00 FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 3520086E Echo: 35200858 IP Header : 20 000000 45000034 D1514000 400671C8 94AB6A51 94AB6402 Protocol Header : 32 000000 0A870BDF 3F9B1F79 3F9C83A6 8019FF45 CC000000 0101080A 3520086E 35200858 IMS Performance Tuning Update 38
  • 39. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – Disconnect – ACK PSH from IMS Connect TCP/IP 4930639 TSO1 PACKET 00000004 13:42:15.866112 Packet Trace From Interface : GIG2 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.866112 Intfx: 2 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: D152 Fragment : DontFragment Offset: 0 TTL : 64 Protocol: TCP CheckSum: 71C7 FFFF Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 2222 () Destination Port: 1111 () Sequence Number : 1067221926 Ack Number: 1067130746 Header Length : 32 Flags: Ack Psh Window Size : 65286 CheckSum: CC29 FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 3520086E Echo: 3520086E IP Header : 20 000000 45000034 D1524000 400671C7 94AB6402 94AB6A51 Protocol Header : 32 000000 0BDF0A87 3F9C83A6 3F9B1F7A 8018FF06 CC290000 0101080A 3520086E 3520086E IMS Performance Tuning Update 39
  • 40. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – Disconnect – ACK PSH FIN from IMS Connect Client TCP/IP 4930640 TSO1 PACKET 00000004 13:42:15.866314 Packet Trace From Interface : GIG1 Device: QDIO Ethernet Full=52 Tod Clock : 2008/03/20 13:42:15.866314 Intfx: 2 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: D153 Fragment : DontFragment Offset: 0 TTL : 64 Protocol: TCP CheckSum: 71C6 FFFF Source : 111.111.111.111 Destination : 222.222.222.222 TCP Source Port : 1111 () Destination Port: 2222 () Sequence Number : 1067221926 Ack Number: 1067130746 Header Length : 32 Flags: Ack Psh Fin Window Size : 65286 CheckSum: CC28 FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 3520086E Echo: 3520086E IP Header : 20 000000 45000034 D1534000 400671C6 94AB6402 94AB6A51 Protocol Header : 32 000000 0BDF0A87 3F9C83A6 3F9B1F7A 8019FF06 CC280000 0101080A 3520086E 3520086E IMS Performance Tuning Update 40
  • 41. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – Disconnect – ACK PSH from IMS Connect TCP/IP 4930641 TSO1 PACKET 00000004 13:42:15.866399 Packet Trace From Interface : LOOPBACK Device: Loopback Full=52 Tod Clock : 2008/03/20 13:42:15.866398 Intfx: 2 Sequence # : 0 Flags: Pkt IpHeader: Version : 4 Header Length: 20 Tos : 00 QOS: Routine Normal Service Packet Length : 52 ID Number: D154 Fragment : DontFragment Offset: 0 TTL : 64 Protocol: TCP CheckSum: 71C5 FFFF Source : 222.222.222.222 Destination : 111.111.111.111 TCP Source Port : 2222 () Destination Port: 1111 () Sequence Number : 1067130746 Ack Number: 1067221927 Header Length : 32 Flags: Ack Psh Window Size : 65349 CheckSum: CBE9 FFFF Urgent Data Pointer: 0000 Option : NOP Option : NOP Option : Timestamp Len: 10 Value: 3520086E Echo: 3520086E IP Header : 20 000000 45000034 D1544000 400671C5 94AB6A51 94AB6402 Protocol Header : 32 000000 0A870BDF 3F9B1F7A 3F9C83A7 8018FF45 CBE90000 0101080A 3520086E 3520086E IMS Performance Tuning Update 41
  • 42. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – So what can go wrong this this picture? – Here is an example: • IMS Connect Client does multiple WRITEs for one message – One IMS Connect Client used 4 WRITEs > One for the LLLL > One for the prefix > One for the transaction > One for the suffix – This caused 4 round trips through TCP/IP > This is easily seen in the TCP/IP trace > ACK PSH – ACK for each WRITE > If you did not specify NODELAYACKS on the Port statement this will hurt even more – This caused 4 trips through IMS Connect code to build the input message • The IMS Connect Client should always use one WRITE per message IMS Performance Tuning Update 42
  • 43. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – So what can go wrong this this picture? – Here is an example: • TCP/IP does multiple interactions for one message – The IMS Connect Client did use one WRITE – TCP/IP was not tuned properly – TCP/IP sent only a few bytes at a time > This resulted in many round trips through TCP/IP > This resulted in many iterations in IMS Connect code – This is easily seen the TCP/IP trace • You need to consult with TCP/IP support IMS Performance Tuning Update 43
  • 44. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – So what can go wrong this this picture? – Here is an example: • IMS Connect Client does a DISCONNECT after each message even though they are using a Persistent Socket – And then a Connect for the next message • This causes much unnecessary processing in IMS Connect, the IMS Connect Client and TCP/IP • This is easily seen in the TCP/IP trace IMS Performance Tuning Update 44
  • 45. IBM Software Group New York City IMS Users Group TCP/IP Tracing and Debugging TCP/IP Flow – For additional information on TCP/IP diagnosis see the following presentation by Deepak Kohli • www.ibm.com/ims • Presentations • IMS Technical Conference Presentations: Fall 2005 • Session B06 – TCP/IP Traces and Displays for IMS Professionals IMS Performance Tuning Update 45
  • 46. IBM Software Group New York City IMS Users Group IMS Connect IMS Performance Tuning Update 46
  • 47. IBM Software Group New York City IMS Users Group IMS Connect Normally messages should flow through IMS Connect VERY quickly – < 1millisecond – There is no logging – There is no locking There are some things which can be tuned – TCP/IP parameters – Port TCB’s – Datastores – RACF IMS Performance Tuning Update 47
  • 48. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Side TCP/IP Parameters – KEEPALIVEOPTIONS • The time in minutes to send a packet so that a connection does not time out or to see if a TCP/IP client has gone away – PORT • Reserve ports for IMS Connect – Be sure to include the NODELAYACKS parameter – SOMAXCONN • The number of connection requests that can be queued because IMS Connect has not yet issued the accept call – Default is 10 – These parameters are specified in the PROFILE.TCPIP • All of the other options should be reviewed to make sure they are optimized for the customer environment IMS Performance Tuning Update 48
  • 49. IBM Software Group New York City IMS Users Group IMS Connect BPECFG – VERY STRONGLY RECOMMENDED internal trace parameters – These will not hurt performance – There will use a small amount of virtual storage – These will be critical for IMS Level 2 to debug processing and performance problems TRCLEV=(*,HIGH,HWS,PAGES=300) TRCLEV=(*,HIGH,BPE,PAGES=300) IMS Performance Tuning Update 49
  • 50. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG – HWS control card • RACF= (Y/N) - Specifies whether or not ICON should do USERID/Password checking for each input message – There can be a performance problem if RACF statistics are being kept for a USERID – If RACF=Y using standard RACF tuning techniques – IMS Connect does not cache RACF ACEE’s > ICON calls RACF for each message from a new user on a socket > IMS Connect Extensions can do security checking and does cache ACEE’s – It is much better to set the value to N and do RACF checking yourself in the ICON User Message Exit – Even if the value is Y the IMS Connect User Message Exit can set the “Trusted User” flag and tell IMS Connect not to issue the RACF call IMS Performance Tuning Update 50
  • 51. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG (continued) – TCP/IP control card • ECB= Performance enhancement if TCP/IP >= 3.7 (everyone is) – The default is N – Set it to Y • IPV6= Use IPV6 if available (z/OS1.4) – If you are on z/OS 1.4 or higher set this to Y even if you are not on IPV6 – it will improve performance IMS Performance Tuning Update 51
  • 52. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG (continued) – TCP/IP control card (continued) • NODELAY= Specifies (Y) if IMS Connect should use the TCP_NODELAY option for the USS SETSOCKET interface call – Introduced with APAR PK08183 (ICON 9.1) – A value of ‘Y’ will disable the “Nagle Algorithm” which adds a 0.2 second delay before sending subsequent partially filled packets to the end user > Disabling the algorithm could result in increased throughput and decrease output message response time – The default is ‘N’ IMS Performance Tuning Update 52
  • 53. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG (continued) – TCP/IP control card (continued) • PORTID= List of from 1 to 50 ports for ICON to listen on – TCP/IP says that multiple ports are really not needed for performance > But they can be useful for availability – IMS Connect has one TCB per port > Listening on multiple ports will increase ICON multi-threading IMS Performance Tuning Update 53
  • 54. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG (continued) – TCP/IP control card (continued) • PORT=(ID=xxxx,KEEPAV=nn,EDIT=xxxxxxxx) – This is an alternate was of defining Ports to ICON – Introduced in ICON V10 via APAR PK72652 – A Port can not be listed in both PORTID and PORT – There can be multiple PORT statements – one for each Port – up to 50 – ID is the Port number – KEEPAV is the override value, in seconds, for the TCP/IP default keepalive value > This is the value that tells TCP/IP how long to wait before telling IMS Connect that a TCP/IP session has gone away > The TCP/IP default can be very long and IMS Connect will not know that a session as ended IMS Performance Tuning Update 54
  • 55. IBM Software Group New York City IMS Users Group IMS Connect HWSCFG (continued) – Datastore Control Cards • Each Datastore is an OTMA TMEMBER • Each OTMA TMEMBER is a separate TCB • Specifying multiple datastores to the same IMS copy will increase OTMA multi-tasking • The IMS Connect User Message Exit or IMS Connect Extensions can route the messages across the datastores with no impact to the IMS Connect Client IMS Performance Tuning Update 55
  • 56. IBM Software Group New York City IMS Users Group IMS Connect XCF GROUP ICON1 GR1 IMSA OTMANM=IMSAOTMA DS1 MB1 DS2 MB2 MB4 MB5 DS3 XCF GROUP GR2 ICON2 IMSB OTMANM=IMSBOTMA DS1 MB3 DS2 DS3 MB6 IMS Performance Tuning Update 56
  • 57. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Tracing and Debugging – There are 4 IMS tools for tracing and debugging messages in IMS Connect • TCP/IP Packet Trace • The VIEWHWS command • The IMS Connect internal trace – This is not easy to format to get a complete picture – This will wrap when it is full so timing is critical – This is not documented – This can only be used by Level 2 • The IMS Connect Recorder Trace IMS Performance Tuning Update 57
  • 58. IBM Software Group New York City IMS Users Group IMS Connect VIEWHWS Command – The DELDUMMY is an IMS Connect Client that just connected – The Status is RECV • IMS Connect is waiting for the first message from the IMS Connect Client for this connection – IMS Connect has been waiting for the first message for 19 seconds – If the first message is not received by the Timeout value (180 seconds) the connection will be disconnected by IMS Connect – IMS Connect will not timeout a RECV session for an established Client 14.06.29 STC02699 R 22,VIEWHWS 14.06.29 STC02699 *28 HWSC0000I *IMS CONNECT READY* ICONA 14.06.29 STC02699 HWSC0001I HWS ID=ICONA Racf=N 14.06.29 STC02699 HWSC0001I Maxsoc=2000 Timeout=18000 14.06.29 STC02699 HWSC0001I Datastore=DS1 Status=ACTIVE 14.06.29 STC02699 HWSC0001I Group=GR1 Member=MB1 14.06.29 STC02699 HWSC0001I Target Member=IMSAOTMA 14.06.29 STC02699 HWSC0001I RACF APPL NAME= 14.06.29 STC02699 HWSC0001I NO ACTIVE IMSPLEX 14.06.29 STC02699 HWSC0001I Port=22222 Status=ACTIVE 14.06.29 Clientid Userid Trancode Status Second ClntPort IP-Address DELDUMMY RECV 19 0 111.111.111.111 CLIENT01 RECV 221 0 111.111.111.112 CLIENT02 CONN 12167 0 111.111.111.113 Total Clients=3 RECV=2 CONN=3 XMIT=0 OTHER=0 IMS Performance Tuning Update 58
  • 59. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is a sequential data set tracing messages into and out of IMS Connect – It is started and stopped via IMS Connect Commands – If it fills up it stops • It does not wrap – The data is most easily printed with IDCAMS //S1 EXEC PGM=IDCAMS,REGION=4096K //SYSPRINT DD SYSOUT=* //SYSIN DD * PRINT IDS(USER1.RECORDER.TRACE) /* IMS Performance Tuning Update 59
  • 60. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – The first 96 bytes is the prefix 000000 00000000 C9C3D6D5 D9C30052 00000877 15413472 0108224F 00000000 00000000 *....ICONRC.............|........* 000020 C8E6E2E3 C5E2E3F1 C2D3F991 09C5646E C2D3F991 09E57CEE 00000000 00000000 *HWSTEST1BL9..E.>BL9..V@.........* 000040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC9D7C2 *............................*IPB* – The eyecatcher is ICONxx where xx= • RC – message received from ICON Client • SN – message sent to ICON Client • TO – timeout sent to ICON Client • ER – error in processing • AR – input message from Client processed by XML adapter • AE – input message from Client rejected by XML adapter • AX – output message to Client processed by XML adapter IMS Performance Tuning Update 60
  • 61. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – The first 96 bytes is the prefix 000000 00000000 C9C3D6D5 D9C30052 00000877 15413472 0108224F 00000000 00000000 *....ICONRC.............|........* 000020 C8E6E2E3 C5E2E3F1 C2D3F991 09C5646E C2D3F991 09E57CEE 00000000 00000000 *HWSTEST1BL9..E.>BL9..V@.........* 000040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC9D7C2 *............................*IPB* – The time of the record is at offset x’10 • X’HHMMSSTT’ local time – The date of the record is at offset x’14’ • x’CCYYDDDF’ – CC – Century – x’01’ = 21st century – YY – Year – DDD – Julian day – F – Packed sign – The Clientid is at offset x’20’ IMS Performance Tuning Update 61
  • 62. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – The first 96 bytes is the prefix 000000 00000000 C9C3D6D5 D9C30052 00000877 15413472 0108224F 00000000 00000000 *....ICONRC.............|........* 000020 C8E6E2E3 C5E2E3F1 C2D3F991 09C5646E C2D3F991 09E57CEE 00000000 00000000 *HWSTEST1BL9..E.>BL9..V@.........* 000040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC9D7C2 *............................*IPB* – The *IPB at offset x’003C’ indicates the Input Buffer follows 0002E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CD6D7C2 *............................*OPB* – The *OPB at offset x’02FC’ indicates that the Output Buffer follows IMS Performance Tuning Update 62
  • 63. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the input buffer for a message from the IMS Connect Client 000060 000000B2 00500000 5CE2C1D4 D7D3E55C 00000000 00001000 C8E6E2E3 C5E2E3F1 *.....&..*SAMPLE*........HWSTEST1* 000080 00200040 E3D9C1D5 F0F0F0F1 C9C4E2C4 C5E2E3F1 40404040 40404040 E4E2C3D9 *... TRAN0001IMSDEST1 USER* 0000A0 F0F0F0F1 40404040 40404040 5C5C5C5C 5C5C5C5C 005A0000 E3D9C1D5 F0F0F0F1 *0001 ********.!..TRAN0001* 0000C0 40839389 A2A34040 40404040 40404040 40404040 40404040 40404040 40404040 * ..... * 0000E0 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 000100 40404040 40404040 40404040 40400004 00000000 00000000 00000000 00000000 * ..................* 000120 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000140 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000160 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000180 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0001A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0001C0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0001E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* ... – The format is LLLLLLZZirmLLZZtrancode datasuffix – The ‘********’ at offset x’00AC’ is IMS Connect overlaying the input password • This occurs even if the prefix is not an IRM – If this was input from ITRA the OTMA prefix would have been pre-built IMS Performance Tuning Update 63
  • 64. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the output buffer to OTMA in the ITOCRC record 000300 01400000 00000000 00000000 0000A0F0 00000000 00000708 00000000 00010000 *. .............0................* 000320 00480020 00400000 00000000 00000000 00000000 00000000 00000000 00000000 *..... ..........................* 000340 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00004040 *.............................. * 000360 40404040 40400000 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 * ....F...USER0001.. * 000380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0003A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0003C0 00000000 00000000 00000000 00000000 00000100 0000C9C4 E2C4C5E2 E3F10000 *......................IMSDEST1..* 0003E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000400 00000000 00005C5C 5C5C5C5C 5C5C1000 00000000 00004040 40404040 40400000 *......********........ ..* 000420 00000200 00000000 00000000 00004040 40404040 40400000 00000000 00000000 *.............. ..........* 000440 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000460 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000480 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0004A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0004C0 00000000 00000000 00000000 00000000 0000005A 0000E3D9 C1D5F0F0 F0F140C4 *...................!..TRAN0001 D* 0004E0 C1E2C140 40404040 40404040 40404040 40404040 40404040 40404040 40404040 *ATA * 000500 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 000520 40404040 40404040 40404040 00000000 00000000 00000000 00000000 00000000 * ....................* 000540 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000560 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000580 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC5D5C4 *............................*END* – The format is OTMALLZZtrancode data – The ICON OTMA User Data is mapped in macro HWSOMPFX • Record is written before key fields are been valued by IMS Connect • You will have to look in the ITOCSN record or OTMA TPIPE trace to see them IMS Performance Tuning Update 64
  • 65. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the input buffer from OTMA in the ITOCSN message 000060 01800000 0000F9F9 F9F94040 4040A0F0 00000001 00000000 00000000 00010000 *......9999 .0................* 000080 00481020 00404040 40404040 40400000 00000000 00000000 00000000 00000000 *..... ..................* 0000A0 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 *......BL9..V.................. * 0000C0 40404040 40400000 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 * ....F...USER0001.. * 0000E0 51005001 80555555 55555555 55555555 55555555 55555555 55555555 55555555 *..&.............................* 000100 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 000120 555594A4 A0A2ADA1 A7155555 55555555 55550100 0000C9D4 E2C4C5E2 E3F1C8E6 *......................IMSDEST1HW* 000140 E2E3C5E2 E3F1F9F0 F0F14040 4040C2D3 F99109C1 652E0000 00000000 00000000 *STEST19001 BL9..A............* 000160 00002754 CD480000 00000000 00001000 20000000 00004040 40404040 40400000 *...................... ..* 000180 00000200 00000000 00000000 00004040 40404040 40400000 00000000 00000000 *.............. ..........* 0001A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0001C0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0001E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000220 00000000 00000000 00000000 00000000 00000054 0300C940 D3D6E5C5 40C9D4E2 *......................I LOVE IMS* 000240 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 000260 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 000280 40404040 40400000 00000000 00000000 00000000 00000000 00000000 00000000 * ..........................* 0002A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 0002C0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* – The format is OTMALLZZreply – The IMS Connect OTMA User Data is now complete IMS Performance Tuning Update 65
  • 66. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the output buffer to the IMS Connect Client in the ITOCSN message 000300 00540300 C940D3D6 E5C540C9 D4E24040 40404040 40404040 40404040 40404040 *....I LOVE IMS * 000320 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 000340 40404040 40404040 40404040 40404040 40404040 000C0000 5CC3E2D4 D6D2E85C * ....*CSMOKY** 000360 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 000380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* ... 000580 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC5D5C4 *............................*END* – The format is LLZZreplyCSMOKY • This is what is sent from the IMS Connect sample user message exit HWSSMPL0 –There is no LLLL • The output could be anything the client understands • It would be nice to pass back the input prefix for correlation IMS Performance Tuning Update 66
  • 67. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the prefix and input buffer from the ICON Client for a Resume TPIPE 00000 00000000 C9E3D6C3 D9C30052 00000877 14485764 0104212F 00000000 00000000 *....ITOCRC......................* 00020 C3D3C9C5 D5E3F0F1 BB980D5C B650B241 BB980D61 7CCA4E80 00000000 00000000 *CLIENT01...*.&...../@.+.........* 00040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CC9D7C2 *............................*IPB* 00060 0000006C 00500000 5CE2C1D4 D7D3C55C 00000000 42FF0000 C3D3C9C5 D5E3F0F1 *...%.&..*SAMPLE*........CLIENT01* 00080 004001D9 40404040 40404040 C4E2F140 40404040 C3D3C9C5 D5E3F0F1 40404040 *. .R DS1 CLIENT01 * 000A0 40404040 40404040 40404040 40404040 40404040 00140000 40404040 40404040 * .... * – This is the output buffer to OTMA for the Resume TPIPE 002E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CD6D7C2 *............................*OPB* 00300 01100000 28000000 00000000 0000A0E0 00000000 00000000 00000000 00010000 *...............................* 00320 000C0240 C3D3C9C5 D5E3F0F1 006AC614 09024040 40404040 40400903 40404040 *... CLIENT01..F... .. * 00340 40404040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * ............................* 00360 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 00380 00000000 00000000 00000000 00000000 00000000 00000100 0000C4E2 F1404040 *..........................DS1 * • The x’01’ at offset x’0301’ indicates a command • The x’28’ at offset x’0304’ is the Resume TPIPE on HOLDQ command • The TPIPE name is in the State data – offset x’0324’ IMS Performance Tuning Update 67
  • 68. IBM Software Group New York City IMS Users Group IMS Connect IMS Connect Recorder Trace – This is the prefix and start of the input buffer for a Duplicate Client 00000 00000000 C9E3D6C3 E2D50052 00000877 14485757 0104212F 00000000 00000000 *....ITOCSN......................* 00020 C4C5D3C4 E4D4D4E8 BB980D5C A3D50A62 00000000 00000000 BB980D5C A3D52C82 *DELDUMMY...*.N.............*.N..* 00040 00000000 00000000 BB980D5C A4031B02 00010000 00000000 00000000 5CC9D7C2 *...........*................*IPB* 00060 01204000 28010000 00000000 0000A0E0 00000000 00000000 00000000 00010000 *.. ............................* – This is the output buffer for the Duplicate Client • As sent by the IMS Connect sample user message exits • The Return Code and Reason Code are at offset x’0C’ in the output message 002E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 5CD6D7C2 *............................*OPB* 00300 00140000 5CD9C5D8 E2E3E25C 00000008 00000038 00000000 00000000 00000000 *....*REQSTS*....................* IMS Performance Tuning Update 68
  • 69. IBM Software Group New York City IMS Users Group IMS Connect Tracing and Debugging – IMS Connect User Exit • Should do WTO’s of messages received and messages sent – Input from the IMS Connect Client – Output from the exit to IMS Connect to send to OTMA – Input from OTMA – Output to send to the IMS Connect Client • WTO’s should be done for error conditions even if the debug flag is not on • HWSJAVA0 will have to be modified to handle IMS TM Resource Adapter messages IMS Performance Tuning Update 69
  • 70. IBM Software Group New York City IMS Users Group IMS Connect Tracing and Debugging – IMS Connect User Exit • Should pass the debug flag to OTMA in the OTMA User Data • Should be able to handle a debug flag from OTMA for asynchronous output • Can use the EXPREA_UFLAG1 field to pass the debug flag from the READ phase of the message from the IMS Connect Client to the EXPXMT_UFLAG1 in the XMIT phase of the message from OTMA IMS Performance Tuning Update 70
  • 71. IBM Software Group New York City IMS Users Group IMS Connect PK57574 is an important IMS Connect performance APAR – It uses a hash technique to search the list of IMS Connect Clients on a Port – This was a sequential search and there can be 1,000’s of Clients on a Port IMS Performance Tuning Update 71
  • 72. IBM Software Group New York City IMS Users Group IMS Connect There are many ways to design for availability with IMS Connect – Multiple IMS Connects on multiple LPARS can connect to multiple IMS copies on multiple LPARS • The IMS copies can have shared queues and shared databases so that any transaction can run on any IMS copy • The IMS Connect User Exit can be coded to route messages to available IMS copies – IMS Connect supports VIPA and Dynamic VIPA • There are considerations for Resume TPIPE and asynchronous output – IMS Connect will work with SYSPLEX Distributor • Will work fine for CM1 messages and IOPCB output from CM0 • Supports OTMA Supermember for Resume TPIPE and asynchronous output and undeliverable output IMS Performance Tuning Update 72
  • 73. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Performance Tuning Update 73
  • 74. IBM Software Group New York City IMS Users Group IMS Connect Extensions There are 3 IMS tools that are very useful for debugging and tuning in an SOA environment – IMS Connect tools • IMS Connect Extensions (CEX) • BMC Energizer for IMS Connect – Log Analysis tools • IMS Performance Analyzer (IMSPA) • BMC Log Analyzer for IMS – IMS Problem Investigator (IMSPI) All of the IBM tools are all supported by the same IBM partner (Fundi) and they all work together IMS Performance Tuning Update 74
  • 75. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions – IMS Connect Extensions provides features and functions to enhance the usability, control, and operation of IMS Connect • Comprehensive event recording for IMS Connect internal events, which can be used for basic reporting and problem analysis • Allow users to monitor and display IMS Connect activity and utilization in real time • Enhance IMS Connect availability by dynamically managing workloads • Improve system security by providing a greater degree of access control IMS Performance Tuning Update 75
  • 76. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions – Event Collection • IMS Connect Extensions gets control for all significant IMS Connect events • Events are recorded on Active Journals • User sets level of event recording • Active journals archived when full • Archive journals can be used for: – Producing summary and detail activity reports > Almost at the IMS Connect internal trace level – Input to IMS Performance Analyzer for more extensive reporting IMS Performance Tuning Update 76
  • 77. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions – Status monitor • IMS Connect Extensions constantly records statistics for each IMS Connect system – Current Activities – Processing rates – At the end of every 20 seconds • Displayed on the Status Monitor in two views: – System View – Port View IMS Performance Tuning Update 77
  • 78. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions – User Exit management • Connect Extensions allows you to manage user exits without interrupting IMS Connect execution – ADD loads and initializes a new user exit – RELOAD loads a new copy of an existing exit – DELETE logically deletes a user exit – DISABLE prevents an exit from being used – ENABLE enables a previously disabled exit IMS Performance Tuning Update 78
  • 79. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions – Message management • Connect Extensions allows you to manage your messages – Efficient RACF checking and caching of ACEE’s – Choosing available datastores – Message flood management IMS Performance Tuning Update 79
  • 80. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Extensions IMS User Message Exit Connect response time Client receives incoming TXN DS availability Send to Region availability IMS Exit End to End Port Task availability IMS IMS Connect and IMS Receive Exit Message Counts response time from IMS TXN Counts IMS Connect User Message Exit sends IMS response Time response response time Client IMS Performance Tuning Update 80
  • 81. IBM Software Group New York City IMS Users Group IMS Connect Extensions New features in IMS Connect Extensions2.2 (and some via SPE in IMS Connect Extensions 2.1) – Eclipse-based operational management GUI (V2R2 only) – Rules-based routing – Extended status information (V2R2 only) – TCP/IP information (similar to NETSTAT) for active sessions(V2R2 only) – Routing intelligence improved to use IMS flood-warn state – IMS V11 command support (V2R2 only) – IMS V11 event collection (V2R2 only) – Improved management of IMS Connect transaction options – Persistent trace: activate IMS Connect Extensions tracing on every system restart – Verify user service exit function – Batch Command utility refresh of Connect Extensions options – Mixed-case password support – Simplified configuration with setup dialog (V2R2 only) IMS Performance Tuning Update 81
  • 82. IBM Software Group New York City IMS Users Group IMS Connect Extensions Eclipse GUI – New IMS Connect interface: provides all IMS Connect Extensions operational management capabilities – Does not provide definition management (still ISPF only) – Shipped as part of the product libraries – Connects via the same TCP/IP port as the IMS Connect Extensions’ ISPF dialog 82 IMS Performance Tuning Update 82
  • 83. IBM Software Group New York City IMS Users Group IMS Connect Extensions File Option Help ---------------------- Eclipse or ISPF _ Stop IMS 1 link _ View TOKYO sessions _ Stop NY sessions F1 – Help F2 - Scroll Tokyo New York IMS IMS IMS Connect IMS Connect IMS IMS IMS Connect IMS Connect IMS IMS 83 IMS Performance Tuning Update 83
  • 84. IBM Software Group New York City IMS Users Group IMS Performance Tuning Update 84
  • 85. IBM Software Group New York City IMS Users Group 85 IMS Performance Tuning Update 85
  • 86. IBM Software Group New York City IMS Users Group 86 IMS Performance Tuning Update 86
  • 87. IBM Software Group New York City IMS Users Group IMS Performance Tuning Update 87
  • 88. IBM Software Group New York City IMS Users Group IMS Connect Extensions Network Status Information Connection: Client ID . . . . . : DUDCLI01 Connection ID . . . : 0011EFF1 Local address . . . : 172.17.69.25..3941 Remote address . . . : 172.17.69.25..10707 Status: State . . . . . . . : ClosWait Start time . . . . . : 2009-07-21 02.59.36.022797 Last activity . . . : 2009-07-21 02.59.36.045867 Counts: Bytes sent . . . . . : 0 Bytes received . . . : 100 Segments sent . . . : 2 Segments received . : 4 Seg retransmitted . : 0 Round trip time (ms) : 0 Round trip variance : 1124 IMS Performance Tuning Update 88
  • 89. IBM Software Group New York City IMS Users Group 89 IMS Performance Tuning Update 89
  • 90. IBM Software Group New York City IMS Users Group IMS Connect Extensions Rules-Based Routing – The simplest way to gain the benefits of IMS Connect Extensions’ routing – Create rules that, for a given DESTID, determine a primary and fallback collection of candidate datastores – IMS Connect Extensions will balance workload between the datastores in the primary collection – If none of the datastores in the primary collection are available or if all datastores in that collection are in flood, then IMS Connect Extensions spreads workload between the fallback collection – Works for transactional messages, Send Only, Resume TPIPE, Synchronous callout, and Asynchronous callout – Benefits: improved performance, redundancy, better capacity management 90 IMS Performance Tuning Update 90
  • 91. IBM Software Group New York City IMS Users Group IMS Connect Extensions Use generic routing to utilize multiple TCB’s in OTMA 1000msg/s 4x250msg/s IMS Connect OTMA Datastore A TCB Messages DESTID=X Datastore B TCB Datastore C TCB Datastore D TCB IMS Connect IMS Connect OTMA Extensions converts a Datastore E TCB single DESTID to Datastore F TCB multiple datastores Datastore G TCB Datastore H TCB 91 IMS Performance Tuning Update 91
  • 92. IBM Software Group New York City IMS Users Group IMS Connect Extensions Setting Routing Behavior File Menu Help —————————————————————————————————————————————————————————————————————————————— Definitions Option ===> 1 System Definitions Specify definitions for IMS Connect systems 2 User Exits Specify User Message Exits 3 Datastores Define datastore processing options 4 Datastore Groups Group datastores for collective control 5 Affinity Lists Associate datastores for Affinity processing 6 Applications Group transactions for Application control 7 Transactions Define transaction processing options 8 System Groups Group IMS Connect systems for collective control 9 Routing Rules Define rules for message routing 10 Routing Lists Associate datastores for rules based routing Two new definition types have been added to IMS Connect Extensions: one lets you set routing behavior, the other lets you define collections of datastores to use as routing candidates 92 IMS Performance Tuning Update 92
  • 93. IBM Software Group New York City IMS Users Group IMS Connect Extensions Setting Routing Behavior File Menu Settings Help —————————————————————————————————————————————————————————————————————————————— Routing Rules Row 1 from 4 Command ===> Scroll ===> CSR Filter . . . (Blank or pattern) Enter "/" to select action --- Applies to --- Original / Name Description Type Definition Datastore DEF01 Default behavior 1 ALL IMS1 DEF02 Default behavior 2 ALL IMS2 PLEX1RT1 Routing rule for dev plex GROUP DEV IMSDEV HWS1RT1 Override routing rule for HWS1 SYSTEM RAPI04 IMS1 ******************************* Bottom of data ******************************** Routing rules describe which datastores to use as primary targets and fallback targets for a given original datastore (DESTID) A rule that is specific to an IMS Connect system will override a rule with the same condition and message type specified for a group or for all systems; a rule specified for a group will override a rule that applies to all systems. 93 IMS Performance Tuning Update 93
  • 94. IBM Software Group New York City IMS Users Group IMS Connect Extensions Setting Routing Behavior File Menu Settings Help —————————————————————————————————————————————————————————————————————————————— EDIT Routing Rule Command ===> Name . . . . : DEF01 Description . . Default behavior 1 Apply rule to: The rule applies to all systems, 3 1. System . . . . . + when a message specifies the 2. Group . . . . . . + 3. All systems original datastore ‘IMS1’ Condition: Original Datastore . IMS1 (IRM_IMSDestId) Rules specific to a ---- Routing lists ----- system, overwrite / Message types Target + Fallback + Y Send-Receive Transactions RL0001 RL0002 rules for the group, Y Send-Only requests RL0001 RL0002 rules for a group Y Resume TPIPE requests RL0001 RL0002 Y Synchronous call-out resume TPIPE requests RL0001 RL0002 overwrite rules for Y Synchronous call-out responses RL0001 RL0002 all systems Press PF1 for help on the various options You can set different collections of target and fallback datastores for each message type. 94 IMS Performance Tuning Update 94
  • 95. IBM Software Group New York City IMS Users Group File Menu Settings Help —————————————————————————————————————————————————————————————————————————————— EDIT System Definition Command ===> Name . . . . : RAPI04 Description . . NY inventory gateway More: + Console settings: Port number . . . . . 9978 Message recall count 255 Host name . . . . . . LOCALHOST Product features: / Activate IMS Connect Extensions / Activate Mixed Case Passwords / Activate Event Collection Collection level . . . . 4 Log Record number . . A0 (A0-FF) / Activate Publisher API Maximum clients . . . 99 (0-99) Enter "/" to edit Journal Data Set template details Active Data Set 'CEX999.QAREA001.ACTIVE.&ID' Archive Data Set 'CEX888.CEX.JRNLARCH.&JDATE..&TIME' / Activate Commands Activate Access Control Security applid . . . Activate PassTicket Generation / Activate Advanced Features / Activate Pacing Interval count . . . . 2 Warning threshold . . 0 Reject threshold . . 0 / Activate Security Activate ACEE Cache Ageing interval . . . 60 (Min) Activate IMS Connect validation Security class . . . FACILITY / Activate Rules-Based Routing Activate Transaction Routing Define Applications for RAPI04 / Activate Workload Balancing You can determine whether rules- based routing is active for a given You can still use the traditional “transaction system. routing” mechanism. 95 IMS Performance Tuning Update 95
  • 96. IBM Software Group New York City IMS Users Group IMS Connect Extensions Flood-warning and routing – All forms of routing in IMS Connect Extensions will now use Flood Warning to help determine the best datastore for processing a message – Datastores in flood warn state are never used unless there are no other datastores (which are not themselves in flood warn state) available – For rules-based routing, if all of the available datastores in the primary collection are in flood-warning, the fallback collection is scanned for available datastores that are not in flood-warning. If no such datastores are available in the fallback collection, a primary datastore in flood warning is nevertheless used. – For primary routing, a primary datastore in flood-warn is never used. 96 IMS Performance Tuning Update 96
  • 97. IBM Software Group New York City IMS Users Group IMS Connect Extensions IMS Connect Transaction Options – IMS Connect clients can specify transaction parameters that: • Balance between responsiveness, resource usage, and reliability • Handle duplicate client sessions – Problem: • Tweaking and optimizing these values may require modifying every client that can potentially access IMS Connect • Customers may want to temporarily change parameter values for various reasons (peak-capacity, faults, etc) 97 IMS Performance Tuning Update 97
  • 98. IBM Software Group New York City IMS Users Group IMS Connect Extensions Transaction Options File Menu Settings Help ────────────────────────────────────────────────────────────────────────────── EDIT Transaction Command ===> Name . . . . : PAYROLL Description . . Application . . . SIMPLE + Activate Transaction Timer Message timeout . . 00 (default) ACK/NAK timeout . . 00 (default) Activate Transaction Expiration Activate Client ID Cancellation Set the timeout value for messages, as / Activate Transaction Routing well as ACKs and NAKs for a given transaction code (V2R1) Override Application options Route transactions to: Sets transaction expiration and Client ID 1 1. All Datastores cancellation options (V2R2) 2. Datastore . . . . . . + 3. Datastore Group . . . + 4. Affinity List . . . . + Routing Error processing: 1. Use the original datastore in the message request 2. Reject the transaction 98 IMS Performance Tuning Update 98
  • 99. IBM Software Group New York City IMS Users Group XCF IMS Performance Tuning Update 99
  • 100. IBM Software Group New York City IMS Users Group XCF Tuning XCF requires two things – Having enough buffers – Having adequate bandwidth in the coupling facilities or CTC’s Your MVS performance tuners have to do this job If OTMA or IMS Connect detects a temporary XCF buffer shortage they will issue a warning message and try again 08.21.47 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 08.21.48 STC99999 DFS2389I XCF SEND RC=0000000C Reason=00000004 IMS Performance Tuning Update 100
  • 101. IBM Software Group New York City IMS Users Group OTMA IMS Performance Tuning Update 101
  • 102. IBM Software Group New York City IMS Users Group OTMA Tuning RACF is important in an OTMA environment – Standard RACF tuning applies – There are important performance APARs for cleaning up expired ACEE’s • PK73647 – IMS V9 • PK79669 – IMS V10 • PK79670 – IMS V11 – If the OTMA Client ACEE timeout value is too small it can cause performance problems in OTMA • There are commands to force the refresh of a Userid IMS Performance Tuning Update 102
  • 103. IBM Software Group New York City IMS Users Group OTMA OTMA Tracing and Debugging – There are 4 tools for tracing and debugging messages in OTMA • OTMT Table Trace • /DIS commands • OTMA TPIPE trace and DFSERA10 • IMS log records and DFSERA10 IMS Performance Tuning Update 103
  • 104. IBM Software Group New York City IMS Users Group OTMA OTMT Table Trace – /TRA SET ON TABLE OTMT OPTION LOG VOLUME HIGH – This is not easy to format to get meaningful data – This is documented • The documentation does not help – This can only be used by Level 2 IMS Performance Tuning Update 104
  • 105. IBM Software Group New York City IMS Users Group OTMA /DIS OTMA Output (IMS V10) – OTMA is the server – The rest are OTMA clients – The TIB count shows the number of • CM0 messages waiting to be queued in IMS • CM1 messages waiting to process and send their reply – And get an ACK if SL1 – Or contunuing IMS conversations • Orphan TIBS /DIS OTMA GROUP/MEMBER XCF-STATUS USER-STATUS SECURITY TIB INPT SMEM DRUEXIT T/O OTMAZZS -IMSA ACTIVE SERVER FULL -IMSA N/A 0 -CSQ1 ACTIVE ACCEPT TRAFFIC NONE 150 200 -CSQ1 CSQ1DRU0 5 -ICON1 ACTIVE ACCEPT TRAFFIC CHECK 0 5000 SM01 -ICON1 HWSYDRU0 10 IMS Performance Tuning Update 105
  • 106. IBM Software Group New York City IMS Users Group OTMA /DIS TMEMBER xxxxxxxx TPIPE ALL Output – There is no input message count – The ENQCT and DEQCT are only for output messages queued to and dequeued by the client • CM1 messages are not counted at all – They are not queued – CM1 message TPIPE’s are the Port number – CM0 message TPIPE’s are the Client Name /DIS TMEMBER ICON1 TPIPE ALL MEMBER/TPIPE ENQCT DEQCT QCT STATUS CSQ1 -6001 0 0 0 (CM1) -6002 0 0 0 (CM1) -CLIENT01 19 18 1 TRA -CLIENT02 99 98 1 WAIT_A -CLIENT03 15 0 15 IMS Performance Tuning Update 106
  • 107. IBM Software Group New York City IMS Users Group OTMA /DIS TMEMBER xxxxxxxx TPIPE ALL Output – Enhanced in IMS V10 • Displays input message counts – even for CM1 messages!!! – Wraps after 65,535 /DIS TMEMBER ICON1 TPIPE ALL MEMBER/TPIPE ENQCT DEQCT QCT INPCT STATUS SMEM CSQ1 -6001 0 0 0 4956 TRA SM01 (CM1) -6002 0 0 0 65534 (CM1) -CLIENT01 19 18 1 768 TRA -CLIENT02 99 98 1 1234 WAIT_A -CLIENT03 15 0 15 45678 IMS Performance Tuning Update 107
  • 108. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Traces – The OTMA TMEMBER/TPIPE traces record OTMA inputs and outputs in x’6701’ log records • They can be formatted with DFSERA10 using EXITR=DFSERA30 – Only the first segment of a multi-segment message is traced – The ID= in the trace record identifies the record type IMS Performance Tuning Update 108
  • 109. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER Trace – Activated with /TRA SET ON TMEMBER xxxx command • ID=YMEM has 9 possible reasons – Message available from XCF – SRVresynch ACK – Client accepts resynch command from OTMA – SRVresynch NACK – Client rejects resynch command from OTMA – TBresynch received – Client wants to resynch a TPIPE – REPRESYNCH received – Client agrees to OTMA request to resynch a TPIPE – Multiple-segment message being assembled by OTMA – Resume TPIPE received (not HOLD Queue) – Resume TPIPE on Hold Queue received – A NACK has been sent to the client (FORCE – traced even if trace is not on) • ID=YMOM – A NACK has been sent to the client (FORCE – traced even if trace is not on) • ID=YPSI – DFS1283E or DFS1284E has been sent – Bad ACK/NACK received – FORCE • ID=YSND – XCF message sent to OTMA client IMS Performance Tuning Update 109
  • 110. IBM Software Group New York City IMS Users Group OTMA OTMA TPIPE Trace – Activated with /TRA SET ON TMEMBER xxxx TPIPE yyyy (or ALL) command • ID=TIB0 - Input message before DFSYIOE0 • ID=TIB2 - Input message after DFSYIOE0* • ID=QAB0 - CM0 output message before ACK from client (if any) • ID=QAB2 - CM0 output message after ACK (if any) and DFSYIOE0* • ID=SLM0 - CM1 output message before ACK from client (if any) • ID=SLM2 - CM1 output message after ACK (if any) and DFSYIOE0* • ID=MEM0 – Resume TPIPE received (retrieve asynchronous output) • * - TIB2, QAB2, and SLM2 are produced only if there is a DFSYIOE0 exit IMS Performance Tuning Update 110
  • 111. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – YMEM for a CM1 SL0 input message • Note the STCK time at TMAMHCOR+8 (OMHDRCOR+8) INTERNAL TRACE RECORD ID = YMEM SEGNO=00 RECNO = 00000201 TIME 15:41:34.720 DATE 2008.224 MCI PREF 7F5FDDD0 000000 01400000 0000F9F9 F9F94040 4040A0F0 00000001 00000708 00000000 00010000 *. ....9999 .0................* STATE 7F5FDDF0 000000 00480020 00400000 00000000 00000000 00000000 00000000 00000000 00000000 *..... ..........................* 7F5FDE10 000020 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 *......BL9J.V.................. * 7F5FDE30 000040 40404040 40400000 * .. * SECURITY 7F5FDE38 000000 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 00000000 00000000 *..F...USER0001.. ........* 7F5FDE58 000020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 7F5FDE78 000040 SAME AS ABOVE 7F5FDE98 000060 00000000 00000000 0000 *.......... * USER SGM 7F5FDEA2 000000 01000000 C9D4E2C4 C5E2E3F1 C3D3D5E3 F0F0F0F1 F9F0F0F1 40404040 C2D3F991 *....IMSDEST1CLNT00019001 BL9J* 7F5FDEC2 000020 09C1652E 00000000 00000000 00000000 2754CD48 00000000 00000000 10002000 *.A..............................* 7F5FDEE2 000040 00000000 40404040 40404040 00000000 02000000 00000000 00000000 40404040 *.... ................ * 7F5FDF02 000060 40404040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * ............................* 7F5FDF22 000080 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 7F5FDF42 0000A0 TO 7F5FDF82 0000E0 SAME AS ABOVE APPL SGM 7F5FDFA2 000000 005A0000 E3D9C1D5 F0F0F0F1 40C4C1E3 C1404040 40404040 40404040 40404040 *....TRAN0001 DATA * 7F5FDFC2 000020 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 7F5FDFE2 000040 40404040 40404040 40404040 40404040 40404040 40404040 4040 * * MTE 21AA48B0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 01000008 00230003 58000000 00000000 *HWSMTEST ................* 21AA48D0 000020 00000000 00000000 00000000 000007D8 01282BE8 00000000 55030000 00000003 *...............Q...Y............* 21AA48F0 000040 21A120C8 21AA4908 00000000 00000000 D7D3C5E7 21AA4908 *...H............PLEX.... * MCB 21A120C8 000000 00000000 00000000 00000000 2607F168 26081030 2617C028 21A40CA0 25968BD0 *..............1..........U...O..* 21A120E8 000020 0077CC08 80000000 262E24A0 262E24A0 00000000 7FFFFFFF 00000000 00000000 *...................."...........* 21A12108 000040 00000000 00000000 00000000 00000000 00000000 00000000 40404040 40404040 *........................ * 21A12128 000060 00000000 00000065 00000000 00000000 00000000 00000000 *........................ * IMS Performance Tuning Update 111
  • 112. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – TIB0 for a CM1 SL0 input message (page 1) INTERNAL TRACE RECORD ID = TIB0 SEGNO=00 RECNO = 00000204 TIME 15:41:34.741 DATE 2008.224 YTIB 262CE318 000000 E8E3C9C2 E8E3C9C2 00000000 262CE060 26360EB0 F9F0F0F1 40404040 21AA48B0 *YTIBYTIB.......-....9001 ....* 262CE338 000020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 262CE358 000040 00000000 00000000 00000000 00000000 00000000 00000100 00060023 00020100 *................................* 262CE378 000060 00080023 00034840 D5800000 40404040 40404040 C2D3F991 0C334224 2636AD08 *....... N... BL9J........* 262CE398 000080 00000000 E3D9C1D5 F0F0F0F1 00000000 00000000 00000000 00000000 00000000 *....TRAN0001....................* 262CE3B8 0000A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 262CE3D8 0000C0 0000036A 7F5FDD78 2636ADA0 2636ADC0 2636AE08 2636AE72 7F5FDFA2 00000000 *...."¬.................."¬.S....* 262CE3F8 0000E0 0F59808B 00000000 00000000 00000000 00000000 00000000 *........................ * TPIPE 26360EB0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 F9F0F0F1 40404040 D9C9F178 00000000 *HWSMTEST 9001 RI1.....* 26360ED0 000020 00000000 00000000 00000000 21AA48B0 262CE318 26363B10 00000001 00000000 *..................T.............* 26360EF0 000040 80400000 00000000 00000000 00000000 00000000 00000000 26360E58 26360E88 *. .............................H* 26360F10 000060 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 26360F30 000080 00000000 00000000 *........ * MSG PREF 2636AD08 000000 026A0000 0040D580 85C60040 F9F0F0F1 40404040 E3D9C1D5 F0F0F0F1 C2D3F991 *..... N.EF. 9001 TRAN0001BL9J* 2636AD28 000020 0C334224 00000000 262CE060 C8E6E2E3 F1E34040 40404040 40404040 00000000 *...........-HWSMTEST ....* 2636AD48 000040 00000000 00000000 00000000 09100098 00000000 00000000 00000000 00000000 *...............Q................* 2636AD68 000060 00000000 00000000 262CE060 00000000 00000000 00000000 C8E6E2D4 E3C5E2E3 *...........-............HWSMTEST* 2636AD88 000080 40404040 40404040 D7D3C5E7 00000000 00000000 00000000 01400000 0000F9F9 * PLEX............. ....99* 2636ADA8 0000A0 F9F94040 4040A0F0 00000001 00000708 00000000 00010000 00481020 00400000 *99 .0..................... ..* 2636ADC8 0000C0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000C2D3 *..............................BL* 2636ADE8 0000E0 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 40404040 40400000 *9J.V.................. ..* 2636AE08 000100 006AC614 0902C1F1 F5F7F8F4 F2400903 40404040 40404040 51005001 80555555 *..F...USER0001.. ..&.....* 2636AE28 000120 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 2636AE48 000140 55555555 55555555 55555555 55555555 55555555 55555555 555594A4 A0A2ADA1 *..........................MU.S..* 2636AE68 000160 A7155555 55555555 55550100 0000C8E6 E2E3F1D7 C9F1C3D3 D5E3F0F0 F0F1F9F0 *X.............IMSDEST1CLNT000190* 2636AE88 000180 F0F14040 4040C2D3 F99109C1 652E0000 00000000 00000000 00002754 CD480000 *01 BL9J.A....................* 2636AEA8 0001A0 00000000 00001000 20000000 00004040 40404040 40400000 00000200 00000000 *.............. ..........* 2636AEC8 0001C0 00000000 00004040 40404040 40400000 00000000 00000000 00000000 00000000 *...... ..................* 2636AEE8 0001E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 2636AF08 000200 TO 2636AF28 000220 SAME AS ABOVE 2636AF48 000240 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 2636AF68 000260 SAME AS ABOVE IMS Performance Tuning Update 112
  • 113. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – TIB0 for a CM1 SL0 input message (page 2) • YTIB is the control block that anchors the message – Because this is a CM1 message it exists until the IOPCB reply is sent • TPIPE is the TPIPE control block • MSG PREF is the message prefix – The first part is mapped by macro DFSYPRE – The second part is mapped by macro DFSYMSG – these are the OTMA headers • TMEMBER – Member Table Entry – macro DFSYMTE • MCB – Member Control Block – macro DFSYMCB • QTPPCB - PORTION OF TPPCB RESERVED FOR QUEUE MANAGER – macro ICLI QPCBASE=0 I/O BUFF 7F5FDFA2 000000 005A0000 E3D9C1D5 F0F0F0F1 40C4C1E3 C1404040 40404040 40404040 40404040 *....TRAN0001 DATA * 7F5FDFC2 000020 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 7F5FDFE2 000040 40404040 40404040 40404040 40404040 40404040 40404040 4040 * * TMEMBER 21AA48B0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 01000008 00230003 58000000 00000000 *HWSMTEST ................* 21AA48D0 000020 00000000 00000000 00000000 000007D8 01282BE8 00000000 55030000 00000003 *...............Q...Y............* 21AA48F0 000040 21A120C8 21AA4908 00000000 00000000 D7D3C5E7 21AA4908 *...H............PLEX.... * MCB 21A120C8 000000 DE5D8FD0 00000000 00000000 2607F168 26081030 2617C028 21A40CA0 25968BD0 *.)............1..........U...O..* 21A120E8 000020 0077CC08 80000000 262E24A0 262E24CC 00000000 7FFFFFFF 00000000 00000000 *...................."...........* 21A12108 000040 00000000 00000000 00000000 00000000 00000001 00000000 40404040 40404040 *........................ * 21A12128 000060 00000000 00000065 00000000 00000000 00000000 00000000 *........................ * QTPPCB 262CE1D8 000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *............................ * IMS Performance Tuning Update 113
  • 114. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – SLM0 for a CM1 SL0 output message (page 1) INTERNAL TRACE RECORD ID = SLM0 SEGNO=00 RECNO = 0000022B TIME 15:41:35.085 DATE 2008.224 YTIB 262CE318 000000 E8E3C9C2 E8E3C9C2 00000000 262CE060 26360EB0 F9F0F0F1 40404040 21AA48B0 *YTIBYTIB.......-....9001 ....* 262CE338 000020 00000000 20BF9060 00000000 00000000 00000000 00000000 00000000 00000000 *.......-........................* 262CE358 000040 00000000 00000000 00000000 00000000 00000000 00000100 00060023 00020100 *................................* 262CE378 000060 00080023 00034840 D5801080 40404040 40404040 C2D3F991 0C334224 2636AD08 *....... N... BL9J........* 262CE398 000080 00000000 E3D9C1D5 F0F0F0F1 00000000 00000000 00000000 00000000 00000000 *....TRAN0001....................* 262CE3B8 0000A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 262CE3D8 0000C0 0000036A 7F5FDD78 2636ADA0 2636ADC0 2636AE08 2636AE72 7F5FDFA2 21A16048 *...."¬.................."¬.S..-.* 262CE3F8 0000E0 0F59808B C2D3F991 0FD78F2B 00000000 00000000 00000000 *....BL9J.P.............. * TPIPE 26360EB0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 F9F0F0F1 40404040 D9C9F178 00000000 *HWSMTEST 9001 RI1.....* 26360ED0 000020 00000000 00000000 00000000 21AA48B0 262CE318 26363B10 00000001 00000000 *..................T.............* 26360EF0 000040 80400000 00000000 00000001 00000000 00000000 00000000 26360E58 26360E88 *. .............................H* 26360F10 000060 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 26360F30 000080 00000000 00000000 *........ * MSG PREF 2636AD08 000000 026A0000 0040D580 85C60040 F9F9F9F9 40404040 E3D9C1D5 F0F0F0F1 C2D3F991 *..... N.EF. 9999 TRAN0001BL9J* 2636AD28 000020 0C334224 80000000 262CE060 C8E6E2E3 F1E34040 40404040 40404040 C9D4E3C5 *...........-HWSMTEST IMTE* 2636AD48 000040 40404040 00000000 00000000 09100098 00000000 00000000 00000000 00000000 * ...........Q................* 2636AD68 000060 00000000 00000000 262CE060 00000000 00000000 C9D4E3C5 C8E6E2D4 E3C5E2E3 *...........-........IMTEHWSMTEST* 2636AD88 000080 40404040 40404040 D7D3C5E7 00000000 00000000 00000000 01400000 0000F9F9 * PLEX............. ....99* 2636ADA8 0000A0 F9F94040 4040A0F0 00000001 00000708 00000000 00010000 00481020 00404040 *99 .0..................... * 2636ADC8 0000C0 40404040 40400000 00000000 00000000 00000000 00000000 00000000 0000C2D3 * ........................BL* 2636ADE8 0000E0 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 40404040 40400000 *9J.V.................. ..* 2636AE08 000100 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 51005001 80555555 *..F...USER0001.. ..&.....* 2636AE28 000120 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 2636AE48 000140 55555555 55555555 55555555 55555555 55555555 55555555 555594A4 A0A2ADA1 *..........................MU.S..* 2636AE68 000160 A7155555 55555555 55550100 0000C9D4 E2C4C5E2 E2F1C3D3 D5E3F0F0 F0F1F9F0 *X.............IMSDEST1CLNT000190* 2636AE88 000180 F0F14040 4040C2D3 F99109C1 652E0000 00000000 00000000 00002754 CD480000 *01 BL9J.A....................* 2636AEA8 0001A0 00000000 00001000 20000000 00004040 40404040 40400000 00000200 00000000 *.............. ..........* 2636AEC8 0001C0 00000000 00004040 40404040 40400000 00000000 00000000 00000000 00000000 *...... ..................* 2636AEE8 0001E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 2636AF08 000200 TO 2636AF28 000220 SAME AS ABOVE 2636AF48 000240 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 2636AF68 000260 SAME AS ABOVE IMS Performance Tuning Update 114
  • 115. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – SLM0 for a CM1 SL0 output message (page 2) I/O BUFF 2637415A 000000 00540300 C940D3D6 E5C540C9 D4E24040 40404040 40404040 40404040 40404040 *....I LOVE IMS * 2637417A 000020 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 2637419A 000040 40404040 40404040 40404040 40404040 40404040 * * TMEMBER 21AA48B0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 01000008 00230003 58000000 00000000 *HWSMTEST ................* 21AA48D0 000020 00000000 00000000 00000000 000007D8 01282BE8 00000000 55030000 00000003 *...............Q...Y............* 21AA48F0 000040 21A120C8 21AA4908 00000000 00000000 D7D3C5E7 21AA4908 *...H............PLEX.... * MCB 21A120C8 000000 DE5D8FD0 00000000 00000000 2607F168 26081030 2617C028 21A40CA0 25968BD0 *.)............1..........U...O..* 21A120E8 000020 0077CC08 80000000 262E24A0 262E24CC 00000000 7FFFFFFF 00000000 00000000 *...................."...........* 21A12108 000040 00000000 00000000 00000000 00000000 00000001 00000000 40404040 40404040 *........................ * 21A12128 000060 00000000 00000065 00000000 00000000 00000000 00000000 *........................ * QTPPCB 20BF90FC 000000 04000018 25F690C8 00000000 262CE288 03D00054 00420080 25F69048 *.....6.H......SH.........6.. * IMS Performance Tuning Update 115
  • 116. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – YSND for a CM1 SL0 output message INTERNAL TRACE RECORD ID = YSND SEGNO=00 RECNO = 0000022E TIME 15:41:35.088 DATE 2008.224 MCI PREF 219EEF6A 000000 01800000 0000F9F9 F9F94040 4040A0F0 00000001 00000000 00000000 00010000 *......9999 .0................* STATE 26373FA8 000000 00481020 00404040 40404040 40400000 00000000 00000000 00000000 00000000 *..... ..................* 26373FC8 000020 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 *......BL9J.V.................. * 26373FE8 000040 40404040 40400000 * .. * SECURITY 26373FF0 000000 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 51005001 80555555 *..F...USER0001.. ..&.....* 26374010 000020 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 26374030 000040 55555555 55555555 55555555 55555555 55555555 55555555 555594A4 A0A2ADA1 *..........................MU.S..* 26374050 000060 A7155555 55555555 5555 *X......... * USER SGM 2637405A 000000 01000000 C9D4E2D4 C5E2E3F1 C3D3D5E3 F0F0F0F1 F9F9F9F9 40404040 C2D3F991 *....IMSDEST1CLNT00019999 BL9J* 2637407A 000020 09C1652E 00000000 00000000 00000000 2754CD48 00000000 00000000 10002000 *.A..............................* 2637409A 000040 00000000 40404040 40404040 00000000 02000000 00000000 00000000 40404040 *.... ................ * 263740BA 000060 40404040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * ............................* 263740DA 000080 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 263740FA 0000A0 TO 2637413A 0000E0 SAME AS ABOVE APPL SGM 2637415A 000000 00540300 C940D3D6 E5C540C9 D4E24040 40404040 40404040 40404040 40404040 *....I LOVE IMS * 2637417A 000020 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040 * * 2637419A 000040 40404040 40404040 40404040 40404040 40404040 * * MTE 21AA48B0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 01000008 00230003 58000000 00000000 *HWSMTEST ................* 21AA48D0 000020 00000000 00000000 00000000 000007D8 01282BE8 00000000 55030000 00000003 *...............Q...Y............* 21AA48F0 000040 21A120C8 21AA4908 00000000 00000000 D7D3C5E7 21AA4908 *...H............PLEX.... * MCB 21A120C8 000000 DE5D8FD0 00000000 00000000 2607F168 26081030 2617C028 21A40CA0 25968BD0 *.)............1..........U...O..* 21A120E8 000020 0077CC08 80000000 262E24A0 262E24CC 00000000 7FFFFFFF 00000000 00000000 *...................."...........* 21A12108 000040 00000000 00000000 00000000 00000000 00000001 00000000 40404040 40404040 *........................ * 21A12128 000060 00000000 00000065 00000000 00000000 00000000 00000000 *........................ * IMS Performance Tuning Update 116
  • 117. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – YSND for a CM1 SL0 Commit Confirm after syncpoint is complete – IMS Connect will not send the response to the IMS Connect Client (ITOCSN) until this message is received INTERNAL TRACE RECORD ID = YSND SEGNO=00 RECNO = 00000232 TIME 15:41:35.094 DATE 2008.224 MCI PREF 219EE236 000000 01080080 0000F9F9 F9F94040 4040A0E0 00000001 00000000 00000000 00010000 *......9999 ..................* STATE 2636ADC0 000000 00481020 00404040 40404040 40400000 00000000 00000000 00000000 00000000 *..... ..................* 2636ADE0 000020 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 00000000 00004040 *......BL9J.V.................. * 2636AE00 000040 40404040 40400000 * .. * SECURITY 2636AE08 000000 006AC614 0902E4E2 C5D9F0F0 F0F10903 40404040 40404040 51005001 80555555 *..F...USER0001.. ..&.....* 2636AE28 000020 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 2636AE48 000040 55555555 55555555 55555555 55555555 55555555 55555555 555594A4 A0A2ADA1 *..........................MU.S..* 2636AE68 000060 A7155555 55555555 5555 *X......... * USER SGM 2636AE72 000000 01000000 C9D4E2C4 C5E2E3F1 C3D3D5E3 F0F0F0F1 F9F9F9F9 40404040 C2D3F991 *....IMSDEST1CLNT00019999 BL9J* 2636AE92 000020 09C1652E 00000000 00000000 00000000 2754CD48 00000000 00000000 10002000 *.A..............................* 2636AEB2 000040 00000000 40404040 40404040 00000000 02000000 00000000 00000000 40404040 *.... ................ * 2636AED2 000060 40404040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * ............................* 2636AEF2 000080 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 2636AF12 0000A0 TO 2636AF52 0000E0 SAME AS ABOVE MTE 21AA48B0 000000 C8E6E2D4 E3C5E2E3 40404040 40404040 01000008 00230003 58000000 00000000 *HWSMTEST ................* 21AA48D0 000020 00000000 00000000 00000000 000007D8 01282BE8 00000000 55030000 00000003 *...............Q...Y............* 21AA48F0 000040 21A120C8 21AA4908 00000000 00000000 D7D3C5E7 21AA4908 *...H............PLEX.... * MCB 21A120C8 000000 DE5D8FD0 00000000 00000000 2607F168 26081030 2617C028 21A40CA0 25968BD0 *.)............1..........U...O..* 21A120E8 000020 0077CC08 80000000 262E24A0 262E24CC 00000000 7FFFFFFF 00000000 00000000 *...................."...........* 21A12108 000040 00000000 00000000 00000000 00000000 00000001 00000000 40404040 40404040 *........................ * 21A12128 000060 00000000 00000065 00000000 00000000 00000000 00000000 *........................ * IMS Performance Tuning Update 117
  • 118. IBM Software Group New York City IMS Users Group OTMA OTMA TMEMBER/TPIPE Trace Example – Other flows might include OTMA Client ACKs, OTMA ACKs, YQABs for CM0 messages and other things – You need to know the basic flows for an OTMA client to interpret what is happening and what may have gone wrong – Practice when there is no problem to make sure you can explain the records that you see – Keep these “good” flows for comparison to when you are having a problem IMS Performance Tuning Update 118
  • 119. IBM Software Group New York City IMS Users Group OTMA IMS log records – IMS log records can be a good source of information for SOA flows – The key thing to know is that the OTMA prefix is stored in the IMS log prefix for the x’01’ (and x’03’ and some x’31’) log records • It is the x’87’ message segment item type • It is mapped by two macros – DFSYPRE for the first part of the segment item – DFSYMSG for the second part of the segment item IMS Performance Tuning Update 119
  • 120. IBM Software Group New York City IMS Users Group OTMA This is the start of the x’01’ log record – The x’87’ APPC (OTMA) segment item starts at x’0090’ with the LL of x’026A’ – The OTMA prefix (DFSYMSG) starts at offset x’0128’ 01 RECORD - 2008-08-11 13:41:34.744955 UTC 00000000 000000 03E00000 01D18194 04000016 04000016 03D01001 C9D4E2F1 40404040 C2D3F991 *.....JAM............IMS1 BL9J* 00000020 000020 0F3BFC2B C9D4E2F1 40404040 C2D3F991 0F3BFC2B 80000100 00000000 00000000 *....IMS1 BL9J................* 00000040 000040 00408100 88000000 00000000 00000000 00010000 00000000 00000000 00000001 *. A.H...........................* 00000060 000060 FDFFFFFF 262CE060 E3D9C1D5 F0F0F0F1 00000000 00000000 00000000 00000000 *.......-TRAN0001................* 00000080 000080 00108600 0350F400 00000000 00000000 026A8700 0040D580 85C60040 F9F9F9F9 *..F..&4...........G.. N.EF. 9999* 000000A0 0000A0 40404040 E3D9C1D5 F0F0F0F1 C2D3F991 0C334224 00000000 262CE060 C8E6E2E3 * TRAN0001BL9J...........-HWST* 000000C0 0000C0 F1E34040 40404040 40404040 C9D4E2F1 40404040 00000000 00000000 09100098 *1T IMS1 ...........Q* 000000E0 0000E0 00000000 00000000 00000000 00000000 00000000 00000000 262CE060 00000000 *...........................-....* 00000100 000100 00000000 C9D4E2F1 C8E6E2D4 E3C5E2E3 40404040 40404040 D7D3C5E7 00000000 *....IMS1HWSMTEST PLEX....* 00000120 000120 00000000 00000000 01400000 0000F9F9 F9F94040 4040A0F0 00000001 00000708 *......... ....9999 .0........* 00000140 000140 00000000 00010000 00481020 00400000 00000000 00000000 00000000 00000000 *............. ..................* 00000160 000160 00000000 00000000 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 *..............BL9J.V............* 00000180 000180 00000000 00004040 40404040 40400000 006AC614 0902E4E2 C5D9F0F0 F0F10903 *...... ....F...USER0001..* 000001A0 0001A0 40404040 40404040 51005001 80555555 55555555 55555555 55555555 55555555 * ..&.....................* 000001C0 0001C0 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 000001E0 0001E0 55555555 55555555 555594A4 A0A2ADA1 A7155555 55555555 55550100 0000C9D4 *..........MU.S..X.............IM* 00000200 000200 E2C4C5E2 E3F1C3D3 D5E3F0F0 F0F1F9F0 F0F14040 4040C2D3 F99109C1 652E0000 *SDEST1CLNT00019001 BL9J.A....* 00000220 000220 00000000 00000000 00002754 CD480000 00000000 00001000 20000000 00004040 *.............................. * 00000240 000240 40404040 40400000 00000200 00000000 00000000 00004040 40404040 40400000 * ................ ..* 00000260 000260 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 00000280 000280 TO 000002C0 0002C0 SAME AS ABOVE 000002E0 0002E0 00000000 00000000 00000000 00000000 00000000 00000000 00000016 8800C1F1 *............................H.A1* IMS Performance Tuning Update 120
  • 121. IBM Software Group New York City IMS Users Group OTMA How do you find the x’01’ record? – If you have a TMEMBER/TPIPE trace you can find the STCK in the TMAMHCOR field (for IMS Connect and MQ – not OTMA C/I) – Then use DFSERA10 to find all records with that STCK • OPTION PRINT PARM=(DATA=X'C2D3F99109E51DEE'),E=DFSERA70 01 RECORD - 2008-08-11 13:41:34.744955 UTC 00000000 000000 03E00000 01D18194 04000016 04000016 03D01001 C9D4E2F1 40404040 C2D3F991 *.....JAM............IMS1 BL9J* 00000020 000020 0F3BFC2B C9D4E2F1 40404040 C2D3F991 0F3BFC2B 80000100 00000000 00000000 *....IMS1 BL9J................* 00000040 000040 00408100 88000000 00000000 00000000 00010000 00000000 00000000 00000001 *. A.H...........................* 00000060 000060 FDFFFFFF 262CE060 E3D9C1D5 F0F0F0F1 00000000 00000000 00000000 00000000 *.......-TRAN0001................* 00000080 000080 00108600 0350F400 00000000 00000000 026A8700 0040D580 85C60040 F9F9F9F9 *..F..&4...........G.. N.EF. 9999* 000000A0 0000A0 40404040 E3D9C1D5 F0F0F0F1 C2D3F991 0C334224 00000000 262CE060 C8E6E2E3 * TRAN0001BL9J...........-HWST* 000000C0 0000C0 F1E34040 40404040 40404040 C9D4E2F1 40404040 00000000 00000000 09100098 *1T IMS1 ...........Q* 000000E0 0000E0 00000000 00000000 00000000 00000000 00000000 00000000 262CE060 00000000 *...........................-....* 00000100 000100 00000000 C9D4E2E1 C8E6E2D4 E3C5E2E3 40404040 40404040 D7D3C5E7 00000000 *....IMS1HWSMTEST PLEX....* 00000120 000120 00000000 00000000 01400000 0000F9F9 F9F94040 4040A0F0 00000001 00000708 *......... ....9999 .0........* 00000140 000140 00000000 00010000 00481020 00400000 00000000 00000000 00000000 00000000 *............. ..................* 00000160 000160 00000000 00000000 00000000 0000C2D3 F99109E5 1DEE0000 00000000 00000000 *..............BL9J.V............* 00000180 000180 00000000 00004040 40404040 40400000 006AC614 0902E4E2 C5D9F0F0 F0F10903 *...... ....F...USER0001..* 000001A0 0001A0 40404040 40404040 51005001 80555555 55555555 55555555 55555555 55555555 * ..&.....................* 000001C0 0001C0 55555555 55555555 55555555 55555555 55555555 55555555 55555555 55555555 *................................* 000001E0 0001E0 55555555 55555555 555594A4 A0A2ADA1 A7155555 55555555 55550100 0000C9D4 *..........MU.S..X.............IM* 00000200 000200 E2C4C5E2 E3F1C3D3 D5E3F0F0 F0F1F9F0 F0F14040 4040C2D3 F99109C1 652E0000 *SDEST1CLNT00019001 BL9J.A....* 00000220 000220 00000000 00000000 00002754 CD480000 00000000 00001000 20000000 00004040 *.............................. * 00000240 000240 40404040 40400000 00000200 00000000 00000000 00004040 40404040 40400000 * ................ ..* 00000260 000260 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *................................* 00000280 000280 TO 000002C0 0002C0 SAME AS ABOVE 000002E0 0002E0 00000000 00000000 00000000 00000000 00000000 00000000 00000016 8800C1F1 *............................H.A1* IMS Performance Tuning Update 121
  • 122. IBM Software Group New York City IMS Users Group OTMA The Source and Destination fields at offset x’60’ and x’68’ are very important for OTMA input (x’01’), message switch (x’03’) and output (x’03’) log records – For input messages and “synchronous” message switch records the Source is x’FDFFFFFFxxxxxxxx’ • xxxxxxxx is the address of an OTMA CLB control block associated with a TIB • The CLB points to an OTMA CTB • The CTB points to an OTMA CNT • Use macro ICLI CLBBASE=0,CTBBASE=0,CNTBASE=0 to map them • You need a dump of the IMS Control Region to see this data 01 RECORD - 2008-08-11 13:41:34.744955 UTC 00000000 000000 03E00000 01D18194 04000016 04000016 03D01001 C9D4E2F1 40404040 C2D3F991 *.....JAM............IMS1 BL9J* 00000020 000020 0F3BFC2B C9D4E2F1 40404040 C2D3F991 0F3BFC2B 80000100 00000000 00000000 *....IMS1 BL9J................* 00000040 000040 00408100 88000000 00000000 00000000 00010000 00000000 00000000 00000001 *. A.H...........................* 00000060 000060 FDFFFFFF 262CE060 E3D9C1D5 F0F0F0F1 00000000 00000000 00000000 00000000 *.......-TRAN0001................* IMS Performance Tuning Update 122
  • 123. IBM Software Group New York City IMS Users Group OTMA The Source and Destination fields at offset x’60’ and x’68’ are very important for OTMA input (x’01’), message switch (x’03’) and output (x’03’) log records – For “asynchronous” message switch records the Source is x’FEFFFFFFxxxxxxxx’ 01 RECORD - 2008-08-11 13:41:34.744955 UTC 00000000 000000 03E00000 01D18194 04000016 04000016 03D01001 C9D4E2F1 40404040 C2D3F991 *.....JAM............IMS1 BL9J* 00000020 000020 0F3BFC2B C9D4E2F1 40404040 C2D3F991 0F3BFC2B 80000100 00000000 00000000 *....IMS1 BL9J................* 00000040 000040 00408100 88000000 00000000 00000000 00010000 00000000 00000000 00000001 *. A.H...........................* 00000060 000060 FEFFFFFF 263CF060 E3D9C1D5 F0F0F0F2 00000000 00000000 00000000 00000000 *.......-TRAN0002................* IMS Performance Tuning Update 123
  • 124. IBM Software Group New York City IMS Users Group OTMA IMS log records – Q. Now that you have the x’01’ log record what do you do? – A. Using the IMS Unit-of-Work (UOW) and Recovery Token (RT) find the rest of the log records for this message • This is standard IMS log record analysis – See the following presentation by Deepak Kohli • www.ibm.com/ims • Presentations • IMS Technical Conference Presentations: Fall 2005 • Session B66 – Using IMS Log Records for IMS Diagnosis – Part 1 & 2 IMS Performance Tuning Update 124
  • 125. IBM Software Group New York City IMS Users Group OTMA Once you have the UOW(s) and Recovery Token(s) associated with an OTMA message you can extract all of the records with one execution of DFSERA10 CONTROL CNTL STOPAFT=EOF OPTION PRINT PARM=(DATA=X'C2D3F9910F3BFC2B'),E=DFSERA70 OPTION PRINT PARM=(DATA=X'C2D3F9916F61BF2C'),E=DFSERA70 OPTION PRINT PARM=(DATA=X'0000000500000000'),E=DFSERA70 OPTION PRINT PARM=(DATA=X'0000000600000000'),E=DFSERA70 OPTION PRINT O=5,T=X,V=67D0,L=2,COND=M,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=LA3A,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=LA3B,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=MEM0,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=QAB0,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=QAB2,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=SLM0,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=SLM2,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=SND0,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=TIB0,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=TIB2,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=YMEM,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=YMOM,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=YPSI,L=4,COND=E,E=DFSERA30 OPTION PRINT O=5,T=X,V=6701,L=2,COND=M,E=DFSERA30 OPTION PRINT O=9,T=C,V=YSND,L=4,COND=E,E=DFSERA30 IMS Performance Tuning Update 125
  • 126. IBM Software Group New York City IMS Users Group OTMA /TRA SET ON PROGRAM xxxx will give you a trace of IMS TM calls – This trace can be very useful when debugging CM1/CM0 problems – These are written as x’6701’ log records • Type LA3A is when the call is first issued – You can see if the ALTPCB was Express or non-Express • Type LA3B is after the return from the call – So you can check the status in the IO or ALT PCB – The PST control block is also dump as part of the trace record • At offset x’1C0’ and x’290’ in the PST you can clearly see the call function – GU, GN, CHNG, ISRT, PURG etc. • The problem is that this is not always the call that was issued for this trace record!!! • You must use the encoded function at offset x’1C4’ – These codes are documented in the FUNBASE section of the IDLI macro > IDLI FUNBASE=0 (see next page) • This is an example where the characters are ISRT but the function is x’01’(GU) - 0001C0 C9E2D9E3 0140BA07 IMS Performance Tuning Update 126
  • 127. IBM Software Group New York City IMS Users Group OTMA *********************************************************************** 02047000 *********************************************************************** 02048000 ** ** 02049000 ** F U N C T I O N C O D E S D S E C T ** 02050000 ** ** 02051000 *********************************************************************** 02052000 *********************************************************************** 02053000 FUNCNOTV EQU X'00' FUNCTION IS NOT DEFINED. @BA36202 02056000 FUNCGU EQU X'01' ENCODED FUNCTION FOR 'GU' CALL @BA36202 02057000 FUNCGN EQU X'03' ENCODED FUNCTION FOR 'GN' CALL @BA36202 02058000 FUNCGNP EQU X'04' ENCODED FUNCTION FOR 'GNP' CALL @BA36202 02059000 FUNCGUP EQU X'05' FUNCTION IS NOT DEFINED. @BA36202 02060000 FUNCGND EQU X'06' FUNCTION IS NOT DEFINED. @BA36202 02061000 FUNCGNX EQU X'08' FUNCTION IS NOT DEFINED. @BA36202 02062000 FUNCGP EQU X'0C' FUNCTION IS NOT DEFINED. @BA36202 02063000 FUNCGBT EQU X'0D' FUNCTION IS NOT DEFINED. @BA36202 02064000 FUNCGB EQU X'0E' FUNCTION IS NOT DEFINED. @BA36202 02065000 FUNCGL EQU X'0F' FUNCTION IS NOT DEFINED. @BA36202 02066000 FUNCGHU EQU X'01' ENCODED FUNCTION FOR 'GHU' CALL @BA36202 02067000 FUNCGHN EQU X'03' ENCODED FUNCTION FOR 'GHN' CALL @BA36202 02068000 FUNCGHNP EQU X'04' ENCODED FUNCTION FOR 'GHNP' CALL @BA36202 02069000 FUNCGHNX EQU X'08' FUNCTION IS NOT DEFINED. @BA36202 02070000 FUNCGHP EQU X'0C' FUNCTION IS NOT DEFINED. @BA36202 02071000 FUNCGHBT EQU X'0D' FUNCTION IS NOT DEFINED. @BA36202 02072000 FUNCGHB EQU X'0E' FUNCTION IS NOT DEFINED. @BA36202 02073000 FUNCGHL EQU X'0F' FUNCTION IS NOT DEFINED. @BA36202 02074000 FUNCDRTY EQU X'20' FUNCTION TYPE FOR DLET OR REPL @BA36202 02075000 FUNCREPL EQU X'21' ENCODED FUNCTION FOR 'REPL' CALL @BA36202 02076000 FUNCDLET EQU X'22' ENCODED FUNCTION FOR 'DLET' CALL @BA36202 02077000 FUNCDLTP EQU X'23' ENCODED FUNCT FOR DLET AND REPL @BA36202 02078000 FUNCISTY EQU X'40' FUNCTION TYPE FOR 'ISRT' CALL @BA36202 02079000 FUNCISRT EQU X'41' ENCODED FUNCTION FOR 'ISRT' CALL @BA36202 02081000 FUNCASRT EQU X'42' ENCODED FUNCTION FOR 'ASRT' CALL @BA36202 02082000 FUNCIHPS EQU X'43' FUNCTION IS NOT DEFINED. @BA36202 02083000 FUNCSETO EQU X'50' ENCODED FUNCTION FOR 'SETO' CALL @KN21195 02083500 FUNCDEQ EQU X'60' ENCODED FUNCTION FOR 'DEQ' CALL @BA36202 02084000 FUNCXRST EQU X'66' ENCODED FUNCTION FOR 'XRST' CALL @BMR0877 02084100 FUNCLOG EQU X'65' ENCODED FUNCTION FOR 'LOG' CALL @BA36202 02085000 FUNCINQY EQU X'67' ENCODED FUNCTION FOR 'INQY' CALL @BA36202 02085010 FUNCRELD EQU X'70' ENCODED FUNCT FOR 'RELOAD' CALL @BA36202 02085050 FUNCSYNC EQU X'79' ENCODED FUNCTION FOR 'SYNC' CALL @BMH0214 02085059 FUNCORCH EQU X'7A' ENCODED FUNCTION FOR 'ORCH' CALL @OLRCH01 02085060 IMS Performance Tuning Update 127
  • 128. IBM Software Group New York City IMS Users Group OTMA IMS log records – If OTMA finds an error it may write an x’67D0’ log record • ILOGREC RECID=67D0 or DFS67D0 • The error will also produce a DFSxxxxx error message – If OTMA detects an error in an application call it may return an AX status code to the application and write an x’67D0’ log record with a return code • This is usually an OTMA exit error • Return codes are shown on the next foil IMS Performance Tuning Update 128
  • 129. IBM Software Group New York City IMS Users Group OTMA X’67D0 return codes for an AX status code Code: Explanation: X'1C' An internal interface error occurred. X'20' DFSYDRU0 overrides exceed the maximum limit. X'24' DFSYDRU0 specifies an invalid destination. X'28' DFSYDRU0 specifies an invalid return code. X'2C' DFSYPRX0 returned an invalid XCF member name. X'30' DFSYPRX0 required an XCF member name that was not returned. X'34' DFSYPRX0 returned an invalid return code. X'38' The destination is in a different client, and the XCF member name from DFSYDRU0 is invalid. X'3C' DFSYDRU0 returned an invalid user data length. IMS Performance Tuning Update 129
  • 130. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging – WTO can be used to display debug messages but you have to be careful – OTMA exits must be reentrant and normal WTO calls modify themselves • You must use WTO MF=(E,WTOMSG) • Control region 0C4’s will result if this is not done – OTMA exits can be called in cross-memory • You must determine if this is the case and branch to WTO • Control region 0F8’s will result if this is not done – This is shown on the next two foils IMS Performance Tuning Update 130
  • 131. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging LA R7,WTOMSG BAL R10,DOWTO DOWTO DS 0H L R5,ADDRWORK ADDRESS OF 512-BYTE WORK AREA LTR R5,R5 IS THERE A WORK AREA? BZR R10 NO - JUST RETURN USING WTOD,R5 TELL ASSEMBLER MVC WTOMSGE,WTOMSGL MOVE MESSAGE MASK MVC WTOTEXT,0(R7) MOVE MESSAGE TEXT EPAR R0 PRIMARY ASID ESAR R1 SECONDARY ASID CLR R0,R1 PRIMARY = SECONDARY? BNE DOWTOX NO - IN XM MODE WTO MF=(E,WTOMSGE) WTO TRACE MESSAGE BR R10 RETURN TO CALLER DOWTOX DS 0H WTO MF=(E,WTOMSGE),LINKAGE=BRANCH BR R10 RETURN TO CALLER DROP R5 TELL ASSEMBLER EJECT IMS Performance Tuning Update 131
  • 132. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging WTOMSGL WTO ‘ ', X ROUTCDE=(11),DESC=(7),MF=L WTOD DSECT WTOMSGE DS 0CL58 REENTRANT WTO MESSAGE DS CL2 TEXT LENGTH DS CL2 MCSFLAGS WTOTEXT DS CL50 DS CL4 ROUTING AND DESCRIPTOR CODES IMS Performance Tuning Update 132
  • 133. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging – DFSYIOE0 can look at the debug flag in the OTMA User Data and WTO messages in and out – This includes any editing this exit may do to the messages IMS Performance Tuning Update 133
  • 134. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging – There are fields in the TSCD which are reserved for user processing • TSCDXFLG – 1 byte • TSCDXPTR – Full word – These fields can be used for communication among the OTMA User Exits • A “dummy” message could be sent to OTMA to turn on Debug for an OTMA exit – DFSYIOE0, DFSYPRX0, dru exits • DFSYIOE0 would see the message and set a bit in TSCDXFLG – The “dummy” message could then be discarded • The OTMA exit would check the bit and issue debug messages IMS Performance Tuning Update 134
  • 135. IBM Software Group New York City IMS Users Group OTMA OTMA User Exit Debugging – DLT0 could be used to issue a CHNG call to a “dummy” destination • e.g. “IOEDBON” to turn on debugging for DFSYIOE0 – DFSYPRX0 would be invoked and set a bit in TSCDXFLG – DFSYPRX0 would also set RC=8 to say this is a non-OTMA destination and the DLT0 call would just get an A1 status code so no real transaction is needed IMS Performance Tuning Update 135
  • 136. IBM Software Group New York City IMS Users Group Enhanced Performance Tools IMS Performance Tuning Update 136
  • 137. IBM Software Group New York City IMS Users Group Enhanced Performance Tools There are 3 IMS tools that are very useful for debugging and tuning in an SOA environment – IMS Connect tools • IMS Connect Extensions (CEX) • BMC Energizer for IMS Connect – Log Analysis tools • IMS Performance Analyzer (IMSPA) • BMC Log Analyzer for IMS – IMS Problem Investigator (IMSPI) All of the IBM tools are all supported by the same IBM partner (Fundi) and they all work together IMS Performance Tuning Update 137
  • 138. IBM Software Group New York City IMS Users Group IMS Performance Analyzer IMSPA produces performance reports from IMS logs, IMS DC Monitor records, and IMS Connect Extensions journals I will just show one of the standard IMSPA reports – There are many more standard reports There are customizable reports IMS Performance Tuning Update 138
  • 139. IBM Software Group New York City IMS Users Group IMS Performance Analyzer Transaction transit report can include time in IMS Connect (PreOTMA Time) and time in OTMA (OTMAproc Time) Without IMS Connect Extensions, IMS before arriving on the IMS queue Connect and OTMA performance cannot be obtained IMS Performance Analyzer 4.1 combined tran list ________________________________ 0LIST0001 Printed at 19:33:38 12Dec2007 Data from 13.57.52 12Dec2007 CON Tran CON Resp PreOTMA OTMAproc IMS Tran InputQ Process Total PostOTMA Start Trancode OTMA Time Time Time Start Time Time IMS Time Time 13.57.52.714 IMSTRANS CONNECT 1.810 0.000 1.803 13.57.54.517 0.000 0.001 0.001 0.006 13.57.52.964 IMSTRANS CONNECT 1.575 0.000 1.574 13.57.54.538 0.000 0.001 0.001 0.000 13.57.52.972 IMSTRANS CONNECT 1.588 0.000 1.588 13.57.54.548 0.009 0.002 0.011 0.000 13.57.53.091 IMSTRANS CONNECT 1.716 0.002 1.714 13.57.54.806 0.000 0.001 0.001 0.000 13.57.53.567 IMSTRANS CONNECT 1.839 0.000 1.839 13.57.55.403 0.000 0.002 0.002 0.000 13.57.54.044 IMSTRANS CONNECT 1.800 0.000 1.799 13.57.55.836 0.006 0.001 0.007 0.001 IMS Connect time can be OTMA is the source of the IMS is showing millisecond under 0.001 seconds problem response times IMS Performance Tuning Update 139
  • 140. IBM Software Group New York City IMS Users Group IMS Performance Analyzer IMS Problem Investigator ISPF Reports Extracts dialog Transaction activity merged into a single view Data sources IMS PA Transaction IMS log Index User records OMEGAMON TRF Extractor TRF records data set IMS monitor IMS Connect SMF Extensions IRLM long-lock WebSphere MQ DB2 log journal records CQS log log extract Transaction activity logged in multiple locations IMS-related subsystems IMS Connect Extensions WebSphere DB2 IMS Connect IMS DB/TM CQS MQ IMS transaction IMS Performance Tuning Update 140
  • 141. IBM Software Group New York City IMS Users Group IMS Performance Analyzer – V4.2 Analyzes OMEGAMON monitor-level instrumentation Transaction response time breakdown, CPU time, and other resource usage statistics Full Function and Fast Path database DL/I call count and elapsed time DB2 database call count and elapsed time OMEGAMON TRF DLI Call Summary _____________________________ Func Call Total Average Minimum Maximum Calls per Calls per Trancode Database Code Count Elapsed Elapsed Elapsed Elapsed second minute ________ ________ ____ ________ __________ __________ __________ __________ __________ __________ BANKING CUSTOMER ISRT 3 0.101742 0.033914 0.003939 0.093401 0.00 0.04 REPL 1 0.001066 0.001066 0.001066 0.001066 0.00 0.01 DLET 1 0.005595 0.005595 0.005595 0.005595 0.00 0.01 GU 7 0.571528 0.081646 0.000915 0.557844 0.00 0.10 GHU 3 0.004219 0.001406 0.001343 0.001520 0.00 0.04 Total 15 0.684150 0.045610 0.000915 0.557844 0.00 0.23 ORDER ORDERS ISRT 3 0.023952 0.007984 0.001385 0.016799 0.00 0.04 REPL 1 0.000120 0.000120 0.000120 0.000120 0.00 0.01 DLET 1 0.000239 0.000239 0.000239 0.000239 0.00 0.01 GU 5 0.024809 0.004961 0.001622 0.015286 0.00 0.07 GHU 2 0.002255 0.001127 0.000727 0.001528 0.00 0.03 Total 12 0.051375 0.004281 0.000120 0.016799 0.00 0.18 IMS Performance Tuning Update 141
  • 142. IBM Software Group New York City IMS Users Group IMS Performance Analyzer – V4.2 Cold start analysis takes the IMS log and tell you what would be lost in the event of a cold start – Input messages (transactions) that would be discarded – Incomplete units-of-work with database changes and ESAF connections Gap analysis identifies periods of time where log records are not being cut, potentially highlighting an external system event that may have caused IMS to slow down. Form improvements: – Analyze service levels with distributions. For example, request the percentage of transactions with response time that exceeded the allowable threshold – Profile database updates and ESAF activity by transaction IMS Performance Tuning Update 142
  • 143. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI is an ISPF application – Batch reports can also be produced – REXX EXECs can also be used IMSPI filters, merges, tracks, displays and reports on records – IMS logs •Including Omegamon II TRF records – IMS DC Monitor – CQS logs – DB2 logs – MQSeries logs – IMS Connect Extensions journals – SMF IRLM Long Lock records There are MANY more options and facilities than I will show in this presentation IMS Performance Tuning Update 143
  • 144. IBM Software Group New York City IMS Users Group IMS Problem Investigator • IMS PI allows you to interactively browse and analyze log records • Instantaneous view of the logs • Navigate by the time of day • Select records to drill down right to the values of individual flag bits IMS Performance Tuning Update 144
  • 145. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI Main Menu File Help __________________________________________________________________________ V2R1M0 IMS Problem Investigator - Primary Option Menu Option ===> _____________________________________________________________ 0 Profile Customize your dialog profile 1 Process Analyze log files (IMS Log, Monitor, CQS, Connect, OMEGAMON) 2 Filter Define record filtering criteria 3 Forms Define record presentation formats 4 Systems Specify IMS and Connect systems and files 5 Groups Specify groups of IMS and Connect systems (Sysplexes) 6 Connect Process IMS Connect Extensions archive journals 7 User Log Define and prepare User Log records for formatting X Exit Quit IMS Problem Investigator Control Data Set . . . 'SNATHAN.IMSPI.CDS' + +-----------------------------------------------------------------+ | IBM IMS Problem Investigator for z/OS | | Licensed Materials - Property of Fundi | | 5655-R02 Copyright Fundi Software 2003, 2007. | | All rights reserved. Use, duplication or disclosure restricted. | +-----------------------------------------------------------------+ IMS Performance Tuning Update 145
  • 146. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI Input Selection File Menu Edit Help ────────────────────────────────────────────────────────────────── Process Log Files More: < > Generated index Command ===> Scroll ===> CSR Select a Log File to browse. IMS Release 910 + Zone IMS logs / Log File Rel + Filter + Zone s ‘USER.CA01.INDEX' SLOW_TX +8 s 'IPI000.QADATA.SLDSP.I9DE.D06311.T0548030.V00' s 'IPI000.QADATA.SLDSP.I9DF.D06311.T0548280.V00' DB2 log s 'IPI000.QADATA.TCN000N.DB2P.ARCHLOG1.A0000826' s 'IPI000.QADATA.TCN000N.I9DE.IMSMON' s 'IPI000.QADATA.TCN000N.I9DF.IMSMON' Monitor files s 'IPI000.QADATA.TCN000N.SMFDUMP.VBS' SMF file One index file can refer to multiple logs on different subsystems IMS Performance Tuning Update 146
  • 147. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI Filtering Log Records File Menu Edit Object Lists Help ─────────────────────────────────────────────────────────────────── Conditions Row 1 to 1 of 1 Command ===> Scroll ===> CSR Code: CA01 IMS transaction accounting index / Field Name + Oper Value + TRANSACTION_PROCESSING_TIME GT 0.3 ******************************* Bottom of data ******************************** “Show accounting records with processing time greater than 0.3 seconds” IMS Performance Tuning Update 147
  • 148. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI List of transactions matching the filter File Menu Edit Mode Navigate Filter Time Labels Options Help ─────────────────────────────────────────────────────────────────── BROWSE JCH.CA01.INDEX Record 00002488 More: < > Command ===> Scroll ===> CSR Forwards / Backwards . . 00.00.00.000100 Time of Day . . 14.41.55.532866 Code Description Date 2006-11-07 Tuesday Time (+8) Track all / ---- ------------------------------------------------------ ---------------- records from TX CA01 IMS transaction accounting index TranCode=DSN8PP 05.33.51.461511 CA01 IMS transaction accounting index TranCode=DSN8PP 05.34.12.744752 transaction CA01 IMS transaction accounting index TranCode=DSN8PP 05.34.32.058505 CA01 IMS transaction accounting index TranCode=DSN8PP 05.34.51.741893 CA01 IMS transaction accounting index TranCode=DSN8PP 05.35.09.250895 CA01 IMS transaction accounting index TranCode=DSN8PS 05.36.21.228573 CA01 IMS transaction accounting index TranCode=DSN8PS 05.36.50.751332 CA01 IMS transaction accounting index TranCode=DSN8PS 05.37.49.038430 CA01 IMS transaction accounting index TranCode=DSN8PS 05.37.55.082883 CA01 IMS transaction accounting index TranCode=DSN8PS 05.38.03.562376 CA01 IMS transaction accounting index TranCode=DSN8PS 05.38.08.273151 CA01 IMS transaction accounting index TranCode=DSN8PS 05.38.11.009607 CA01 IMS transaction accounting index TranCode=DSN8PT 05.38.55.350488 CA01 IMS transaction accounting index TranCode=DSN8PT 05.39.02.144529 CA01 IMS transaction accounting index TranCode=DSN8PT 05.39.11.703720 CA01 IMS transaction accounting index TranCode=DSN8CS 05.42.59.634801 CA01 IMS transaction accounting index TranCode=DSN8CS 05.44.52.678102 CA01 IMS transaction accounting index TranCode=DSN8CS 05.44.58.104215 CA01 IMS transaction accounting index TranCode=DSN8PS 05.45.13.821636 CA01 IMS transaction accounting index TranCode=DSN8PS 05.45.16.253392 CA01 IMS transaction accounting index TranCode=DSN8PS 05.45.18.786698 ******************************** Bottom of Data ******************************* IMS Performance Tuning Update 148
  • 149. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMSPI Extended Tracking Code Description Date 2006-11-07 Tuesday Time (+8) / ---- ------------------------------------------------------ ---------------- CA01 IMS transaction accounting index TranCode=DSN8CS 05.44.58.104215 01 Input Message TranCode=DSN8CS 05.44.58.104215 35 Input Message Enqueue TranCode=DSN8CS 05.44.58.108221 08 Application Start TranCode=DSN8CS Region=0001 05.44.58.108974 5607 Start of UOR Program=DSN8IC0 Region=0001 05.44.58.108975 4E10 MPP Scheduling start Region=0001 05.44.54.732193 4E14 Scheduling IWAIT start Region=0001 05.44.54.732197 4E15 Scheduling IWAIT end Region=0001 05.44.58.108412 4E64 DLA30 start Database=I/O PCB Region=0001 05.44.58.108978 31 DLI GU TranCode=DSN8CS Region=0001 05.44.58.112730 4E65 DLA30 end Region=0001 SC=' ' 05.44.58.112774 4E42 Transaction Originator TranCode=DSN8CS Region=0001 05.44.58.112775 4E11 MPP Scheduling end TranCode=DSN8CS Region=0001 05.44.58.112782 5616 Start of protected UOW Region=0001 05.44.58.112988 Records for the problem transaction from 4E60 DLI Call start Region=0001 05.44.58.203604 4E61 DLI Call end Region=0001 05.44.58.203609 across all selected subsystems now appear 4E48 External Subsystem Call start Region=0001 05.44.58.203704 4E49 External Subsystem Call end Region=0001 05.44.58.210453 5600 Sign-on to ESAF Region=0001 SSID=DB2P 05.44.58.210476 5600 Thread created for ESAF SSID=DB2P 05.44.58.210508 4E48 External Subsystem Call start Region=0001 05.44.58.211852 4E49 External Subsystem Call end Region=0001 05.44.58.212268 4E48 External Subsystem Call start Region=0001 05.44.58.212333 0020 DB2 Unit of Recovery Control - Begin UR 05.44.58.212768 0020 DB2 Exclusive Lock on Pageset Partition or DBD 05.44.58.212768 0010 DB2 Savepoint 05.44.58.212784 0020 DB2 Type 2 Index Update 05.44.58.212784 0020 DB2 Delete from a Data Page 05.44.58.212816 0020 DB2 Update Area Map/Spacemap Bit Changes or Whole Byte 05.44.58.212832 4E49 External Subsystem Call end Region=0001 05.44.58.212952 4E60 DLI Call start Region=0001 05.44.58.213039 4E64 DLA30 start Database=I/O PCB Region=0001 Func=ISRT 05.44.58.213050 4E65 DLA30 end Region=0001 SC=' ' 05.44.58.213116 4E61 DLI Call end Region=0001 05.44.58.213117 IMS Performance Tuning Update 149
  • 150. IBM Software Group New York City IMS Users Group IMS Problem Investigator Code Description Date 2007-02-09 Friday Time (Local) XML message received ---- ------------------------------------------------ ---------------- from SOAP gateway A03C Prepare READ Socket 14.30.02.363636 A049 READ Socket 0.000060 A0A9 RXML Trace for XML Adapter 0.000062 Message converted to A03D Message Exit called for RXML 0.000004 COBOL byte stream A03E Message Exit returned from RXML 0.000046 A0A4 Event Collection IRM Trace 1.537365 A03D Message Exit called for READ 0.000015 A0A3 Event Collection OTMA Trace 0.000044 A03E Message Exit returned from READ TranCode=IVTNO 0.000004 A0A3 Event Collection OTMA Trace 0.000155 Message sent to OTMA A041 Message sent to OTMA Datastore=XCFMI9D1 0.000006 01 Input Message TranCode=IVTNO Source=Connect 0.001290 35 Input Message Enqueue TranCode=IVTNO 0.000140 31 DLI GU TranCode=IVTNO Region=0001 0.000103 Message in IMS 5E SB Handler requests Image Capture Region=0001 0.000808 5E SB Handler requests Image Capture Region=0001 0.001135 03 Output Message Response LTerm=8801 Source=Connect 0.059450 Response from IMS 31 DLI GU TranCode=IVTNO Region=0001 0.000032 33 Free Message 0.000140 5610 Start Phase 1 Syncpoint Region=0001 0.000038 37 Syncpoint Region=0001 0.000061 33 Free Message 0.000094 Message in OTMA 5612 End of Phase 2 Syncpoint Program=DFSIVP1 0.001072 A0A3 Event Collection OTMA Trace 0.003010 A042 Message received from OTMA Datastore=XCFMI9D1 0.000017 Message converted to A0A3 Event Collection OTMA Trace 0.000039 XML A042 Message received from OTMA Datastore=XCFMI9D1 0.000004 A0A3 Event Collection OTMA Trace 0.000272 A03D Message Exit called for XMIT 0.000023 A0A6 Event Recording EXIT Output Message Trace 0.000039 A03E Message Exit returned from XMIT 0.000003 XML message sent to A04A WRITE Socket 0.855510 SOAP Gateway A048 Trigger Event 0.000080 IMS Performance Tuning Update 150
  • 151. IBM Software Group New York City IMS Users Group IMS Problem Investigator IMS Performance Tuning Update 151
  • 152. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 Time slicing: Process large log files with the responsiveness and convenience of a small extract file Transaction index: Extend the insights of transaction performance recorded by IMS PA to help analyze transactions in IMS PI Unit of work tracking: Isolate a single unit of recovery (TU) within a transaction (TX) with a long program switch sequence Improved coverage with support for the DB2 and MQ logs Extract to CSV: Import the required fields from selected log record types into a DB2 database or PC spreadsheet OMEGAMON TRF: transaction accounting, DLI and DB2 calls Future Requirement: OMEGAMON ATF – transaction call trace IMS Performance Tuning Update 152
  • 153. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 – Time Slicing Process large log files with the speed and convenience of a small extract Process Log Files Row 21 of 537 More: < > Command ===> Scroll ===> CSR Select a Log File to browse. IMS Release 101 + Zone / Log File Rel + Filter + Zone WW ‘IMSP.INDEX' 101 CA01 ‘IMSP.SLDS' 101 01 ‘IMSP.OMEGAMON.ATF WW ‘DB2.LOG.ARCHIVE' Specify the time period you need to investigate. PI goes directly to that point in the log(s), no waiting for a long sequential scan. Time Slicing Row 1 of 4 More: < > Command ===> Scroll ===> CSR Specify the time of day and duration that you want to analyze. Select the slice (first row) to merge and process all the files, or select one file. ————————————————— Time Slice ——————————————————— Time Date Duration Merges all 4 files, / Type HH.MM.SS.thmiju YYYY-MM-DD HH.MM.SS.thmiju and displays only S ALL 16.50.00.000000 2009-07-10 00.01.00.000000 the required —————————————————— Log Files ——————————————————— Type Start Time Date Duration 1minute interval Coverage IDX 16.00.12.407506 2009-07-10 Fri 1.28.50.420549 COMPLETE IMS 16.00.02.466524 2009-07-10 Fri 1.53.37.334729 COMPLETE ATF 16.00.15.965444 2009-07-10 Wed 1.47,45.176289 COMPLETE DB2 12.28.46.529264 2009-07-10 Sat 5.59.59.999999 COMPLETE ******************************* Bottom of data ******************************** IMS Performance Tuning Update 153
  • 154. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 – Transaction Index All the accumulated information from the IMS log about each transaction is condensed into a single transaction index record – use the index as input into: 1. IMSPA reporting, instead of re-processing large SLDS log files 2. IMSPI, to enable more intelligent problem detection, as follows: Conditions Row 1 to 2 of 2 Command ===> Scroll ===> PAGE Locate all MQATREQ1 Code: CA01 Transaction transactions with / Field Name + Oper Value + processing time > 1 sec TRANCODE EQ 'MQATREQ1' PROCESS GT 1.0 ******************************* Bottom of data ******************************** Track against the index record to display all associated transaction log records: BROWSE JCH.INDEX Record 00427482 More: < > Command ===> FILTER Scroll ===> PAGE Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU Code Description Date 2009-07-10 Friday Time (Relative) / --- ------------------------------------------------------ ---------------- TX CA01 Transaction 16.50.06.570728 TranCode=MQATREQ1 Program=MQATPGM Userid=FUNTRM07 LTerm=FUNTRM07 Terminal=SC0TCP07 Region=0001 OrgUOWID=IADG/C476657D88074C60 RecToken=IADG/0000002300000000 CPU=0.033756 InputQ=0.000734 Process=1.204564 TotalTm=1.305298 RegTyp=MPP DBCalls=5 --------------------------------------------------------------------------- 01 Input Message TranCode=MQATREQ1 +0.000000 35 Input Message Enqueue TranCode=MQATREQ1 +0.000032 08 Application Start TranCode=MQATREQ1 Region=0001 +0.000631 5607 Start of UOR Program=MQATPGM Region=0001 +0.000632 05 DLI (GU) TranCode=MQATREQ1 Region=0001 +0.000638 31 DLI GU TranCode=MQATREQ1 Region=0001 +0.000697 IMS Performance Tuning Update 154
  • 155. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 – OM ATF: DLI Calls OMEGAMON V4.2 ATF provides IMS transaction call trace for all DLI, DB2 and MQ calls BROWSE JCH.INDEX Record 00427482 More: < > Command ===> Scroll ===> PAGE Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU Code Description Date 2009-07-10 Friday Time (Relative) / ---- ------------------------------------------------------ ---------------- S 01 DLI GHU Database=DI21PART ELAPSE=0.000153 +0.004704 01 DLI REPL Database=DI21PART ELAPSE=0.000516 +0.004994 50 Database Update Database=DI21PART REGION=0001 +0.005074 01 DLI GHU Database=DI21PART ELAPSE=0.000617 +0.005128 01 DLI DLET Database=DI21PART ELAPSE=0.000415 +0.005191 50 Database Update Database=DI21PART Region=0001 +0.005246 50 Database Update Database=DI21PART Region=0001 +0.005382 01 DLI ISRT Database=DI21PART ELAPSE=0.000298 +0.005468 50 Database Update Database=DI21PART Region=0001 +0.005515 50 Database Update Database=DI21PART Region=0001 +0.005561 For each DLI call, see elapsed time, functon and status codes, SSAs: BROWSE JCH.INDEX Record 00427491 Line 00000000 ********************************* Top of data ********************************* +0012 Code... 01 DLI GHU +001C STCK... C476657D89349520 LSN.... 000000000000079C Date... 2009-07-10 Friday Time... 16.50.06.575433.320 +0013 ATRSTYPE... 62 ATRPST..... 0001 +0016 ATRCOR..... C9C1C4C70001C476656EDA182902 +0024 ATRCRE..... C9C1C4C7404040400000002300000000 +0034 ATRSTART... C476657D89349520 ATRELAPS... 0.000153 secs +0084 DATABASE... 'DI21PART' +0094 ATRDLIS.... Function and Status Codes +0094 ATRVERB.... 'GHU ' ATRRC...... ' ' +0128 ATRDXV..... Segment Search Argument +0000 D7C1D9E3 D9D6D6E3 4DD7C1D9 E3D2C5E8 *PARTROOT(PARTKEY* +0010 40407ED4 D8C140E3 C5E2E340 D2C5E85D * =MQA TEST KEY)* ********************************* End of data ********************************* IMS Performance Tuning Update 155
  • 156. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 – OM ATF – DB2 Calls For each DB2 call, see: • Elapsed time • SQL call type • SQL statement number in BROWSE DBRM JCH.INDEX Record 00427518 More: < > Command ===> Scroll ===> CSR Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU Code Description Date 2009-07-10 Friday Time (Relative) / ---- ------------------------------------------------------ ---------------- 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.008198 5600 Sign-on to ESAF Region=0001 SSID=DB3A +0.008592 5600 Thread created for ESAF SSID=DB3A +0.008609 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.008612 82 DB2 OPEN SSID=DB3A STMT=12 PROGRAM=MQATPGM +0.009321 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000910 +0.009753 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000847 +0.010468 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000415 +0.010800 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000417 +0.011090 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000314 +0.011419 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000276 +0.011735 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000198 +0.012053 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000372 +0.012696 82 DB2 FETCH SSID=DB3A STMT=13 ELAPSE=0.000516 +0.012981 82 DB2 UPDATE TranCode=MQATREQ1 Region=0001 SSID=DB3A 0020 DB2 Unit of Recovery Control - Begin UR Also notice the DB2 +0.013298 +0.015319 0020 DB2 Update In-Place in a Data Page 82 DB2 CLOSE SSID=DB3A STMT=14 PROGRAM=MQATPGM log events+0.015335 (0020) +0.015677 82 DB2 DELETE SSID=DB3A STMT=15 ELAPSE=0.000116 merged in+0.016000 sequence 0010 DB2 Savepoint +0.016071 0020 DB2 Delete from a Data Page with the other ATF +0.016167 82 DB2 ISRT SSID=DB3A STMT=16 ELAPSE=0.000219 +0.016481 0020 DB2 Insert into a Data Page and IMS log record +0.016647 events IMS Performance Tuning Update 156
  • 157. IBM Software Group New York City IMS Users Group IMS Problem Investigator – V2.2 – OM ATF – MQ Calls For each MQ call, see: • Elapsed time • MQ call type • Queue name BROWSE JCH.INDEX Record 00427518 More: < > Command ===> Scroll ===> CSR Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU Code Description Date 2009-07-10 Friday Time (Relative) / ---- ------------------------------------------------------ ---------------- 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=CSQ6 +0.039065 5600 Sign-on to ESAF Region=0001 SSID=CSQ6 +0.039240 5600 Thread created for ESAF SSID=CSQ6 +0.039256 87 Generic ESS TranCode=MQATREQ1 Region=0001 +0.039260 88 MQ CONN SSID=CSQ6 +0.039471 88 MQ OPEN SSID=CSQ6 QNAME=PROD_IN_QUEUE +0.039874 88 MQ GET SSID=CSQ6 QNAME=PROD_IN_QUEUE +0.040321 88 MQ GET SSID=CSQ6 QNAME=PROD_IN_QUEUE +0.075995 88 MQ GET SSID=CSQ6 QNAME=PROD_IN_QUEUE +0.101618 88 MQ CLOS SSID=CSQ6 QNAME=PROD_IN_QUEUE +0.572026 88 MQ OPEN SSID=CSQ6 QNAME=PROD_OUT_QUEUE +0.572366 88 MQ PUT SSID=CSQ6 QNAME=PROD_OUT_QUEUE +0.572817 88 MQ CLOS SSID=CSQ6 QNAME=PROD_OUT_QUEUE +0.576998 88 MQ DISC SSID=CSQ6 QNAME=PROD_OUT_QUEUE +0.590434 IMS Performance Tuning Update 157
  • 158. IBM Software Group New York City IMS Users Group IMS Problem Investigator – Track and Merge We have reached end of the transaction we are tracking. Observe all the syncpoint events in the IMS and DB2 logs, as well as the OMEGAMON ATF call trace, and end of transaction events: BROWSE JCH.INDEX Record 00427560 More: < > Command ===> Scroll ===> CSR Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU Code Description Date 2009-07-10 Friday Time (Relative) / ---- ------------------------------------------------------ ---------------- 05 DLI ISRT TranCode=MQATREQ1 Region=0001 +0.590831 5600 Commit Prepare starting Region=0001 SSID=CSQ6 +0.591778 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=CSQ6 +0.593882 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.593957 0020 DB2 Unit of Recovery Control - End Commit Phase 1 +0.594135 03 Output Message Response LTerm=FUNTRM07 +0.596907 35 Output Message Enqueue LTerm=FUNTRM07 Region=0001 +0.596939 37 Syncpoint Region=0001 +0.596976 37 Syncpoint message transfer Region=0001 +0.597013 33 Free Message +0.597053 31 Communications GU LTerm=FUNTRM07 +0.597269 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=CSQ6 +0.599075 36 Output Message Dequeue LTerm=FUNTRM07 +0.601916 33 Free Message +0.601980 5600 Commit Continue completed Region=0001 SSID=CSQ6 +0.602623 87 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.602626 05 DLI Detail TranCode=MQATREQ1 Region=0001 Func=ASRT +0.591748 0020 DB2 Unit of Recovery Control - Begin Commit Phase 2 +0.602711 0020 DB2 Unit of Recovery Control - End Commit Phase 2 +0.604407 5600 Commit Continue completed Region=0001 SSID=DB3A +0.605222 5612 End of Phase 2 Syncpoint Program=MQATPGM Region=0001 +0.605234 07 Application Terminate TranCode=MQATREQ1 Region=0001 +0.605362 F1 Transaction End TranCode=MQATREQ1 Region=0001 +0.605699 04 Summary Completion TranCode=MQATREQ1 Region=0001 +0.605699 ******************************** Bottom of Data ******************************* IMS Performance Tuning Update 158
  • 159. IBM Software Group New York City IMS Users Group Conclusion IMS Performance Tuning Update 159
  • 160. IBM Software Group New York City IMS Users Group Conclusion Debugging and tuning an IMS TCP/IP environment takes a LOT of work – There are many areas to be looked at – There are many potential problems and bottlenecks – Application programmers can be very inventive – There are many tools to help IMS Performance Tuning Update 160