4. Messaging and Queuing
Telephone answering machine
Punched Cards and processing
Clerical Data Input and processing
Information Management Systems (IMS)
Customer Information Control System (CICS)
Telecommunications Access Methods (TCAM)
Virtual Telecommunications Access Method
5. Messaging and Queuing
Messaging is
an availability adaptation technique used for the transfer of information between two entities without regard for the immediate availability and accessibility of either
Queuing
a time adaptation technique user for saving information until the intended message receiver is ready to receive it
Messaging and queuing is both asynchronous and connection less
Note: entities can be program, persons, systems etc
6. Requirements
Distributed applications and systems
Increase in program to program communication
Heterogeneous Platforms
Multiple communication protocols
Maximize resource utilization
Development complexity and resource skill set
7. Websphere MQ (MQ Series until '02)
IBM middleware for messaging & queuing
Launched on MVS/ESA in 1992
Currently available on all major platforms
Windows
UNIX (AIXLinuxSun Solaris, HP-UX)
AS/400
Z/OS
Common API (MQI) on all platforms
Supports multiple programming languages
C, C++, JAVA, VB, COBOL,PL/1,Assembler
Supports JMS standard APIs for messaging
9. MQ Message
String of bytes meaningful to the entity using it.
Two parts to MQ messages
Application Data
Content and structure defined and understood by the application program
Message Descriptor
Contains control information like type of message and priority which are understood and used by MQ
Default length is 4 MB; the max length is 100 MB
Length can be restricted by the definition of MQ objects and the system storage
Messages can be segmented or grouped
11. MQ Message Descriptor
Version
Message ID/Correlation ID
Persistent/Non persistent
Priority
Date and time
Life time of a message
Coded Character Set Id
Format
Sender application and type
Report options/Feedback
Back out counter
Segmentation/Grouping information
13. MQ Object Names
Up to 48 characters for all objects except channels
Up to 20 characters for channels
Uppercase A- Z
Lowercase a - z
Numerics 0 - 9
Period (.)
Underscore (_)
Forward slash (/)*
Percent sign (%)*
Blanks are not allowed
* - If used, names should be enclosed in double quotes since these are special characters
14. Managing MQ Objects
Control commands typed in through key board.
MQSC commands through key board or from files
Programmable Command Format (PCF) Messages
MQ Administration Interface (MQAI) call from a program
MQ Explorer snap-in and MQ services snap-in running under Microsoft management Console*
* - Applicable to Windows platform only
15. Message Queue Manager (MQM)
Manages queues of messages for applications
Provides application programming interface MQI
Uses existing network facilities to transfer messages
Coordinates database and queue data updates
Segments messages and assembles them
Allows message grouping and disassembling
Can send message to more than one destination
Provides administrative functions on objects
Special events like triggers and instrumentation events are generated
17. Process definition
Defines an application that starts in response to a trigger
Attributes include application type, application ID (path) and data specific to the application
18. Channels
Logical communication link
Communication path from one MQM to another
Used to move messages between MQMs
Shields applications from underlying protocols
Message Channels connects two MQMs
Uses Message Channel Agents (MCA)
Unidirectional
Requires a sender, receiver channels and a protocol
MQI Channels connects MQ client to server
Bidirectional
Used for MQ calls and responses only
19. Namelist
Contains a list of MQ objects; queue, cluster, authinfo
Typically used by trigger monitors to identify a group of queues
Used in clustered environment to inform the availability of an MQ object to clusters
Maintained independent of application and can be modified without stopping it
Multiple applications can use a Namelist
23. System Default Objects
Object Name Description
SYSTEM.ADMIN.CHANNEL.EVENT Event queue for channels
SYSTEM.ADMIN.COMMAND.QUEUE Administration command queue. Used for remote MQSC commands and PCF commands.
SYSTEM.ADMIN.PERFM.EVENT Event queue for performance events
SYSTEM.ADMIN.QMGR.EVENT Event queue for queue manager events
SYSTEM.AUTH.DATA.QUEUE The queue that holds access control lists for the queue manager.
SYSTEM.CHANNEL.INITQ Channel initiation queue
SYSTEM.CHANNEL.SYNCQ The queue that holds the synchronization data for channels.
SYSTEM.CICS.INITIATION.QUEUE Default CICS initiation queue.
SYSTEM.CLUSTER.COMMAND.QUEUE The queue used to carry messages to the repository queue manager.
SYSTEM.CLUSTER.REPOSITORY.QUEUE The queue used to store all repository information
SYSTEM.CLUSTER.TRANSMIT.QUEUE The transmission queue for all messages to all clusters
SYSTEM.DEAD.LETTER.QUEUE Dead-letter (undelivered-message) queue.
24. System Default Objects
Object Name Description
SYSTEM.DEFAULT.ALIAS.QUEUE Default alias queue
SYSTEM.DEFAULT.INITIATION.QUEUE Default initiation queue
SYSTEM.DEFAULT.LOCAL.QUEUE Default local queue
SYSTEM.DEFAULT.MODEL.QUEUE Default model queue
SYSTEM.DEFAULT.REMOTE.QUEUE Default remote queue
SYSTEM.MQSC.REPLY.QUEUE MQSC command reply-to queue. This is a model queue.
SYSTEM.PENDING.DATA.QUEUE Support deferred messages in JMS.
SYSTEM.DEFAULT.NAMELIST Default namelist.
SYSTEM.DEFAULT.PROCESS Default process definition.
25. System Default Objects
Object Name Description
SYSTEM.DEFAULT.AUTHINFO. CRLLDAP Default authentication information object.
SYSTEM.AUTO.RECEIVER Dynamic receiver channel.
SYSTEM.AUTO.SVRCONN Dynamic server-connection channel.
SYSTEM.DEF.CLUSRCVR Default receiver channel for the cluster
SYSTEM.DEF.CLUSSDR Default sender channel for the cluster
SYSTEM.DEF.RECEIVER Default receiver channel.
SYSTEM.DEF.REQUESTER Default requester channel.
SYSTEM.DEF.SENDER Default sender channel.
SYSTEM.DEF.SERVER Default server channel.
SYSTEM.DEF.SVRCONN Default server-connection channel.
SYSTEM.DEF.CLNTCONN Default client-connection channel.
28. MQM Administration
Create MQ Manager
CRTMQM
e.g. CRTMQM /q TESTQMGR
/q – sets the MQM as default
Start MQ Manager
STRMQM
e.g. STRMQM TESTQMGR
Stop MQ Manager
ENDMQM
e.g. ENDMQM TESTQMGR
Delete MQ Manager
DLTMQM
e.g. ENDMQM TESTQMGR
29. MQM Administration
Stop MQ Manager ENDMQM Options
By default MQM is quiesced and so takes time
-c to notifies all applications to stop but doesn't wait for MQM to end
-c is same as issuing ENDMQM with no options
-w waits until apps stop and MQM ends
-i immediate and doesn't wait for apps to disconnect
-p pre-emptive and should be used with caution
30. Managing MQM Objects
MQSC Commands by running RUNMQSC
Verify a command without running it
Run a command on a local queue manager
Run a command on a remote queue manager
To display MQM properties
STRMQM TESTQMGR (if not started already)
RUNMQSC or RUNMQSC TESTQMGR
display QMGR (MQSC Command)
END
RUNMQSC can accept commands through key board or from a file
RUNMQSC < command.in > execution.out
31. Managing Queues
Create a local queue
DEFINE QLOCAL(TEST.LOCAL.QUEUE)
Display a local queue
DISPLAY QLOCAL(TEST.LOCAL.QUEUE)
Alter a local queue
ALTER QLOCAL(TEST.LOCAL.QUEUE) + GET(DISABLED)
Delete a local queue
DELETE QLOCAL(TEST.LOCAL.QUEUE) PURGE
32. Managing Trigger Processes
Create a Process
DEFINE PROCESS(PROC1) REPLACE +
DESCR('test proc') APPLTYPE(WINDOWS)+
APPLICID('c:testprog.exe')
Display a Process
DISPLAY PROCESS(PROC1)
Alter a Process
ALTER PROCESS(PROC1)USERDATA('12')
Delete a Process
DELETE PROCESS(PROC1)
33. Enabling Triggers
Enabling trigger in a queue
DEFINE QLOCAL(TEST.LOCAL.QUEUE)REPLACE +
TRIGGER TRIGTYPE(first)PROCESS(PROC1)+
INITQ(SYSTEM.DEFAULT.INITIATION.QUEUE)
Trigger Types
every
first
N messages
34. Managing Channels
Create a Channel
DEFINE CHANNEL(MQMA.MQMB)CHLTYPE(sdr)+
XMITQ(MQMB) TRPTYPE(tcp)+
CONNAME(SERVER1.XYZ.COM)
Display a Channel
DISPLAY CHANNEL(MQMA.MQMB)
Alter a Channel
ALTER CHANNEL(MQMA.MQMB)+ CONNAME(SERVER2.XYZ.COM)
Delete a Channel
DELETE CHANNEL(MQMA.MQMB)
35. Managing Namelist
Create a Namelist
DEFINE NAMELIST(CLUSTERNL)+
NAMES(CLUSA,CLUSB)+
NLTYPE(CLUSTER)REPLACE
Display a Namelist
DISPLAY NAMELIST(CLUSTERNL)
Alter a Namelist
DEFINE NAMELIST(CLUSTERNL)+
NAMES(CLUSA,CLUSB,CLUSC)+
NLTYPE(CLUSTER)
Delete a Namelist
DELETE NAMELIST(CLUSTERNL)
38. Distributed MQ
Transferring messages between Queue Managers
Queue Managers can be on the same machine or world apart
Queue Managers can be on the same platform or hetrogenous platforms
Implemented using Distributed Queue Management (DQM) features of MQ including MQ Clusters
39. Distributed MQ Components
Local and remote Queue Managers and queues
Local is one to which an application is connected
Remote queue definition
Local definition of a queue in a remote MQM
Transmission Queues
Special local queue to store message temporarily before transmitting to remote destination
Message channel agents (MCA)
Software that handles sending and receiving
Channels
One way communication link between MQMs
40. Distributed MQ Components
Dead letter queue
Special queue to store undelivered messages
Transport Service
Communication protocol independent of MQ
MQ supports multiple protocols
Platform dependent
Channel Initiators and Listeners
Initiators are trigger monitors for sender channels
Listeners start receiver channels on receiving request from the sender MCA
Channel Exit Programs
Used for additional processing like encryption
41. Distributed MQ Components
M C A
M C A
M C A
M C A
Channel
Channel
Transport Service (TCP)
Transport Service (TCP)
App Q
App Q
App Q
App Q
XMIT Q
XMIT Q
QM1
QM2
Channel Initiator
Listener
Init Q
45. Queue Addressing & Aliases
Queue Name + Queue Manager Name + Transmission Queue (if remote queue)
Remote Queue Definition
Resolves the remote queue location
Queue Manager Alias Definition
Specifies alternative names for the message descriptor of a “send” message
Reply-to Queue Alias Definition
Specifies alternative names for the reply information in the message descriptor
Aliases are characterized by a blank RNAME in the remote queue definition
46. Remote Queue Definition
Remote Queue Definition
Define QREMOTE(Q1)RNAME(Q1)+ RQMNAME(QMB)XMITQ(QMB)
Resolves the remote queue address using the remote queue manager, remote queue name and the transmission queue
If the transmission queue is not specified, a transmission queue with the name of the remote queue manager will be used
If not available, the default transmission queue of the queue manager will be used
47. Queue Manager Alias
Queue Manager Alias
Define QREMOTE(QMB_Test)RNAME()+ RQMNAME(QMB)
Define QREMOTE(QMB_Test)RNAME()+ RQMNAME(QMB)XMITQ(QMC)
Messages with MQM QMB_Test in the descriptor will be modified to QMB before being forwarded
XMITQ parameter can be used to steer the message to use a specified channel
If transmission queue is not specified, the logic detailed in the previous slide will be used
48. Reply-to Queue Alias
Can be used to alter the reply-to queue and queue manager without changing the application
At the sender queue manager
Define QREMOTE(QMA_Reply) + RNAME(Answer_QMA)RQMNAME(QMA_Return)
Define QREMOTE(QMA_Return)RNAME()+ RQMNAME(QMA)
In “PUT” call leave the “reply to queue manager” blank and set the queue name to QMA_Reply
At the replying queue manager
Define QLOCAL(QMA_Return)USAGE(XMITQ)
49. Reaching Remote Queue Managers
Point to point
Multi-hopping
Using shared channels
Using multiple channels
Using MQ clusters
50. DMQ Commands
Start Channel Initiators
RUNMQCHI -q IQ (Control Command)
START CHINIT INITQ(IQ) (MQSC Command)
Stopping Channel Initiators
Alter the init queue to disable GET from queue
Start MQM Listeners
RUNMQLSR -t TCP -m MQM-p 1414 (control Command)*
START LISTENER IPADDR(X.X.X.X) PORT(nnnn) TRPTYPE(tcp) (MQSC Command)
*-This requires editing of /etc/services and /etc/inetd.conf files to add
MQSeries 1414/tcp #MQ Listener
MQSeries stream tcp nowait mqm /usr/mqm/bin/amqcrsta -q mqm.name
55. Cluster Advantages
Reduced system administration due to reduced number of definitions
Increased availability and scalability due to easy administration and multiple instances of queues
Work load balancing using multiple instances of queues
56. Cluster Component Details
MQ Cluster is a network of logically related MQMs and need to be unique in an enterprise
Cluster Queue Manager is a MQM that is a member of a MQ Cluster and should be unique in the cluster
Cluster Queue is a queue hosted in a cluster queue manager and is available to all MQM in a cluster
Repository is a collection of information about the queue managers that are part of a cluster
Full Repository is one which stores a complete set of information about all MQMs in the cluster
Partial Repository is one which stores information which is required to deal with interested MQMs
57. Cluster Component Details
Repository Queue manager is a cluster queue manager which stores a full repository
SYSTEM.CLUSTER.REPOSITORY.QUEUE
Cluster Receiver Channel is the receiving end of a channel on which cluster MQMs can receive messages from other MQMs in the cluster
Cluster Sender Channel is the sending end of a channel on which cluster MQMs can send cluster information to one of the full repositories
Cluster Transmission Queue transmits all messages from a cluster MQM to any MQM in the cluster
SYSTEM.CLUSTER.TRANSMIT.QUEUE
Command Queue is used to make requests and get responses from full repositories
58. Clustered MQ Components
Cluster Receiver Channel
Cluster Receiver Channel
Transport Service (TCP)
Transport Service (TCP)
App Q
App Q1
App Q
App Q1
XMIT Q
QM1
QM2
Full Repository
Full Repository
Cluster Sender Channel
Cluster - CLUA
XMIT Q
60. Adding new QM (QM3) to Cluster
RUNMQSC DEFINE CHANNEL(TO.QM3)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME(QM3.COM)+ CLUSTER(CLUA) DEFINE CHANNEL(TO.QM1)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM1.COM)+ CLUSTER(CLUA) Note: QM3 doesn't store a full repository and points to QM1 repository
61. Adding new QM on DHCP to Cluster
RUNMQSC DEFINE CHANNEL(TO.QM3)+ CHLTYPE(CLUSRCVR)+ TRPTYPE(tcp)+ CONNAME('')+ CLUSTER(CLUA) DEFINE CHANNEL(TO.+QMNAME+)+ CHLTYPE(CLUSSDR)+ TRPTYPE(tcp)+ CONNAME(QM2.COM)+ CLUSTER(CLUA) Note: the channel definition TO.+QMNAME+ will be replaced by TO.QM2
62. Verifying a Cluster
Each MQM require a channel initiator to monitor the system defined init Q SYSTEM.CHANNEL.INITQ
START CHINIT
Each MQM require a channel listener to listen to incoming network requests
START LISTENER PORT(1414)+ TRPTYPE(TCP)
Verify all the available MQMs in the cluster
DISPLAY CLUSQMGR(*)
Verify all the channel statuses
DISPLAY CHSTATUS(*)
63. Load Balancing Using Cluster
Define the same queue in multiple queue managers in the cluster
MQ series will use the default algorithm (round robin) to identify MQM and transfer message
Default algorithm can be changed using the cluster workload user exit
73. Transaction Management
ACID unit of work
Single phase commit
Two phase commit with XA specification compliant resource managers
BackoutCount and BackoutThreshold attributes to avoid looping
74. Message Queuing Interface (MQI)
Call Purpose
MQCONN Connect to a Queue Manager
MQDISC Disconnect from a Queue Manager
MQOPEN Open a specific Queue
MQCLOSE Close a Queue
MQPUT Put a message on a queue
MQGET Get a message from a queue
MQPUT1 MQOPEN + MQPUT + MQCLOSE
MQINQ Inquire Properties of an object
MQSET Set properties of an object
MQCONNX Standard or fast path bindings
MQBEGIN Begin an unit of work
MQCMIT Commit an unit of work
MQBACK Back out to the start of unit of work
75. MQI Data Structures
Name Purpose
MQBO Specifies options for MQBEGIN call
MQCNO Specifies options for MQCONNX call
MQDH Describes the data in a distribution-list message
MQGMO Specifies options for MQGET call
MQMD Message descriptor details to control message in a MQGET or MQPUT call
MQOD Object descriptor which is required in a MQOPEN call
MQOR Object record identifies the destinations to work with in a distribution list
MQPMO Specifies options for MQPUT call
76. MQ Programming Sequence
MQCONN
MQOPEN
BASIC OP
MQCLOSE
MQDISC
Connect to MQM
Open MQ Objects
Perform n operations
Close MQ Objects
Disconnect from MQM
78. JAVA Programming Environment
Required jar file
com.ibm.mq.jar
Normally found in the MQ installation directory
Environment variable set-up
Add com.ibm.mq.jar to the classpath
CLASSPATH=install_dirlibcom.ibm.mq.jar
install_dir is platform dependent
Import required package in the code
Import com.ibm.mq.*;
80. MQ Series JAVA Classes
MQEnvironment
Contains the static variables that control the environment in which a MQQueueManager is constructed
MQQueueManager
Represents the MQ series queue manager
MQQueue
Provides inquiry, set,put,get operations for queues
MQMessage
Represents the descriptor and data of a MQ message
MQPutOptions
Contains options which control MQQueue.put() method
MQGetOptions
Contains options which control MQQueue.get() method
MQException
Contains completion codes and error code constants
81. Setting up the Connection
MQEnvironment.hostname=“TEST.SERVER.COM” MQEnvironment.channel=“TEST.CLIENT.CHNL” MQEnvironment.port=1638 MQEnvironment.userId=”userABC” MQEnvironment.password=”password” MQEnvironment.properties.put(MQC.TRANSPORT_PROPERTY,MQC.TRANSPORT_MQSERIES) MQQueueManager qMgr = new MQQueueManager(“TEST”);
82. Interacting with Queues - Put
Int openOptions = MQC.MQOO_OUTPUT|MQC.MQOO_FAIL_IF_QUIESCING; MQQueue queue = qMgr.accessQueue(“TestQ”,openOptions); MQPutMessageOptions pmo = new MQPutMessageOptions(); MQMessage outMsg = new MQMessage(); outMsg.format = MQC.MQFMT_STRING; String msgString = “Test message”; outMsg.writeString(msgString); queue.put(outMsg,pmo);
83. Interacting with Queues - Get
Int openOptions = MQC.MQOO_INPUT_SHARED|MQC.MQOO_FAIL_IF_QUIESCING; MQQueue queue = qMgr.accessQueue(“TestQ”,openOptions); MQGetMessageOptions gmo = new MQGetMessageOptions(); gmo.options = gmo.options + MQC.MQGMO_SYNCPOINT; gmo.options = gmo.options + MQC.MQGMO_WAIT; gmo.waitInterval = 3000; MQMessage inMsg = new MQMessage(); queue.put(inMsg,gmo);
87. Security Services
Identification and Authentication
Access Control
Confidentiality
Data Integrity
Non repudiation
Managing security is more concerned with managing risk to a level that is acceptable to the business.
90. Cryptography
Concept
Convert plain text to ciphertext called encryption
Transmit ciphertext to receiver
Convert ciphertext to plain text called decryption
Symmetric algorithms share same key and involves key distribution problem
Asymmetric algorithms use different keys; Public Key Cryptography
Strength of encryption determined by key size; 512 bits, 768 bits, 1024 bits
Encryption algorithms can be on data blocks or on each byte of data
91. Message Digest
Concept
Fixed size numeric representation of contents of message
Computed by hash function which is one way
No two different message will have the same value
Also called Message Authentication Code (MAC)
Message digest is sent along with the message
Receiver generates a digest and compares with what was received
92. Digital Signatures
Concept
Formed by encrypting a representation of message
Encryption done using the private key of the signatory
Normally done on the message digest for efficiency
Digital Signature sent along with the message
Receiver retrieves the message digest using the public key of the signatory
Computes the digest from the message received and the compares against the digest retrieved
Can involve man in the middle attack with public key distribution
93. Digital Certificates
Concept
Binds public key to a owner and prevents impersonation
Also known as public key certificates
Involves trusted third party called Certification Authority
Digital Certificates include
Owner's public key
Owner's distinguished name: CN,O,OU,L,ST,C
Distinguished name of the issuer
Date from which the certificate is valid
Expiry date of the certificate
Version number
Serial number
Need to make a request and receive digital certificate from a CA
94. Digital Certificates
Concept
Binds public key to a owner and prevents impersonation
Involves trusted third party called Certification Authority
Also known as public key certificates
Digital Certificates include
Owner's public key
Owner's distinguished name: CN,O,OU,L,ST,C
Distinguished name of the issuer
Date from which the certificate is valid
Expiry date of the certificate
Version number
Serial number
Need to make a request and receive certificate from a CA
95. Public Key Infrastructure (PKI)
System of facilities, policies and services that support the use of Public Key Cryptography
Typically involves Certification Authorities (CA) and Registration Authorities (RA) which does
Issuing digital certificates
Validation digital certificates
Revoking digital certificates
Distribution public keys
RA does the verification of information provided in the request for certificate to CA
97. MQ Security Considerations
Authority to administer Websphere MQ
Authority to work with Websphere MQ objects
Channel Security
MQ Clusters
MQ Publish/Subscribe
MQ Internet Pass-Thru
Link level security
Application level security
98. MQ Security Options
MQM can verify the administration authority
MQM can verify the authority to access objects
MCA can authenticate a partner MCA
Messages can be encrypted before sending and decrypted when receiving
User-id in the message descriptor can be used to verify the authority to access MQM objects
COA and COD report options to implement non- repudiation functions in an application