David Ware, profile picture
Uploaded byDavid Ware
PDF, PPTX5,098 views

IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe

The document provides an introduction to IBM MQ's publish/subscribe feature, detailing its administration, management, and the structure of topic trees. It explains the differences between publish/subscribe and point-to-point messaging, outlines the significance of topics, publishers, subscriptions, and subscribers, and discusses best practices for designing topic trees. Additionally, it covers configuration and security considerations for topic objects and the management of topics within IBM MQ.

Embed presentation

Download as PDF, PPTX
© 2015 IBM Corporation AME-2271 IBM MQ: An introduction to using and developing with MQ publish/subscribe David Ware
2 Agenda • Publish/Subscribe in IBM MQ • Administration of publish/subscribe • Management of publish/subscribe • Subscriptions and publications • Topologies
What is publish/subscribe?
Topic point-to-point Message consumer Message consumer Message consumer Message producer 4 How does it compare to point-to-point? publish/subscribe Subscription Subscription Subscription Message consumer Message consumer Message consumer Message producer
5 But which subscriptions receive the messages? Subscription Subscription Subscription Subscription Publisher Publisher Publisher Subscription Subscription • Publishing and subscribing is based on ‘topics’ • Green messages go to green subscribers • Orange messages go to orange subscribers • But nobody wants a blue message! Publisher
Publish/Subscribe in IBM MQ
7 IBM MQ’s publish/subscribe over the years IBM MQ Publish/Subscribe SupportPac MA0C IBM Integration Bus® IBM MQ Command message based publish/subscribe API JMS publish/subscribe API Native MQI publish/subscribe API Publish/Subscribe brokers IBM MQ Publish/Subscribe APIs XMS publish/subscribe API MQ V 8MQ V 7MQ V 6MQ V 5.1
8 Terminology in IBM MQ MQ’s use of publish/subscribe terminology: • Topic This can mean topic strings, topic objects and even topic nodes, not to mention JMS topics! • Publisher A publisher is an application that puts messages to a topic as opposed to a queue. – A publisher can either open a topic object or a topic string. • Publication A publication is simply a message that is put to a topic rather than a queue • Subscription A subscription is an artefact that exists on a queue manager that describes where copies of messages published for a particular topic string are delivered to – A subscription identifies a queue for those messages • Subscriber A subscriber is an application that consumes messages from a subscription
Topics
10 Publish/Subscribe in IBM MQ • The queue manager holds a view of all the topic strings you are using in a hierarchical construct known as the topic tree (see next page). This topic tree is the central control point for all publish/subscribe. As a user you will interact with the topic tree in several different ways. • You can configure the behaviour of the topic tree by defining topic objects and changing attributes on them. Of course you only need to do this if you want to change the default behaviour. You may not need any topic objects – we will look at this in more detail later. • When using the MQI API you can programmatically interface with the topic tree as a subscriber using MQSUB and as a publisher using MQOPEN and MQPUT. • Applications using other APIs that already support publish/subscribe (e.g. JMS) continue to work as before. • You can monitor the use of your topic tree by such applications using the Topic status command, the Subscription status command and the commands to display connections and their handles.
11 It’s all about the topic tree Price Fruit Apples Oranges Vegetables Potatoes Topic string Topic node /Price/Fruit/Apples /Price/Fruit/Oranges /Price/Vegetables/Potatoes /Price /Price/Fruit /Price/Vegetables
12 Topic strings and topic tree • A topic string is broken up into ‘/’ delimited parts. Each part is represented as a node in IBM MQ’s topic tree. • As IBM MQ becomes aware of new topic strings, new topic nodes are automatically created in the topic tree. • Topic strings can be any characters you choose. You can, and should, add structure to you topic strings using the ‘/’ character. This produces a topic tree with a hierarchical structure, as the example on this foil shows. Although this hierarchical topic tree was created by the use of the topic strings shown, we generally picture it as a tree such as this. • There are some special characters, apart from the ‘/’ character that you should avoid in your topic strings. These are ‘#’, ‘+’, ‘*’ and ‘?’. We will look at these in more detail later when we discuss wildcards.
13 Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges •Subscriptions are attached to matching nodes in the topic tree •Publications identify the relevant topic node Publisher /Price/Fruit/Oranges •A copy of the publication is delivered to the queue identified by each matching subscription Subscription /Price/Fruit/Apples Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/# •Wildcarding subscriptions at the topic node level can receive messages from multiple topic strings Matching publications to subscriptions
14 Designing your topic tree structure • Make it extendable. • Understand a rapidly changing set of topic strings. • Avoid excessively wide or deep dynamic topic trees. • Use structure where appropriate. • Limit it to subscribable content. is my first topic /This/is/my/first/topic /This/is/my/second/topic second topic Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic2 Topic3 Topic4 Topic1000 Topic1 Topic2 Topic3 Topic3 Topic1000 AppA AppB AppC This
15 Designing your topic tree structure • No ‘standard’ way of designing the shape of a topic tree • Depends on the specific data • A well structured topic hierarchy should be intuitive to the user (but should also consider the later design considerations!) • General recommendation is to describe the high level / broad information, then break this down into finer detail • Start by looking at the ‘big picture’ • Topic Tree structure may evolve over time as user’s requirements become clearer – Build expandability into the design (e.g. always branch off from the root to allow future branches) • If using distributed pub/sub, avoid clustering the root node – consider using high level topics such as ‘/global’, ‘/local’ or cluster name to make their scope clear. • It’s possible to combine a topic hierarchy with message properties and selection strings to best fit the usage pattern • Potentially use where normally a subscriber may not require the further selection criteria but occasionally they may – E.g. Don’t encode the price into the topic for the rare occasion where a subscriber wants to know when it reaches £100, add it as a message property and allow a selector to be used. • Use of selectors can have an impact on performance • Selectors are not flowed around a multiple queue manager publish/subscribe topology – Selector parsing is only performed on the queue managers where the subscriptions exist
16 Why worry about the size and shape of the topic tree? • First, it makes your administration simpler! The less topic objects and topic nodes there are, the less there are to worry about. • Second, IBM MQ can efficiently scale to tens or hundreds of thousands of topic nodes but certain topic designs can cause IBM MQ pain, which impacts the user. • IBM MQ V8 has made significant improvements • It’s best to understand what’s happening internally… • When an application opens a topic string… • The topic string is looked up in a hash table – The more topic strings the bigger the hash table • If found, job done • If the topic string is not found the topic string is stripped down to its ‘/’ delimited parts and every new topic node that is referenced in the topic string is dynamically created. Each new node is linked to its parent. • For every topic node created, that branch of the topic tree is walked backwards to discover – The node’s configuration – The set of wildcard subscriptions that may match it • The more topic nodes the more memory consumed by the queue manager • The further left in a topic string that wildcard characters are used the more subscription processing required. • For this reason: • Very broad topic trees put stress on certain parent topic nodes • Very deep topic trees add unnecessary topic nodes and levels of the tree that need to be walked. • Very many topics can put a strain on memory resources of the system • Subscriptions with # wildcards at the start or middle of the topic string put them higher up the tree and can therefore mean they will need to be checked for almost all topic nodes being created
17 Why worry about the size and shape of the topic tree? Selectors • When an application publishes a message on a topic string • For every subscription associated with the topic node: – A decision is made to deliver a copy of the message or not, if the subscription has a selector the message properties are parsed to assess its suitability • For this reason: • By example, if there is a single topic with a thousand subscriptions, all with selectors, each publication will result in a thousand checks against those selctors. • However, if there are one hundred topics, each with ten subscriptions, all with selectors, each publication will result in only ten checks. • Both would result in the same number of publications, the latter with a lot less work
18 Why worry about the size and shape of the topic tree? Topic tree management • And finally, in the background… • Periodically the topic nodes are scanned to discover if they are still needed – An unneeded topic node is a leaf topic with no publishers or subscriptions currently directly attached – And the topic node has not been used for a period of time (minimum 30 minutes by default, controlled by the TREELIFE queue manager attribute) • In the case of unneeded topics they are deleted and unlinked from their parent – If new topic strings are continually being created, old ones need to be cleaned up at the same pace to prevent a problem. – From IBM MQ V8, use DISPLAY PUBSUB ALL to show the TPCOUNT for the queue manager. This is the current number of topic nodes in the tree. If this number continually grows, a problem may be present. • This scanning is necessary to keep a cap on memory use and a cluttered topic tree when viewing. • For this reason: • The bigger the topic hierarchy, the more additional work incurred. • Try not to have topics that are never or very rarely reused. – E.g. Don’t encode a unique job ID into a topic string, add it as a message property. • Don’t encode too many additional ‘/’ delimited layers in the tree where unnecessary.
Configuration
20 VEGOBJ TOPICSTR(‘/Price/Vegetables’) FRUITOBJ TOPICSTR(‘/Price/Fruit’) Topic Objects Price Fruit Apples Oranges Vegetables Potatoes SYSTEM.BASE.TOPIC TOPICSTR(‘’)  You start with a base object defined for the ‘ ’ node … the rest are optional.  They provide hook points in the topic tree to configure specific pub/sub behaviour for a branch.  A dynamically created topic node inherits its attributes from administered topic objects associated with topic nodes above it in the topic tree.  Topic objects are a point of administration associated with a node in the topic tree.
21 Topic Objects • Topic objects are a point of administration for a node or sub tree in the topic tree • SYSTEM.BASE.TOPIC is at the root of the topic tree • An administrator can define a topic object tied to any (non wildcard) topic string • Topic object definitions are configured in the same way as queue definitions • They provide a ‘hook’ in the Topic Tree to configure: • Security / access controls • Scope (local to queue manager or global) • …many other configurable attributes • A dynamically created topic node inherits its attributes from administered topic objects associated with topic nodes above it in the topic tree • Similarly one administered topic object can be configured to inherit some attributes from others above it • Configuring topic objects is optional • Only define topic objects where they are needed, additional objects will only add administrative overhead and possible confusion
22 FRUITOBJ PUB(ASPARENT) DEFPSIST(YES) … Topic object attributes Fruit Oranges SYSTEM.BASE.TOPIC PUB(ENABLED) DEFPSIST(NO) … Price • Many attributes can be set on topic objects to effect a publisher or subscriber’s behaviour. • Dynamic nodes inherit their behaviour from nodes above. Create a topic object for topic string ‘/Price/Fruit’ DEFINE TOPIC(FRUITOBJ) TOPICSTR(‘/Price/Fruit’) DEFPSIST(YES) Attributes default to inherit settings from above (e.g. ‘ASPARENT’). (So by default, a new object does nothing) Publish a message to topic string ‘/Price/Fruit/Oranges’ Are publications enabled? What message persistence to use? PUB = ? PUB = ENABLED PUB = ? DEFPSIST = ? DEFPSIST = YES Publisher PUB = ? YES ENABLED
23 SYSTEM.BASE.TOPIC FRUITOBJ Topic Security • Access control is set as for queues, but for a defined topic object, not a topic string! • Authority checks performed on the topic tree • Walk up the tree, just like attributes. • Keep checking until an authorisation is found or we run out of topic tree. Apples Bob Price VEGOBJ Jane Vegetables Subscription /Price/Fruit/Apples Bob Jane? Jane? Jane Subscription /Price/Fruit/Apples Fruit Bob?
24 Topic Security • To set access control on a branch of the topic tree: • First define a topic object that represents that point in the topic tree. • Define the access control based on the name of that topic object. • For example: • DEFINE TOPIC(FRUITOBJ) TOPICSTR(‘/Price/Fruit’) • setmqaut -m QMGR1 -t topic -n FRUITOBJ -p BOB +sub • Authority check are also performed on a subscription’s destination queue • MQ checks the subscriber has authority to PUT to that queue • (hang on for more on subscription queues) • Pick a suitable layer of the topic hierarchy and set access control at this point. • Think hard before adding additional access control at higher levels in the tree as this can cause confusion and grant very wide authorisations.
Managing topics
26 Managing topics • Displaying topic object definitions • This shows how administered topic objects are configured 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY TOPIC(*) 1 : DISPLAY TOPIC(*) AMQ8633: Display topic details. TOPIC(FRUITOBJ) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BASE.TOPIC) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.ADMIN.STREAM) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.DEFAULT.STREAM) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.DEFAULT.SUBPOINT) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.DEFAULT.TOPIC) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(VEGOBJ) TYPE(LOCAL) DISPLAY TOPIC(FRUITOBJ) 2 : DISPLAY TOPIC(FRUITOBJ) AMQ8633: Display topic details. TOPIC(FRUITOBJ) TYPE(LOCAL) TOPICSTR(/Price/Fruit) DESCR(Price of fruit) CLUSTER( ) CLROUTE(DIRECT) DURSUB(ASPARENT) PUB(ASPARENT) SUB(ASPARENT) DEFPSIST(YES) DEFPRTY(ASPARENT) DEFPRESP(ASPARENT) ALTDATE(2015-02-03) ALTTIME(08.44.48) PMSGDLV(ASPARENT) NPMSGDLV(ASPARENT) PUBSCOPE(ASPARENT) SUBSCOPE(ASPARENT) PROXYSUB(FIRSTUSE) WILDCARD(PASSTHRU) MDURMDL( ) MNDURMDL( ) MCAST(ASPARENT) COMMINFO( ) USEDLQ(ASPARENT) CUSTOM( )
27 Managing topic definitions • DISPLAY TOPIC(*) TOPICSTR • Display all locally defined topic objects on this queue manager, including the topic string they are associated with. • Although not yet covered in this presentation, but useful to list here, when using ‘clustered topics’ use the following command to show all the clustered topic objects known by this queue manager: • DISPLAY TOPIC(*) TYPE(CLUSTER) TOPICSTR • DISPLAY TOPIC(<name>) • Show all configured attributes of the topic definition. This includes where a value is to be inherited from a higher topic object or not (check for ‘ASPARENT’ or ‘ ‘ values). • (See the next slide for how to view the inherited values of a topic)
28 Managing topics • Displaying the topic tree • This shows how the topic nodes in the topic tree behave 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY PUBSUB ALL 1 : display PUBSUB all AMQ8723: Display pub/sub status details. QMNAME(QMGR1) TYPE(LOCAL) STATUS(ACTIVE) SUBCOUNT(5) TPCOUNT(11) DISPLAY TPSTATUS(‘#’) TOPICSTR WHERE(SUBCOUNT GT 0) 22 : DISPLAY TPSTATUS('#') TOPICSTR where(SUBCOUNT GT 0) AMQ8754: Display topic status details. TOPICSTR(/Price/Vegetables/Potatoes) SUBCOUNT(1) AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Oranges) SUBCOUNT(2) AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Apples) SUBCOUNT(1) DISPLAY TPSTATUS('/Price/Fruit/Apples') 23 : DISPLAY TPSTATUS('/Price/Fruit/Apples') AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Apples) ADMIN( ) CLUSTER( ) COMMINFO(SYSTEM.DEFAULT.COMMINFO.MULTICAST) MDURMDL(SYSTEM.DURABLE.MODEL.QUEUE) MNDURMDL(SYSTEM.NDURABLE.MODEL.QUEUE) CLROUTE(NONE) DEFPSIST(YES) DEFPRTY(0) DEFPRESP(SYNC) DURSUB(YES) PUB(ENABLED) SUB(ENABLED) PMSGDLV(ALLDUR) NPMSGDLV(ALLAVAIL) RETAINED(NO) MCAST(DISABLED) PUBCOUNT(0) SUBCOUNT(1) PUBSCOPE(ALL) SUBSCOPE(ALL) USEDLQ(YES) V8
29 Managing topic status • DISPLAY PUBSUB ALL • Returns TPCOUNT which gives an idea of the total number of topic nodes currently in the topic tree of this queue manager (Added in IBM MQ V8). • This can be used to see if the tree is continually growing (introducing a memory usage risk) or stable. If growing it may indicate a problem in the topic tree design (too many unique topic strings). Investigate the ‘TREELIFE’ attribute of the queue manager for ways of improving this. • DISPLAY TPSTATUS(‘#’) • This will return an entry for every node in the topic tree. • Note the use of quoted topic string wildcard notation (‘#’) to achieve this, not the traditional MQSC wildcarding (*). • DISPLAY TPSTATUS(‘#’) TOPICSTR WHERE(SUBCOUNT GT 0) • Perhaps more useful, this command will only return those topic nodes with one or more subscriptions registered against it. • DISPLAY TPSTATUS(‘<topic string>’) • Displaying an individual topic node (using its topic string value) will return all resolved attributes of that node. This replaces any inheritable values with what they resolve to in the topic tree (so you won’t see ‘ASPARENT’ in this output). • It also returns current status for this node, such as the number of registered subscriptions (SUBCOUNT) and currently connected publishers (PUBCOUNT).
Applications
31 Applications and topics • When creating subscriptions or opening topics to publish on, do I use a topic string or a topic object? • A topic string, a topic object or both. • So which should I use? • Using the topic string is probably the easiest, it’s closest to what the application writer is expecting – Sub( - , ‘/Price/Fruit/Apples’ ) → • Using a topic object maps the operation to the topic string of that topic object – Sub( FRUITOBJ, ‘’ ) → • If you use both, you get both! – The topic string is appended to the topic string of the object – Sub( FRUITOBJ, ‘Apples’ ) → • If in doubt, check the topic tree for which nodes exist and are actually being used /Price/Fruit/Apples /Price/Fruit /Price/Fruit/Apples
Subscriptions
33 Subscription types • There are many different types of subscriptions: • Administered or application created • Durable or non-durable • Managed or unmanaged subscription queues • These different aspects of a subscription can be combined, don’t assume it’s one or the other…
34 Subscription types Subscription creation and deletion • Application created subscriptions • Applications use an API to dynamically create and delete subscriptions • Administratively created subscriptions • An administrator defines subscriptions that can be accessed by applications • Applications can either use the publish/subscribe APIs to access these subscriptions or access their associated queue using point-to-point APIs. Admin Application
35 Types of Subscriptions Subscription creation and deletion • Application created subscriptions • Applications use an API to dynamically create and delete subscriptions – For example: – MQI uses the MQSUB to create and MQCLOSE to delete – JMS uses TopicSession.createDurableSubscriber() and TopicSession.unsubscribe() • Administratively created subscriptions • An administrator defines subscriptions that can be accessed by applications – For example, in MQSC: – DEFINE SUB(‘SUB1’) TOPICSTR(‘/Price/Fruit/Apples’)….. – DELETE SUB(‘SUB1’) • Applications can either use the publish/subscribe APIs to access these subscriptions or access their associated queue using point-to-point APIs.
36 Subscription types Subscription lifetime • Durable subscriptions • The lifetime of the subscription is independent of any application • Non-durable subscriptions • The lifetime of the subscription is bounded by the creating application – Subscriptions are automatically deleted when the application closes Admin Application Durable Non-durable Admin Application
37 Types of Subscriptions Subscription lifetime • Durable subscriptions • The lifetime of the subscription is independent of any application – Explicit creation and deletion of the subscription is required – Every durable subscription must be uniquely named within a queue manager – Subscriptions can be set to expire • Non-durable subscriptions • The lifetime of the subscription is bounded by the creating application – Subscriptions are automatically deleted when the application closes • These cannot be administratively created
38 Subscription types Subscription queue management • A subscription maps a topic to a queue. The queue relationship is either explicit or implicit… • Managed subscription queue • The subscription automatically creates and deletes a queue for the use of queuing any matching publications. • Unmanaged subscription queue • When the subscription is created the name and location of an existing queue must be provided by you. Durable Non-durable Admin Application Managed Unmanaged Durable Non-durable Durable Non-durable Admin Application (Not JMS) (Not JMS)
39 Types of Subscriptions Subscription queue management • Over simplifying things, a subscription essentially maps a topic to a queue • The queue is either explicit or implicit… • Managed subscription queue • The creation of the subscription automatically creates a queue to be used by this subscription for any matching publications. • Deleting the subscription automatically deletes the managed queue • JMS and XMS application created subscriptions are always managed subscriptions • Unmanaged subscription queue • When the subscription is created the name and location of an existing queue is provided • When the subscription is deleted the queue is not affected • Multiple subscriptions can point to the same unmanaged queue • Unmanaged subscriptions can be administratively created or programmatically using the MQI interface
Subscription 40 Accessing a subscription’s messages Via the subscription • An application opens the subscription • A ‘true’ pub/sub application • Works with managed and unmanaged subscription queues • Limited to one attached consuming application at a time • Unless you’re using JMS cloned/shared subscriptions • Generally better publish/subscribe status feedback Via the queue • An application opens the queue associated with the subscription • This is really a point-to-point application • Only works with unmanaged subscription queues • Allows more freedom in what can be done • For example, multiple concurrent consuming applications are possible or multiple subscriptions using a single queue. Subscription Managed queue Unmanaged queue Consuming application Consuming application Consuming application Subscription Subscription Consuming application
41 Controlling a subscription’s specific queue • Manually create the subscription through administration • For example, using runmqsc: DEFINE SUB(…) TOPICSTR(‘…’) DEST(name of queue) DESTQMGR(location of queue) … • Programmatically create the subscription • Through certain APIs – For example MQI, using the MQSUB verb – Not supported with JMS • Through programmatically sending an administrative command – By generating an MQCMD_CREATE_SUBSCRIPTION PCF command message • Through the queued publish/subscribe mechanism – Matches the pre MQ V7 model of sending subscription registration messages (MQPS_REGISTER_SUBSCRIBER) to a queue manager – Provided for compatibility with pre-V7 systems
Subscription types 42 • What there isn’t is a ‘persistent’ or a ‘non-persistent’ subscription • Any subscription type can receive persistent or non-persistent publications • Obviously, persistent publications to a non-durable subscription are not very persistent!
43 Managing subscriptions • Displaying subscriptions • This shows the subscriptions on a queue manager 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY SUB(*) TOPICSTR 1 : DISPLAY SUB(*) TOPICSTR AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002306) SUB(SUB1) TOPICSTR(/Price/Fruit/Apples) AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202004183D5310000006) SUB(SYSTEM.DEFAULT.SUB) TOPICSTR() AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002312) SUB(SUB3) TOPICSTR(/Price/Vegetables/Potatoes) SUBID(414D5120514D4752312020202020202007183D532000230C) SUB(SUB2) TOPICSTR(/Price/Fruit/Oranges) DISPLAY SUB(SUB1) 2 : DISPLAY SUB(SUB1) AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002306) SUB(SUB1) TOPICSTR(/Price/Fruit/Apples) TOPICOBJ( ) DEST(SYSTEM.MANAGED.DURABLE.533D180705230020) DESTQMGR(QMGR1) PUBAPPID( ) SELECTOR( ) SELTYPE(NONE) USERDATA( ) PUBACCT(16010515000000DEA960DF651724E4B97C192FE803000000000B) DESTCORL(414D5120514D4752312020202020202007183D5320002306) DESTCLAS(MANAGED) DURABLE(YES) EXPIRY(UNLIMITED) PSPROP(MSGPROP) PUBPRTY(ASPUB) REQONLY(NO) SUBSCOPE(ALL) SUBLEVEL(1) SUBTYPE(ADMIN) VARUSER(ANY) WSCHEMA(TOPIC) SUBUSER(xxxx) CRDATE(2015-02-03) CRTIME(09:19:15) ALTDATE(2015-02-03) ALTTIME(09:19:15) DISPLAY SBSTATUS(SUB1) 3 : DISPLAY SBSTATUS(SUB1) AMQ8099: WebSphere MQ subscription status inquired. SUB(SUB1) SUBID(414D5120514D4752312020202020202007183D5320002306) SUBUSER(xxxx) RESMDATE(2015-02-03) RESMTIME(09:19:15) LMSGDATE( ) LMSGTIME( ) ACTCONN(000000000000000000000000000000000000000000000000) DURABLE(YES) MCASTREL( , ) NUMMSGS(73) SUBTYPE(ADMIN) TOPICSTR(/Price/Fruit/Apples) DISPLAY QLOCAL(SYSTEM.MANAGED.DURABLE.533D180705230020)
44 Managing subscriptions • DISPLAY PUBSUB ALL • Gives an idea of the total number of subscriptions currently registered on this queue manager (Added in IBM MQ V8). • DISPLAY SUB(*) TOPICSTR • List all the subscriptions, including the topic string that they are subscribing to. • DISPLAY SUB(SUB1) • Display the configuration of a subscription. • DISPLAY SBSTATUS(SUB1) • Display the status of a subscription. This includes things like the last time it was resumed (opened) and approximately how message messages have matched this subscription since it was created (or since a queue manager restart, if later). • DISPLAY SBSTATUS(*) WHERE(ACTCONN NE '000000000000000000000000000000000000000000000000') • Another attribute returned by SBSTATUS is ACTCONN, this is the connection ID if the subscription is currently in use. The above query will return all in use subscriptions. The last 16 characters of ACTCONN can be used with DISPLAY CONN to show further application information. • DISPLAY QLOCAL(<subscription’s DEST>) • One property returned by DISPLAY SUB is the DEST, this is the queue where messages will be delivered. Standard queue queries can be used on that queue.
Publishing
46 Publication, success or failure? • Point-to-point is nice and simple: • Did the message get onto the queue? • Was it persistent and transacted? ConsumerProducer
47 Publication, success or failure? • Point-to-point is nice and simple: • Did the message get onto the queue? • Was it persistent and transacted? • Publish/subscribe is not so clear cut… • Persistence and transactions still ensures integrity of successful publications. • But if one or more subscriptions can’t receive the publication, should the publish fail? Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges
48 Publication, success or failure? • Should those subscriptions impact the others, should the publisher know? • What if the subscriptions are durable and the publication is persistent? • Controlled at the topic level • Persistent Message Delivery (PMSGDLV) and Non-persistent Message Delivery (NPMSGDLV). – ALL, ALLDUR, ALLAVAIL • Don’t forget that being able to DLQ a publication is still counted as a success! • USEDLQ on the topic to fine tune this behaviour. • And finally, remember – when there are no subscriptions, no-one gets it. That’s still a successful publish! Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges V7.1 DLQ
49 Publication, success or failure? • The publish/subscribe model encourages separation between the publishers of information and each of the subscribers to the information. • If a subscriber cannot receive a message published to a topic (e.g. it’s subscription queue is currently full), should that affect the other subscribers? And should the publisher be aware of the problem? • The default behaviour in IBM MQ depends on the persistence of the message and the durability of the subscription: • Persistent messages (PMSGDLV=ALLDUR): – The publish will fail if it cannot be delivered to one or more durable subscriptions. Failures to any non-durable subscriptions are acceptable. • Non-persistent messages (NPMSGDLV=ALLAVAIL): – Any failures to subscriptions are acceptable, the publish will succeed (even if no-one can receive the message). • This behaviour can be changed through configuration of the PMSGDLV and NPMSGDLV attributes of a topic. • If a dead letter queue is configured, a failure to put a publication to a subscription’s queue will result in an atempt to DLQ it. If that is successful the put to the subscription is classed as a success. • This can be controlled through the use of the USEDLQ option on a topic (available from WebSphere MQ V7.1 onwards).
50 Retained publications Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges • When a message is published to a topic string, it is delivered to each matching subscription registered at that time. • Subscriptions created after that point will not receive the message only newly published ones. • Unless publications are retained
51 Retained publications • When a message is published to a topic string, it is delivered to each matching subscription registered at that time. • Subscriptions created after that point will not receive the message only newly published ones. • Unless publications are retained • Every time a message is published, the most recent publication for each topic string is retained by the queue manager. • When a new subscription is created, any matching retained message is delivered to it. • Take care, using retained can be subtle • Don’t confuse it with persistent publications Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges
52 Retained publications • Retained publications can be useful if information on a topic is infrequently published, and a new subscriber needs to see the latest update as soon as they are created. • These messages are stored by the queue manager on a queue. The same recommendation of not building a very deep application queue applies equally here. Therefore, calculate the total number of topic strings where publications would be retained when considering using retained publications. • Retained publications in a multi queue manager pus/sub topology (not yet covered in this presentation) introduce complexities that need to be understood. • If you are only connected to the system from time to time, or if regular updates are not what you require, you can choose to only get publications when you request them (MQI only) • You make a subscription in the normal way, additionally using the MQSO_PUBLICATIONS_ONLY_REQUEST option. Then whenever you want the latest state, you make a request for it using the MQSUBRQ verb. This will just send you the latest publication on the topic you have subscribed on. • If you have a wild-carded subscription you will get the latest publication on each topic string that matches your subscription. When using the MQI, the field called NumPubs in the MQSRO structure will detail how many publications have been sent to you.
Topologies
54 QMgr CHANNEL QMgrQMgr GetGetPut Distributed queuing • We all know how point-to-point works • Even across multiple queue managers
55 Distributed publish/subscribe • We know how everything revolves around the topic tree, dynamically built up in a queue manager QMgr Topics Subscription Publisher Subscription Subscription Subscription
56 QMgrQMgr Topics Publisher QMgrQMgr Topics Subscription QMgrQMgr Topics Subscription Subscription Distributed publish/subscribe • We know how everything revolves around the topic tree, dynamically built up in a queue manager • Queue managers can work together to share their topic tree knowledge between them • Enabling publications to be propagated to subscriptions on different queue managers • The applications stay the same, the changes are at the configuration level. Subscription
57 Distributed publish/subscribe topologies • Publish/subscribe topologies can either be created as a defined hierarchy or more dynamically as a cluster Subscription Publisher Subscription Subscription Publisher Subscription Subscription Subscription
58 Distributed publish/subscribe topologies Subscription Publisher Subscription Subscription • A hierarchy manually defines the relationship between each queue manager. • Messages are flowed via those relationships.
59 Pub/Sub Hierarchies • By default publications will flow between queue managers from publishers to subscribers in a hierarchy. Points to consider: • Consider the scalability of the higher nodes in the hierarchy, especially the root as more publication traffic is expected to flow through these. • As for clusters, all topic strings subscribed to are propagated throughout the cluster, not just to the immediate neighbours – Rapid change to the set of subscriptions can become a limiting factor • Propagating is asynchronous, so early publications may not be sent following a subscription to a new topic string
60 Distributed publish/subscribe topologies • A cluster can be used for publish/subscribe through simple configuration. • Any queue manager can publish and subscribe to topics • Published messages can go direct between queue managers Publisher Subscription Subscription Subscription Publisher Subscription Subscription Subscription • Or be routed via specific queue managersV8 See AME-2272 for much more on topologies…
61 Pub/Sub Clusters • Publications will not flow between queue managers in a cluster unless the branch of the topic tree is clustered. • This is achieved by setting the ‘CLUSTER’ property of an administered topic object at the root of that topic tree to the name of the cluster. • The topic object can be defined in a single queue manager in the cluster, clustering it will automatically share the topic object definition with all other queue managers in the cluster. Points to consider: • A pub/sub cluster behaves very differently to a pure point-to-point cluster, plan accordingly: – All queue managers automatically know of all other queue managers – All queue managers with subscriptions to clustered topics will create channels to all other queue managers in the cluster – The above affects queue managers that have no publishers or subscribers but are in the same cluster as the topics. • All clustered topic strings subscribed to are propagated throughout the cluster – Rapid change to the set of subscriptions can become a limiting factor • Propagating is asynchronous, so early publications may not be sent following a subscription to a new topic string
62 Summary • Publish/Subscribe in IBM MQ • Administration of publish/subscribe • Management of publish/subscribe • Subscriptions and publications • Topologies
63 Notices and Disclaimers Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
64 Notices and Disclaimers (con’t) Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. • IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®, urban{code}®, Watson, WebSphere®, Worklight®, X- Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
Thank You Your Feedback is Important! Access the InterConnect 2015 Conference CONNECT Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe

More Related Content

PPT
IBM WebSphere MQ Introduction
byejlp12
 
PPT
IBM Websphere MQ Basic
byPRASAD BHATKAR
 
PDF
IBM MQ: Using Publish/Subscribe in an MQ Network
byDavid Ware
 
PDF
IBM MQ - What's new in 9.2
byDavid Ware
 
PPTX
IBM MQ Overview (IBM Message Queue)
byJuarez Junior
 
PDF
Websphere MQ (MQSeries) fundamentals
byBiju Nair
 
PDF
IBM MQ - High Availability and Disaster Recovery
byMarkTaylorIBM
 
PPT
SSL Implementation - IBM MQ - Secure Communications
bynishchal29
 
IBM WebSphere MQ Introduction
byejlp12
 
IBM Websphere MQ Basic
byPRASAD BHATKAR
 
IBM MQ: Using Publish/Subscribe in an MQ Network
byDavid Ware
 
IBM MQ - What's new in 9.2
byDavid Ware
 
IBM MQ Overview (IBM Message Queue)
byJuarez Junior
 
Websphere MQ (MQSeries) fundamentals
byBiju Nair
 
IBM MQ - High Availability and Disaster Recovery
byMarkTaylorIBM
 
SSL Implementation - IBM MQ - Secure Communications
bynishchal29
 

What's hot

PDF
Websphere MQ admin guide
byRam Babu
 
PPT
IBM MQ Online Tutorials
byBigClasses.com
 
PPTX
Building an Active-Active IBM MQ System
bymatthew1001
 
PDF
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
byDavid Ware
 
PDF
MQ Security Overview
byMarkTaylorIBM
 
PPTX
IBM MQ on cloud and containers
byRobert Parker
 
PPTX
MQ Infrastructure of Today and Tomorrow
byProlifics
 
PDF
IBM MQ and Kafka, what is the difference?
byDavid Ware
 
PDF
Domino policies deep dive
byMartijn de Jong
 
PDF
The Ultimate Administrator’s Guide to HCL Nomad Web
bypanagenda
 
PPTX
IBM MQ Whats new - including 9.3 and 9.3.1
byRobert Parker
 
PPTX
Apache kafka
byViswanath J
 
PPTX
Introduction to Apache Kafka
byJeff Holoman
 
PDF
IBM MQ Basics
bysubbareddyalamur
 
PPTX
Message Broker System and RabbitMQ
byUniversity of Alabama at Birmingham
 
PPTX
What's new with MQ on z/OS 9.3 and 9.3.1
byMatt Leming
 
PPTX
Getting up to speed with MirrorMaker 2 | Mickael Maison, IBM and Ryanne Dolan...
byHostedbyConfluent
 
PDF
IBM MQ - better application performance
byMarkTaylorIBM
 
PPTX
Overview - ESBs and IBM Integration Bus
byJuarez Junior
 
PDF
IBM WebSphere MQ V8 Security Features: Deep Dive
byMorag Hughson
 
Websphere MQ admin guide
byRam Babu
 
IBM MQ Online Tutorials
byBigClasses.com
 
Building an Active-Active IBM MQ System
bymatthew1001
 
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
byDavid Ware
 
MQ Security Overview
byMarkTaylorIBM
 
IBM MQ on cloud and containers
byRobert Parker
 
MQ Infrastructure of Today and Tomorrow
byProlifics
 
IBM MQ and Kafka, what is the difference?
byDavid Ware
 
Domino policies deep dive
byMartijn de Jong
 
The Ultimate Administrator’s Guide to HCL Nomad Web
bypanagenda
 
IBM MQ Whats new - including 9.3 and 9.3.1
byRobert Parker
 
Apache kafka
byViswanath J
 
Introduction to Apache Kafka
byJeff Holoman
 
IBM MQ Basics
bysubbareddyalamur
 
Message Broker System and RabbitMQ
byUniversity of Alabama at Birmingham
 
What's new with MQ on z/OS 9.3 and 9.3.1
byMatt Leming
 
Getting up to speed with MirrorMaker 2 | Mickael Maison, IBM and Ryanne Dolan...
byHostedbyConfluent
 
IBM MQ - better application performance
byMarkTaylorIBM
 
Overview - ESBs and IBM Integration Bus
byJuarez Junior
 
IBM WebSphere MQ V8 Security Features: Deep Dive
byMorag Hughson
 

Viewers also liked

PPT
IBM WebSphere MQ: Using Publish/Subscribe in an MQ Network
byDavid Ware
 
PPTX
Pub/Sub Messaging
byPeter Hanzlik
 
PPTX
Interconnect 2017: 6893 Keep out the bad guys by securing your MQ messaging e...
byRobert Parker
 
PPTX
Interconnect 2017: 6885 Deploying IBM MQ in the cloud
byRobert Parker
 
PDF
Where is My Message
byMatt Leming
 
PPTX
New Tools and Interfaces for Managing IBM MQ
byMatt Leming
 
PPTX
A scalable and reliable matching service slide
bysomnath goud
 
PPTX
Effective administration of IBM Integration Bus - Sanjay Nagchowdhury
byKaren Broughton-Mabbitt
 
PPTX
MQTC 2016: Monitoring and Tracking MQ and Applications
byRobert Parker
 
PDF
Mq ssl channels_on_windows
bykarthickmsit
 
PPTX
CTU 2017 - I168 IBM MQ in the cloud
byRobert Parker
 
PDF
MQTC 2016: IBM MQ Security deep dive including AMS
byRobert Parker
 
PPTX
Whats New in IBM Integration Bus Interconnect 2017
bybthomps1979
 
PDF
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
byDavid Ware
 
PDF
IBM Design Thinking + Agile + DevOps Interconnect 2017
byDavid Luke
 
DOCX
A scalable and reliable matching service for content based publish subscribe ...
bysomnath goud
 
PDF
InterConnect 2016: IBM MQ self-service and as-a-service
byDavid Ware
 
PPTX
A scalable and reliable matching service for content based
bysyeda yasmeen
 
PDF
Common DataPower use cases, incl Caching with XC-10 appliance.
bysflynn073
 
IBM WebSphere MQ: Using Publish/Subscribe in an MQ Network
byDavid Ware
 
Pub/Sub Messaging
byPeter Hanzlik
 
Interconnect 2017: 6893 Keep out the bad guys by securing your MQ messaging e...
byRobert Parker
 
Interconnect 2017: 6885 Deploying IBM MQ in the cloud
byRobert Parker
 
Where is My Message
byMatt Leming
 
New Tools and Interfaces for Managing IBM MQ
byMatt Leming
 
A scalable and reliable matching service slide
bysomnath goud
 
Effective administration of IBM Integration Bus - Sanjay Nagchowdhury
byKaren Broughton-Mabbitt
 
MQTC 2016: Monitoring and Tracking MQ and Applications
byRobert Parker
 
Mq ssl channels_on_windows
bykarthickmsit
 
CTU 2017 - I168 IBM MQ in the cloud
byRobert Parker
 
MQTC 2016: IBM MQ Security deep dive including AMS
byRobert Parker
 
Whats New in IBM Integration Bus Interconnect 2017
bybthomps1979
 
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
byDavid Ware
 
IBM Design Thinking + Agile + DevOps Interconnect 2017
byDavid Luke
 
A scalable and reliable matching service for content based publish subscribe ...
bysomnath goud
 
InterConnect 2016: IBM MQ self-service and as-a-service
byDavid Ware
 
A scalable and reliable matching service for content based
bysyeda yasmeen
 
Common DataPower use cases, incl Caching with XC-10 appliance.
bysflynn073
 

Similar to IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe

PDF
3425 - Using publish/subscribe to integrate applications
byTimothy McCormick
 
PDF
IBM Publish Subscribe in a Network
byIBM Systems UKI
 
PPTX
Message queuing telemetry transport (mqtt)and part 3 and summarizing
byHamdamboy
 
PPTX
Message queuing telemetry transport (mqtt)and part 3 and summarizing
byHamdamboy (함담보이)
 
PPTX
Message queuing telemetry transport (mqtt) topic parameters
byHamdamboy
 
PDF
Introduction to Apache Kafka
byvishnu rao
 
PDF
WM2011G19.pdf
byweblogicfans
 
PDF
1463401 rc214-mqtt-update
byEugenio Lysei
 
PDF
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities
byPeter Broadhurst
 
PPTX
Enterprise messaging with jms
bySridhar Reddy
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PPT
test
bytechweb08
 
PDF
MQTT in Reactive Blocks
byBitreactive
 
PPTX
MQTT(Message queuing and telemetry transport)
byprashantchopra30
 
PPT
test
bytechweb08
 
3425 - Using publish/subscribe to integrate applications
byTimothy McCormick
 
IBM Publish Subscribe in a Network
byIBM Systems UKI
 
Message queuing telemetry transport (mqtt)and part 3 and summarizing
byHamdamboy
 
Message queuing telemetry transport (mqtt)and part 3 and summarizing
byHamdamboy (함담보이)
 
Message queuing telemetry transport (mqtt) topic parameters
byHamdamboy
 
Introduction to Apache Kafka
byvishnu rao
 
WM2011G19.pdf
byweblogicfans
 
1463401 rc214-mqtt-update
byEugenio Lysei
 
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities
byPeter Broadhurst
 
Enterprise messaging with jms
bySridhar Reddy
 
test
bytechweb08
 
test
bytechweb08
 
test
bytechweb08
 
test
bytechweb08
 
test
bytechweb08
 
test
bytechweb08
 
test
bytechweb08
 
MQTT in Reactive Blocks
byBitreactive
 
MQTT(Message queuing and telemetry transport)
byprashantchopra30
 
test
bytechweb08
 

More from David Ware

PDF
IBM MQ What's new - Sept 2022
byDavid Ware
 
PDF
Fault tolerant and scalable ibm mq
byDavid Ware
 
PDF
IBM MQ High Availability 2019
byDavid Ware
 
PDF
IBM MQ Update, including 9.1.2 CD
byDavid Ware
 
PDF
Whats new in MQ V9.1
byDavid Ware
 
PDF
What's new in IBM MQ, March 2018
byDavid Ware
 
PDF
Designing IBM MQ deployments for the cloud generation
byDavid Ware
 
PDF
InterConnect 2016: What's new in IBM MQ
byDavid Ware
 
PPT
IBM WebSphere MQ: Managing Workloads, Scaling and Availability with MQ Clusters
byDavid Ware
 
IBM MQ What's new - Sept 2022
byDavid Ware
 
Fault tolerant and scalable ibm mq
byDavid Ware
 
IBM MQ High Availability 2019
byDavid Ware
 
IBM MQ Update, including 9.1.2 CD
byDavid Ware
 
Whats new in MQ V9.1
byDavid Ware
 
What's new in IBM MQ, March 2018
byDavid Ware
 
Designing IBM MQ deployments for the cloud generation
byDavid Ware
 
InterConnect 2016: What's new in IBM MQ
byDavid Ware
 
IBM WebSphere MQ: Managing Workloads, Scaling and Availability with MQ Clusters
byDavid Ware
 

Recently uploaded

PPTX
Managed Splunk Partner vs In-House: Cost, Risk & Value Comparison
byssuser6ab5a5
 
PDF
KoderXpert – Odoo, Web & AI Solutions for Growing Businesses
byDhvanil Trivedi
 
PDF
Here’s the case study that shows how companies lose ₹2–6 Cr annually due to m...
bysiddhantdazzlo
 
PPTX
GDS Integration Solution | GDS Integration Service
byyugababu033
 
PDF
Imed Eddine Bouchoucha | computer engineer | software Architect
byImed Bouchoucha
 
PDF
INTRODUCTION TO DATABASES, MYSQL, MS ACCESS, PHARMACY DRUG DATABASE.pdf
bylikudas9861
 
PPTX
Deep Dive into Durable Functions, presented at Cloudbrew 2025
byJoonas Westlin
 
PPTX
AI Clinic Management Software for Pulmonology Clinics Bringing Clarity, Contr...
byEasy Clinic
 
PPTX
application security presentation 2 by harman
byharmanideapad
 
PDF
Constraints First - Why Our On-Prem Ticketing System Starts With Limits, Not ...
bySpirit Face
 
PPTX
Magnet-AXIOM_overview_tool_cyber_tool.pptx
byashishtjoseph
 
PDF
Why Zoho Notebook’s AI-Fueled Upgrade Matters for Knowledge Workers in 2026
byEvoluz Global Solutions
 
PDF
IT Estate Modernization: Transform Your Infrastructure for the Future .pdf
byAstuteBusiness
 
DOCX
How to Change Classic SharePoint to Modern SharePoint (An Updated Guide)
bySharepoint Designs
 
PDF
Combinatorial Interview Problems with Backtracking Solutions - From Imperativ...
byPhilip Schwarz
 
PDF
Blueprint to build quality before the code exists - StackConnect Milan 2025
byLudovicoBesana1
 
PDF
API_SECURITY CONSULTANCY SERVICES IN USA
bydavidjohansen1597
 
PDF
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 
PPTX
AI Clinic Management Software for Otolaryngology Clinics Bringing Precision, ...
byEasy Clinic
 
PPTX
Application Security – Static Application Security Testing (SAST)
byLalit Jagotra
 
Managed Splunk Partner vs In-House: Cost, Risk & Value Comparison
byssuser6ab5a5
 
KoderXpert – Odoo, Web & AI Solutions for Growing Businesses
byDhvanil Trivedi
 
Here’s the case study that shows how companies lose ₹2–6 Cr annually due to m...
bysiddhantdazzlo
 
GDS Integration Solution | GDS Integration Service
byyugababu033
 
Imed Eddine Bouchoucha | computer engineer | software Architect
byImed Bouchoucha
 
INTRODUCTION TO DATABASES, MYSQL, MS ACCESS, PHARMACY DRUG DATABASE.pdf
bylikudas9861
 
Deep Dive into Durable Functions, presented at Cloudbrew 2025
byJoonas Westlin
 
AI Clinic Management Software for Pulmonology Clinics Bringing Clarity, Contr...
byEasy Clinic
 
application security presentation 2 by harman
byharmanideapad
 
Constraints First - Why Our On-Prem Ticketing System Starts With Limits, Not ...
bySpirit Face
 
Magnet-AXIOM_overview_tool_cyber_tool.pptx
byashishtjoseph
 
Why Zoho Notebook’s AI-Fueled Upgrade Matters for Knowledge Workers in 2026
byEvoluz Global Solutions
 
IT Estate Modernization: Transform Your Infrastructure for the Future .pdf
byAstuteBusiness
 
How to Change Classic SharePoint to Modern SharePoint (An Updated Guide)
bySharepoint Designs
 
Combinatorial Interview Problems with Backtracking Solutions - From Imperativ...
byPhilip Schwarz
 
Blueprint to build quality before the code exists - StackConnect Milan 2025
byLudovicoBesana1
 
API_SECURITY CONSULTANCY SERVICES IN USA
bydavidjohansen1597
 
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 
AI Clinic Management Software for Otolaryngology Clinics Bringing Precision, ...
byEasy Clinic
 
Application Security – Static Application Security Testing (SAST)
byLalit Jagotra
 

IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe

  • 1.
    © 2015 IBMCorporation AME-2271 IBM MQ: An introduction to using and developing with MQ publish/subscribe David Ware
  • 2.
    2 Agenda • Publish/Subscribe inIBM MQ • Administration of publish/subscribe • Management of publish/subscribe • Subscriptions and publications • Topologies
  • 3.
  • 4.
    Topic point-to-point Message consumer Message consumer Messageconsumer Message producer 4 How does it compare to point-to-point? publish/subscribe Subscription Subscription Subscription Message consumer Message consumer Message consumer Message producer
  • 5.
    5 But which subscriptionsreceive the messages? Subscription Subscription Subscription Subscription Publisher Publisher Publisher Subscription Subscription • Publishing and subscribing is based on ‘topics’ • Green messages go to green subscribers • Orange messages go to orange subscribers • But nobody wants a blue message! Publisher
  • 6.
  • 7.
    7 IBM MQ’s publish/subscribeover the years IBM MQ Publish/Subscribe SupportPac MA0C IBM Integration Bus® IBM MQ Command message based publish/subscribe API JMS publish/subscribe API Native MQI publish/subscribe API Publish/Subscribe brokers IBM MQ Publish/Subscribe APIs XMS publish/subscribe API MQ V 8MQ V 7MQ V 6MQ V 5.1
  • 8.
    8 Terminology in IBMMQ MQ’s use of publish/subscribe terminology: • Topic This can mean topic strings, topic objects and even topic nodes, not to mention JMS topics! • Publisher A publisher is an application that puts messages to a topic as opposed to a queue. – A publisher can either open a topic object or a topic string. • Publication A publication is simply a message that is put to a topic rather than a queue • Subscription A subscription is an artefact that exists on a queue manager that describes where copies of messages published for a particular topic string are delivered to – A subscription identifies a queue for those messages • Subscriber A subscriber is an application that consumes messages from a subscription
  • 9.
  • 10.
    10 Publish/Subscribe in IBMMQ • The queue manager holds a view of all the topic strings you are using in a hierarchical construct known as the topic tree (see next page). This topic tree is the central control point for all publish/subscribe. As a user you will interact with the topic tree in several different ways. • You can configure the behaviour of the topic tree by defining topic objects and changing attributes on them. Of course you only need to do this if you want to change the default behaviour. You may not need any topic objects – we will look at this in more detail later. • When using the MQI API you can programmatically interface with the topic tree as a subscriber using MQSUB and as a publisher using MQOPEN and MQPUT. • Applications using other APIs that already support publish/subscribe (e.g. JMS) continue to work as before. • You can monitor the use of your topic tree by such applications using the Topic status command, the Subscription status command and the commands to display connections and their handles.
  • 11.
    11 It’s all aboutthe topic tree Price Fruit Apples Oranges Vegetables Potatoes Topic string Topic node /Price/Fruit/Apples /Price/Fruit/Oranges /Price/Vegetables/Potatoes /Price /Price/Fruit /Price/Vegetables
  • 12.
    12 Topic strings andtopic tree • A topic string is broken up into ‘/’ delimited parts. Each part is represented as a node in IBM MQ’s topic tree. • As IBM MQ becomes aware of new topic strings, new topic nodes are automatically created in the topic tree. • Topic strings can be any characters you choose. You can, and should, add structure to you topic strings using the ‘/’ character. This produces a topic tree with a hierarchical structure, as the example on this foil shows. Although this hierarchical topic tree was created by the use of the topic strings shown, we generally picture it as a tree such as this. • There are some special characters, apart from the ‘/’ character that you should avoid in your topic strings. These are ‘#’, ‘+’, ‘*’ and ‘?’. We will look at these in more detail later when we discuss wildcards.
  • 13.
    13 Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges •Subscriptions areattached to matching nodes in the topic tree •Publications identify the relevant topic node Publisher /Price/Fruit/Oranges •A copy of the publication is delivered to the queue identified by each matching subscription Subscription /Price/Fruit/Apples Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/# •Wildcarding subscriptions at the topic node level can receive messages from multiple topic strings Matching publications to subscriptions
  • 14.
    14 Designing your topictree structure • Make it extendable. • Understand a rapidly changing set of topic strings. • Avoid excessively wide or deep dynamic topic trees. • Use structure where appropriate. • Limit it to subscribable content. is my first topic /This/is/my/first/topic /This/is/my/second/topic second topic Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic1 Topic2 Topic2 Topic3 Topic3 Topic4 Topic4 Topic5 Topic5 Topic6 Topic6 Topic7 Topic7 Topic8 Topic8 Topic9 Topic9 Topic10 Topic10 Topic1 Topic2 Topic3 Topic4 Topic1000 Topic1 Topic2 Topic3 Topic3 Topic1000 AppA AppB AppC This
  • 15.
    15 Designing your topictree structure • No ‘standard’ way of designing the shape of a topic tree • Depends on the specific data • A well structured topic hierarchy should be intuitive to the user (but should also consider the later design considerations!) • General recommendation is to describe the high level / broad information, then break this down into finer detail • Start by looking at the ‘big picture’ • Topic Tree structure may evolve over time as user’s requirements become clearer – Build expandability into the design (e.g. always branch off from the root to allow future branches) • If using distributed pub/sub, avoid clustering the root node – consider using high level topics such as ‘/global’, ‘/local’ or cluster name to make their scope clear. • It’s possible to combine a topic hierarchy with message properties and selection strings to best fit the usage pattern • Potentially use where normally a subscriber may not require the further selection criteria but occasionally they may – E.g. Don’t encode the price into the topic for the rare occasion where a subscriber wants to know when it reaches £100, add it as a message property and allow a selector to be used. • Use of selectors can have an impact on performance • Selectors are not flowed around a multiple queue manager publish/subscribe topology – Selector parsing is only performed on the queue managers where the subscriptions exist
  • 16.
    16 Why worry aboutthe size and shape of the topic tree? • First, it makes your administration simpler! The less topic objects and topic nodes there are, the less there are to worry about. • Second, IBM MQ can efficiently scale to tens or hundreds of thousands of topic nodes but certain topic designs can cause IBM MQ pain, which impacts the user. • IBM MQ V8 has made significant improvements • It’s best to understand what’s happening internally… • When an application opens a topic string… • The topic string is looked up in a hash table – The more topic strings the bigger the hash table • If found, job done • If the topic string is not found the topic string is stripped down to its ‘/’ delimited parts and every new topic node that is referenced in the topic string is dynamically created. Each new node is linked to its parent. • For every topic node created, that branch of the topic tree is walked backwards to discover – The node’s configuration – The set of wildcard subscriptions that may match it • The more topic nodes the more memory consumed by the queue manager • The further left in a topic string that wildcard characters are used the more subscription processing required. • For this reason: • Very broad topic trees put stress on certain parent topic nodes • Very deep topic trees add unnecessary topic nodes and levels of the tree that need to be walked. • Very many topics can put a strain on memory resources of the system • Subscriptions with # wildcards at the start or middle of the topic string put them higher up the tree and can therefore mean they will need to be checked for almost all topic nodes being created
  • 17.
    17 Why worry aboutthe size and shape of the topic tree? Selectors • When an application publishes a message on a topic string • For every subscription associated with the topic node: – A decision is made to deliver a copy of the message or not, if the subscription has a selector the message properties are parsed to assess its suitability • For this reason: • By example, if there is a single topic with a thousand subscriptions, all with selectors, each publication will result in a thousand checks against those selctors. • However, if there are one hundred topics, each with ten subscriptions, all with selectors, each publication will result in only ten checks. • Both would result in the same number of publications, the latter with a lot less work
  • 18.
    18 Why worry aboutthe size and shape of the topic tree? Topic tree management • And finally, in the background… • Periodically the topic nodes are scanned to discover if they are still needed – An unneeded topic node is a leaf topic with no publishers or subscriptions currently directly attached – And the topic node has not been used for a period of time (minimum 30 minutes by default, controlled by the TREELIFE queue manager attribute) • In the case of unneeded topics they are deleted and unlinked from their parent – If new topic strings are continually being created, old ones need to be cleaned up at the same pace to prevent a problem. – From IBM MQ V8, use DISPLAY PUBSUB ALL to show the TPCOUNT for the queue manager. This is the current number of topic nodes in the tree. If this number continually grows, a problem may be present. • This scanning is necessary to keep a cap on memory use and a cluttered topic tree when viewing. • For this reason: • The bigger the topic hierarchy, the more additional work incurred. • Try not to have topics that are never or very rarely reused. – E.g. Don’t encode a unique job ID into a topic string, add it as a message property. • Don’t encode too many additional ‘/’ delimited layers in the tree where unnecessary.
  • 19.
  • 20.
    20 VEGOBJ TOPICSTR(‘/Price/Vegetables’) FRUITOBJ TOPICSTR(‘/Price/Fruit’) Topic Objects Price Fruit Apples Oranges Vegetables Potatoes SYSTEM.BASE.TOPIC TOPICSTR(‘’) You start with a base object defined for the ‘ ’ node … the rest are optional.  They provide hook points in the topic tree to configure specific pub/sub behaviour for a branch.  A dynamically created topic node inherits its attributes from administered topic objects associated with topic nodes above it in the topic tree.  Topic objects are a point of administration associated with a node in the topic tree.
  • 21.
    21 Topic Objects • Topicobjects are a point of administration for a node or sub tree in the topic tree • SYSTEM.BASE.TOPIC is at the root of the topic tree • An administrator can define a topic object tied to any (non wildcard) topic string • Topic object definitions are configured in the same way as queue definitions • They provide a ‘hook’ in the Topic Tree to configure: • Security / access controls • Scope (local to queue manager or global) • …many other configurable attributes • A dynamically created topic node inherits its attributes from administered topic objects associated with topic nodes above it in the topic tree • Similarly one administered topic object can be configured to inherit some attributes from others above it • Configuring topic objects is optional • Only define topic objects where they are needed, additional objects will only add administrative overhead and possible confusion
  • 22.
    22 FRUITOBJ PUB(ASPARENT) DEFPSIST(YES) … Topic object attributes Fruit Oranges SYSTEM.BASE.TOPIC PUB(ENABLED) DEFPSIST(NO) … Price •Many attributes can be set on topic objects to effect a publisher or subscriber’s behaviour. • Dynamic nodes inherit their behaviour from nodes above. Create a topic object for topic string ‘/Price/Fruit’ DEFINE TOPIC(FRUITOBJ) TOPICSTR(‘/Price/Fruit’) DEFPSIST(YES) Attributes default to inherit settings from above (e.g. ‘ASPARENT’). (So by default, a new object does nothing) Publish a message to topic string ‘/Price/Fruit/Oranges’ Are publications enabled? What message persistence to use? PUB = ? PUB = ENABLED PUB = ? DEFPSIST = ? DEFPSIST = YES Publisher PUB = ? YES ENABLED
  • 23.
    23 SYSTEM.BASE.TOPIC FRUITOBJ Topic Security • Accesscontrol is set as for queues, but for a defined topic object, not a topic string! • Authority checks performed on the topic tree • Walk up the tree, just like attributes. • Keep checking until an authorisation is found or we run out of topic tree. Apples Bob Price VEGOBJ Jane Vegetables Subscription /Price/Fruit/Apples Bob Jane? Jane? Jane Subscription /Price/Fruit/Apples Fruit Bob?
  • 24.
    24 Topic Security • Toset access control on a branch of the topic tree: • First define a topic object that represents that point in the topic tree. • Define the access control based on the name of that topic object. • For example: • DEFINE TOPIC(FRUITOBJ) TOPICSTR(‘/Price/Fruit’) • setmqaut -m QMGR1 -t topic -n FRUITOBJ -p BOB +sub • Authority check are also performed on a subscription’s destination queue • MQ checks the subscriber has authority to PUT to that queue • (hang on for more on subscription queues) • Pick a suitable layer of the topic hierarchy and set access control at this point. • Think hard before adding additional access control at higher levels in the tree as this can cause confusion and grant very wide authorisations.
  • 25.
  • 26.
    26 Managing topics • Displayingtopic object definitions • This shows how administered topic objects are configured 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY TOPIC(*) 1 : DISPLAY TOPIC(*) AMQ8633: Display topic details. TOPIC(FRUITOBJ) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BASE.TOPIC) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.ADMIN.STREAM) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.DEFAULT.STREAM) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.BROKER.DEFAULT.SUBPOINT) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(SYSTEM.DEFAULT.TOPIC) TYPE(LOCAL) AMQ8633: Display topic details. TOPIC(VEGOBJ) TYPE(LOCAL) DISPLAY TOPIC(FRUITOBJ) 2 : DISPLAY TOPIC(FRUITOBJ) AMQ8633: Display topic details. TOPIC(FRUITOBJ) TYPE(LOCAL) TOPICSTR(/Price/Fruit) DESCR(Price of fruit) CLUSTER( ) CLROUTE(DIRECT) DURSUB(ASPARENT) PUB(ASPARENT) SUB(ASPARENT) DEFPSIST(YES) DEFPRTY(ASPARENT) DEFPRESP(ASPARENT) ALTDATE(2015-02-03) ALTTIME(08.44.48) PMSGDLV(ASPARENT) NPMSGDLV(ASPARENT) PUBSCOPE(ASPARENT) SUBSCOPE(ASPARENT) PROXYSUB(FIRSTUSE) WILDCARD(PASSTHRU) MDURMDL( ) MNDURMDL( ) MCAST(ASPARENT) COMMINFO( ) USEDLQ(ASPARENT) CUSTOM( )
  • 27.
    27 Managing topic definitions •DISPLAY TOPIC(*) TOPICSTR • Display all locally defined topic objects on this queue manager, including the topic string they are associated with. • Although not yet covered in this presentation, but useful to list here, when using ‘clustered topics’ use the following command to show all the clustered topic objects known by this queue manager: • DISPLAY TOPIC(*) TYPE(CLUSTER) TOPICSTR • DISPLAY TOPIC(<name>) • Show all configured attributes of the topic definition. This includes where a value is to be inherited from a higher topic object or not (check for ‘ASPARENT’ or ‘ ‘ values). • (See the next slide for how to view the inherited values of a topic)
  • 28.
    28 Managing topics • Displayingthe topic tree • This shows how the topic nodes in the topic tree behave 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY PUBSUB ALL 1 : display PUBSUB all AMQ8723: Display pub/sub status details. QMNAME(QMGR1) TYPE(LOCAL) STATUS(ACTIVE) SUBCOUNT(5) TPCOUNT(11) DISPLAY TPSTATUS(‘#’) TOPICSTR WHERE(SUBCOUNT GT 0) 22 : DISPLAY TPSTATUS('#') TOPICSTR where(SUBCOUNT GT 0) AMQ8754: Display topic status details. TOPICSTR(/Price/Vegetables/Potatoes) SUBCOUNT(1) AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Oranges) SUBCOUNT(2) AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Apples) SUBCOUNT(1) DISPLAY TPSTATUS('/Price/Fruit/Apples') 23 : DISPLAY TPSTATUS('/Price/Fruit/Apples') AMQ8754: Display topic status details. TOPICSTR(/Price/Fruit/Apples) ADMIN( ) CLUSTER( ) COMMINFO(SYSTEM.DEFAULT.COMMINFO.MULTICAST) MDURMDL(SYSTEM.DURABLE.MODEL.QUEUE) MNDURMDL(SYSTEM.NDURABLE.MODEL.QUEUE) CLROUTE(NONE) DEFPSIST(YES) DEFPRTY(0) DEFPRESP(SYNC) DURSUB(YES) PUB(ENABLED) SUB(ENABLED) PMSGDLV(ALLDUR) NPMSGDLV(ALLAVAIL) RETAINED(NO) MCAST(DISABLED) PUBCOUNT(0) SUBCOUNT(1) PUBSCOPE(ALL) SUBSCOPE(ALL) USEDLQ(YES) V8
  • 29.
    29 Managing topic status •DISPLAY PUBSUB ALL • Returns TPCOUNT which gives an idea of the total number of topic nodes currently in the topic tree of this queue manager (Added in IBM MQ V8). • This can be used to see if the tree is continually growing (introducing a memory usage risk) or stable. If growing it may indicate a problem in the topic tree design (too many unique topic strings). Investigate the ‘TREELIFE’ attribute of the queue manager for ways of improving this. • DISPLAY TPSTATUS(‘#’) • This will return an entry for every node in the topic tree. • Note the use of quoted topic string wildcard notation (‘#’) to achieve this, not the traditional MQSC wildcarding (*). • DISPLAY TPSTATUS(‘#’) TOPICSTR WHERE(SUBCOUNT GT 0) • Perhaps more useful, this command will only return those topic nodes with one or more subscriptions registered against it. • DISPLAY TPSTATUS(‘<topic string>’) • Displaying an individual topic node (using its topic string value) will return all resolved attributes of that node. This replaces any inheritable values with what they resolve to in the topic tree (so you won’t see ‘ASPARENT’ in this output). • It also returns current status for this node, such as the number of registered subscriptions (SUBCOUNT) and currently connected publishers (PUBCOUNT).
  • 30.
  • 31.
    31 Applications and topics •When creating subscriptions or opening topics to publish on, do I use a topic string or a topic object? • A topic string, a topic object or both. • So which should I use? • Using the topic string is probably the easiest, it’s closest to what the application writer is expecting – Sub( - , ‘/Price/Fruit/Apples’ ) → • Using a topic object maps the operation to the topic string of that topic object – Sub( FRUITOBJ, ‘’ ) → • If you use both, you get both! – The topic string is appended to the topic string of the object – Sub( FRUITOBJ, ‘Apples’ ) → • If in doubt, check the topic tree for which nodes exist and are actually being used /Price/Fruit/Apples /Price/Fruit /Price/Fruit/Apples
  • 32.
  • 33.
    33 Subscription types • Thereare many different types of subscriptions: • Administered or application created • Durable or non-durable • Managed or unmanaged subscription queues • These different aspects of a subscription can be combined, don’t assume it’s one or the other…
  • 34.
    34 Subscription types Subscription creationand deletion • Application created subscriptions • Applications use an API to dynamically create and delete subscriptions • Administratively created subscriptions • An administrator defines subscriptions that can be accessed by applications • Applications can either use the publish/subscribe APIs to access these subscriptions or access their associated queue using point-to-point APIs. Admin Application
  • 35.
    35 Types of Subscriptions Subscriptioncreation and deletion • Application created subscriptions • Applications use an API to dynamically create and delete subscriptions – For example: – MQI uses the MQSUB to create and MQCLOSE to delete – JMS uses TopicSession.createDurableSubscriber() and TopicSession.unsubscribe() • Administratively created subscriptions • An administrator defines subscriptions that can be accessed by applications – For example, in MQSC: – DEFINE SUB(‘SUB1’) TOPICSTR(‘/Price/Fruit/Apples’)….. – DELETE SUB(‘SUB1’) • Applications can either use the publish/subscribe APIs to access these subscriptions or access their associated queue using point-to-point APIs.
  • 36.
    36 Subscription types Subscription lifetime •Durable subscriptions • The lifetime of the subscription is independent of any application • Non-durable subscriptions • The lifetime of the subscription is bounded by the creating application – Subscriptions are automatically deleted when the application closes Admin Application Durable Non-durable Admin Application
  • 37.
    37 Types of Subscriptions Subscriptionlifetime • Durable subscriptions • The lifetime of the subscription is independent of any application – Explicit creation and deletion of the subscription is required – Every durable subscription must be uniquely named within a queue manager – Subscriptions can be set to expire • Non-durable subscriptions • The lifetime of the subscription is bounded by the creating application – Subscriptions are automatically deleted when the application closes • These cannot be administratively created
  • 38.
    38 Subscription types Subscription queuemanagement • A subscription maps a topic to a queue. The queue relationship is either explicit or implicit… • Managed subscription queue • The subscription automatically creates and deletes a queue for the use of queuing any matching publications. • Unmanaged subscription queue • When the subscription is created the name and location of an existing queue must be provided by you. Durable Non-durable Admin Application Managed Unmanaged Durable Non-durable Durable Non-durable Admin Application (Not JMS) (Not JMS)
  • 39.
    39 Types of Subscriptions Subscriptionqueue management • Over simplifying things, a subscription essentially maps a topic to a queue • The queue is either explicit or implicit… • Managed subscription queue • The creation of the subscription automatically creates a queue to be used by this subscription for any matching publications. • Deleting the subscription automatically deletes the managed queue • JMS and XMS application created subscriptions are always managed subscriptions • Unmanaged subscription queue • When the subscription is created the name and location of an existing queue is provided • When the subscription is deleted the queue is not affected • Multiple subscriptions can point to the same unmanaged queue • Unmanaged subscriptions can be administratively created or programmatically using the MQI interface
  • 40.
    Subscription 40 Accessing a subscription’smessages Via the subscription • An application opens the subscription • A ‘true’ pub/sub application • Works with managed and unmanaged subscription queues • Limited to one attached consuming application at a time • Unless you’re using JMS cloned/shared subscriptions • Generally better publish/subscribe status feedback Via the queue • An application opens the queue associated with the subscription • This is really a point-to-point application • Only works with unmanaged subscription queues • Allows more freedom in what can be done • For example, multiple concurrent consuming applications are possible or multiple subscriptions using a single queue. Subscription Managed queue Unmanaged queue Consuming application Consuming application Consuming application Subscription Subscription Consuming application
  • 41.
    41 Controlling a subscription’sspecific queue • Manually create the subscription through administration • For example, using runmqsc: DEFINE SUB(…) TOPICSTR(‘…’) DEST(name of queue) DESTQMGR(location of queue) … • Programmatically create the subscription • Through certain APIs – For example MQI, using the MQSUB verb – Not supported with JMS • Through programmatically sending an administrative command – By generating an MQCMD_CREATE_SUBSCRIPTION PCF command message • Through the queued publish/subscribe mechanism – Matches the pre MQ V7 model of sending subscription registration messages (MQPS_REGISTER_SUBSCRIBER) to a queue manager – Provided for compatibility with pre-V7 systems
  • 42.
    Subscription types 42 • Whatthere isn’t is a ‘persistent’ or a ‘non-persistent’ subscription • Any subscription type can receive persistent or non-persistent publications • Obviously, persistent publications to a non-durable subscription are not very persistent!
  • 43.
    43 Managing subscriptions • Displayingsubscriptions • This shows the subscriptions on a queue manager 5724-H72 (C) Copyright IBM Corp. 1994, 2014. Starting MQSC for queue manager QMGR1. DISPLAY DISPLAY SUB(*) TOPICSTR 1 : DISPLAY SUB(*) TOPICSTR AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002306) SUB(SUB1) TOPICSTR(/Price/Fruit/Apples) AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202004183D5310000006) SUB(SYSTEM.DEFAULT.SUB) TOPICSTR() AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002312) SUB(SUB3) TOPICSTR(/Price/Vegetables/Potatoes) SUBID(414D5120514D4752312020202020202007183D532000230C) SUB(SUB2) TOPICSTR(/Price/Fruit/Oranges) DISPLAY SUB(SUB1) 2 : DISPLAY SUB(SUB1) AMQ8096: WebSphere MQ subscription inquired. SUBID(414D5120514D4752312020202020202007183D5320002306) SUB(SUB1) TOPICSTR(/Price/Fruit/Apples) TOPICOBJ( ) DEST(SYSTEM.MANAGED.DURABLE.533D180705230020) DESTQMGR(QMGR1) PUBAPPID( ) SELECTOR( ) SELTYPE(NONE) USERDATA( ) PUBACCT(16010515000000DEA960DF651724E4B97C192FE803000000000B) DESTCORL(414D5120514D4752312020202020202007183D5320002306) DESTCLAS(MANAGED) DURABLE(YES) EXPIRY(UNLIMITED) PSPROP(MSGPROP) PUBPRTY(ASPUB) REQONLY(NO) SUBSCOPE(ALL) SUBLEVEL(1) SUBTYPE(ADMIN) VARUSER(ANY) WSCHEMA(TOPIC) SUBUSER(xxxx) CRDATE(2015-02-03) CRTIME(09:19:15) ALTDATE(2015-02-03) ALTTIME(09:19:15) DISPLAY SBSTATUS(SUB1) 3 : DISPLAY SBSTATUS(SUB1) AMQ8099: WebSphere MQ subscription status inquired. SUB(SUB1) SUBID(414D5120514D4752312020202020202007183D5320002306) SUBUSER(xxxx) RESMDATE(2015-02-03) RESMTIME(09:19:15) LMSGDATE( ) LMSGTIME( ) ACTCONN(000000000000000000000000000000000000000000000000) DURABLE(YES) MCASTREL( , ) NUMMSGS(73) SUBTYPE(ADMIN) TOPICSTR(/Price/Fruit/Apples) DISPLAY QLOCAL(SYSTEM.MANAGED.DURABLE.533D180705230020)
  • 44.
    44 Managing subscriptions • DISPLAYPUBSUB ALL • Gives an idea of the total number of subscriptions currently registered on this queue manager (Added in IBM MQ V8). • DISPLAY SUB(*) TOPICSTR • List all the subscriptions, including the topic string that they are subscribing to. • DISPLAY SUB(SUB1) • Display the configuration of a subscription. • DISPLAY SBSTATUS(SUB1) • Display the status of a subscription. This includes things like the last time it was resumed (opened) and approximately how message messages have matched this subscription since it was created (or since a queue manager restart, if later). • DISPLAY SBSTATUS(*) WHERE(ACTCONN NE '000000000000000000000000000000000000000000000000') • Another attribute returned by SBSTATUS is ACTCONN, this is the connection ID if the subscription is currently in use. The above query will return all in use subscriptions. The last 16 characters of ACTCONN can be used with DISPLAY CONN to show further application information. • DISPLAY QLOCAL(<subscription’s DEST>) • One property returned by DISPLAY SUB is the DEST, this is the queue where messages will be delivered. Standard queue queries can be used on that queue.
  • 45.
  • 46.
    46 Publication, success orfailure? • Point-to-point is nice and simple: • Did the message get onto the queue? • Was it persistent and transacted? ConsumerProducer
  • 47.
    47 Publication, success orfailure? • Point-to-point is nice and simple: • Did the message get onto the queue? • Was it persistent and transacted? • Publish/subscribe is not so clear cut… • Persistence and transactions still ensures integrity of successful publications. • But if one or more subscriptions can’t receive the publication, should the publish fail? Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges
  • 48.
    48 Publication, success orfailure? • Should those subscriptions impact the others, should the publisher know? • What if the subscriptions are durable and the publication is persistent? • Controlled at the topic level • Persistent Message Delivery (PMSGDLV) and Non-persistent Message Delivery (NPMSGDLV). – ALL, ALLDUR, ALLAVAIL • Don’t forget that being able to DLQ a publication is still counted as a success! • USEDLQ on the topic to fine tune this behaviour. • And finally, remember – when there are no subscriptions, no-one gets it. That’s still a successful publish! Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges V7.1 DLQ
  • 49.
    49 Publication, success orfailure? • The publish/subscribe model encourages separation between the publishers of information and each of the subscribers to the information. • If a subscriber cannot receive a message published to a topic (e.g. it’s subscription queue is currently full), should that affect the other subscribers? And should the publisher be aware of the problem? • The default behaviour in IBM MQ depends on the persistence of the message and the durability of the subscription: • Persistent messages (PMSGDLV=ALLDUR): – The publish will fail if it cannot be delivered to one or more durable subscriptions. Failures to any non-durable subscriptions are acceptable. • Non-persistent messages (NPMSGDLV=ALLAVAIL): – Any failures to subscriptions are acceptable, the publish will succeed (even if no-one can receive the message). • This behaviour can be changed through configuration of the PMSGDLV and NPMSGDLV attributes of a topic. • If a dead letter queue is configured, a failure to put a publication to a subscription’s queue will result in an atempt to DLQ it. If that is successful the put to the subscription is classed as a success. • This can be controlled through the use of the USEDLQ option on a topic (available from WebSphere MQ V7.1 onwards).
  • 50.
    50 Retained publications Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges •When a message is published to a topic string, it is delivered to each matching subscription registered at that time. • Subscriptions created after that point will not receive the message only newly published ones. • Unless publications are retained
  • 51.
    51 Retained publications • Whena message is published to a topic string, it is delivered to each matching subscription registered at that time. • Subscriptions created after that point will not receive the message only newly published ones. • Unless publications are retained • Every time a message is published, the most recent publication for each topic string is retained by the queue manager. • When a new subscription is created, any matching retained message is delivered to it. • Take care, using retained can be subtle • Don’t confuse it with persistent publications Price Fruit Apples Oranges Subscription /Price/Fruit/Oranges Publisher /Price/Fruit/Oranges Subscription /Price/Fruit/Oranges
  • 52.
    52 Retained publications • Retainedpublications can be useful if information on a topic is infrequently published, and a new subscriber needs to see the latest update as soon as they are created. • These messages are stored by the queue manager on a queue. The same recommendation of not building a very deep application queue applies equally here. Therefore, calculate the total number of topic strings where publications would be retained when considering using retained publications. • Retained publications in a multi queue manager pus/sub topology (not yet covered in this presentation) introduce complexities that need to be understood. • If you are only connected to the system from time to time, or if regular updates are not what you require, you can choose to only get publications when you request them (MQI only) • You make a subscription in the normal way, additionally using the MQSO_PUBLICATIONS_ONLY_REQUEST option. Then whenever you want the latest state, you make a request for it using the MQSUBRQ verb. This will just send you the latest publication on the topic you have subscribed on. • If you have a wild-carded subscription you will get the latest publication on each topic string that matches your subscription. When using the MQI, the field called NumPubs in the MQSRO structure will detail how many publications have been sent to you.
  • 53.
  • 54.
    54 QMgr CHANNEL QMgrQMgr GetGetPut Distributed queuing • Weall know how point-to-point works • Even across multiple queue managers
  • 55.
    55 Distributed publish/subscribe • Weknow how everything revolves around the topic tree, dynamically built up in a queue manager QMgr Topics Subscription Publisher Subscription Subscription Subscription
  • 56.
    56 QMgrQMgr Topics Publisher QMgrQMgr Topics Subscription QMgrQMgr Topics Subscription Subscription Distributed publish/subscribe • Weknow how everything revolves around the topic tree, dynamically built up in a queue manager • Queue managers can work together to share their topic tree knowledge between them • Enabling publications to be propagated to subscriptions on different queue managers • The applications stay the same, the changes are at the configuration level. Subscription
  • 57.
    57 Distributed publish/subscribe topologies •Publish/subscribe topologies can either be created as a defined hierarchy or more dynamically as a cluster Subscription Publisher Subscription Subscription Publisher Subscription Subscription Subscription
  • 58.
    58 Distributed publish/subscribe topologies Subscription Publisher Subscription Subscription •A hierarchy manually defines the relationship between each queue manager. • Messages are flowed via those relationships.
  • 59.
    59 Pub/Sub Hierarchies • Bydefault publications will flow between queue managers from publishers to subscribers in a hierarchy. Points to consider: • Consider the scalability of the higher nodes in the hierarchy, especially the root as more publication traffic is expected to flow through these. • As for clusters, all topic strings subscribed to are propagated throughout the cluster, not just to the immediate neighbours – Rapid change to the set of subscriptions can become a limiting factor • Propagating is asynchronous, so early publications may not be sent following a subscription to a new topic string
  • 60.
    60 Distributed publish/subscribe topologies •A cluster can be used for publish/subscribe through simple configuration. • Any queue manager can publish and subscribe to topics • Published messages can go direct between queue managers Publisher Subscription Subscription Subscription Publisher Subscription Subscription Subscription • Or be routed via specific queue managersV8 See AME-2272 for much more on topologies…
  • 61.
    61 Pub/Sub Clusters • Publicationswill not flow between queue managers in a cluster unless the branch of the topic tree is clustered. • This is achieved by setting the ‘CLUSTER’ property of an administered topic object at the root of that topic tree to the name of the cluster. • The topic object can be defined in a single queue manager in the cluster, clustering it will automatically share the topic object definition with all other queue managers in the cluster. Points to consider: • A pub/sub cluster behaves very differently to a pure point-to-point cluster, plan accordingly: – All queue managers automatically know of all other queue managers – All queue managers with subscriptions to clustered topics will create channels to all other queue managers in the cluster – The above affects queue managers that have no publishers or subscribers but are in the same cluster as the topics. • All clustered topic strings subscribed to are propagated throughout the cluster – Rapid change to the set of subscriptions can become a limiting factor • Propagating is asynchronous, so early publications may not be sent following a subscription to a new topic string
  • 62.
    62 Summary • Publish/Subscribe inIBM MQ • Administration of publish/subscribe • Management of publish/subscribe • Subscriptions and publications • Topologies
  • 63.
    63 Notices and Disclaimers Copyright© 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
  • 64.
    64 Notices and Disclaimers(con’t) Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. • IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®, urban{code}®, Watson, WebSphere®, Worklight®, X- Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
  • 65.
    Thank You Your Feedbackis Important! Access the InterConnect 2015 Conference CONNECT Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.