Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's new in IBM MQ, March 2018

1,148 views

Published on

This presentation gives an overview of the many updates to the IBM MQ family of messaging products over the year leading up to March 2018. Learn how MQ has been continuously delivering new features and capabilities, enabling enterprise level messaging in ever more cloud and on-prem solutions, whether you're building your own MQ environment, using the MQ Appliance or looking to consume MQ as a service. This presentation touches on the main updates made to MQ during the 9.0.x releases.

Published in: Software
  • Be the first to comment

What's new in IBM MQ, March 2018

  1. 1. What's New in IBM MQ: A Technical Deep-Dive David Ware STSM, IBM MQ Chief Architect www.linkedin.com/in/dware1
  2. 2. Exactly once delivery MQ is designed from the ground up to ensure that when messages of worth are being processed and exactly once, transactional, messages are required, that really means exactly once MQ’s key value capabilities and why they matter… The world depends on reliable, secure messaging IBM MQ is the most reliable and trusted messaging solution Your bank transfers complete without losing your money Your credit card information is kept safe You move your customer data between systems without exposing it to hackers Your developers focus on the business needs, not transaction recovery or security Scalable Scales to meet the needs of your business, handling billions of messages per day Simple Your applications are focused on the business, not fixing connectivity problems or tracking their progress Think 2018 / © 2018 IBM Corporation
  3. 3. Secure MQ is secure by design, from fine grain authentication and authorization of messaging resources, to encryption of data from one application to another, without the application even knowing or the administrators gaining access The world depends on reliable, secure messaging IBM MQ is the most reliable and trusted messaging solution Your bank transfers complete without losing your money Your credit card information is kept safe You move your customer data between systems without exposing it to hackers Your developers focus on the business needs, not transaction recovery or security Reliable Only MQ has 25 years of proven reliability and robustness when used by the most demanding customers in the most demanding systems, without you even knowing it’s there Connectivity Simple queues and topics provide simple application messaging but MQ provides true routing capabilities to join applications and systems together, locally and globally MQ’s key value capabilities and why they matter… Think 2018 / © 2018 IBM Corporation
  4. 4. Run IBM MQ in any location or cloud exactly as you need it On-premise, software and the MQ Appliance Run it yourself in any cloud, public or private Let IBM host it for you with its new managed MQ service in IBM Cloud Private cloud AWSAWS AWSAzure AWS … Linux Windows Solaris HPE AIX IBMi IBM Z Appliance NEW Think 2018 / © 2018 IBM Corporation IBM Cloud Private
  5. 5. Recent advances Think 2018 / © 2018 IBM Corporation
  6. 6. MQ’s focus Running MQ in a cloud Enabling applications High availability and scalability Managing MQ in a cloud native way Think 2018 / © 2018 IBM Corporation
  7. 7. Mix and Match Both are available under the same license. Both can interoperate, just like any previous version of MQ. Continuous Delivery New CD versions of MQ are released approximately every four months, incrementally introducing new product capabilities. Intended for those that can continually integrate. Long Term Support Approximately every two years a new LTS version is released, rolling up many of the CD capabilities into a release with 5+3 support attached. Required by those looking for fixed function. In 2016 MQ introduced a dual Long Term Support and a Continuous Delivery model IBM MQ: long term support and continuous delivery 2016 2017 201820152014 V8.0.0 V9.0.0 LTS V9.0.1 CD V9.0.2 CD V9.0.3 CD V9.0.4 CD V9.0.5 CD IBM MQ Appliance M2000 IBM MQ Appliance M2001 NEW Statement of Direction: With the next release of MQ, IBM intends to initiate deployment of a new LTS release cycle MQ on IBM Cloud
  8. 8. Think 2018 / © 2018 IBM Corporation MQ on IBM Cloud MQ as a Service
  9. 9. MQ on IBM Cloud Provision queue managers directly into IBM Cloud IBM owns the infrastructure and the responsibility to keep the systems up to date and running The customer owns the configuration and the monitoring of the messaging Try the service for free at: console.bluemix.net/catalog/services/mq Hosted on NEW Think 2018 / © 2018 IBM Corporation
  10. 10. MQ on IBM Cloud Hardware, virtualization, servers, network, storage Queues, topics, channels, clustering, applications MQ installation, basic configuration, security, maintenanceManaged & operated by IBM Configured & monitored by the customer NEW Hosted on Provision queue managers directly into IBM Cloud IBM owns the infrastructure and the responsibility to keep the systems up to date and running The customer owns the configuration and the monitoring of the messaging Try the service for free at: console.bluemix.net/catalog/services/mq Think 2018 / © 2018 IBM Corporation
  11. 11. On-premises software install MQ Appliance BYOL cloud install MQ on IBM Cloud Data centre Networking Servers Storage Hypervisor Virtual machine OS patching MQ patching Data centre Networking Servers Storage Hypervisor Virtual machine OS patching MQ patching Data centre Networking Servers Storage Hypervisor Virtual machine OS patching MQ patching Data centre Networking Servers Storage Hypervisor Virtual machine OS patching MQ patching QM failover Queues, Topics Clustering IBM / VendorCustomer QM availability/restart Msg, Conn, Chl monitoring QM failover QM availability/restart QM failover QM availability/restart QM failover QM availability/restart Queues, Topics Clustering Msg, Conn, Chl monitoring Queues, Topics Clustering Msg, Conn, Chl monitoring Queues, Topics Clustering Msg, Conn, Chl monitoring ApplicationApplicationApplicationApplication System-level monitoringSystem-level monitoringSystem-level monitoringSystem-level monitoring Think 2018 / © 2018 IBM Corporation
  12. 12. Bring your own license to the Cloud Think 2018 / © 2018 IBM Corporation
  13. 13. MQ in Containers MQ has been supporting Docker containers since 2015 with images on Docker Hub and Docker Store and sample setups on Github More recently it has been demonstrating how to get the most from containers using Kubernetes And now MQ Advanced is available as a fully supported product with IBM Cloud Private, a Kubernetes-based solution from IBM Statement of Direction IBM intends to offer clients the ability to purchase an entitlement that is based on the container size in Virtual Processor Cores and the number of hours that MQ was deployed for in each container Think 2018 / © 2018 IBM Corporation
  14. 14. Getting up and running with MQ quickly is important, but running it in the best possible way is critical MQ is publishing quick starts and preconfigured environments to show best practice Preconfigured with essential MQ resources to get going as quickly as possible Examples so far: IBM MQ on AWS Quick Start MQ on IBM Cloud MQ for Developers on Docker Helm charts for MQ in Kubernetes IBM MQ was the first IBM product to work with Amazon to publish an AWS Quick Start, deploying a highly available MQ setup in AWS in less than thirty minutes Getting started quickly with MQ Think 2018 / © 2018 IBM Corporation
  15. 15. High availability and scalability Think 2018 / © 2018 IBM Corporation
  16. 16. HA, there are no excuses MQ delivers HA through the ability to build horizontally scaled, active-active systems and typically active-passive HA of the data itself*, the messages. Traditionally active-passive HA has been achieved through HA clusters or multi instance queue managers. Both rely on highly available infrastructure to be setup and relied on. The MQ Appliance changed this with a fully integrated HA solution, providing built in machine to machine data replication and failover. * z/OS shared queue provides active-active HA of the message data! shared storage Think 2018 / © 2018 IBM Corporation
  17. 17. HA, there are no excuses MQ delivers HA through the ability to build horizontally scaled, active-active systems and typically active-passive HA of the data itself*, the messages. Traditionally active-passive HA has been achieved through HA clusters or multi instance queue managers. Both rely on highly available infrastructure to be setup and relied on. The MQ Appliance changed this with a fully integrated HA solution, providing built in machine to machine data replication and failover. storage storage storage Think 2018 / © 2018 IBM Corporation * z/OS shared queue provides active-active HA of the message data!
  18. 18. MQ HA Group Node 2 Node 3Node 1 Synchronous data replication Replicated Data Queue Managers Linux only, MQ Advanced HA solution with no need for a shared file system or HA cluster MQ configures the underlying resources to make setup and operations natural to an MQ user Three-way replication for quorum support Synchronous data replication for once and once only transactional delivery of messages Active/passive queue managers with automatic takeover Per queue manager control to support active/active utilisation of nodes Per queue manager IP address to provide simple application setup Supported on RHEL v7 x86-64 only Monitoring App Network New in V9.0.4 CD MQ Advanced for Linux
  19. 19. MQ HA Group Node 2 Node 3Node 1 Synchronous data replication Replicated Data Queue Managers Monitoring App NetworkLinux only, MQ Advanced HA solution with no need for a shared file system or HA cluster MQ configures the underlying resources to make setup and operations natural to an MQ user Three-way replication for quorum support Synchronous data replication for once and once only transactional delivery of messages Active/passive queue managers with automatic takeover Per queue manager control to support active/active utilisation of nodes Per queue manager IP address to provide simple application setup Supported on RHEL v7 x86-64 only New in V9.0.4 CD MQ Advanced for Linux
  20. 20. MQ HA Group Node 2 Node 3Node 1 Replicated Data Queue Managers App NetworkRecommended deployment pattern: Spread the workload across multiple queue managers and distribute them across all three nodes Even better, more than one queue manager per node for better failover distribution Use MQ Clusters for additional routing of messages to work around problems MQ licensing is aligned to maximise benefits One full IBM MQ Advanced license and two High Availability Replica licenses (previously named Idle Standby) App App New in V9.0.4 CD MQ Advanced for Linux
  21. 21. Connecting Applications at Scale Binding an application to a specific queue manager restricts future changes, impacting availability, scalability and maintenance of the system Run applications remote from the queue managers, connecting as MQ clients For horizontal scaling, create groups of matching queue managers and connect applications to the group, not each queue manager Now you can add and remove queue managers without changing the application Use one of multiple ways to map applications to queue managers and their location. For example Infrastructure level routing Client Channel Definition Tables (CCDTs) App App App decoupled CCDT QM1 QM2 QM3 HTTP server V9.0.0 LTS
  22. 22. Node 2Node 1 data replication Replicated Data Queue Managers App Manual failover 9.0.5 CD MQ Advanced adds the ability to build a looser coupled pair of nodes for data replication but no automatic failover, typically for Disaster Recovery Data replication can be Asynchronous for systems separated by a high latency network Synchronous for systems on a low latency network No automatic takeover means no need for a third node to provide a quorum New in V9.0.5 CD MQ Advanced for Linux
  23. 23. Think 2018 / © 2018 IBM Corporation Supporting applications
  24. 24. Messages from one application can be received by any other application, independent of API or protocol. APIsProtocolsMultiple APIs and Protocols IBM MQ supports multiple APIs and multiple client protocols. Both proprietary and open. APIs: MQI, JMS, MQ Light, REST … Protocols: MQ, AMQP, MQTT, HTTP These support a wide range of application styles, from the simplest of messaging needs through to the most sophisticated MQ is simply the broker of messages produced from any API, protocol or language MQI Exposes the full set of MQ capabilities Uses the MQ protocol JMS Supports the full JMS API for use in many JSE or JEE environments MQ Light A simple pub/sub messaging API Uses the AMQP 1.0 protocol MQ MQ’s highly reliable and performant messaging protocol MQTT MQ Advanced supports the MQTT protocol Open source Eclipse Paho clients AMQP Support for AMQP 1.0 enables support for open source clients such as Qpid Proton HTTP A very simple but secure messaging API over REST MQTT Support for the MQTT API for IoT devices Think 2018 / © 2018 IBM Corporation
  25. 25. Applications can be built using a wide range of clients APIsProtocolsMultiple APIs and Protocols IBM MQ supports multiple APIs and multiple client protocols. Both proprietary and open. APIs: MQI, JMS, MQ Light, REST … Protocols: MQ, AMQP, MQTT, HTTP These support a wide range of application styles, from the simplest of messaging needs through to the most sophisticated MQ is simply the broker of messages produced from any API or protocol. MQI Java .Net MQ Light MQTT paho AMQP Qpid MQ Go MQ nodejs CPAN perl community IBM supported … … MQTT Think 2018 / © 2018 IBM Corporation MQI Exposes the full set of MQ capabilities Uses the MQ protocol JMS Supports the full JMS API for use in many JSE or JEE environments MQ Light A simple pub/sub messaging API Uses the AMQP 1.0 protocol MQ MQ’s highly reliable and performant messaging protocol MQTT MQ Advanced supports the MQTT protocol Open source Eclipse Paho clients AMQP Support for AMQP 1.0 enables support for open source clients such as Qpid Proton HTTP A very simple but secure messaging API over REST MQTT Support for the MQTT API for IoT devices
  26. 26. Developer engagement MQ can be hard to get going with, we know! How do we make it easier for new application teams to pick up MQ and integrate with their development practices? Early wins • A better getting started experience with simple introductions and exercises • MQ JMS jars available through Maven and MQ client available through simple download • New open source language bindings for MQI NodeJS, Go • Spring Boot starter for MQ To enable a user instructed to use MQ for the first time, to go from zero understanding to running a sample application in a sandbox environment with a fundamental understanding of MQ concepts in 2 hours To enable an application developer, instructed to use MQ for the first time, to go from zero understanding to writing their first MQ application in the language and environment of their choice within an afternoon Mission Statement Tell us where you see a need more focusThink 2018 / © 2018 IBM Corporation
  27. 27. LearnMQ Finding it hard to get developers started with MQ? Point them to: developer.ibm.com/ messaging/learn-mq Totally new to MQ? Learn the basics Step-by-step guide to getting up and running with MQ Tutorials on building your applications Think 2018 / © 2018 IBM Corporation
  28. 28. Think 2018 / © 2018 IBM Corporation Connectivity
  29. 29. Bridging to MQ As well as connecting a wide array of applications directly to an MQ system, there are a growing set of bridges and connectors between MQ and external systems Salesforce Integrate MQ’s publish/subscribe with Salesforce. Exchange Salesforce events and MQ publications using the MQ Bridge for Salesforce with no need for your backend applications to connect to Salesforce directly. Blockchain Use MQ messages to query and update a Blockchain ledger. Connects to Hyperledger Fabric networks in IBM Cloud and locally. Supported for use with V9.0.x MQ Advanced queue managers Kafka IBM MQ sink and source connectors are currently being openly developed by IBM and provides as-is, allowing you to connect your MQ systems with your Kafka clusters www.confluent.io/product/connectors Salesforce Blockchain Kafka Think 2018 / © 2018 IBM Corporation
  30. 30. Managing MQ Think 2018 / © 2018 IBM Corporation
  31. 31. Managing MQ Management of systems is evolving towards cattle not pets. What does that mean for MQ? Consistent configuration and operations across multiple queue managers Automated deployment Configuration as code Self service Collection and analysis of diagnostic data Simple integration into standard devops and automation tooling Think 2018 / © 2018 IBM Corporation
  32. 32. MQ Web Console Point a browser at the MQ installation to create and manage queue managers and their resources Provides a very simple way to access MQ resources Think 2018 / © 2018 IBM Corporation
  33. 33. HTTPS POST: https://host:port/ibmmq/v1/qmgr/QM1/queue { “name”: ”QUEUE.1”, “type”: ”local”, “cluster”: { “name”: “CLUSTER1” } } HTTP Response: 201 JSON payload when defining/updating objects URL represents target object for command HTTP response indicates success/failure HTTPS GET: https://hostname:portNumber/ibmmq/v1/qmgr { “qmgr”: [ { “qmgrName”: “QMTEST01”, “status”: “running” }, { “qmgrName”: “QMTEST02”, “status”: “ended” } ] } JSON payload returned when querying Restful administration MQ has supported scripting and programmatic administration for many years, but it requires MQ knowledge and tooling. MQ has been increasing support for a RESTful administrative API to enable much of what’s available today with MQSC and PCF. But in a more intuitive way to many, using the URL and structured JSON payloads to define the operations. Being over HTTPS also enables the embedding of MQ administrative operations into many environments and tools that previously would not be possible Each 9.0.x CD release has extended the level of REST support, and it continues… Think 2018 / © 2018 IBM Corporation
  34. 34. Restful administration MQ has supported scripting and programmatic administration for many years, but it requires MQ knowledge and tooling. MQ has been increasing support for a RESTful administrative API to enable much of what’s available today with MQSC and PCF. But in a more intuitive way to many, using the URL and structured JSON payloads to define the operations. Being over HTTPS also enables the embedding of MQ administrative operations into many environments and tools that previously would not be possible Each 9.0.x CD release has extended the level of REST support, and it continues… 9.0.5 adds support for monitoring your Managed File Transfer system with agent and transfer status lists over REST NEW HTTPS GET: https://host:port/ibmmq/v1/mft/transfer {"transfer": [ { "destinationAgent": {"name": "AGENT.X.BANK"}, "originator": { "host": "192.168.99.1", "userId": "ramsubbarao" }, "sourceAgent": {"name": "TESTAGENT"}, "statistics": { "endTime": "2018-01-08T16:22:15.569Z", "numberOfFileFailures": 0, "numberOfFileSuccesses": 2, "numberOfFileWarnings": 0, "numberOfFiles": 2, "startTime": "2018-01-08T16:22:15.242Z" }, "status": { "state": "successful" }, "id": "414D51204D465444454D4F3320202020513E525A21109908" } Think 2018 / © 2018 IBM Corporation
  35. 35. Error logs output JSON for easy parsing Managing diagnostic data The need to centrally collect and analyse diagnostic data is increasing, using tools such as Splunk, Elasticsearch and Grafana MQ generates a wide range of information and has demonstrated how this can be collected using off the shelf tooling Subscribing to metrics in MQ V9 makes that even easier MQ 9.0.x has seen enhancements to the error log data it generates to aid such solutions Universal timestamps and severity levels Separated inserts JSON output Multiple logs Syslog output Forward MQ error logs to ElasticSearch or Splunk Publish MQ statistics to Prometheus and Grafana { "ibm_messageId":"AMQ6287I", "ibm_arithInsert1":0, "ibm_arithInsert2":0, "ibm_commentInsert1":"Linux 4.13.0-36-generic (MQ Linux (x86-64 platform) 64-bit)", "ibm_commentInsert2":"/opt/mqm (Installation1)", "ibm_commentInsert3":"9.0.5.0 (p905-L180228.1)", "ibm_datetime":"2018-03-04T13:18:27.506Z", "ibm_serverName":"QM905", "type":"mq_log", "host":"david-VirtualBox", "loglevel":"INFO", "module":"amqxeida.c:6238", "ibm_sequence":"1520169507_506462655", "ibm_processId":2119, "ibm_threadId":1, "ibm_version":"9.0.5.0", "ibm_processName":"strmqm", "ibm_userName":"david", ”ibm_installationName":"Installation1", "ibm_installationDir":"/opt/mqm", "message":"AMQ6287I: IBM MQ V9.0.5.0 (p905-L180228.1).” } NEW Think 2018 / © 2018 IBM Corporation
  36. 36. Run IBM MQ in any location or cloud exactly as you need it On-premise, software and the MQ Appliance Run it yourself in any cloud, public or private Let IBM host it for you with its new managed MQ service in IBM Cloud AWSAWS AWSAzure AWS … Linux Windows Solaris HPE AIX IBMi IBM Z Appliance Think 2018 / © 2018 IBM Corporation Private cloud IBM Cloud Private
  37. 37. Notices and disclaimers © 2018 International Business Machines Corporation. No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event, shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted per the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer follows any law. Think 2018 / © 2018 IBM Corporation
  38. 38. Notices and disclaimers continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products about this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a purpose. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com and [names of other referenced IBM products and services used in the presentation] are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml. . Think 2018 / © 2018 IBM Corporation
  39. 39. Thank you Think 2018 / © 2018 IBM Corporation David Ware STSM, IBM MQ Chief Architect www.linkedin.com/in/dware1
  40. 40. Think 2018 / © 2018 IBM Corporation

×