IBM Integration Bus High Availability Overview

Peter Broadhurst
Peter BroadhurstCo-Founder, Head of Engineering at Kaleido, a ConsenSys business
IBM Integration Bus
High Availability Overview
Peter Broadhurst
IBM Messaging & Integration
© 2013, 2014 International Business Machines Corporation 2
Introduction
These charts provide a high-level overview of IIB HA topologies:
• Comparison of active/active and active/passive HA
• Solutions for active/passive HA failover with IBM Integration Bus
• Solutions for active/active processing with IBM Integration Bus
• Adding Global Cache to active/active processing
• Combining all of the above
Only HTTP and JMS (MQ) workloads are shown
© 2013, 2014 International Business Machines Corporation 3
Active/active vs. active/passive
Scale High Availability Design Considerations
Active/Active All N instances contribute to
the processing capacity of
the system
New requests can be serviced
immediately after the planned
or unplanned termination of
an active instance.
Often referred to as
continuous availability.
Each instance must be able to
operate independently, without
relying on the availability of any
other instance in the environment.
The order of processing for two
items of work cannot be
guaranteed, as there are multiple
instances that might perform each
item of work
Active/Passive Only one of N instances
contributes to the processing
capacity of the system
There is a failover period after
the planned or unplanned
termination of an active
instance. This failover period
commonly lasts for a small
number of minutes,
depending on the technology
used.
An infrastructure for HA failover
must exist, that ensures only one
instance is ever activated, as well
as detecting when one instance
fails to initiate the failover.
The active and the passive
system must have identical copies
of persistent data, such as
persisted transaction state, and
persisted messages. In IBM
Integration Bus this is achieved by
sharing a filesystem between the
two machines.
© 2013, 2014 International Business Machines Corporation 4
Active/passive high availability
Machine 1 Machine 2
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
Shared
file system
HA
failover
• Single instance of the IIB runtime
• Single instance of all state
– Configuration, MQ messages and Transaction coordinator
• Highly available due to automatic fail-over. Two options available for HA failover:
– Out-of-the-box with Multi-Instance capability,
– Or using external HA cluster software
JMS workloads
HTTP workloads
© 2013, 2014 International Business Machines Corporation 5
Option 1: Out of the box Multi-Instance failover
• IP address of each machine is different
– IP address redirect is required for HTTP workloads
– MQ client libraries automatically handle IP redirect for JMS workloads
• Requires highly available network-attached storage (NAS). Examples:
– IBM GPFS
– Veritas Cluster File System
– Highly available NFSv4
• More information on choosing a suitable NAS:
– https://ibm.biz/BdFxfz
Machine 1 – IP addr 1 Machine 2 – IP addr 2
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
Network-
attached
file system (NAS)
TCP/ IP
Failover
managed
via
NAS file
locks
IP redirect (Gateway / Load Balancer)
JMS workloads
TCP/ IP
© 2013, 2014 International Business Machines Corporation 6
Option 2: HA cluster failover
• Allows use of direct-attached storage
• IP address failed over by HA cluster software
• Requires HA cluster software. Examples:
– IBM PowerHA (HACMP)
– Veritas Cluster Server (VCS)
– Microsoft Cluster Service (MSCS)
– Red Hat Cluster
Machine 1 – IP addr 1 Machine 2 – IP addr 1
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
Direct-attached
file system
switched by HA
Failover
managed
via
health
checking
Fibre/SCSI
HA cluster Software HA cluster Software
Heartbeat
Healthcheck Healthcheck
JMS workloads
HTTP workloads
Fibre/SCSI
IP addr
failover
© 2013, 2014 International Business Machines Corporation 7
Multi-instance or HA cluster?
Multi-instance queue manager
• Integrated into the IIB and MQ products
• Faster failover than HA cluster*
• Delay before queue manager restart is much shorter*
• Runtime performance of networked storage must be considered
• IP address of standby instance is different to primary
• No automatic fail-back to primary hardware when restored
• More susceptible to MQ and OS defects
HA cluster
• Capable of handling a wider range of failures
• Failover historically rather slow, but some HA clusters are improving
• Some customers frustrated by unnecessary failovers
• Require MC91 SupportPac or equivalent configuration
• Extra product purchase and skills required
Storage distinction
• Multi-instance queue manager typically uses NAS
• HA clustered queue manager typically uses SAN
*depends on NAS file-system tuning and specific customer environment
© 2013, 2014 International Business Machines Corporation 8
Active/active topologies
Machine 1 Machine 2
IIB Integration Node
MQ Messaging
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
WESB AppTargetWESB AppTargetIntegration Server
Gateway / Load Balancer for HTTP workloads
HTTP workloads
JMS workloads
• Each Integration Node operates independently
• Continuous availability of the service during a failure
– Individual in-flight requests on the failed node receive errors, but the service stays available
• HTTP workloads require external workload balancing
– Hardware load balancer is the most common solution
• JMS workloads require load balancing
– Via an MQ cluster when remote applications have their own MQ queue managers
– Options for direct MQ client attachment (over TCP/IP) described here: https://ibm.biz/BdFxfS
© 2013, 2014 International Business Machines Corporation 9
Adding Global Cache to active/active topologies
Machine 1 Machine 2
IIB Integration Node
MQ Messaging
Global
Cache
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
WESB AppTargetWESB AppTargetIntegration Server
• Highly available in-memory state store
– High performance compared to disk persistence (DB/MQ Messaging)
– High availability through redundancy – cannot be recovered if all nodes are stopped concurrently
– Built with WebSphere eXtreme Scale technology
• Alternative to using MQ or a Database for state storage/retrieval
– Correlation state between asynchronous request/reply flows
• Only needed if replies might be routed via a different runtime, otherwise simply use memory (LocalEnvironment etc.)
– Regularly accessed reference data, such as routing tables
© 2013, 2014 International Business Machines Corporation 10
Global
Cache
Putting it all together
• Active/active processing. HA fail-over for state and in-flight message recovery
• Individual queues/flows can be configured active/passive if required
– Single flow instance for message ordering or file-based processing
– Single correlation state queue shared between active flow instances
Machine 1 Machine 2
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
IIB Integration Node
MQ Messaging
WESB AppTargetWESB AppTargetIntegration Server
Shared
Filesystem
Failover
Failover
Gateway / Load Balancer for HTTP workloads
JMS workloads
HTTP workloads
© 2013, 2014 International Business Machines Corporation 11
More information
• Knowledge Center starting points:
IIB Active/passive HA: https://ibm.biz/BdFxfe
IIB Active/active HA for HTTP: https://ibm.biz/BdFxfb
IIB Global cache: https://ibm.biz/BdFxfp
MQ HA Cluster configurations: https://ibm.biz/BdFxf8
• Wiki article with detailed description of IIB topology choices
Tailored to customers migrating from WebSphere WESB, but useful to all
https://ibm.biz/BdFxfa
• MQDev blog on attaching MQ clients to active/active qmgrs:
https://ibm.biz/BdFxfS
• Testing and support statement for multi-instance
https://ibm.biz/BdFxfz
1 of 11

Recommended

IBM MQ High Availability 2019 by
IBM MQ High Availability 2019IBM MQ High Availability 2019
IBM MQ High Availability 2019David Ware
4.5K views42 slides
IBM Integration Bus & WebSphere MQ - High Availability & Disaster Recovery by
IBM Integration Bus & WebSphere MQ - High Availability & Disaster RecoveryIBM Integration Bus & WebSphere MQ - High Availability & Disaster Recovery
IBM Integration Bus & WebSphere MQ - High Availability & Disaster RecoveryRob Convery
9.8K views59 slides
IBM MQ - High Availability and Disaster Recovery by
IBM MQ - High Availability and Disaster RecoveryIBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster RecoveryMarkTaylorIBM
9K views68 slides
IBM WebSphere MQ Introduction by
IBM WebSphere MQ Introduction IBM WebSphere MQ Introduction
IBM WebSphere MQ Introduction ejlp12
10.9K views231 slides
414: Build an agile CI/CD Pipeline for application integration by
414: Build an agile CI/CD Pipeline for application integration414: Build an agile CI/CD Pipeline for application integration
414: Build an agile CI/CD Pipeline for application integrationTrevor Dolby
964 views36 slides
IBM Websphere MQ Basic by
IBM Websphere MQ BasicIBM Websphere MQ Basic
IBM Websphere MQ BasicPRASAD BHATKAR
3K views10 slides

More Related Content

What's hot

IBM MQ Clustering (2017 version) by
IBM MQ Clustering (2017 version)IBM MQ Clustering (2017 version)
IBM MQ Clustering (2017 version)MarkTaylorIBM
1.4K views41 slides
Overview - ESBs and IBM Integration Bus by
Overview - ESBs and IBM Integration BusOverview - ESBs and IBM Integration Bus
Overview - ESBs and IBM Integration BusJuarez Junior
3.4K views18 slides
IBM MQ Whats new - including 9.3 and 9.3.1 by
IBM MQ Whats new - including 9.3 and 9.3.1IBM MQ Whats new - including 9.3 and 9.3.1
IBM MQ Whats new - including 9.3 and 9.3.1Robert Parker
501 views52 slides
IBM Cloud Integration Platform High Availability - Integration Tech Conference by
IBM Cloud Integration Platform High Availability - Integration Tech ConferenceIBM Cloud Integration Platform High Availability - Integration Tech Conference
IBM Cloud Integration Platform High Availability - Integration Tech ConferenceRobert Nicholson
1.3K views36 slides
IBM MQ Overview (IBM Message Queue) by
IBM MQ Overview (IBM Message Queue)IBM MQ Overview (IBM Message Queue)
IBM MQ Overview (IBM Message Queue)Juarez Junior
2.2K views14 slides
IBM DataPower Gateway - Common Use Cases by
IBM DataPower Gateway - Common Use CasesIBM DataPower Gateway - Common Use Cases
IBM DataPower Gateway - Common Use CasesIBM DataPower Gateway
97.5K views91 slides

What's hot(20)

IBM MQ Clustering (2017 version) by MarkTaylorIBM
IBM MQ Clustering (2017 version)IBM MQ Clustering (2017 version)
IBM MQ Clustering (2017 version)
MarkTaylorIBM1.4K views
Overview - ESBs and IBM Integration Bus by Juarez Junior
Overview - ESBs and IBM Integration BusOverview - ESBs and IBM Integration Bus
Overview - ESBs and IBM Integration Bus
Juarez Junior3.4K views
IBM MQ Whats new - including 9.3 and 9.3.1 by Robert Parker
IBM MQ Whats new - including 9.3 and 9.3.1IBM MQ Whats new - including 9.3 and 9.3.1
IBM MQ Whats new - including 9.3 and 9.3.1
Robert Parker501 views
IBM Cloud Integration Platform High Availability - Integration Tech Conference by Robert Nicholson
IBM Cloud Integration Platform High Availability - Integration Tech ConferenceIBM Cloud Integration Platform High Availability - Integration Tech Conference
IBM Cloud Integration Platform High Availability - Integration Tech Conference
Robert Nicholson1.3K views
IBM MQ Overview (IBM Message Queue) by Juarez Junior
IBM MQ Overview (IBM Message Queue)IBM MQ Overview (IBM Message Queue)
IBM MQ Overview (IBM Message Queue)
Juarez Junior2.2K views
IBM MQ Online Tutorials by BigClasses.com
IBM MQ Online TutorialsIBM MQ Online Tutorials
IBM MQ Online Tutorials
BigClasses.com11.2K views
IBM MQ - What's new in 9.2 by David Ware
IBM MQ - What's new in 9.2IBM MQ - What's new in 9.2
IBM MQ - What's new in 9.2
David Ware1.3K views
An Introduction to the Message Queuing Technology & IBM WebSphere MQ by Ravi Yogesh
An Introduction to the Message Queuing Technology & IBM WebSphere MQAn Introduction to the Message Queuing Technology & IBM WebSphere MQ
An Introduction to the Message Queuing Technology & IBM WebSphere MQ
Ravi Yogesh1.7K views
Red Hat OpenShift Container Platform Overview by James Falkner
Red Hat OpenShift Container Platform OverviewRed Hat OpenShift Container Platform Overview
Red Hat OpenShift Container Platform Overview
James Falkner16.3K views
Azure architecture by Amal Dev
Azure architectureAzure architecture
Azure architecture
Amal Dev4.8K views
Building an Active-Active IBM MQ System by matthew1001
Building an Active-Active IBM MQ SystemBuilding an Active-Active IBM MQ System
Building an Active-Active IBM MQ System
matthew1001383 views
WebSphere application server 8.5.5 - quick overview by Chris Sparshott
WebSphere application server 8.5.5 - quick overviewWebSphere application server 8.5.5 - quick overview
WebSphere application server 8.5.5 - quick overview
Chris Sparshott3.8K views
Whats New in Integration What's New in IBM Integration Bus and IIB on Cloud by Rob Convery
Whats New in Integration What's New in IBM Integration Bus and IIB on Cloud Whats New in Integration What's New in IBM Integration Bus and IIB on Cloud
Whats New in Integration What's New in IBM Integration Bus and IIB on Cloud
Rob Convery2.7K views
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters by David Ware
IBM MQ: Managing Workloads, Scaling and Availability with MQ ClustersIBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
David Ware7.5K views
Designing IBM MQ deployments for the cloud generation by David Ware
Designing IBM MQ deployments for the cloud generationDesigning IBM MQ deployments for the cloud generation
Designing IBM MQ deployments for the cloud generation
David Ware770 views
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe by David Ware
IBM MQ: An Introduction to Using and Developing with MQ Publish/SubscribeIBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
David Ware4.9K views

Similar to IBM Integration Bus High Availability Overview

IBM MQ High Availabillity and Disaster Recovery (2017 version) by
IBM MQ High Availabillity and Disaster Recovery (2017 version)IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)MarkTaylorIBM
2K views66 slides
What's new in informix v11.70 by
What's new in informix v11.70What's new in informix v11.70
What's new in informix v11.70am_prasanna
190 views24 slides
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr... by
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...Hendrik van Run
91 views52 slides
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M... by
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...Peter Broadhurst
3.9K views19 slides
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee... by
Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...xKinAnx
2.6K views28 slides
Could the “C” in HPC stand for Cloud? by
Could the “C” in HPC stand for Cloud?Could the “C” in HPC stand for Cloud?
Could the “C” in HPC stand for Cloud?IBM India Smarter Computing
360 views12 slides

Similar to IBM Integration Bus High Availability Overview(20)

IBM MQ High Availabillity and Disaster Recovery (2017 version) by MarkTaylorIBM
IBM MQ High Availabillity and Disaster Recovery (2017 version)IBM MQ High Availabillity and Disaster Recovery (2017 version)
IBM MQ High Availabillity and Disaster Recovery (2017 version)
MarkTaylorIBM2K views
What's new in informix v11.70 by am_prasanna
What's new in informix v11.70What's new in informix v11.70
What's new in informix v11.70
am_prasanna190 views
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr... by Hendrik van Run
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
2689 - Exploring IBM PureApplication System and IBM Workload Deployer Best Pr...
Hendrik van Run91 views
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M... by Peter Broadhurst
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...
Peter Broadhurst3.9K views
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee... by xKinAnx
Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...
xKinAnx2.6K views
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive by xKinAnx
Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep diveAccelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive
xKinAnx1.1K views
Hyper-V’s Virtualization Enhancements - EPC Group by EPC Group
Hyper-V’s Virtualization Enhancements - EPC GroupHyper-V’s Virtualization Enhancements - EPC Group
Hyper-V’s Virtualization Enhancements - EPC Group
EPC Group 928 views
Geographically Distributed Multi-Master MySQL Clusters by Continuent
Geographically Distributed Multi-Master MySQL ClustersGeographically Distributed Multi-Master MySQL Clusters
Geographically Distributed Multi-Master MySQL Clusters
Continuent2.5K views
1480-techintrotoiib-150224130001-conversion-gate01.pptx by BalakoteswaraReddyM
1480-techintrotoiib-150224130001-conversion-gate01.pptx1480-techintrotoiib-150224130001-conversion-gate01.pptx
1480-techintrotoiib-150224130001-conversion-gate01.pptx
W23 - Advanced Performance Tactics for WebSphere Performance by Hendrik van Run
W23 - Advanced Performance Tactics for WebSphere PerformanceW23 - Advanced Performance Tactics for WebSphere Performance
W23 - Advanced Performance Tactics for WebSphere Performance
Hendrik van Run95 views
Virtualizing Tier One Applications - Varrow by Andrew Miller
Virtualizing Tier One Applications - VarrowVirtualizing Tier One Applications - Varrow
Virtualizing Tier One Applications - Varrow
Andrew Miller664 views
PCA_Admin_Presentation-1.pptx by ssuser21ded1
PCA_Admin_Presentation-1.pptxPCA_Admin_Presentation-1.pptx
PCA_Admin_Presentation-1.pptx
ssuser21ded176 views
WSI35 - WebSphere Extreme Scale Customer Scenarios and Use Cases by Hendrik van Run
WSI35 - WebSphere Extreme Scale Customer Scenarios and Use CasesWSI35 - WebSphere Extreme Scale Customer Scenarios and Use Cases
WSI35 - WebSphere Extreme Scale Customer Scenarios and Use Cases
Hendrik van Run125 views
Multi Tenancy In The Cloud by rohit_ainapure
Multi Tenancy In The CloudMulti Tenancy In The Cloud
Multi Tenancy In The Cloud
rohit_ainapure20.8K views
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica... by Andrew Miller
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Andrew Miller682 views
Hhm 3474 mq messaging technologies and support for high availability and acti... by Pete Siddall
Hhm 3474 mq messaging technologies and support for high availability and acti...Hhm 3474 mq messaging technologies and support for high availability and acti...
Hhm 3474 mq messaging technologies and support for high availability and acti...
Pete Siddall498 views
Simplifying Hyper-V Management for VMware Administrators by 5nine
Simplifying Hyper-V Management for VMware AdministratorsSimplifying Hyper-V Management for VMware Administrators
Simplifying Hyper-V Management for VMware Administrators
5nine108 views
VMworld Europe 2014: Advanced SQL Server on vSphere Techniques and Best Pract... by VMworld
VMworld Europe 2014: Advanced SQL Server on vSphere Techniques and Best Pract...VMworld Europe 2014: Advanced SQL Server on vSphere Techniques and Best Pract...
VMworld Europe 2014: Advanced SQL Server on vSphere Techniques and Best Pract...
VMworld972 views

More from Peter Broadhurst

Kaleido Platform Overview and Full-stack Blockchain Services by
Kaleido Platform Overview and Full-stack Blockchain ServicesKaleido Platform Overview and Full-stack Blockchain Services
Kaleido Platform Overview and Full-stack Blockchain ServicesPeter Broadhurst
1.1K views94 slides
The Emerging Role of ZKP in the Privacy Stack of Production Business Networks... by
The Emerging Role of ZKP in thePrivacy Stack ofProduction Business Networks...The Emerging Role of ZKP in thePrivacy Stack ofProduction Business Networks...
The Emerging Role of ZKP in the Privacy Stack of Production Business Networks...Peter Broadhurst
612 views23 slides
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin... by
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...Peter Broadhurst
151 views23 slides
What does it take to deploy large scale decentralized applications? by
What does it take to deploy large scale decentralized applications?What does it take to deploy large scale decentralized applications?
What does it take to deploy large scale decentralized applications?Peter Broadhurst
234 views12 slides
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re... by
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...Peter Broadhurst
3.3K views36 slides
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities by
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging CapabilitiesIBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging CapabilitiesPeter Broadhurst
1.6K views38 slides

More from Peter Broadhurst(6)

Kaleido Platform Overview and Full-stack Blockchain Services by Peter Broadhurst
Kaleido Platform Overview and Full-stack Blockchain ServicesKaleido Platform Overview and Full-stack Blockchain Services
Kaleido Platform Overview and Full-stack Blockchain Services
Peter Broadhurst1.1K views
The Emerging Role of ZKP in the Privacy Stack of Production Business Networks... by Peter Broadhurst
The Emerging Role of ZKP in thePrivacy Stack ofProduction Business Networks...The Emerging Role of ZKP in thePrivacy Stack ofProduction Business Networks...
The Emerging Role of ZKP in the Privacy Stack of Production Business Networks...
Peter Broadhurst612 views
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin... by Peter Broadhurst
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...
Enterprise Blockchain: Lessons from Global Innovators in Production (All Thin...
Peter Broadhurst151 views
What does it take to deploy large scale decentralized applications? by Peter Broadhurst
What does it take to deploy large scale decentralized applications?What does it take to deploy large scale decentralized applications?
What does it take to deploy large scale decentralized applications?
Peter Broadhurst234 views
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re... by Peter Broadhurst
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...
Peter Broadhurst3.3K views
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities by Peter Broadhurst
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging CapabilitiesIBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities
IBM IMPACT 2014 AMC-1866 Introduction to IBM Messaging Capabilities
Peter Broadhurst1.6K views

Recently uploaded

Automated Testing of Microsoft Power BI Reports by
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsRTTS
10 views20 slides
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P... by
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...NimaTorabi2
16 views17 slides
Flask-Python.pptx by
Flask-Python.pptxFlask-Python.pptx
Flask-Python.pptxTriloki Gupta
9 views12 slides
Introduction to Gradle by
Introduction to GradleIntroduction to Gradle
Introduction to GradleJohn Valentino
6 views7 slides
Dapr Unleashed: Accelerating Microservice Development by
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice DevelopmentMiroslav Janeski
15 views29 slides
Ports-and-Adapters Architecture for Embedded HMI by
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMIBurkhard Stubert
33 views19 slides

Recently uploaded(20)

Automated Testing of Microsoft Power BI Reports by RTTS
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
RTTS10 views
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P... by NimaTorabi2
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...
NimaTorabi216 views
Dapr Unleashed: Accelerating Microservice Development by Miroslav Janeski
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice Development
Miroslav Janeski15 views
Ports-and-Adapters Architecture for Embedded HMI by Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert33 views
Supercharging your Python Development Environment with VS Code and Dev Contai... by Dawn Wages
Supercharging your Python Development Environment with VS Code and Dev Contai...Supercharging your Python Development Environment with VS Code and Dev Contai...
Supercharging your Python Development Environment with VS Code and Dev Contai...
Dawn Wages5 views
JioEngage_Presentation.pptx by admin125455
JioEngage_Presentation.pptxJioEngage_Presentation.pptx
JioEngage_Presentation.pptx
admin1254558 views
FOSSLight Community Day 2023-11-30 by Shane Coughlan
FOSSLight Community Day 2023-11-30FOSSLight Community Day 2023-11-30
FOSSLight Community Day 2023-11-30
Shane Coughlan7 views
Transport Management System - Shipment & Container Tracking by Freightoscope
Transport Management System - Shipment & Container TrackingTransport Management System - Shipment & Container Tracking
Transport Management System - Shipment & Container Tracking
Freightoscope 5 views
Understanding HTML terminology by artembondar5
Understanding HTML terminologyUnderstanding HTML terminology
Understanding HTML terminology
artembondar57 views
Quality Engineer: A Day in the Life by John Valentino
Quality Engineer: A Day in the LifeQuality Engineer: A Day in the Life
Quality Engineer: A Day in the Life
John Valentino7 views
tecnologia18.docx by nosi6702
tecnologia18.docxtecnologia18.docx
tecnologia18.docx
nosi67025 views
Advanced API Mocking Techniques Using Wiremock by Dimpy Adhikary
Advanced API Mocking Techniques Using WiremockAdvanced API Mocking Techniques Using Wiremock
Advanced API Mocking Techniques Using Wiremock
Dimpy Adhikary5 views

IBM Integration Bus High Availability Overview

  • 1. IBM Integration Bus High Availability Overview Peter Broadhurst IBM Messaging & Integration
  • 2. © 2013, 2014 International Business Machines Corporation 2 Introduction These charts provide a high-level overview of IIB HA topologies: • Comparison of active/active and active/passive HA • Solutions for active/passive HA failover with IBM Integration Bus • Solutions for active/active processing with IBM Integration Bus • Adding Global Cache to active/active processing • Combining all of the above Only HTTP and JMS (MQ) workloads are shown
  • 3. © 2013, 2014 International Business Machines Corporation 3 Active/active vs. active/passive Scale High Availability Design Considerations Active/Active All N instances contribute to the processing capacity of the system New requests can be serviced immediately after the planned or unplanned termination of an active instance. Often referred to as continuous availability. Each instance must be able to operate independently, without relying on the availability of any other instance in the environment. The order of processing for two items of work cannot be guaranteed, as there are multiple instances that might perform each item of work Active/Passive Only one of N instances contributes to the processing capacity of the system There is a failover period after the planned or unplanned termination of an active instance. This failover period commonly lasts for a small number of minutes, depending on the technology used. An infrastructure for HA failover must exist, that ensures only one instance is ever activated, as well as detecting when one instance fails to initiate the failover. The active and the passive system must have identical copies of persistent data, such as persisted transaction state, and persisted messages. In IBM Integration Bus this is achieved by sharing a filesystem between the two machines.
  • 4. © 2013, 2014 International Business Machines Corporation 4 Active/passive high availability Machine 1 Machine 2 IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server Shared file system HA failover • Single instance of the IIB runtime • Single instance of all state – Configuration, MQ messages and Transaction coordinator • Highly available due to automatic fail-over. Two options available for HA failover: – Out-of-the-box with Multi-Instance capability, – Or using external HA cluster software JMS workloads HTTP workloads
  • 5. © 2013, 2014 International Business Machines Corporation 5 Option 1: Out of the box Multi-Instance failover • IP address of each machine is different – IP address redirect is required for HTTP workloads – MQ client libraries automatically handle IP redirect for JMS workloads • Requires highly available network-attached storage (NAS). Examples: – IBM GPFS – Veritas Cluster File System – Highly available NFSv4 • More information on choosing a suitable NAS: – https://ibm.biz/BdFxfz Machine 1 – IP addr 1 Machine 2 – IP addr 2 IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server Network- attached file system (NAS) TCP/ IP Failover managed via NAS file locks IP redirect (Gateway / Load Balancer) JMS workloads TCP/ IP
  • 6. © 2013, 2014 International Business Machines Corporation 6 Option 2: HA cluster failover • Allows use of direct-attached storage • IP address failed over by HA cluster software • Requires HA cluster software. Examples: – IBM PowerHA (HACMP) – Veritas Cluster Server (VCS) – Microsoft Cluster Service (MSCS) – Red Hat Cluster Machine 1 – IP addr 1 Machine 2 – IP addr 1 IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server Direct-attached file system switched by HA Failover managed via health checking Fibre/SCSI HA cluster Software HA cluster Software Heartbeat Healthcheck Healthcheck JMS workloads HTTP workloads Fibre/SCSI IP addr failover
  • 7. © 2013, 2014 International Business Machines Corporation 7 Multi-instance or HA cluster? Multi-instance queue manager • Integrated into the IIB and MQ products • Faster failover than HA cluster* • Delay before queue manager restart is much shorter* • Runtime performance of networked storage must be considered • IP address of standby instance is different to primary • No automatic fail-back to primary hardware when restored • More susceptible to MQ and OS defects HA cluster • Capable of handling a wider range of failures • Failover historically rather slow, but some HA clusters are improving • Some customers frustrated by unnecessary failovers • Require MC91 SupportPac or equivalent configuration • Extra product purchase and skills required Storage distinction • Multi-instance queue manager typically uses NAS • HA clustered queue manager typically uses SAN *depends on NAS file-system tuning and specific customer environment
  • 8. © 2013, 2014 International Business Machines Corporation 8 Active/active topologies Machine 1 Machine 2 IIB Integration Node MQ Messaging IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server WESB AppTargetWESB AppTargetIntegration Server Gateway / Load Balancer for HTTP workloads HTTP workloads JMS workloads • Each Integration Node operates independently • Continuous availability of the service during a failure – Individual in-flight requests on the failed node receive errors, but the service stays available • HTTP workloads require external workload balancing – Hardware load balancer is the most common solution • JMS workloads require load balancing – Via an MQ cluster when remote applications have their own MQ queue managers – Options for direct MQ client attachment (over TCP/IP) described here: https://ibm.biz/BdFxfS
  • 9. © 2013, 2014 International Business Machines Corporation 9 Adding Global Cache to active/active topologies Machine 1 Machine 2 IIB Integration Node MQ Messaging Global Cache IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server WESB AppTargetWESB AppTargetIntegration Server • Highly available in-memory state store – High performance compared to disk persistence (DB/MQ Messaging) – High availability through redundancy – cannot be recovered if all nodes are stopped concurrently – Built with WebSphere eXtreme Scale technology • Alternative to using MQ or a Database for state storage/retrieval – Correlation state between asynchronous request/reply flows • Only needed if replies might be routed via a different runtime, otherwise simply use memory (LocalEnvironment etc.) – Regularly accessed reference data, such as routing tables
  • 10. © 2013, 2014 International Business Machines Corporation 10 Global Cache Putting it all together • Active/active processing. HA fail-over for state and in-flight message recovery • Individual queues/flows can be configured active/passive if required – Single flow instance for message ordering or file-based processing – Single correlation state queue shared between active flow instances Machine 1 Machine 2 IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server IIB Integration Node MQ Messaging WESB AppTargetWESB AppTargetIntegration Server Shared Filesystem Failover Failover Gateway / Load Balancer for HTTP workloads JMS workloads HTTP workloads
  • 11. © 2013, 2014 International Business Machines Corporation 11 More information • Knowledge Center starting points: IIB Active/passive HA: https://ibm.biz/BdFxfe IIB Active/active HA for HTTP: https://ibm.biz/BdFxfb IIB Global cache: https://ibm.biz/BdFxfp MQ HA Cluster configurations: https://ibm.biz/BdFxf8 • Wiki article with detailed description of IIB topology choices Tailored to customers migrating from WebSphere WESB, but useful to all https://ibm.biz/BdFxfa • MQDev blog on attaching MQ clients to active/active qmgrs: https://ibm.biz/BdFxfS • Testing and support statement for multi-instance https://ibm.biz/BdFxfz