Integration Solution Patterns
© WSO2 2013. Not for redistribution. Commercial in Confidence.
May 2013
Nadeesha Gamage
Dakshitha Ratnayake
About WSO2
•  Providing the only complete open source componentized cloud platform
•  Dedicated to removing all the stumbling blocks to enterprise agility
•  Enabling you to focus on business logic and business value
•  Recognized by leading analyst firms as visionaries and leaders
•  Gartner cites WSO2 as visionaries in all 3 categories of application
infrastructure
•  Forrester places WSO2 in top 2 for API Management
•  Global corporation with offices in USA, UK & Sri Lanka
•  200+ employees and growing
•  Business model of selling comprehensive support & maintenance for our products
150+ globally positioned support customers
Integration
Image Source -
http://blog.bootstraptoday.com/2013/01/03/considering-saas-3-tips-for-your-data-integration-strategy/
Integration – Why?
•  Business applications rarely live in isolation.
•  Users expect instant access to all business functions an
enterprise can offer.
•  Functionality may reside in different systems.
•  Requires disparate applications to be connected into a larger,
integrated solution.
Integration – How?
•  Architecting integration solutions is a complex task.
•  There are many conflicting drivers and even more possible 'right'
solutions.
•  No "cookbook" for enterprise integration solutions.
•  Asynchronous messaging architectures have proven to be the
best strategy for enterprise integration because they allow for a
loosely coupled solution that overcomes the limitations of
remote communication, such as latency and unreliability.
Why Patterns?
Patterns are an excellent way to ensure:
1.  Everyone understands the model taken – poses a design
problem and presents a solution
2.  Common vocabulary
3.  Use of best practices - experience base that architects have
gained by repeatedly building solutions and learning from
mistakes.
4.  Faster development time – reuse of patterns and not reinvent
patterns
Outline
§  Enterprise Integration Patterns (EIP) – Gregor Hohpe
§  Message Translator, Content-Based Router, etc.
§  Implementation of EIPs in WSO2 ESB
§  Low level patterns
§  Using functional components such as mediators, sequences
etc.
§  High level solution patterns
§  ESB Patterns – Concentrator, Security Proxy etc.
§  Overall Integration Patterns
§  e.g. API Gateway, Trusted subsystem etc.
§  Also see Integration Styles from EIP
§  Implementation of integration patterns using WSO2
products
Enterprise Integration Patterns
http://www.eaipatterns.com/
"The core language of EAI, defined in the
book Enterprise Integration Patterns by
Gregor Hohpe and Bobby Woolf, is also the
core language of defining ESB flows and
orchestrations, as seen in the ESB's
developer tooling."
Enterprise Integration Patterns
§  EIPs are patterns using asynchronous messaging as the style
of integration.
§  They are technology-agnostic which means it does not matter
what programming language or operating system you use.
§  Categorization of Patterns
•  Messaging Channels
•  Message Constructions
•  Message Routing
•  Message Transformation
•  Messaging Endpoints
•  System Management
WSO2 ESB is…
•  A lightweight, high performance ESB
•  Feature rich and standards compliant
•  SOAP and WS-* standards
•  REST support
•  Domain specific protocol support (eg: FIX)
•  User friendly and highly extensible
•  100% free and open source with commercial support
WSO2 ESB - Functional Components
•  Mediators
•  Sequences
•  Endpoints
•  Proxy Services
•  Event Sources
•  Priority Executors
•  Message Stores
•  Tasks
•  Local Entries
•  Registry
More on Functional Components
•  Each functional component serves a specific
purpose
•  Functional components can be mixed and
matched to implement various integration
scenarios and patterns
•  Configuring WSO2 ESB for a given scenario
requires:
•  Identifying the right set of components
•  Putting them together in the optimal manner
Mediators
http://docs.wso2.org/wiki/display/ESB460/Mediators
Rich mediator library, wherein most cases a single
mediator can be directly mapped to an EIP.
List of Mediators
•  Send
•  Log
•  Property
•  Sequence
•  Event
•  Drop
•  Enrich
•  Filter
•  Out
•  In
•  Switch
•  Router
•  Conditional Router
•  XSLT
•  URLRewrite
•  XQuery
•  Header
•  Fault
•  Cache
•  Clone
•  Store
•  Iterate
•  Aggregate
•  Callout
•  Transaction
•  Throttle
•  RMSequence
•  DBReport
•  DBLookup
•  Rule
•  Entitlement
•  OAuth
•  Class
•  POJOCommand
•  Script
•  Spring
•  Smooks
Sequences
•  A chain of mediators
•  Messages are sent through all the
mediators in the sequence, in the order
they appear
Enterprise Integration Patterns with WSO2 ESB
Complete Guide -
http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise
+Integration+Patterns+with+WSO2+ESB
The guide shows how each pattern in the patterns catalog can be
simulated using various constructs in WSO2 ESB.
A selected few of the patterns will be discussed in this webinar.
Message Channel
A message channel is a logical channel which is used to connect
the applications.
One application writes messages to the channel and the other one
(or others) reads that message from the channel.
Message queue and message topic are examples of message
channels.
•  Queues -
•  With WSO2 ESB, a common pattern is to persist the
message from an incoming HTTP request into a persistent
message queue, and then process from there.
•  WSO2 Message Broker, ActiveMQ etc. can provide the
persistent queue.
•  Topics – Supported by ESB and Message Broker
•  WSO2 ESB has an event distribution model and eventing
support.
•  WSO2 Message Broker, the Apache QPid-based broker,
provides higher performance as well as supporting the JMS
API.
Message Channel continued
Message Translator
Message translator transforms messages in one format to another.
Can be implemented through several Mediators in WSO2 ESB –
• Enrich
• XSLT
• Smooks
• Payloadfactory
• XQuery
Publish-Subscribe Channel
•  This type of channel broadcasts an event or notification to all
subscribed receivers.
•  This is in contrast with a point-to-point channel. Each subscriber
receives the message once.
•  The most common implementation of this pattern is messaging
topic.
•  Event mediator.
•  Topics – WSO2 ESB/ WSO2 Message Broker
Publish-Subscribe Channel continued
Dead Letter Channel
The Dead Letter Channel describes how messaging system
determines how to deliver a dead message to the specified
recipient.
This may be caused by connection problems or other exception like
overflowed memory or disc space.
Dead Letter Channel continued
Content-Based Router
Content-Based Router examines message contents and
route messages based on data contained in the
message.
Content-Based Router continued
Message routing is simulated by Switch and Send mediators
of the WSO2 ESB.
The Switch Mediator depicts the Router and observes the
content of the message while the Send Mediator is used to
send the message to a selected recipient.
Each case defined should decide on routing the message to
the appropriate service.
Content Enricher
Content Enricher as the name suggests enrich message with
missing information. Usually an external data source like a
database or a web service is used.
Content Enricher continued
Message Filter
Message Filter EIP checks an incoming message against a certain
criteria that the message should adhere to.
If the criteria is not met, the filter will discard the message.
Else, it will route the message to the output channel.
The Filter mediator in the ESB will check on the criteria.
Splitter/Aggregator
The Splitter EIP - Processes messages containing multiple
elements.
Breaks out the composite message into a series of individual
messages.
The Aggregator EIP - Combines the results of individual messages
so that they can be processed as a whole.
Splitter/Aggregator continued
Splitter
Aggregator
Composed Message Processor
The Composed Msg. Processor EIP is used to process a
composite message.
It maintains the overall message flow when processing a message
consisting of multiple elements.
Composed Message Processor continues
•  Use of Sequences
•  Named Sequences
•  Sequences in Proxy Services – In Sequence, Out
Sequence, Fault Sequence
WSO2 ESB Configuration Documented as EIPs
ESB Patterns
•  It’s worth understanding the way the ESB is being used:
•  ESB are not the same as network routers
•  Can be used in multiple ways for Routing, Traffic filtering,
Transformation, Protocol Switching, Load balancing.
•  More like a Swiss-army knife
•  Be clear about how you intend to use it
•  The same ESB can be implementing multiple patterns
•  Anti-Patterns
•  Implement all your business logic in the ESB
The Concentrator Pattern
EAI Hub
•  Request Transformation
•  ESB can split and clone messages.
•  Response Transformation
•  ESB can aggregate messages.
•  Service chaining
•  ESB can act as a lightweight orchestration engine.
Federated ESB Pattern
Enterprise ESB
Routing and
Audit
Department
ESB
Department
ESB
Department
ESB
Overall Integration Patterns
•  API Gateway
•  Service Broker/Trusted Subsystem
•  Service Firewall
•  Master Data Management
•  Security Gateway
•  Dual Channel/ Claim Check
WSO2 Product Stack
API Gateway
Business Problem
•  Different Service platforms run on a variety of server hardware,
operating systems and development platforms.
•  Services expose different communication and security patterns.
•  Complexity in exposing the SOA to internal and external
consumers.
API Gateway
Solution
•  An Approach to simplify and normalize interaction with
heterogeneous services.
•  Define a unified generic service layer on-top of the existing
services to expose to the consumers.
•  API Gateway Pattern is able to solve this business problem.
API Gateway
Service Broker/ Trusted Subsystem
Business Problem
•  Backend services need to be relieved from authentication of
service clients.
•  Security authentication needs to be handled before the call
reach the backend service.
Solution
•  User authentication to be handled at the ESB layer.
•  Backend service trust the ESB.
•  ESB access backend service on-behalf of the authenticated
user.
Service Broker/ Trusted Subsystem
Service Firewall Pattern
Business Problem
•  To expose internal business services to external service
consumers.
•  To ensure that only authorized users access the services
exposed.
•  To validate the user credentials before reaching the internal
network.
Solution
•  Deploy a separate ESB to validate user requests.
•  Achieve this by using a Service Firewall Pattern.
Service Firewall Pattern
Security Gateway Pattern
Master Data Management (MDM)
Business Problem
•  Business organizations have legacy applications and databases
that store and manage master data.
•  Managing master data is critical to make successful business
decisions.
•  Present IT infrastructure is depended on SOA.
•  Need a way to connect the Master Data with the SOA.
Master Data Management (MDM)
Traditional Approach
Master Data Management (MDM)
Solution
•  To expose Master data as services in a SOA.
•  Master Data Management Pattern is able to solve this problem.
Dual Channeling
Business Problem
•  When large files are required to go through a workflow process.
•  Moving files through the workflow is inefficient, creates
additional traffic, reduce the throughput.
Dual Channeling
Solution
•  Dual Channel pattern avoids the above mentioned problems by
having two channels.
•  For the metadata on file
•  For the actual file
•  A light-weight message with the file related metadata, to process
within the workflow.
•  Metadata contains the pointer reference to the actual file.
Dual Channeling
Dual Channeling
Resources
•  http://wso2.com/blogs/architecture
•  http://www.eaipatterns.com/
•  http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise
+Integration+Patterns+with+WSO2+ESB
Questions?
Engage with WSO2
•  Helping you get the most out of your deployments
•  From project evaluation and inception to development
and going into production, WSO2 is your partner in
ensuring 100% project success
59
lean . enterprise . middleware

Integration Solution Patterns

  • 1.
    Integration Solution Patterns ©WSO2 2013. Not for redistribution. Commercial in Confidence. May 2013 Nadeesha Gamage Dakshitha Ratnayake
  • 2.
    About WSO2 •  Providingthe only complete open source componentized cloud platform •  Dedicated to removing all the stumbling blocks to enterprise agility •  Enabling you to focus on business logic and business value •  Recognized by leading analyst firms as visionaries and leaders •  Gartner cites WSO2 as visionaries in all 3 categories of application infrastructure •  Forrester places WSO2 in top 2 for API Management •  Global corporation with offices in USA, UK & Sri Lanka •  200+ employees and growing •  Business model of selling comprehensive support & maintenance for our products
  • 3.
    150+ globally positionedsupport customers
  • 4.
  • 5.
    Integration – Why? • Business applications rarely live in isolation. •  Users expect instant access to all business functions an enterprise can offer. •  Functionality may reside in different systems. •  Requires disparate applications to be connected into a larger, integrated solution.
  • 6.
    Integration – How? • Architecting integration solutions is a complex task. •  There are many conflicting drivers and even more possible 'right' solutions. •  No "cookbook" for enterprise integration solutions. •  Asynchronous messaging architectures have proven to be the best strategy for enterprise integration because they allow for a loosely coupled solution that overcomes the limitations of remote communication, such as latency and unreliability.
  • 7.
    Why Patterns? Patterns arean excellent way to ensure: 1.  Everyone understands the model taken – poses a design problem and presents a solution 2.  Common vocabulary 3.  Use of best practices - experience base that architects have gained by repeatedly building solutions and learning from mistakes. 4.  Faster development time – reuse of patterns and not reinvent patterns
  • 8.
    Outline §  Enterprise IntegrationPatterns (EIP) – Gregor Hohpe §  Message Translator, Content-Based Router, etc. §  Implementation of EIPs in WSO2 ESB §  Low level patterns §  Using functional components such as mediators, sequences etc. §  High level solution patterns §  ESB Patterns – Concentrator, Security Proxy etc. §  Overall Integration Patterns §  e.g. API Gateway, Trusted subsystem etc. §  Also see Integration Styles from EIP §  Implementation of integration patterns using WSO2 products
  • 9.
    Enterprise Integration Patterns http://www.eaipatterns.com/ "Thecore language of EAI, defined in the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf, is also the core language of defining ESB flows and orchestrations, as seen in the ESB's developer tooling."
  • 10.
    Enterprise Integration Patterns § EIPs are patterns using asynchronous messaging as the style of integration. §  They are technology-agnostic which means it does not matter what programming language or operating system you use. §  Categorization of Patterns •  Messaging Channels •  Message Constructions •  Message Routing •  Message Transformation •  Messaging Endpoints •  System Management
  • 11.
    WSO2 ESB is… • A lightweight, high performance ESB •  Feature rich and standards compliant •  SOAP and WS-* standards •  REST support •  Domain specific protocol support (eg: FIX) •  User friendly and highly extensible •  100% free and open source with commercial support
  • 12.
    WSO2 ESB -Functional Components •  Mediators •  Sequences •  Endpoints •  Proxy Services •  Event Sources •  Priority Executors •  Message Stores •  Tasks •  Local Entries •  Registry
  • 13.
    More on FunctionalComponents •  Each functional component serves a specific purpose •  Functional components can be mixed and matched to implement various integration scenarios and patterns •  Configuring WSO2 ESB for a given scenario requires: •  Identifying the right set of components •  Putting them together in the optimal manner
  • 14.
    Mediators http://docs.wso2.org/wiki/display/ESB460/Mediators Rich mediator library,wherein most cases a single mediator can be directly mapped to an EIP.
  • 15.
    List of Mediators • Send •  Log •  Property •  Sequence •  Event •  Drop •  Enrich •  Filter •  Out •  In •  Switch •  Router •  Conditional Router •  XSLT •  URLRewrite •  XQuery •  Header •  Fault •  Cache •  Clone •  Store •  Iterate •  Aggregate •  Callout •  Transaction •  Throttle •  RMSequence •  DBReport •  DBLookup •  Rule •  Entitlement •  OAuth •  Class •  POJOCommand •  Script •  Spring •  Smooks
  • 16.
    Sequences •  A chainof mediators •  Messages are sent through all the mediators in the sequence, in the order they appear
  • 17.
    Enterprise Integration Patternswith WSO2 ESB Complete Guide - http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise +Integration+Patterns+with+WSO2+ESB The guide shows how each pattern in the patterns catalog can be simulated using various constructs in WSO2 ESB. A selected few of the patterns will be discussed in this webinar.
  • 18.
    Message Channel A messagechannel is a logical channel which is used to connect the applications. One application writes messages to the channel and the other one (or others) reads that message from the channel. Message queue and message topic are examples of message channels.
  • 19.
    •  Queues - • With WSO2 ESB, a common pattern is to persist the message from an incoming HTTP request into a persistent message queue, and then process from there. •  WSO2 Message Broker, ActiveMQ etc. can provide the persistent queue. •  Topics – Supported by ESB and Message Broker •  WSO2 ESB has an event distribution model and eventing support. •  WSO2 Message Broker, the Apache QPid-based broker, provides higher performance as well as supporting the JMS API. Message Channel continued
  • 20.
    Message Translator Message translatortransforms messages in one format to another. Can be implemented through several Mediators in WSO2 ESB – • Enrich • XSLT • Smooks • Payloadfactory • XQuery
  • 21.
    Publish-Subscribe Channel •  Thistype of channel broadcasts an event or notification to all subscribed receivers. •  This is in contrast with a point-to-point channel. Each subscriber receives the message once. •  The most common implementation of this pattern is messaging topic. •  Event mediator. •  Topics – WSO2 ESB/ WSO2 Message Broker
  • 22.
  • 23.
    Dead Letter Channel TheDead Letter Channel describes how messaging system determines how to deliver a dead message to the specified recipient. This may be caused by connection problems or other exception like overflowed memory or disc space.
  • 24.
  • 25.
    Content-Based Router Content-Based Routerexamines message contents and route messages based on data contained in the message.
  • 26.
    Content-Based Router continued Messagerouting is simulated by Switch and Send mediators of the WSO2 ESB. The Switch Mediator depicts the Router and observes the content of the message while the Send Mediator is used to send the message to a selected recipient. Each case defined should decide on routing the message to the appropriate service.
  • 27.
    Content Enricher Content Enricheras the name suggests enrich message with missing information. Usually an external data source like a database or a web service is used.
  • 28.
  • 29.
    Message Filter Message FilterEIP checks an incoming message against a certain criteria that the message should adhere to. If the criteria is not met, the filter will discard the message. Else, it will route the message to the output channel. The Filter mediator in the ESB will check on the criteria.
  • 30.
    Splitter/Aggregator The Splitter EIP- Processes messages containing multiple elements. Breaks out the composite message into a series of individual messages. The Aggregator EIP - Combines the results of individual messages so that they can be processed as a whole.
  • 31.
  • 32.
    Composed Message Processor TheComposed Msg. Processor EIP is used to process a composite message. It maintains the overall message flow when processing a message consisting of multiple elements.
  • 33.
    Composed Message Processorcontinues •  Use of Sequences •  Named Sequences •  Sequences in Proxy Services – In Sequence, Out Sequence, Fault Sequence
  • 34.
    WSO2 ESB ConfigurationDocumented as EIPs
  • 35.
    ESB Patterns •  It’sworth understanding the way the ESB is being used: •  ESB are not the same as network routers •  Can be used in multiple ways for Routing, Traffic filtering, Transformation, Protocol Switching, Load balancing. •  More like a Swiss-army knife •  Be clear about how you intend to use it •  The same ESB can be implementing multiple patterns •  Anti-Patterns •  Implement all your business logic in the ESB
  • 36.
  • 37.
    EAI Hub •  RequestTransformation •  ESB can split and clone messages. •  Response Transformation •  ESB can aggregate messages. •  Service chaining •  ESB can act as a lightweight orchestration engine.
  • 38.
    Federated ESB Pattern EnterpriseESB Routing and Audit Department ESB Department ESB Department ESB
  • 39.
    Overall Integration Patterns • API Gateway •  Service Broker/Trusted Subsystem •  Service Firewall •  Master Data Management •  Security Gateway •  Dual Channel/ Claim Check
  • 40.
  • 41.
    API Gateway Business Problem • Different Service platforms run on a variety of server hardware, operating systems and development platforms. •  Services expose different communication and security patterns. •  Complexity in exposing the SOA to internal and external consumers.
  • 42.
    API Gateway Solution •  AnApproach to simplify and normalize interaction with heterogeneous services. •  Define a unified generic service layer on-top of the existing services to expose to the consumers. •  API Gateway Pattern is able to solve this business problem.
  • 43.
  • 44.
    Service Broker/ TrustedSubsystem Business Problem •  Backend services need to be relieved from authentication of service clients. •  Security authentication needs to be handled before the call reach the backend service. Solution •  User authentication to be handled at the ESB layer. •  Backend service trust the ESB. •  ESB access backend service on-behalf of the authenticated user.
  • 45.
  • 46.
    Service Firewall Pattern BusinessProblem •  To expose internal business services to external service consumers. •  To ensure that only authorized users access the services exposed. •  To validate the user credentials before reaching the internal network. Solution •  Deploy a separate ESB to validate user requests. •  Achieve this by using a Service Firewall Pattern.
  • 47.
  • 48.
  • 49.
    Master Data Management(MDM) Business Problem •  Business organizations have legacy applications and databases that store and manage master data. •  Managing master data is critical to make successful business decisions. •  Present IT infrastructure is depended on SOA. •  Need a way to connect the Master Data with the SOA.
  • 50.
    Master Data Management(MDM) Traditional Approach
  • 51.
    Master Data Management(MDM) Solution •  To expose Master data as services in a SOA. •  Master Data Management Pattern is able to solve this problem.
  • 52.
    Dual Channeling Business Problem • When large files are required to go through a workflow process. •  Moving files through the workflow is inefficient, creates additional traffic, reduce the throughput.
  • 53.
    Dual Channeling Solution •  DualChannel pattern avoids the above mentioned problems by having two channels. •  For the metadata on file •  For the actual file •  A light-weight message with the file related metadata, to process within the workflow. •  Metadata contains the pointer reference to the actual file.
  • 54.
  • 55.
  • 56.
    Resources •  http://wso2.com/blogs/architecture •  http://www.eaipatterns.com/ • http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise +Integration+Patterns+with+WSO2+ESB
  • 57.
  • 58.
    Engage with WSO2 • Helping you get the most out of your deployments •  From project evaluation and inception to development and going into production, WSO2 is your partner in ensuring 100% project success
  • 59.