Wso2 integration platform deep dive eu con 2016

WSO2 Integration Platform Deep
Dive
Chanaka Fernando - Senior Technical Lead
Ramith Jayasinghe - Senior Technical Lead

Agenda
• Concepts in Integration
• Demonstration
• Product roadmaps/features
• Q&A

Evolution of Integration
Point to Point Integration
● Custom code, data formats for
majority of integration ( data
extraction, business rule
processing, data loading)
● Costly over time (maintenance,
complexity)
● Tight Coupling
● Scalability is an issue
● Less reusability

Evolution of Integration
Use of Enterprise Service Bus ● Consistency: Clear set of guidelines
(EIPs) results in repeatable and
successful integration
efforts/projects.
● Reusability: Tested and
preconfigured components provides
repeatable approach to
integration.(transports/connectors/m
ediators etc)
● Effectiveness: A common integration
framework (with related abstract
concepts) improves integration
developer's productivity.

Connected Business
An internally and externally connected business.
Image courtesy hRp://jdamico.net/wp-content/uploads/2010/04/collaborate-with-B2B-channel-partners.jpg
ESB were used to connect
internal systems in the past.
Current need is to connect
internal/external parties quickly
and flexible manner.
E.g. Employees, Internal/External
systems, Partners, Customers.

Connected Business
Integration is a key factor for having frictionless interactions, inside
and outside the organization.

Integration and APIs - The Close Cousins
• APIs cannot replace Integration
– Integration of internal services, systems, data and cloud apis
• Cannot mangle SOA for API Management needs
• Using SOA and API in combination is the key to success of
Connected Business.

Integration and APIs - The Close Cousins

WSO2 Integration + API Management Platform
In practise API management and
integration co-exists providing
flexible, faster ways to connect
required parties such as users,
internal/external systems, employees
etc.

Fundamentals of WSO2 ESB
Message Processing units
- Sequences, mediators
Message entry points
- Proxy Services, APIs, Inbound
endpoints.
Message exit points
- Endpoints

Example Mediators
Name Description
Log Mediator Logs full or part of the message, at various severity levels ( Trace, Debug, etc)
Sequence Mediator Invokes existing sequence - Sequence name can be static or dynamic
Send Mediator Sends a message out, using static information or endpoint definition.
Datamapper Mediator Transform data from one form to another
Switch Mediator Evaluates messages contents against regular expression and invokes the
corresponding mediator (switch-case-default)
Validate Mediator Validates message or parts of message against XML schema (schema can be
local or in registry)
PayloadFactory Mediator Create a message payload from an existing message or from scratch
Fault Mediator Transforms current message into custom Fault message

Mediators
• Transformation Mediators
• Filters, Switches, Iterate, Aggregate, Clone
• Property Mediator
• Script Mediators
• Custom Mediators

Connectors
• A connector is a ready made and convenient
tool to reach publicly available web API’s.
• ‘Cloud to Cloud’ and ‘Cloud to Enterprise’
Integration
• ESB Connector Store : https://store.wso2.com/

Connectors
• Every connector is self-contained and
independent from ESB code
• Dynamically plug in to ESB
• Dynamic Tooling support with WSO2 Developer
Studio
• You can write, ‘your own connector’ and just plug it
in

Proxy Service
• Acts like a virtual service.
• Receives messages and mediates
them before sending them to the
endpoint (usually an actual
service)
• Mainly to expose as SOAP Service

API/HTTP Service
• APIs can accept REST messages which allow clients
to provide additional information on how to
mediate the message
• Defined under a URL Context (e.g: /customer)
• Can define multiple resources under a URL context
• Resources
– Component of API accessed through HTTP call
– Similar to proxy service (in, out, and fault
sequences)
– Can restrict resource’s scope using URL
patterns and URI templates

API/HTTP Service
• URL mapping
– Path mappings (eg: /test/*, /foo/bar/*)
– Extension mappings (eg: *.jsp, *.do)
– Exact mappings (eg: /test, /test/foo)
• URI template
– /order/{orderId}would process /order/A0001
– /dictionary/{char}/{word}would process /dictionary/c/cat
– Use get-property to retrieve exact values
<log level="custom">
<property name="Character" expression="get-property('uri.var.char')"/>
<property name="Word" expression="get-property('uri.var.word')"/>
</log>

Inbound Endpoint
• Creating inbound messaging channels
dynamically.
– Listening Inbound – HTTP, HL7, TCP
– Polling Inbound – JMS, File/VFS
• Injects the inbound messages into an
ESB message flow.

Polling Inbound endpoint in a Cluster
One Consumer Multiple Consumers

Endpoints
• Logical representation of one or more real backend
service, jms queue etc.
• Address, HTTP Endpoints – A logical endpoint for an
existing BE service
• Load balancing / Failover – Group of endpoints for
existing BE services

Message Store and Message Processor
• Asynchronous/One-way Messaging, JMS
• Store and forward
• Guaranteed Delivery and Rate matching
• JMS Message broker as the persistence store – WSO2 MB, Apache
Active MQ etc.

Message Processor in a Cluster
• One consumer per cluster – Guaranteed in-order delivery
• Multiple consumers – Guaranteed delivery
One Consumer Multiple Consumers

Tasks
• Allow configuration of scheduled jobs to execute internal/external
commands
• Inject a message into a proxy service or a named sequence

Service Chaining
• Orchestrating/Combining multiple Services

Enterprise Integration Patterns
https://docs.wso2.com/display/IntegrationPatterns/Enterprise
+Integration+Patterns+with+WSO2+ESB

WSO2 ESB 5.0 – Upcoming release
• Beta version is available (Released on May 2016)
• Improved usability of the ESB through debugging, data
mapping capabilities through developer studio tooling
• The product is released as a tuple of runtime, analytics
and tooling
• Dedicated analytics distribution(DAS) for ESB specific
monitoring with pre built dashboards

WSO2 ESB 5.0 – Statistics/Tracing
• Coarse grained statistics about proxy/sequence/api/endpoint

• Fine grained statistics/tracing on mediators (message level)

• Fine grained tracing on messages

WSO2 ESB 5.0 – Visual Data Mapper
• Visual Data mapper
– Different input/output options (XML, JSON, Files, Schemas)
– Apply functions while mapping the data (Split, Aggregate, Uppercase, Lowercase)

WSO2 ESB 5.0 – Mediation debugger
• Debug mediation flows from the Developer studio

WSO2 ESB – What’s new in 5.0
Other new features
• JMS 2.0 support
• Websocket support
• JMS distributed(XA) transaction support
• Rabbitmq/MQTT performance improvements

WSO2 Integration Cloud (Upcoming)

WSO2 Integration Cloud (Upcoming)
Integration Platform as a Service (iPaaS)
• Running integrations on the cloud
• Analytics integration
• Develop integrations using WSO2 Developer studio

WSO2 Business Process Server
• BPMN 2.0
• WS-BPEL 2.0
• WS-HumanTask 1.1
• BPEL4People
● Define and execute business processes
● Define workflows interacting with People
● Graphical process modeling/administration/monitoring

WSO2 Business Rules Server
● Integrates JSR94
compliant rule engines.
● Embed drools engine.

Fundamentals of WSO2 Message Broker
Messaging Models
● RPC Style
- Request/Response (there is always a response)
- Synchronous (client waits for response)
- Non-persistent (message is lost if something failed)
● Point to Point (Queues)
- A message is delivered only once to a single Broadcast a
message to all the subscribers
● Publish/Subscribe (Topics)
- Broadcast a message to all the subscribers

Messaging Models
● Publish/Subscribe (Topics)
- Broadcast a message to all the subscribers
● Durable Subscriptions
- Same as topic subscriptions
- Subscriber offline - Messages get queued
- Subscriber back online - Messages get delivered
- Unsubscription needed to remove queuing

● Transactional
- Send/Receive multiple messages in a transactions.
- ( send/consume -> rollback, reject, commit)
● Guaranteed Delivery/Acknowledgments , QOS ( MQTT, JMS).
● Persistence
- Messages kept in memory or in reliable storage.

● Dead letter channel
- Broker keeps invalid/rejected messages in this channel.
- Users can inspect/reroute/delete these messages.
● Hierarchical topics
- Map relationships between data being published onto different
topics.
- Can Subscribe to part of the hierarchy.
Games.*
- receives messages published to ‘Games.FootBall’
and ‘Games.Cricket’
Games.Cricket.#
- Receives messages published to
‘Games.Cricket.SriLanka’ and ‘Games.Cricket.UK’

Distributed Queues
● Provides strict or best effort
support for inorder delivery
● There are no guarantee
about the global order seen
across subscribers

Distributed Queues with MB
● Provides strict or best effort
support for inorder delivery
● There are no guarantee
about the global order seen
across subscribers
● Supported Protocols
AMQP/JMS, MQTT

MB Roadmap
• Support for AMQP 1.0 and JMS 2.0
• Improvements to Storage Scheme, Performance improvements
• Analytics for MB
• C5 migration
• Improved GUI
• MQTT + Websockets

Introducing WSO2 Data Services Server
Painpoints:
● Application Silos / Different data
stores
● Cumbersome data access/
transformation logic
● Business logic vs Data access logic
● Repetitive Code
DSS:
● Well defined interfaces (standard based)
● Encapsulated data logic
● Configuration driven
● Loose Coupling between App & Data stores
● Scale as a separate architectural layer.

WSO2 Data Services Server : Features
• Multiple Data Sources
• Nested Queries
• Batch Processing
• Distributed Transactions
• OData (www.odata.org)
– Quickly expose data as REST APIs
– Standardized [URL conventions, Request/Response Headers,
Status Codes ,Payload formats]

Demonstration - Use Case
• As part of an effort to improve traffic monitoring process, City
transport authorities are making buses to send useful
information while it's in operation such as speed, location data
and direction.
• To ensure safety of passengers, officials would like to detect bus
drivers who violate speed limits. Supervisors will investigate
these incidents case-by-case basis.
• If a supervisor determines a particular incident is undue, bus
driver will be placed under further investigation/review.

Demonstration - Solution Architecture

Demonstration - Concepts (OData)
Retrieve duty roster:
GET http://localhost:9765/odata/AllocationData/DriverAllocations/duty_roster(1)
{
"@odata.context": "$metadata#duty_roster/$entity",
"@odata.etag": "8d32bf38-8d8f-30cb-8e87-36ebeaedc2a9",
"vehicleId": 1,
"driverId": 1
}
Retrieve employee information:
GET http://localhost:9765/odata/HRDataAPI/records/employee(1)
{
"@odata.context": "$metadata#employee/$entity",
"@odata.etag": "58bff0d2-09a4-3315-b2a0-667626dbf5c7",
"id": 1,
"supervisorId": 2,
"username": "driver",
"status": "UnderReview",
"name": "Driver"
}
Update Employee status:
PATCH http://localhost:9765/odata/HRDataAPI/records/employee(1)
{ "status" :"UnderReview"}

Demonstration - Concepts (Inbound Endpoints)
<inboundEndpoint name="speed-alert-receiver" onError="fault"
protocol="jms" sequence="BusSpeedViolationsSeq" suspend="false" xmlns="http://ws.apache.org/ns/synapse">
<parameters>
<parameter name="interval">1</parameter>
<parameter name="sequential">true</parameter>
<parameter name="coordination">true</parameter>
<parameter name="transport.jms.Destination">SpeedViolations</parameter>
<parameter name="transport.jms.CacheLevel">3</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName">QueueConnectionFactory</parameter>
<parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>
<parameter name="java.naming.provider.url">repository/conf/jndi.properties</parameter>
<parameter name="transport.jms.SessionAcknowledgement">SESSION_TRANSACTED</parameter>
<parameter name="transport.jms.SessionTransacted">true</parameter>
<parameter name="transport.jms.SubscriptionDurable">false</parameter>
<parameter name="transport.jms.ConnectionFactoryType">queue</parameter>
<parameter name="transport.jms.ReceiveTimeout">100</parameter>
<parameter name="transport.jms.ContentType">application/json</parameter>
</parameters>
</inboundEndpoint>
<inboundEndpoint name="hr-request-receiver" onError="fault"
protocol="http" sequence="HRSeq" suspend="false" xmlns="http://ws.apache.org/ns/synapse">
<parameters>
<parameter name="inbound.http.port">8081</parameter>
</parameters>
</inboundEndpoint>

Demonstration - Concepts (Endpoints)
<endpoint name="DutyRosterEP" xmlns="http://ws.apache.org/ns/synapse">
<http method="get" trace="disable"
uri-template="http://localhost:9765/odata/AllocationData/DriverAllocations/duty_roster({uri.var.busId})"/>
</endpoint>
<endpoint name="GetEmployeeRecordsEP" xmlns="http://ws.apache.org/ns/synapse">
<http method="get" trace="disable" uri-template="http://localhost:9765/odata/HRDataAPI/records/employee({uri.var.employeeId})"/>
</endpoint>
<endpoint name="UpdateEmployStatusEP" xmlns="http://ws.apache.org/ns/synapse">
<http method="patch" trace="disable"
uri-template="http://localhost:9765/odata/HRDataAPI/records/employee({uri.var.employeeId})"/>
</endpoint>
Usage:
<property expression="json-eval($.driverId)" name="uri.var.employeeId"
scope="default" type="STRING"/>
<call>
<endpoint key="GetEmployeeRecordsEP"/>
</call>

Demonstration - Concepts (Data Mapping)
<datamapper
config="gov:datamapper/MapSpeedViolationResultToHRSystem.dmc"
description="Map-Speed-Violation-OutCome-To-HR-System"
inputSchema="gov:datamapper/MapSpeedViolationResult
ToHRSystem_inputSchema.json"
inputType="JSON"
outputSchema=
"gov:datamapper/MapSpeedViolationResultToHRSystem_outputSchem
a.json"
outputType="JSON" />

Demonstration - Concepts (Payload Messages and Service Chains)
<sequence name="BusSpeedViolationsSeq" onError="fault" statistics="enable" trace="disable"
xmlns="http://ws.apache.org/ns/synapse">
<property expression="json-eval($.event.payloadData.id)"
name="uri.var.busId" scope="default" type="STRING"/>
<property expression="json-eval($.event.payloadData.id)"
name="busId" scope="default" type="STRING"/>
<property expression="json-eval($.event.payloadData.timeStamp)"
name="timeStamp" scope="default" type="STRING"/>
<property name="uri.var.busId" scope="default"
type="STRING" value="1"/>
<call>
<endpoint key="DutyRosterEP"/>
</call>
<property expression="json-eval($.driverId)"
name="uri.var.employeeId" scope="default" type="STRING"/>
name="driverId" scope="default" type="STRING"/>
<call>
</call>

<payloadFactory media-type="json">
<format>
{"processDefinitionKey": "trafficViolations","businessKey": "trafficKey","tenantId": "-1234",
"variables": [
{"name": "vehicleId", "value": "$1"},
{ "name": "timeStamp", "value": $2 },
{ "name": "driverId", "value": "$3"},
{ "name": "driverUserName", "value": "$4"},
{ "name": "driverName", "value": "$5"},
{ "name": "supervisorUserName", "value": "$6" }]}
</format>
<args>
<arg evaluator="xml" expression="$ctx:busId" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:timeStamp" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverId" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverUsername" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverName" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:supervisorUserName" xmlns:ns="http://org.apache.synapse/xsd"/>
</args>
</payloadFactory>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<property expression="fn:concat('Basic ', base64Encode('admin:admin'))" name="Authorization" scope="transport"
type="STRING" xmlns:ns="http://org.apache.synapse/xsd"/>
<call> <endpoint key="CreateBPMNWorkFlowEP"/> </call>
<drop/>
</sequence>

<sequence name="HRSeq" statistics="enable" trace="disable"
xmlns="http://ws.apache.org/ns/synapse">
name="uri.var.employeeId" scope="default" type="STRING" />
<datamapper config="gov:datamapper/MapSpeedViolationResultToHRSystem.dmc"
description="Map-Speed-Violation-OutCome-To-HR-System"
inputSchema="gov:datamapper/MapSpeedViolationResultToHRSystem_inputSchema.json"
inputType="JSON"
outputSchema="gov:datamapper/MapSpeedViolationResultToHRSystem_outputSchema.json"
outputType="JSON" />
<property name="FORCE_SC_ACCEPTED" scope="axis2" type="STRING" value="true" />
<property name="OUT_ONLY" scope="default" type="STRING" value="true" />
<call description="">
<endpoint key="UpdateEmployStatusEP" />
</call>
</sequence>

Demonstration - Concepts (BPMN Workflows)

Next generation Integration Platform
• WSO2 Gateway Framework
– An ultra high performance, lightweight and reusable message
Gateway framework that encapsulates the messaging between
source and target systems
– HTTP/s message gateway leveraging Netty, LMAX Disruptor and
WSO2 Pass-Thru messaging architecture.

WSO2 Integration product suite
• Series of products based on WSO2 GW Core framework.

WSO2 Process Center
Process owner /
analyst
Process
participant
Developer
Create / improve
/ standardize
Search / browse
/ follow
View
Develop executable processes
Executable
process is
associated with
Process Center

Thank You!
#WSO2ConEU
Share your feedback for this session
wso2con.com/app

Wso2 integration platform deep dive eu con 2016

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Wso2 integration platform deep dive eu con 2016

Similar to Wso2 integration platform deep dive eu con 2016 (20)

More from Chanaka Fernando

More from Chanaka Fernando (10)

Recently uploaded

Recently uploaded (20)

Wso2 integration platform deep dive eu con 2016