To view recording of this webinar please use the below URL: http://wso2.com/library/webinars/2015/06/exploring-data-integration-capabilities-of-the-wso2-platform/ You will learn the following: How siloed data in an enterprise can be quickly exposed as a service How this data can be integrated with various systems How WSO2 DSS can be used with WSO2 Enterprise Service Bus for data intensive applications that support security, transformation, and distributed transactions

1. Exploring Data Integration Capabilities
of the WSO2 Platform
June 2015
Jagath Ariyarathne
Technical Lead
Ravi Undupitiya
Senior Software Engineer

2. About the Presenters
๏ Jagath Ariyarathne - jagatha@wso2.com
Jagath is a Technical Lead at WSO2. Prior to joining WSO2, he worked for DirectFN (Pvt) Ltd,
where he was involved in the design and development of DirectFN Financial Data
dissemination back-end system. His main focus was on real-time data distribution with low
latency, high performance and bandwidth friendly solutions. He holds a BSc First Class degree
in Computer Science and Engineering from University of Moratuwa, Sri Lanka.
๏ Ravi Undupitiya - ravi@wso2.com
Ravi is a senior software engineer in the integration technology team where he mainly focuses
on the WSO2 Enterprise Service Bus. Prior to joining WSO2 he interned at the Intelligent
Agents and Multimedia Research Group at the University of Southampton where he worked
on a smart grid project which was aimed at helping people make smarter decisions when
consuming energy. Additionally he also dived into the world of Natural Language Processing
and carried out an evaluation of several text classification algorithms.

3. *
Agenda
๏ Background to Data Integration
๏ SOA based approach to Data Integration
๏ WSO2 Data Services Server
๏ WSO2 Enterprise Service Bus
๏Demo
๏Security
๏Transactions

4. *
Why we need Data Integration?
๏ Data is an important asset.
๏ Found in various sources and often siloed within applications.
๏We must leverage this data.
๏ Data consumers on various devices and platforms.
๏Security and privacy.
๏ Needs to be flexible for change.
๏ Legacy system support.

5. *
Data Integration Solutions
๏ Ad-hoc approach to data integration not good!
๏Unmaintainable and messy.
๏Not flexible enough for changes.
๏Need data source specific implementations
๏SOA approach better!
๏Data exposed as a services.
๏Applications consume the data through services.
.NET J2EE C++
Data Sources
.NET J2EE C++
Data Sources

6. *
Data Services on WSO2 Platform
๏ WSO2 Data Services Server (DSS)
๏ Allows rapid development of data services.
๏ Data services are decoupled from the infrastructure where it is stored.
๏ Wizard based UI to generate data service description language.
๏Data service description language
๏ Maps service requests to queries operating on data store.
๏ Maps query results to XML schema definitions.
๏ Supports validation, transformation, eventing etc…

7. DSS Key Features - Any Data Store

8. DSS Key Features - Data Federation

9. DSS Key Features - Server Side Batch Processing

10. DSS Key Features - Client Side Batch Processing

11. DSS Key Features - Array/UDT/Complex Types

12. DSS Key Features - Distributed Transactions

13. DSS Key Features - Notifications
๏Event sources, subscribers, topics
๏Allows a subscriber to be notified of a specific activity in a
data service
๏Event Triggers
๏Input Event Trigger
๏Output Event Trigger

14. DSS Key Features - REST Support

15. DSS Key Features - Caching Support

16. DSS Key Features - Security Support

17. DSS Key Features - Transformation

18. *
Demo 1
๏Exposing MySQL data table as a service.
๏SOAP
๏RESTful

19. *
Data Integration Solutions
๏ Ad-hoc approach to data integration not good!
๏Unmaintainable and messy.
๏Not flexible enough for changes.
๏Need data source specific implementations
๏SOA approach better!
๏Data exposed as a services.
๏Applications consume the data through services.
.NET J2EE C++
Data Sources
.NET J2EE C++
Data Sources

20. *
Layered Approach
๏SOA alone is not enough. Point-to-Point service integration.
๏Enterprise Service Bus for a cleaner layered approach.
Point-to-Point Service Integration (Spaghetti
Integration)
ESB in Action

21. What is an ESB?
๏An ESB is a middleware solution that enables
interoperability, among heterogeneous environments using a
service oriented model.
๏Stateless and Seamless Integration
๏Standard protocols/formats like SOAP, REST, JSON
๏Transports like HTTP/S, JMS, TCP, VFS (and many more)

22. The WSO2 ESB
๏Lightweight and high performance
๏Feature rich and standards compliant
๏SOAP and WS-* standards
๏REST support
๏Domain specific transport support (FIX, HL7)
๏User friendly and highly extensible
๏Configuration driven!
๏100% free and open source

23. WSO2 ESB
๏Apache Synapse
๏A lightweight and open source ESB implementation from
ASF
๏Makes up the mediation engine of WSO2 ESB
๏Multithreaded and asynchronous message processing
๏Based on a number of well known open source projects
๏Apache Axis2
๏Apache HTTP Components

24. WSO2 ESB Fundamentals
๏ Routing
๏ Filtering
๏ Transformation
๏ Protocol Switching
๏ Load Balancing
๏ Quality of Service

25. More Features
๏100% EIP support
๏Connectors
๏Message Stores/Processors
๏JSON Support
๏ JMX based monitoring and control
๏ Statistics collection
๏ Priority based message mediation
๏ XSLT, XPath, XQuery, Smooks, JSONPath
๏ Caching, Throttling
๏ Scripting languages (JavaScript, Groovy)
๏ JDBC connectivity
๏ Drools
๏ Clustering

26. Functional Components
Mediator
Sequence
Endpoints
Local Entry
Proxy Service
REST API
Message Stores/Processors
Templates Connectors
Tasks
Priority Executors
Registry

27. Functional Components
๏Each functional component serves a specific purpose
๏Functional components can be mixed and matched to
implement various integration scenarios and patterns
๏Configuring WSO2 ESB for a given scenario requires:
๏Identifying the right set of components
๏Putting them together in the most optimal manner

28. ESB and DSS
๏ Why not access data stores directly from ESB?
๏ESB has rudimentary DB support
๏DBLookup Mediator
๏DBReport Mediator
๏ESB’s function is to ensure low latency with high throughput
for stateless transactions. Data intensive processing done
on ESB could hinder ESB performance.
๏Separate JVM/servers to handle data intensive tasks.
๏Can deploy DSS features on top of ESB (thanks to WSO2
Carbon platform).

29. Layered Approach
๏ Why access DSS through ESB?
๏ Perfectly ok to access DSS data services directly.
๏ Handling DSS invocations through ESB provides a layer of
abstraction over data layer for integration.
๏Interception point
๏Service Chaining
๏Parallel processing
๏Load balancing
๏Simple transformations
๏Handling Multiple protocols

30. ๏ A merger causes an organization to consolidate their account
databases.
๏ Leverage WSO2 DSS to expose data from multiple data
sources as services.
๏Leverage WSO2 ESB to provide a secured uniform interface
and aggregation.
Demo 2 Use Case
MySQL Database
Excel Data Sheet

31. *
Demo 2
Aggregate data from multiple WSO2 DSS Data Services
using WSO2 ESB
Excel Data Service
RDBMS Data
Service
Excel File
MySQL DB
Data Services ProxyClient
Service
● Aggregate Mediator
● Clone Mediator
● Header Mediator
Request
Response
Multiple
Sources

32. Demo 3 Use Case
๏ A fund transfer between two accounts need to occur in a
transacted manner.
๏ Leverage WSO2 DSS for distributed transaction using
boxcarring feature.
๏ Leverage WSO2 ESB for service chaining.

33. *
Demo 3
Boxcarring sample with WSO2 DSS and ESB
begin_boxcar
execution 1
execution 2
………
execution 10
end_boxcar
begin_boxcar
execution 1
execution 2
abort_boxcar
execution 10
end_boxcar

34. *
Demo 3 cont
Bank Data
Service
Bank DB
(Book Store’s)
Payment Services
Proxy
Book Store
Application
● Customer’s Account Number
● Book Store’s Account Number
● Payment Amount
Bank DB
(Customer’s)
● Customer’s Account Number
● Payment Amount
● Book Store’s Account Number
● Payment Amount
begin_boxcar
end_boxcar
credit_bookstore
debit_customer
Service Chaining

35. Where to begin..
๏ WSO2 ESB Documentation
๏https://docs.wso2.org/display/ESB481/WSO2+Enterprise+Service+Bus+
Documentation
๏ WSO2 DSS Documentation
๏https://docs.wso2.com/display/DSS322/WSO2+Data+Services+Server+D
ocumentation

36. Further Reading
๏Master Data Management using WSO2 Platform
๏http://wso2.com/library/webinars/2013/10/mdm-with-wso2-
enterprise-middleware-platform/

37. Contact us !