SOA Pattern: Data Model Transformation
•Download as PPT, PDF•
5 likes•2,488 views
This document discusses data model transformation, which is a solution for services that need to interact even when using different data formats to represent the same information. It describes how embedding transformation logic directly in applications can limit flexibility and reusability. The document recommends using an external transformation service instead to convert data between formats while keeping provider and consumer concerns separate. It provides examples of how WSO2 products implement external transformation using mediators or the data services server with XSLT.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to SOA Pattern: Data Model Transformation
Similar to SOA Pattern: Data Model Transformation (20)
More from WSO2
More from WSO2 (20)
Recently uploaded
Recently uploaded (20)
SOA Pattern: Data Model Transformation
- 1. Last Updated: Jan. 2014 Associate Technical Lead – Technical Sales Dakshitha Ratnayake SOA Pattern: Data Model Transformation
- 2. 2 About the Presenter ๏ Dakshitha is an Associate Technical Lead in the Solutions Architecture/Technical Sales team. ๏ She has hands on experience in Java/J2EE technologies and has worked on designing and implementing software solutions in the fields of health-care information systems and content management systems for telecommunications providers prior to her employment at WSO2. Prior to joining the Solutions Architecture/Technical Sales team she worked with the WSO2 Developer Studio team.
- 3. 3 About WSO2 ๏ Global enterprise, founded in 2005 by acknowledged leaders in XML, web services technologies, standards and open source ๏ Provides only open source platform-as-a-service for private, public and hybrid cloud deployments ๏ All WSO2 products are 100% open source and released under the Apache License Version 2.0. ๏ Is an Active Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Foundation and W3C. ๏ Driven by Innovation ๏ Launched first open source API Management solution in 2012 ๏ Launched App Factory in 2Q 2013 ๏ Launched Enterprise Store and first open source Mobile solution in 4Q 2013
- 6. The Integration Problem ๏ As organizations move forward with their SOA implementation, they find themselves grappling with significant problems of data integration among a wide variety of data sources. ๏ The mismatch between the data representation that one application provides and the data representation that another application expects forms the core of this long-lived problem. ๏ This data integration problem is particularly troublesome for SOA implementations because it limits the loose coupling between service providers and consumers.
- 7. Integration Needs ๏ Standard Integration ๏ Entity Aggregation / Canonicalization ๏ Standards Compliance
- 8. A Narrowed Down Problem ๏ How can services interact with programs that communicate with different data formats? ๏ Services may use incompatible schemas to represent the same data, hindering service interaction and composition. ๏ Even when organizations use metadata standards such as XML to format messages, the schema of one Service might not match the schema of another, and fields with the same meaning might use differing tags.
- 9. The Straight-Forward Approach ๏ When an application or service must transform data from one format to another in order to achieve some desired data integration goal, the most common and straightforward approach is to embed data transformation logic directly within the application.
- 10. Disadvantages of Using Embedded Transformation ๏ Other applications or Services have no practical way of reusing the embedded transformations. ๏ It's impossible to offload the CPU overhead that the embedded data transformations cause. ๏ The transformation logic is tightly coupled to the business logic within the application, reducing the flexibility and reusability of the applications. ๏ The transformation logic is handled programmatically vs. declaratively through metadata, further impeding the flexibility of the application.
- 11. A Better Approach ๏ An alternative to embedding transformation logic within an application is to use an external transformation service to handle the task. ๏ A transformation service, loosely-coupled in a SOA implementation, accepts incoming data in one format, and outputs data with the same semantic meaning in another format. ๏ The use of data transformation services improves loose coupling by separating the concerns of service providers and consumers.
- 12. Solution for ‘same format different schema’ problem ๏ A data transformation technology can be incorporated to convert data between disparate schema structures. ๏ Mapping logic needs to be developed and deployed so that data compliant to one data model can be dynamically converted to comply to a different data model.
- 13. Solution Pattern: Data Model Transformation ๏ The requirements statement for Data Model Transformation is: How can services interoperate when using different data models for the same type of data?
- 14. Reference Architecture: Standard Integration with XSLT transformation Service 2Service 2 Service 1Service 1 Data Model Transformation Data Model Transformation Schema1.xsdSchema1.xsd Schema2.xsdSchema2.xsd Schema1.xsdSchema1.xsd Schema2.xsdSchema2.xsd Transform.xslTransform.xsl
- 15. WSO2 Solution Architecture: Standard Integration Service 2Service 2 Service 1Service 1 Schema1.xsdSchema1.xsd Schema2.xsdSchema2.xsd Schema1.xsdSchema1.xsd Schema2.xsdSchema2.xsd Transform.xslTransform.xsl
- 16. WSO2 Offerings The WSO2 ESB supports transformations using the following transformation mediators: WSO2 Data Services Server supports transformation using XSLT Transformation, where the user can define the transformation in XSLT.
- 17. Entity Aggregation ๏ There are essentially three reasons for the aggregation of entities in different systems: ๏ Single View of Entity—Having a single view of all common entities spread across enterprise systems. ๏ Horizontal Partitions—There are situations where entity information is distributed across many different services, based on factors such as geographical boundaries. ๏ Cross-Join scenarios—These situations require cross-join among entities present in different services.
- 18. Reference Architecture: Entity Aggregation ERP ServiceERP Service Data Model Transformation (Entity Aggregation) Data Model Transformation (Entity Aggregation) Customer ERPCustID LName Fname OrgID Customer ERPCustID LName Fname OrgID Transform.xslTransform.xsl CustomerAPICustomerAPI CRM ServiceCRM Service Customer CRMCustID LName Fname Address Tel OrgID Customer CRMCustID LName Fname Address Tel OrgID Customer ERPCustID LName Fname OrgID Address Tel Customer ERPCustID LName Fname OrgID Address Tel
- 19. WSO2 Solution Architecture: Entity Aggregation ERP ServiceERP Service Customer ERPCustID LName Fname OrgID Customer ERPCustID LName Fname OrgID Transform.xslTransform.xsl CustomerAPICustomerAPI CRM ServiceCRM Service Customer CRMCustID LName Fname Address Tel OrgID Customer CRMCustID LName Fname Address Tel OrgID Customer ERPCustID LName Fname OrgID Address Tel Customer ERPCustID LName Fname OrgID Address Tel
- 20. Standards Compliance ๏ Adherence to standards insures, to some degree, the loose coupling and composition of services and data exchange mechanisms with a guarantee that the data exchanged with an individual service will be universally comprehensible. ๏ E.g. – HL7, other domain-specific formats
- 21. Demo ERP ServiceERP Service Customer ERPCustID LName Fname OrgID Customer ERPCustID LName Fname OrgID CustomerAPICustomerAPI CRM ServiceCRM Service Customer CRMCustID LName Fname Address Tel OrgID Customer CRMCustID LName Fname Address Tel OrgID Customer ERPCustID LName Fname OrgID Address Tel Customer ERPCustID LName Fname OrgID Address Tel
- 22. Resources ๏ http://searchsoa.techtarget.com/tip/The-role-of-transformation-serv ๏ http://soa.dzone.com/articles/soa-pattern-week-6-canonical ๏ http://www.asprom.com/note/wp20.pdf ๏ http://www.ibm.com/developerworks/data/library/techartic le/dm-0803sauter/ ๏ http://msdn.microsoft.com/en-us/library/ms954596.aspx
- 23. Contact us !
Editor's Notes
- As businesses continued to expand over the years, their demand for more IT systems increased. As mergers and acquisitions became the standard, there was an even greater demand for new systems. As a result, today's enterprises are composed of a plethora of IT systems that are independent of each other. The result is hundreds, if not thousands, of application islands.
- When services and service consumer programs interact, data is transmitted (usually in the form of messages) organized according to some structure and a set of rules. This structure and the associated rules constitute a formal representation (or model) of the data. When different services are designed with different data models representing the same type of data, then they will have a problem sharing this data because the data models are simply incompatible. To address this problem, a technique called data model transformation is applied whereby data model mapping logic is developed so that data exchanged by such services is dynamically converted at runtime from compliance with one data model to another. So successful has this technique been that a corresponding Data Model Transformation pattern was developed.
- An XSLT style sheet containing data model mapping logic (2) is added as a form of intermediary processing that is executed at runtime. With each transmission, the data model of the service 1 is converted from schema1 to the data model compliant with the schema2 used by service 2. This runtime transformation logic can reside with either service architecture or as part of a separate middleware platform.
- through XSLT transformation on the XML payload and message stream, XQuery transformations, message enrichment and message content replacement
- There are essentially three reasons for the aggregation of entities in different systems: Single View of Entity—Having a single view of all common entities spread across enterprise systems. E.g a unified view of a Customer entity, named Customer Information Repository (CIR), is required to build useful solutions in many enterprises, especially in the banking sector. Horizontal Partitions—There are situations where entity information is distributed across many different services, based on factors such as geographical boundaries. Cross-Join scenarios—These situations require cross-join among entities present in different services. Goal of an aggregation service- Acts as a unified source of entities. Provides a holistic view of an entity. Provides a holistic view of the entity model—entities and their relationships with other entities Provides location transparency—consumers of the Entity Aggregation layer do not need to know who owns the information. Enforces business rules that determine the segments of entities retrieved in a given context. Determines the system of record for each data element constituting an entity. Enriches the combined data model across systems—the whole being better than the sum of its parts.
- Business concepts such as Customer and Employee were duplicatedFor example, in many businesses employees must make separate phone calls to HR, Payroll, and Benefits to have their addresses updated in the different systems. An Entity Aggregation solution can be designed to not only act as a single point in accessing information that exists in multiple systems, but it also provides a holistic view of an entity and the entity model.
- Not only do disparate services have different views of an entity, they may also have their own schematic representation of their views. In order to achieve a single unified view of an entity, it is imperative for the EA service to harmonize the schematic differences between different services. To do this, the EA service should first know how each service represents an entity. Additionally, the EA service should define a holistic view of the entity by logically consolidating the various views. Equally important is for the EA service to discover a way to represent the transformation between a service's view of an entity and the holistic view. The following figure shows the customer information represented in more than one service.