Your SlideShare is downloading. ×
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Data Quality meets SOA
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data Quality meets SOA

594

Published on

Data quality functions were already being provided as services for Unix, Windows and Linux, before the analysts of Gartner had invented the term SOA. For the most part, technical reasons were the …

Data quality functions were already being provided as services for Unix, Windows and Linux, before the analysts of Gartner had invented the term SOA. For the most part, technical reasons were the decisive factor for this architecture. In addition to this, the implementation of service-oriented
architectures results in new and changed
requirements for data quality services
and also increases the opportunities and benefits which they can create.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
594
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. WHITE PAPER: SOA WHITE PAPER / Data Quality meets SOA – Making Data Quality available for all Business Processes Data quality functions were already being provided as services for Unix, Windows and Linux, before the analysts of Gartner had invented the term SOA. For the most part, technical reasons were the decisive factor for this architecture. In addition to this, the implementation of service-orien- ted architectures results in new and chan- ged requirements for data quality servi- ces and also increases the opportunities and benefits which they can create. __________________________ All company and product names and logos used in this document are trade names and/or registered trademarks of the respective companies. Page 1
  • 2. WHITE PAPER: SOA The starting point for data quality services In order to consider the importance of service-oriented architectures for the pro- vision and use of data quality functions, it is first of all useful to look at the typi- cal data quality functions themselves. APPLICATION A: A classic application is the validation of a postal address on the basis of reference data, which includes street and place names and the dependencies of the postcodes on places, streets and house numbers. In contrast to simple database access, the correct address should also be found here if the input is incomplete or contains recording or hearing errors. The goal is a high matching accuracy, in order to be able to correct the greatest possible number of incorrect addresses automatically. APPLICATION B: Another application is searching for duplicates in the in-house database. Here too the goal is to quickly and reliably identify a business object, a business partner, a product or a sales oppor- tunity in spite of incomplete, divergent or incorrect input, in order to (a) simplify the search and thereby increase the productivity of the users, and (b) to prevent the creation of duplicates, i.e. multiple entries which refer to the same object in the real world. Consistent, complete and unambiguous mapping of the real objects in the database will be achieved as a result. An important goal in the context of improving Apart from the response time, the integration in a the data quality consists in preventing incomplete wide range of environments already played an or incorrect data from being stored in the data- important role in data quality services. Decoupling base. Possible problems should be detected at data the data quality services from the service consum- entry and then cleaned up either automatically or ers and utilization via a client/server protocol is manually by the user after appropriate feedback. important for reaching this goal. This function had Specialized search indices ensure that a search therefore already been provided and used for data in databases with 1,000,000 to 100,000,000 quality, at least for more sophisticated applications, data records normally only requires a fraction of a before the „invention“ of service-oriented architec- second, even with divergent spelling. Nevertheless, tures as a service. As a result, both the high require- these response times require intelligent caching of ments for the response times could be met and the the indices. This can be provided very efficiently provision of functions guaranteed for a wide range by implementing the software as a central service of environments. which is made available from an in-house server. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 2
  • 3. WHITE PAPER: SOA From the „fat client“ to the 3-layer architecture Layered Architecture LAYERED ARCHITECTURE The typical architecture in the early days of the cli- FAT CLIENT ent/server world consisted of a database which, in addition to the storage of transaction and mas- ROLE ter data, enabled asynchronous communication Name Customer between different system components, thereby Street Supplier allowing these components to be decoupled. Postcode Reseller Messages were written in the database by the sender and read there by the receiver. For this pur- pose, however, regular «polling» of the respective table was required, in order to establish whether new unprocessed messages were available. The business logic was mainly implemented in so- called «fat» clients. Tasks which were executed via batch processes were implemented via back- ground processes which accessed the database. Interactive functions for the validation of addresses or for the detection of duplicates directly at data entry were normally integrated in the graphical user interface. They were usually called via proprietary interfaces. Specifications such as DCE1 or CORBA2, whose goal was the standardization of interfaces for the communication of distributed components, had nothing more than a niche existence. 1 http://en.wikipedia.org/wiki/Distributed_Computing_Environment 2 http://en.wikipedia.org/wiki/CORBA © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 3
  • 4. LAYERED ARCHITECTURE WHITE PAPER: SOA FAT CLIENT CLIENT This situation has fundamentally changed in the past few years. The starting point for this was mainly the ROLE ROLE Name Customer Name Customer establishment of standards within the framework of JEE3 (Java Enterprise Edition) which resulted in the Street Supplier Street Supplier provision of high-performance implementations of Postcode Reseller Postcode Reseller these standards both as commercial products and open source solutions. APPLICATION SERVER SIMPLE INTEGRATION IN THE APPLICATION SERVER – EITHER WITH A JEE OR A .NET ARCHITECTURE – NOW CAME TO THE FORE. For the Windows world, Microsoft followed with the development of .NET4 as a language-independent platform and the .NET Enterprise Services. As a result, a high-performance infrastructure software was available irrespective of the selected platform (JEE, .NET), in order to largely detach the business logic from the presentation layer and implement it in its own layer on the server. This also changed the requirements for data quality services, which were now executed mainly from the business logic on the server side. 3 http://en.wikipedia.org/wiki/Java_Platform,_Enterprise_Edition 4 http://en.wikipedia.org/wiki/.NET_Framework © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 4
  • 5. WHITE PAPER: SOA SOAP as an Enabler for SOA SOA does not become really effective until a stand- Proprietary Protocol ard protocol for the provision and use of services has been established. This gap was closed by the Web Service protocol SOAP5. It is supported by practically all middleware and infrastructure com- ponents, thereby enabling interoperability between service providers, middleware and service con- sumers. As a result, the provision of connectors for the use of proprietary protocols in proprietary middleware is no longer necessary. This therefore lays the basis for the establishment of powerful mid- dleware components. It includes the concept of the Enterprise Service Bus6 (ESB), which enables the loose coupling of different components which play a significant role in the routing of messages, as well as engines which can directly execute defined workflows (BPEL)7 in a business process language. SOAP-based Web Services have therefore devel- oped into a central instrument for the provision of interactive data quality services in modern enter- prise architectures. These mainly concern services which run according to the request/response pat- tern, in which the service consumer initiates a request, e.g. «validate the specified address», and the service makes a direct response with a confir- mation, a correction suggestion or a selection of possible correct addresses. 5 http://en.wikipedia.org/wiki/SOAP 6 http://en.wikipedia.org/wiki/Enterprise_service_bus 7 http://en.wikipedia.org/wiki/BPEL © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 5
  • 6. WHITE PAPER: SOA This procedure corresponds to the interactive char- SOAP Protocol acter of the validation, which should support the user directly at entry of a business object and offer options for intervention in the event of problems. However, this function is no longer implemented in isolation in the presentation layer but normally takes place in the context of a higher-ranking busi- THE CORRELATION BETWEEN THE IMPLEMENTATION OF BUSINESS PROCESSES AND DATA QUALITY FUNCTIONS IMMEDIATELY BECOMES OBVIOUS AND THEREFORE ALSO THE CONTRIBUTION WHICH THEY MAKE TO THE SUCCESS OF THE RESPECTIVE BUSINESS PROCESS. ness process, e.g. the implementation of the order- ing process in an e-business application, the imple- mentation of a process for lead conversion and qualification in a CRM application or a compa- rable process. The correlation between the imple- mentation of business processes and data quality functions immediately becomes obvious and there- fore also the contribution which they make to the success of the respective business process. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 6
  • 7. WHITE PAPER: SOA Customer Cases Orange The customers of the telecommunications company This service-oriented approach makes it possible Orange in France can contact the company via to use the same services in different processes and various channels. They can visit different channels. Irrespective of whether it con- the web portal of Orange in cerns the creation of a new Orange customer or the Internet, call the call center the change of address of an existing customer and of Orange and visit the mobile irrespective of which contact channel a process is phone business of a partner of initiated, the underlying service-oriented architecture Orange. However, irrespective always ensures that the executed processes are of which contact channel the customer chooses, it configured consistently and can access the same must be ensured that the respective processes are services. It is thereby possible to guarantee a con- executed with the same quality. An important com- sistently high quality standard of the address data ponent of this process quality consists in ensuring across the company. that customer addresses are correct. The technical basis is the open source Java appli- cation server JONAS. This JEE server is the central point for the provision of services which are required to implement the business processes of Orange. The Uniserv data quality service for the valida- tion, restructuring and standardization of customer addresses is also provided in this environment. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 7
  • 8. WHITE PAPER: SOA Customer Cases WinGroup AG The German WinGroup AG, a service network for sales and marketing, offers its customers an extensive range of services in the areas of call center, letter- shop, dialog marketing and IT services. In order to guarantee a consistently high quality level of the underly- ing processes in all service areas and customer- specific applications, the subsidiary company, WinLogic, has developed a service-oriented archi- tecture based on an Enterprise Service Bus. This represents the central middleware for docking all applications in the company in the central services. The address validation and the duplicate check of Uniserv are linked here to secure the data quality. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 8
  • 9. WHITE PAPER: SOA Lightweight REST – when SOAP is too unwieldy Even if the SOAP-based protocol for integration The call is via the http protocol, and the call argu- in typical enterprise middleware seems to be the ments in the URL are encoded as a result. In the ideal solution, there are various applications where case of a call from JavaScript, the result is output alternatives such as RESTful8 Services are advan- in the JSONformat9 in the ideal case. This results tageous. This is particularly the case when data in a JavaScript code which can be directly inter- quality services are to be directly activated in a presentation layer which is HTML/AJAX-based. SERVICES CONFIGURED IN THIS Input aids which automatically complete a partial MANNER CAN BE IDEALLY USED IN input by the user or offer suggestions for comple- TYPICAL WEB 2.0 APPLICATIONS. tion of the input based on the partial input are a typical scenario for this case. Input aids are located in the presentation layer by their nature. preted by the JavaScript interpreter of the browser, However, they make considerably higher demands in order to provide the result of the call directly for the response time, since they are called more as JavaScript objects. Services configured in this frequently during the input, usually after each input manner can be ideally used in typical Web 2.0 character. applications. Although the same services used for address valida- tion present themselves for the address input in this implementation, SOAP-based Web Services are not always suitable for this application scenario on account of the overhead which the SOAP protocol entails. RESTful Web Services are extremely lean in comparison to SOAP-based Web Services. 8 http://en.wikipedia.org/wiki/REST 9 http://en.wikipedia.org/wiki/JSON © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 9
  • 10. WHITE PAPER: SOA SOAP - the fine differences A learning curve also has to be overcome for However, this criterion is essential in a data qual- adapting proprietary interfaces to a SOAP-based ity service which must useable in a great variety communication. The packets exchanged in the of environments which may not even be known framework of a SOAP-based communication are beforehand. described in a metaformat, the so-called WSDL In addition to this, the SOAP specification offers (Web Service Description Language).10 two basic options for defining in the WSDL the linkage between the packet structure in XML and DURING THE DEVELOPMENT OF the constructs of the programming language which THE WSDL, IT MUST FIRST OF ALL BE provides or interprets the packet. As the acronym ENSURED WITHOUT FAIL THAT THE DATA suggests, the so-called «rpc-style» corresponds more TYPES USED ARE ACTUALLY SUPPORTED to the conventional Remote Procedure Call (RPC) BY ALL THE TARGET LANGUAGES AND and models operations as method calls which do SYSTEMS. not differ from local calls. They are ideal when the Web Service is to be called from an object- oriented programming language such as Java or During the development of the WSDL, it must first C#. The so-called «document-style» is more suitable of all be ensured without fail that the data types for modelling complex contents as a result which used are actually supported by all the target lan- is represented as an XML document with its own guages and systems, in which the service is to be XML scheme. Validation against the XML scheme consumed. This aspect is relatively non-critical in a using standard XML means is therefore possible on pure in-house development with a homogeneous the one hand, and the result document can be eas- software infrastructure, e.g. JEE or .NET. ily further processed, transformed or processed for the presentation layer using standard XML means or suitable frameworks on the other. Both variants have their advantages and disadvantages. 10 http://en.wikipedia.org/wiki/Web_Services_Description_Language © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 10
  • 11. WHITE PAPER: SOA Both variants may also have to be made avail- Web Services must be scalable. A prerequisite able to services which need to be called from any for this is the described statelessness. In addition context. to this, the Web Service should access global resources as little as possible or not at all. If this is necessary, however, the administration and WEB SERVICES ARE STATELESS BY THEIR synchronization of the access to these resources NATURE. IN THE IDEAL CASE, THIS should never be implemented ad-hoc for the MEANS NO PARTIAL RESULTS; INSTEAD respective WEB service. Resource pools such as THE OVERALL RESULT IS OUTPUT AS A are offered by most application servers or exist as RESULT OF THE CALL, AND TWO SUCCES- open source extensions should be used instead. SIVE CALLS ARE TOTALLY INDEPENDENT Effective and configurable pooling is thereby OF EACH OTHER. enabled. The last two points in particular normally require at Web Services are stateless by their nature. In the least a partial redesign if the existing functionality ideal case, this means no partial results; instead is to be made available as a Web Service. the overall result is output as a result of the call, and two successive calls are totally independent of each other. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 11
  • 12. WHITE PAPER: SOA SOA blurs the distinction between «on premise» and «on demand» The provision of software services and applica- The reference data required for the service must tions via the Internet, referred to by the buzzwords be regularly updated. This means both regularly «Software on Demand», «Software as a Service»11 recurring manual work as well as regular subscrip- or «Cloud Computing», is a theme which is grow- tion charges which have to be paid to the data ing in importance. In many cases, a fundamen- provider. In the case of country-specific postcode tal and profound contradiction between locally directories, this work and expense are incurred for installed software («on premise») and software each country for which addresses are checked. used via the Internet («on demand») is depicted. This is not the case from the SOA perspective, AS A RESULT, AN OPTIMUM SOLUTION, because within the framework of a service-oriented WHICH CAN ALSO BE FLEXIBLY ADAPTED architecture it is not critical how the respective IN RETROSPECT TO CHANGED BASIC service is provided. The provision of a service via CONDITIONS, CAN BE FOUND FOR THE the Internet as an alternative to a locally installed RESPECTIVE USER. server presents interesting new application possi- bilities, particularly in the area of data quality serv- This only makes the use of such solutions interesting ices which carry out validations and corrections by for larger quantities of data. This restriction is not matching and merging against reference data. applicable if the service is provided as an SaaS offer, in which invoicing is based exclusively on the executed transactions. Locally installed services and services used via the Internet can be also com- bined as required or exchanged by using them in a service-oriented architecture. As a result, an opti- mum solution, which can also be flexibly adapted in retrospect to changed basic conditions, can be found for the respective user. 11 http://en.wikipedia.org/wiki/Software_as_a_Service © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 12
  • 13. WHITE PAPER: SOA Conclusions The following conclusions about two aspects can be drawn from the above described experience: ASPECT ONE: What are the requirements for data quality functions which are to be used in an SOA environment? – The functions must be accessible via SOAP as – It should be checked whether an alternative use Services. Integration in practically any infra- scenario, in which the service is provided via structure for the service-oriented implementation the Internet, provides commercial or technical of the business processes is thereby guaran- advantages, and whether the service provider teed. However, care should be taken in the supports such a scenario. detail that mapping between the XML elements of the service description and the respective constructs of the respective server environment can be represented. – If use in the presentation layer is foreseeable, it must be checked whether a RESTful Service implementation is necessary. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 13
  • 14. WHITE PAPER: SOA ASPECT TWO: Which aspects have to be considered if functions for the validation, enhancement and processing of data are to be made SOA-capable? – The use scenarios must be clearly defined. The – The scalability of the service must be provided: question as to whether the application of the if the Web Service requires global resources, services takes place within the framework of e.g. a database connection, these should be the business logic or the presentation logic is administered by means of a suitable resource particularly important. In the first case, integra- pool in the server container. Otherwise, these tion takes place in the application server, the resources quickly become a bottleneck which Enterprise Services Bus or a BPEL engine, in prevents genuine scalability of the service. the second case in a graphical user interface, which can in turn make its own demands. The – The service should be internet-capable, i.e. it decision as to whether SOAP-based and/or should be irrelevant for the functionality of the RESTful services are more appropriate is made service whether it is provided in the local net- depending on this. work or via the Internet. The possible applica- tions are extended enormously as a result. – The target systems and languages in which the service is to be used must be specified. The decision on the degree of complexity of the modelling with respect to the XML structures For further information used and XML data types of the call results is please visit our web page made depending on this. www.uniserv.com or contact us directly: – The service must be designed, so that it is state- less, i.e. it must function without the storage of an internal state between two calls. If a state between the calls is required, it must be trans- We are looking forward for advising and sup- ferred for the follow-up call or made persistent porting you through your project. in a suitable manner. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 14
  • 15. WHITE PAPER: SOA Uniserv Uniserv is the largest specialised supplier of data quality solutions in Europe with an internationally usable software portfolio and services for the quality as- surance of data in business intelligence, CRM applications, data warehousing, eBusiness and direct and database marketing. With several thousand installations worldwide, Uniserv supports hundreds of customers in their endeavours to map the Single View of Customer in their customer data- DATA base. Uniserv employs more than 110 people at its head- MIGRATION PROJECTS quarters in Pforzheim and its subsidiary in Paris, France, E-COMMERCE and serves a large number of prestigious customers in all sectors of industry and commerce, such as ADAC, Al- ERP lianz, BMW, Commerzbank, DBV Winterthur, Deutsche Bank, Deutsche Börse Group, France Telecom, Green- COMPLIANCE peace, GEZ, Heineken, Johnson & Johnson, Nestlé, CRM Payback, PSA Peugeot Citroën as well as Time Life and Union Investment. Further information is available SOA at www.uniserv.com DIRECT MARKETING MDM/CDI ON-PREMISE/ ON-DEMAND BI/BDW CPM Experience: Market position: Employees: OVER 40 YEARS LARGEST MORE THAN 110 PEOPLE EUROPEAN SUPPLIER UNISERV GmbH Contact: Rastatter Straße 13 • 75179 Pforzheim • Germany • T +49 7231 936-0 • +49 7231 936-0 F +49 7231 936-3002 • E info@uniserv.com • www.uniserv.com © Copyright Uniserv • Pforzheim/Germany • All rights reserved. © UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 15

×