Data Management in a Service Oriented Architecture


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Traditional architectures focused first on centralized computing. A mainframe or similar single-box solution served as the only computing platform available. No integration was necessary because all data and programs were located in a single place. As client server and distributed computing solutions became more and more prevalent, the common solution in most organizations was to cost-justify new hardware and software based upon a vertical, business-function oriented view of the world. In our hypothetical example above, the first non-mainframe solution was created specifically to assist in order-processing. As Purchasing, Inventory, Finance, Sales and other departmental satellite systems were created, custom interfaces were built to allow each of these systems to exchange information with others. The mainframe still had some of the critical information necessary for each of these satellites to function, but other times was just used as a transfer mechanism to overcome technical data transfer issues. As this type of system became more and more common, and more and more custom interfaces were being written to support cross-functional system communication, the burden on IT became larger. More time/money was being spent on maintaining and babysitting interfaces, and often the same data was being replicated from system to system (such as customer information between orders, sales and marketing). In addition, data quality problems begin to become more obvious. Sales doesn’t talk to Orders, but it does talk to Inventory, for instance. Synchronization between the common replicated data becomes more complicated. The nightly choreography of information transfer from system to system must become more sophisticated and error-prone. And it does not support near-real-time transfer of information to web systems, or a consolidated enterprise view of activity that could be viewed in a data warehouse. This situation is probably not that strange to any of you. The single-project-focused introduction of new technology and software, while focusing on short term delivery of individual systems, leads to data disorganization.
  • So how does SOA help? First of all, the paradigm of services allows for commonly shared components (such as customer data) to be defined once and shared across the enterprise. And this is not just the definition of who a customer is. This is also the actual persistent customer data . It is stored in the most appropriate location and accessed by whoever needs it. Secondly, the services are not specific to the systems that need the data. In other words, if the Inventory system needs warehouse information, it accesses it using the same service that Purchasing uses. Now this is not to say that Purchasing and Inventory need exactly the same data. But it does mean that the most appropriate owner of warehouse information needs to be able to supply that information to any consumer within the organization that needs it. If Finance suddenly develops a need to track warehouse costs, a certain amount of enterprise-relevant information about warehouses is already available using the enterprise service. Think about what this does to your information management issues from the previous slide. Instead of needing to replicate the same data over and over again, and then having to worry about the currency of the data and update strategies, this solution allows enterprise data to be stored by the most appropriate organization, and made available to any other organization across the enterprise on a real-time basis. The beauty of the services model is that you can separate interface from implementation . The several interfaces between technical systems have been replaced with a single interface that is called on an as-needed basis. This allows IT to focus on technology management – replacing technology as needed behind the service interface if necessary. Addressing scalability, and being able to respond in a coordinated way to business drivers.
  • Data Management in a Service Oriented Architecture

    1. 1. Data Management in a Service Oriented Architecture presented by Evan Terry Ken Karacsony The Clegg Company
    2. 2. Agenda <ul><li>What is SOA? </li></ul><ul><li>Why SOA? </li></ul><ul><li>Organizing for SOA Success </li></ul><ul><ul><li>Organizational Structure </li></ul></ul><ul><ul><li>Strategic Planning </li></ul></ul><ul><li>Data Management in SOA </li></ul><ul><ul><li>Data Modeling </li></ul></ul><ul><ul><li>Data Quality </li></ul></ul><ul><ul><li>Data Stewardship </li></ul></ul><ul><ul><li>Security & Privacy </li></ul></ul><ul><ul><li>Metadata Management </li></ul></ul><ul><li>Questions / Answers </li></ul>Ken Karacsony
    3. 3. What is SOA? Ken Karacsony
    4. 4. SOA is NOT … <ul><li>A new concept or technology </li></ul><ul><li>A silver bullet </li></ul><ul><li>An IT project </li></ul><ul><li>Something that produces an immediate ROI </li></ul><ul><li>Implementing technology (i.e. web services) </li></ul><ul><li>Something that requires a complete re-write and/or replacement of existing systems </li></ul><ul><li>Something that can be implemented by IT in isolation </li></ul><ul><li>Easy to implement </li></ul>Evan Terry
    5. 5. SOA is… <ul><li>Making IT resources and information enterprise , cross-organizational assets </li></ul><ul><li>Based on reusable services that use common logic and data </li></ul><ul><li>A collaborative effort between the business and IT </li></ul><ul><li>Aligning IT (technology) with the business (processes) </li></ul><ul><li>Promoting building blocks (services) that can be used in new and different ways </li></ul><ul><li>Technology-neutral </li></ul><ul><li>Requiring new behavior and organizational structure for the enterprise </li></ul>Evan Terry
    6. 6. SOA Defined A flexible framework for aligning business and IT resources through the adoption of standards, governance, organizational structure and well defined, common business processes , information policies and practices, and technology-independent, loosely coupled services that enables the business to adapt quickly to changing business drivers. Evan Terry
    7. 7. Why SOA? Ken Karacsony
    8. 8. The Business Model is Changing The old… Redundant processes Redundant application logic Redundant data Ken Karacsony
    9. 9. The Business Model is Changing ..and the new Consolidated processes Consolidated data Shared services Ken Karacsony Marketing
    10. 10. A TDWI study on data integration found that 69 percent of companies considered data integration issues to be a high or very high barrier to new application development Ken Karacsony
    11. 11. Tight-Coupling (As Is) Orders Finance Marketing Warranty Inventory Sales Purchasing
    12. 12. Loose-Coupling (To Be) Purchasing Orders Sales Warranty Marketing Finance Inventory Ken Karacsony Purchasing Orders Finance Marketing Warranty Finance Inventory Services Layer
    13. 13. Let’s talk about the SOA enabled organization Evan Terry
    14. 14. Characteristics of an SOA Enabled Organization <ul><li>Identifies and prioritizes information and business process architecture </li></ul><ul><li>Recognizes the need for “bridge” staff, who understand the business, data, applications, and technology </li></ul><ul><li>Determines and standardizes the definition, content and quality of enterprise information and processes </li></ul><ul><li>Create policies and standards for the use of enterprise information </li></ul><ul><li>Centralizes and controls decisions made about enterprise level components </li></ul><ul><ul><li>Similar to the way that technical architecture groups within IT centralize and control decisions about technology </li></ul></ul>Evan Terry
    15. 15. A Traditional IT Organization Chart Market Research R & D Technical Services Application Dev Data Architecture Accounting DBAs Database Designers ETL Teams Metadata Application Development Teams Web Designers Business Analysts Network PC Services H/W Admin Config Mgmt Engineers Designers Testers Planners Statisticians Demographers Analysts Economists Accountants Capital Managers Economists CEO CIO VP Prod Dev CFO R&D Support Business Knowledge MR Support Business Knowledge Evan Terry
    16. 16. SOA Organization Chart – EIO Placement Market Research R & D Enterprise Architecture (IT) Engineers Designers Testers Planners Statisticians Demographers Analysts Economists EIO Business Process Architecture Information Architecture Information Architects Data Quality Analysts Stewards Business Process Specialists Business Analysts CEO CTO VP Prod Dev R&D Support MR Support Enterprise Business Knowledge Departmental Business Knowledge Cross-functional enterprise focus Departmental focus Sets Information Strategy Sets Technical Strategy CIO Cultural Architecture Change Mgmt Training Communication Application Dev Application Development Teams Web Designers Business Analysts Evan Terry
    17. 17. SOA Organization Chart – EA (IT) & EIO Technical Architecture Application Architecture CEO Integration Architecture EIO Business Process Architecture Information Architecture Cultural Architecture Enterprise Architecture (IT) CTO Process Design Business Analysis Inventories of Business Processes Logical Data Modeling Inventories of Enterprise Data (Metadata) Data Sourcing Data Security Data Stewardship Change Mgmt Training Communication Hardware Special Tools COTS Technology Approved S/W Network Infrastructure S/W Program / System Design Languages / S/W Platforms Inventories of Objects and Patterns Integration Design Inventories of Integrations Construction Patterns Enterprise Business Architecture Evan Terry
    18. 18. Data Management in SOA Ken Karacsony
    19. 19. Data Management in SOA <ul><li>Data Modeling – Common Definitions </li></ul><ul><li>Data Quality – Common Interpretation </li></ul><ul><li>Data Stewardship – Common Management </li></ul><ul><li>Security & Privacy – Common Access Control </li></ul><ul><li>Metadata Management – Common Understanding </li></ul>Ken Karacsony
    20. 20. Creating Enterprise Data Models Finance Orders Purchasing Inventory Sales Topical slice Vertical slice Evan Terry
    21. 21. Ensuring Data Quality Define Discover Prevent Remediate Ken Karacsony “Data Quality Cycle 2.0” <ul><li>Define: </li></ul><ul><li>Establish metrics </li></ul><ul><li>Develop plan </li></ul><ul><li>Establish team </li></ul><ul><li>Discover: </li></ul><ul><li>Initiate project </li></ul><ul><li>Apply metrics </li></ul><ul><li>Report issues </li></ul><ul><li>Remediate: </li></ul><ul><li>Root cause </li></ul><ul><li>Clean-up data </li></ul><ul><li>Define prevention </li></ul><ul><li>Prevent: </li></ul><ul><li>Assign ownership </li></ul><ul><li>Remove barriers </li></ul><ul><li>Reward quality </li></ul>
    22. 22. Creating Effective Data Stewardship <ul><li>“ The role of data steward is one of the most widely adopted in business today. It’s also one of the most misunderstood. Too many companies appoint a data steward, give her an automation tool, and think the job is done.” 1 </li></ul>1. Baseline Consulting Ken Karacsony
    23. 23. What does Data Stewardship look like? Roles & Responsibilities Level of Authority <ul><li>Enterprise data models - Enterprise Glossary - Subject area accountability - Data reuse </li></ul><ul><li>Subject area data model - Data quality - Data profiling - Data requirements - Metadata management </li></ul>CIO Subject Area Steward Data Owners <ul><li>Sponsorship - Executive accountability - Data strategy - Governance </li></ul>Ken Karacsony
    24. 24. Assigning Data Security Classifications <ul><li>At a minimum, classifications should consider: </li></ul><ul><ul><li>The enterprise entity and attribute </li></ul></ul><ul><ul><li>The role of the enterprise information consumers </li></ul></ul><ul><ul><li>The current state of the entity in its life-cycle </li></ul></ul><ul><ul><li>The degree of information sharing permitted </li></ul></ul><ul><li>Example of degrees of information sharing: </li></ul><ul><ul><li>Private – Legal or regulatory restrictions exist </li></ul></ul><ul><ul><li>Confidential – Restricted to a select group of consumers within the enterprise </li></ul></ul><ul><ul><li>Internal Use Only – Can be widely shared within the enterprise </li></ul></ul><ul><ul><li>Public – Can be shared with everyone </li></ul></ul>Evan Terry
    25. 25. Assigning Data Security Classifications Product Proposals Vendor Contracts Ordering Inventory Sales De- commission Product Life Cycle Evan Terry Purchasing Proposed Product Contract Purchasing Legal Finance Distribution Sales Marketing Confidential Product Public Internal Use Only Confidential Confidential Confidential Internal Use Public
    26. 26. Managing Essential Metadata <ul><li>More than just “data about data” </li></ul><ul><ul><li>Clarifies business language </li></ul></ul><ul><ul><li>Defines and documents business rules and logic </li></ul></ul><ul><ul><li>Facilitates information sharing </li></ul></ul><ul><ul><li>Creates an inventory of enterprise data </li></ul></ul><ul><ul><li>Supports data consolidation through understanding </li></ul></ul><ul><ul><li>Used in “new hire” training and knowledge transfers </li></ul></ul><ul><ul><li>Captures security classifications and assignments </li></ul></ul>Ken Karacsony
    27. 27. Metadata Management Metadata Ken Karacsony Operational Business Processes Reporting / BI DW Data Integration Service Repository
    28. 28. Questions and Answers Ken Karacsony
    29. 29. We are interested in hearing from you! (Please no vendor solicitations) SOA White Paper Series Ken Karacsony 310.990.9430 [email_address] Evan Terry 310.776.0511 [email_address]