Information management and enterprise architecture


Published on

Practical Enterprise Information Management; building a bridge between EA and the data architects

Published in: Technology, Business

Information management and enterprise architecture

  1. 1. PRACTICAL ENTERPRISE INFORMATION MANAGEMENT Rajesh Nadipalli ( Mar 2012, rev 3
  2. 2. About this Presentation Most organizations today lack a bridge between the data architect who is proud of the physical data model (ER Diagram) and the Enterprise architect who is trying to transform the business based on priorities set by the CIO at a higher level via architectural blue prints. In this presentation I talk about models and approaches to build this bridge and enable organization to have an architecture driven information management In today’s atmosphere there is a mix of Cloud based services (externally hosted), NOSQL databases and custom relational databases, a successful EIM will enable an organization to be really “agile” to meet their business needs
  3. 3. Agenda • EIM - Basics • EIM - Approach • EIM - Tools • References
  4. 4. EIM - BASICS
  5. 5. EIM - Basics EIM - BASICS Enterprise Information Management (EIM) is an initiative to mange data in all forms and treat them as a strategic asset. A good EIM program will result in an integrated, accurate, timely data across your enterprise. A good EIM program will have policies, frameworks, technologies & process to address: • Data models • Data lineage • Data quality • Data profiling • Stewardship
  6. 6. EIM – What’s the need? EIM - BASICS View of the Enterprise based on 2 roles: Business Architect perspective: • What is the system of record for customers? Should that be the Sales, Customer Support, Accounts Receivable system OR a combination? • If a Master Data system is defined, how is this data being propagated? • How fresh is my Data Mart that get data from a Cloud hosted service? IT technology Architecture perspective: • What’s the best way to integrate with cloud services which can be private or public • How do I model NOSQL databases with little formal structure • How to integrate Social Feeds (Tweeter, Facebook) • How to combine this with Legacy and traditional relational databases EIM will address these questions by process, tools and frameworks
  7. 7. EIM – what’s the value? EIM - BASICS To put a direct $ figure on the value of EIM is a challenge. Couple of Examples on what if you don’t have an EIM: • If your Cloud hosted CRM calls a customer “Bill Adams” but your Accounts system calls him “William Adams Junior” and now your marketing system has no clue that they are the same customer and sends him 2 separate communications. What is the cost of annoying your customer or sending him multiple mails? • Your company recently went through an acquisition and you want your sales team to include current products and the newly acquired company product as a bundle offer. Who can identify all the systems that need to be updated? How long and whom do you need to pull in? Are you confident that your dashboards will reflect these changes by next quarterly results?
  9. 9. EIM – Suggested Key Steps EIM - APPROACH Business & Strategy Alignment (EA) Application & Data Architecture (EA) Data Modeling Master Data Management Lifecycle Mgmt, Governance Data Profiling & Quality
  10. 10. – Enterprise Architecture Alignment EIM - APPROACH TOGAF is popular EA framework and recommends ADM which is an Iterative Process Requirements at center and … •Phase A: Vision, Stmt of work • Phase B,C,D: Baseline, Gap analysis, target state •Phase E: Initial implementation plan •Phase F: Detail transition plan, cost benefits, risk •Phase G: Arch Oversight; issue arch contracts •Phase H: Procedures for managing change. For each requirement, TOGAF looks at Business, Application, Data and Technology (hardware) layers to ensure a solution architecture change is holistic. Phase B is Business Architecture, Phase C is Information System Architecture, which is further broken down into Application and Data Architectures.
  11. 11. EIM – EA Alignment (contd) EIM - APPROACHBusiness Architecture (Phase B) • What are the new goals for the organization • What business requirements need to address by these goals • What business functions, process and services will be changed/addedApplication & Data Architecture (Phase C) • What Applications (IT Systems) serve the business functions (as identified in Phase B); which ones will need revisions • What application functions will be impacted • What enterprise data entitles will need to created/updated/stored to meet these changes • What data transformations, interfaces (like ETL, web services) need to be built/changed
  12. 12. EIM – EA Alignment - Example EIM - APPROACHBusiness Architecture (Phase B) • Goal: Improve conversion rate leveraging social network of our customers • Process Changes: • Customer registration (currently only has email), add facebook, twitter, linkedin, account information • Ad campaign team should use social networks for new leads and advertisingApplication & Data Architecture (Phase C) • Application Architecture: • Customer Registration change for new fields • Integration changes to Marketing systems to push this new fields. • Automated ad’s to customer’s social network. (might suggest this to Biz Architect) • Data Architecture: • Additional Columns / Tables to capture new content • Consider a flexible model to accommodate other social media sites This example shows, how the different architects can work together and be effective
  13. 13. EIM – EA Data Model – 3 layers EIM - APPROACH While the EA effort will talk about data architecture changes at a high level, the data architects should build the next level of details in the following 3 layers: Conceptual Model • Business Friendly • Only High level entities and taxonomy • Connected to Business capabilities Logical Data Model • Key entities , attributes • Relationships to other entities, cardinality • Ownership by IT Systems (System of Records) • Interfaces and dependent IT systems Physical Data Model • Entity Relationship Diagram • Physical characteristics (string, number, date, length) • Constraints (Primary, Foreign, Referential) • Data Store (Database name)
  14. 14. EIM – Data Model – Example EIM - APPROACH Conceptual Layer: Customers •Really High Level • Customer Information Customer Sales Rep Customer Basic Info • Customer ID Logical Data Model • Sales Rep ID • Last Contact Date •Multiple Logical entities• Name• ID • 1:N, N:1 relationships• Address• Customer Since Date Customer Locations • Interfaces identified • Customer ID • Ownership defined • Site ID • State CUSTOMER_SALES_REP ID NUMBER (PK) CUST_ID NUMBER (FK) SALES_REP_ID NUMBER (FK) CUSTOMER_DIM LAST_CONTACT_DATE DATE Physical Data Model UPDATE_DATE DATEIDNAME NUMBER (PK) VARCHAR2(100) UPDATE_BY VARCHAR2(50) •Detail field typesADDRESS VARCHAR2(500) • Typical Entity Relationship from DBSTART_DATE DATE CUSTOMER_LOCATIONSUPDATE_DATE DATEUPDATE_BY VARCHAR2(50) ID NUMBER (PK) CUST_ID NUMBER (FK) SITE_ID NUMBER STATE CHAR(3) UPDATE_DATE DATE UPDATE_BY VARCHAR2(50)
  15. 15. EIM – Data Model – how to scale EIM - APPROACH To have a successfully Data Model, enable scaling on both directions: • Vertically: all layers of stack (Conceptual, Logical & Physical) • Horizontally: segmented by line of business (Finance, HR, Marketing..) • This will help you assign data stewards by domain and layer For example say, lets’ take “Customer” and suppose your Company-A just made a recent acquisition of Company-B which has it’s own set of customers • Conceptual Layer: Customer will be one entity for enterprise • Logically Layer: You can have 2 entities “Customer-A” and “Customer-B”, both linked to the same conceptual entity but having different data due to the separate Line of business. • The architecture target state should recommend a consolidation of the two Customer Entities if there is an overlap.
  16. 16. EIM – Master Data Management (MDM) EIM - APPROACH MDM is for list (like Customer, Product) Key aspects for a successful MDM: • Agree on the System of Records • Define Data Stewards • Adoption of MDM data (integrate/sync with dependent systems) • Ability to identify and merge duplicates • Ability to fix (case sensitivity, renames) • Add versioning (type 2 / type 3) • Hierarchies • Reporting
  17. 17. EIM – Lifecycle Management, Governance EIM - APPROACHLifecycle Management • Defines the business processes linked to “Data Entity” at all phases - Creation, Consumption and Archival • What is the impact to data integrity; for example in a SOA based environment System A would rely on System B for details of a related entity; if System B archives the entity, the business users might have an impact.Governance • Identify data stewards • Establish guidelines for process, documentation, permissions & communications. • Data stewards should work with EA to ensure alignment
  18. 18. EIM – Data Profiling, Data Quality EIM - APPROACH Data Profiling • A mature source should be profiled and this sample can then be used as a basis to detect bad data before it gets reported in dashboards. • Quality issues should be proactively be emailed and reviewed by appropriate analyst who can fix the same. • Data correction can also be automated to reduce the manual overhead.
  19. 19. EIM – TOOLS
  20. 20. Tools EIM - TOOLS I plan to add some screenshots of tools in later revisions of this presentation
  21. 21. REFERENCES
  22. 22. References REFERENCES EIM & EA • • • • • MDM • • Data Profiling • •