Mdm dg bestpractices techgig dc final cut - copy


Published on

Published in: Technology, Business

Mdm dg bestpractices techgig dc final cut - copy

  1. 1. www.hitachiconsulting.comMaster Data Management (MDM)Data Governance Leadership and Best PracticesDinesh ChandrasekarPractice Director CRM & MDMHitachi Consulting , GDC © Copyright 2010 Hitachi Consulting 1
  2. 2. Agenda  Impact of Poor Data & Need for DQ  Why MDM & Customer Hub  Customer Data Problems & Solutions  Significance of Data Governance  Data Governance Leadership Strategies  Data Stewardship Best Practices  Open Forum © Copyright 2010 Hitachi Consulting 2
  3. 3. Acronyms  EIM – Enterprise Information Management  EDM – Enterprise Data Management  MDM – Master Data Management  DM – Data Management  DG – Data Governance  DQ – Data Quality  SOR – System of Record  KPI – Key Performance Indicators  UCM – Universal Customer Master  CDH – Customer Data Hub  PDH – Product Data Hub  SH – Supplier Hub & Site Hub  CH – Customer Hub Commercial in Confidence © Copyright 2010 Hitachi Consulting 3
  4. 4. How clean is your Wind Shield ?“ Ultimately, poor data is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point, you either have to stop and clear the windshield or Risk everything.” - Ken Orr Institute Commercial in Confidence © Copyright 2010 Hitachi Consulting 4
  5. 5. Impact of Poor Data Quality“… Fortune 1000 enterprises will lose more money in “Data integration and data quality are operational inefficiency due to data quality issues fundamental prerequisites for the successful than they will spend on data warehouse and CRM implementation of enterprise applications, initiatives.” such as CRM, SCM, and ERP.” Operational Efficiency Customer Service  Increased data management costs  Ineffective Cross-sell/Up-sell  Increased sales order error  Lower call center productivity  Delayed sales cycle time (B2B)  Increased marketing mailing costs  Mediocre campaign response rate  Reduced CRM adoption rate Risk, Compliance Reduced IT Agility Management  Heightened credit risk costs  Increased integration costs  Potential non-compliance risk  Increased the time to bring new projects and services to market  Increased report generation costs  Proliferation of data problems from silos to more applications Commercial in Confidence © Copyright 2010 Hitachi Consulting 5
  6. 6. Fragmented data is the source of the problem Ever proliferating islands of information …in disparate applications covering multiple channels, divisions & functions …duplicated, incomplete, inaccurate data Call Web Fusion SFA Center Partner site App • Key enterprise processes based on unclean / incomplete data Marketing, sales, service & customer retention processes, regulatory compliance, new product introduction,… • Unclean data makes Analytics invalid Fusion ERP 1 ERP2 SCM Legacy App • Error prone integration • Slows enterprise agility and innovation Commercial in Confidence © Copyright 2010 Hitachi Consulting 6
  7. 7. MDM : The source of clean data for the enterprise Nurture one of your most valuable asset  Consolidate/Federate shared information into one place ETL  Cleanse data centrally Web  Share data as a single point of SFA Call Fusion Partner Center site App truth as a service Middleware Application Integration Architecture MDM BI Analytics  Consistency siloed environments (Integrated Best of Breed) Fusion  Lower data management costs ERP 1 ERP2 SCM Legacy App  Better reporting ETL  Enterprise foundation for agility & innovation Commercial in Confidence © Copyright 2010 Hitachi Consulting 7
  8. 8. The New Age Digital Customer © Copyright 2010 Hitachi Consulting
  9. 9. Why Customer Hub ? Unify your Customer View with Customer HubMaximize Customer Retention Provides complete knowledge of customers value and history to improve customer loyalty Ensures effective marketing and selling while avoiding missteps Enables sharing of customer information with applications, business processes and point of contact personnelIncrease Selling Efficiencies Facilitates accurate up-selling and cross-selling of products and services Provides accurate product data which reduces order entry errors and decreases days sales outstanding Delivers full quality customer and product information at the point of contactReduces Cost and Risk Provides clean data to all applications and business processes increasing ROI from existing investments Enables data governance to insure compliance and reduce risk Accelerates time-to-market of new products and services Commercial in Confidence © Copyright 2010 Hitachi Consulting 9
  10. 10. Why Organizations engage in Customer Hub Projects? Benefits GROWTH EFFICIENCY IT AGILITY COMPLIANCE Improve CRM Operational Increase IT resiliency Reduce operational performance to efficiency across in a changing risk and improve increase revenue and multi-functions of an business landscape regulatory market share enterprise compliance CUSTOMERS ON AVERAGE EFFICIENCY OF OPERATIONS EFFICIENCY OF IT EFFICIENCY OF IT OPERATIONS GENERATED 2%-5% INCREASED INCREASE WITH IMPROVED OPERATIONS RESULTING IN RESULTING IN GREATER REVENUE FROM SALES WITH PROCESSES AND DATA GREATER AGILITY OF AGILITY OF BUSINESS MODELS MDM GOVERNANCE BUSINESS MODELS Commercial in Confidence © Copyright 2010 Hitachi Consulting 10
  11. 11. Customer Hub StylesRegistry Style Consolidation Style Transaction Style•Various Source System publish • The Consolidation Style MDM • In this architecture, the Hub stores, enhancestheir data and a Subscribing Hub has a physically and maintains all the relevant (master) dataHub stores only the Foreign instantiated, "golden" record attributes.Keys , Source System Ids and stored in the central Hub • It becomes the authoritative source of truthKey data values needed for and publishes this valuable information back tomatching • The authoring of the data the respective source systems. remains distributed across the • The Hub publishes and writes back the•The Hub runs the cleansing and spoke systems and the master various data elements to the source systemsmatching algorithms and data can be updated based on after the linking, cleansing, matching andassigns unique global identifier events, but is not guaranteed enriching algorithms have done their the matching records , but to be up to date. Upstream, transactional applications can readdoes not send any data back to master data from the MDM Hub, and,the Source Systems •The master data in this case is potentially, all spoke systems subscribe to usually not used for updates published from the central system in a•The Registry Style Hub is to transactions, but rather form of the “ Virtual Golden View supports reporting; however, it •The Hub needs to support merging of masterof the master entity from the can also be used for reference records. Security and visibility policies at theSource Systems” operationally. data attribute level need to be supported by the Transaction Style hub, as well. Simple & Faster Medium Complex Complex Short term Gain Mid term Gain Long term Gain Commercial in Confidence © Copyright 2010 Hitachi Consulting 11
  12. 12. Oracle Enterprise Master Data Management © Copyright 2010 Hitachi Consulting
  13. 13. Gartner Magic Quadrant for Customer Hub Solutions “UCM has the strength of the Oracle name behind it, leading to an impressive number of commitments from blue chip names in the Siebel customer base across a range of industries” John Radcliffe, Gartner, May 2008 Commercial in Confidence © Copyright 2010 Hitachi Consulting 13
  14. 14. Oracle Customer Hub (Siebel UCM) 8.2Best in Class MDM Solution Hyperion DRM for Customer Hub Source Data Governance Manager MDM Aware Apps Systems MDM Analytics Siebel Siebel EBS Application Oracle Customer Integration EBS SAP Data Quality Hub 8.2 Architecture SAP JDE JDE Custom Custom Operational exchanges Unclean to clean data(Initial & Delta load) Hub / Apps Commercial in Confidence © Copyright 2010 Hitachi Consulting 14
  15. 15. Key Components of Oracle Customer Hub © Copyright 2010 Hitachi Consulting
  16. 16. Example of Customer Data Quality IssueA Simple Customer Table Sample Matching Records Non Standard formats Name Address City State Zip Phone Email Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 Robert Williams 36 Jones Av. MA 02106 617555000 Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532(9550) Jason Bourne, 76 East 51st Newton MA 617-536-5480 6175541329 Bourne & Cie. … … … … … … … Mis-fielded data Multiple Names Typos Mixed business and Missing Data contact names Commercial in Confidence 16 © Copyright 2010 Hitachi Consulting
  17. 17. Customer Data Problems today COMPLETENESS CONFORMITY CONSISTENCY DUPLICATION INTEGRITY ACCURACY Commercial in Confidence © Copyright 2010 Hitachi Consulting 17
  18. 18. Oracle Enterprise Data Quality Functionality in a Glance Feature Functionality Examples Oracle Offering Understand data status & Name: LN+FN (CHS, KOR, Profiling/Pattern deduce meaning from JPN); FN+MN+ PN+LN OEDQ Profiling Server Detection unstructured patterns (Latin); Tel# is null 30% Create structured records Address field -> Address Parsing and from unstructured data Line 1, City, State,… OEDQ Parsing & Standardization Spot and correct errors; Nationality: US, USA, Standardization Server transform to std format American-> USA Address Valid address 809 Newel rd, PALO ALTO Validation / identification and 94301 -> 809 Newel Road, OEDQ Cleansing Server Cleansing correction Palo Alto, CA 94303-3453 Matching and Spot / eliminate duplicates & Haidong Song = 宋海东 OEDQ Matching Server Linking identify related entities = Attach additional attributes Haidong Song: “single, Universal DQ Connector + Enrichment and categorizations 1 child, Summit Estate, D&B connector + AIA 2.5 PIP DoNot Mail” for Acxiom * OEDQ is formerly known as Datanomics Data Quality Application Commercial in Confidence © Copyright 2010 Hitachi Consulting 18
  19. 19. Data Governance Leadership Commercial in Confidence © Copyright 2010 Hitachi Consulting 19
  20. 20. Data Governance ( DG )DG is all about establishing thestrategies, objectives and policiesto effectively manage corporatedata by specifying accountabilityon data and its related processesincluding decision rights.For example, DG defines• Who owns the data;• Who creates records;• Who can update them; and also,• Who arbitrates decisions when data management disagreements arise. People, processes and technologies are the building blocks for Data Governance © Copyright 2010 Hitachi Consulting
  21. 21. Data Governance Technology Requirements Define, Communicate & Easily Operate hub EnforceDefine enterprise master data • Execute day-to-day hub operationsDefine and view data policies (Consolidate, Cleanse, Share & Master) Data accountability • Perform data steward tasks, such as Escalation process merge/unmergeAdminister hub Monitor hub operations Fix data issues• Analyze hub DQ metrics • Fix import errors and resubmit corrected data• Track sources of bad data • Proactively watch & repair data• Monitor hub transaction load • Tune data quality rules © Copyright 2010 Hitachi Consulting
  22. 22. Potential Data Governance Leadership Council Leadership Layer Client DG Leadership Council · Sponsorship, Oversight & Approval Roles and Responsibilities Data Governance Committee Executive Layer · Approve Strategy Roadmap · Align Business and IT Goals Subject Area Business Owners IT Domain Owners · Align to Client Strategy Customer/Contact, Booking, Services etc. Client IT Systems · Approve Project Prioritization · Advocate Compliance Management Layer Development · Recommend Strategy and Goals Lead / Business Data Managers IT Architect & Maintenance Technical · Prioritize and Execute Projects Manager Manager · Define Standards and Policies · Advocate Compliance · Act as Subject Matter Experts (SMEs) IT Data IT Application IT Integration Process Stewards Data Stewards Personnel Personnel Personnel Operations/Execution Layer · Sales Process · Source Steward · Stewardship of Data, Data SME · MDM Specialist · Service Process · End User Steward · DBA · Application Leads · DQM Specialist · IT/System/Database Administration (DBAs) · Orders/Bookings · Data Hygiene · ETL Specialist · Technology Leads · DQ Tools · Data Modeler · Project Delivery Specialist · Interface Daily with Customer Groups · Cancellation Steward · Ensure Compliance Consumer Base Business IT Enterprise Wide Commercial in Confidence © Copyright 2010 Hitachi Consulting 22
  23. 23. DG Council Task ForceLeadership Council • Champions of the DG Council provides the Leadership, Sponsorship and Overall Vision & Direction Serves as the Final Authority on all decisions • The council would typically consists of a Chief Sponsor ( MDM )and top leadership from Business & IT (for e.g. CIO, VP Operations etc.)Governance Committee • Defines business strategies and champions the importance of data governance & data quality domain-specific data, processes, and business rules throughout Client Organization • Sets priorities for domain-specific data quality improvement projects • Arbitrates competing interests and makes final decisions regarding issues the Management Layer is unable to resolveBusiness Data Managers & IT Administrators • Responsible for managing specific domain-data sets and is responsible for the data stewardship and quality of that data • Recommend specific data projects to support better Data Governance and Data Quality efforts • Responsible for assigning IT resources to support various data projects and initiatives • Responsible for the upkeep of IT systems and tools to support better Data Management Data Stewards Process Stewards • Stewardship of the data for a particular domain (e.g. Customer) • Responsible for entering data for each business process (e.g. • Perform data cleansing, and other data quality activities for that Sales , Marketing, Order Entry, Service Request etc.) data domain • Aid better data quality by supporting data corrections and • Ensure data standards and compliance communication • Perform audits and security checks • Provide inputs to data collection process improvements for the • Serve as a liaison between IT & business with regards to data specific process domain • Serve as SME for specific data sets within the process domain Commercial in © Copyright 2010 Hitachi Consulting 23
  24. 24. Data Governance Program Activities Data Governance ActivitiesHigh-level Activities Detailed tasks 1. Establish Data Define Data Governance Establish Establish Data Identify DG Council Formalize & Kick off Data Governance Governance Leadership Organization Framework Leadership Council Governance Committee Champions Leadership Organization internally Organization Define & Refine Leadership Nominate Data Roles & Responsibilities Governance Lead 2. Establish Data Establish Governance Refine Data Governance Charter after Define Data Governance Review & Refine Data Governance Charter & Charter & Vision socializing with the Leadership Goals & Objectives Governance Goals & Objectives Vision Define Data Governance Subject Area Owners & IT Domain Owners Foundations & Framework Communicate Charter & Vision to their teams 3. Establish the Data Identify Business Data Identify IT Management Define Data Governance Review & Refine Data Governance Define Standards, Governance Framework Managers for Customer Master Resources Framework Process Framework Processes Policies & Procedures Processes Establish Data Governance Define Stewardship Compliance & Monitoring Framework Roles & Responsibilities 4. Operationalize Align standards with vision & Establish processes to manage Define/Refine additional policies Standards & Policies strategy; Refine standards; and monitor standards & policies around audit & security 5. Establish the Identify and Align Identify/Recruit Identify IT, Technical Define & Refine Stewardship Formalize the operational Data Stewardship Processes Process Stewards Data Stewards & Project Resources Processes including DQ Processes Governance Organization & Organization 6. Formalize & Kick Off Publish, Communicate and Kick Off Data Formalize & Kickoff Customer Customer Master Data Governance Organization across the Enterprise Data Governance Initiative Governance Initiative Commercial in Confidence © Copyright 2010 Hitachi Consulting 24
  25. 25. Process Definitions and Improvement Activities Process Definitions & Improvement Activities High-level Activities Detailed tasks 1. Establish Data Refer & Align with Data Governance Processes Governance Roadmap 2. Refine Program/ Identify Current Program Refine/Redefine Program Identify Current Change Project Management management Framework Management Framework Management Framework Processes Identify project Management Refine/Redefine Change Establish Change processes in place and refine/ Management Framework Control Processes adopt to MDM/DG projects 3. Refine Business Inventory current Business Processes Identify process improvements Processes to support with touch point to customer data for each process MDM/DG Processes Refine/Redefine business process to Implement Identified align better with future state MDM Changes Commercial in Confidence © Copyright 2010 Hitachi Consulting 25
  26. 26. Metrics Definition & Monitoring Activities Metrics Definitions & Monitoring Activities High-level Activities Detailed tasks 1. Establish Governance Identify & Define Governance Operationalize Monitor & Report Governance Metrics & Stewardship Metrics Governance Metrics & Stewardship Metrics 2. Establish Data Quality Identify & Define Data Quality Operationalize DQ Metrics for each system Metrics Metrics for Customer Domain (Oracle CRM on Demand , BRM etc..) Monitor & Report Governance & Stewardship Metrics 3. Refine System SLAs Refine/Define System SLAs Operationalize System Monitor & Report System and System Metrics and Metrics SLAs Metrics SLAs and Metrics Commercial in Confidence © Copyright 2010 Hitachi Consulting 26
  27. 27. Data Governance – Key Takeaways Establish Data Governance Leadership Council Establish Data Governance procedures  To ensure data standards and compliance around  Data Consolidation  Data Cleansing  Data Governance  Data Sharing  Data Protection  Data Analysis  Data Decay Commercial in Confidence © Copyright 2010 Hitachi Consulting
  28. 28. Some Examples of DG Council Action Items  Addition of any global languages needs DGC approval  Rules to curtail data decay need to be formalized .e.g.. All golden records that are not updated for the last 6 months needs revisit from customer calls.  Hierarchy Management of customers needs to be visited occasionally, as new branches can be added to accounts.  Exception management process (DQ Assistant)related functionality needs revision and monitoring from DGC.  Any updates for Transports and Connectors w.r.t. change, upgrade etc needs DGC approval  Any changes to Authorization and Registry services needs approval of DGC Commercial in Confidence © Copyright 2010 Hitachi Consulting 28
  29. 29. Customer HubData Stewardship Best Practices Commercial in Confidence © Copyright 2010 Hitachi Consulting 29
  30. 30. Data Stewardship with OCH 8.2 v … © Copyright 2010 Hitachi Consulting
  31. 31. Data Stewardship with OCH 8.2 v Data Steward performs the following operations on a day to day basis using the Data Stewardship application screens provided with OCH 8.2 o Suspect Match o Merge Request o Incoming Duplicate Overview o Guided Merge & Unmerge o Incomplete Records o Survivorship Rules o Data Decay Management The idea is to present the features available and supported by Oracle Customer Hub 8.2 v This is only sample set of functionalities and you may choose to explore other options and enhancements available with the product Commercial in Confidence © Copyright 2010 Hitachi Consulting 31
  32. 32. Merge UC Matching Threshold Scores M Merging UCM calculates Process Matching UCM process the Record is updated Record is sent back Threshold score record based on based on to boundary Record is sent back to based on the the Matching Survivorship Rules system boundary system defined attributes Threshold  There are 3 possible outcomes: Threshold Type Threshold Score Description Auto Threshold >= 90 UCM will automatically merge the two records (Auto-merge) (except for Sales Records) Manual Threshold <90 and =>70 UCM will flag the records to have a Data Steward review and determine whether or not to merge Auto Threshold <70 UCM will create a new record and publish the (Create New Record) record to the boundary systems © Copyright 2010 Hitachi Consulting
  33. 33. Merge Criteria used within UCM UCM Merging Process  Threshold Score:  90% or above - the incoming record will merge with the existing record using the survivorship rules*  Less than 90% greater than 70% - the incoming record will be potentially merged depending on the Data Steward’s decision If the Matching Threshold score falls within this range, the Survivorship Rules will apply * Sales Records will never be auto merged Matching Threshold Accounts Attributes Survivorship Rules • Account Name >=90% • Recent – Incoming value will always survive • Main Phone • History – Existing value will always • Address <90% survive • City • Source – The value from the • State >=70% source will survive., External • Postal Code Systems or Siebel. <70% © Copyright 2010 Hitachi Consulting
  34. 34. Create and Merge Accounts Data Stewards needs to review the record within the “Incoming Duplicates” screen when a Matching Threshold score is within the range of >= 70 and < 90 Data Stewards will determine if the record needs to be merged with another record or should be treated as a new record Matching Threshold Survivorship Accounts Attributes Rules Link and >=90% Update • Account Name • Main Phone <90% • Address Data Steward • City >=70% • State Create • Postal Code New <70% Create New Record © Copyright 2010 Hitachi Consulting
  35. 35. Incoming Duplicate ProcessManual Link and Update Process Create and Merge Accounts Data Steward logs onto Data Steward Data Steward Data Steward Record Incoming queries for their reviews Yes selects “Link and Matches? Duplicates record incoming record Update” Screen in UCM No UCM updates Data Steward record using selects “Create” Survivorship Rules UCM updates record as a new End record  All Data Stewards will see the same records within the “Incoming Duplicates” Screen © Copyright 2010 Hitachi Consulting
  36. 36. Link and Update a Record After reviewing the record information, the Data Steward can return to the “Incoming Duplicates” Screen to “Link and Update” or “Create New” When a Data Steward selects “Link & Update”, UCM will update the record based on the predefined survivorship rules Link and Update © Copyright 2010 Hitachi Consulting
  37. 37. Create a New Record After reviewing the record information, the Data Steward can return to the “Incoming Duplicates” Screen to “Link and Update” or “Create New” If the Data Steward selects “ Create New”, UCM will update the record as a new record and no survivorship rules are applied Create New © Copyright 2010 Hitachi Consulting
  38. 38. Guided Merge and Un Merge ProcessUCM Existing Duplicates Create and Merge Accounts  The “Existing Duplicates” screen is only used when records are loaded into UCM using a batch process  Only potential duplicates will be displayed in the “Existing Duplicates” screen  Potential duplicates can be view “Duplicate Contacts” under Administration- Data Quality and “Existing Duplicates” under Administration – Universal Customer screen. Potential Duplicate Records Merge Button © Copyright 2010 Hitachi Consulting
  39. 39. Unmerging Records Unmerging Records  The Unmerge Profile Screen is where the account and contact records can be unmerged: Records that were merged within the “existing Duplicate” screen Un Merge Button © Copyright 2010 Hitachi Consulting
  40. 40. Merge, Un Merge and Reject Records Reject Button Guided Merge Button Merge Button © Copyright 2010 Hitachi Consulting
  41. 41. Guided MergeGuided Merge allows end-user to review duplicate records and propose merge bypresenting three versions of the duplicate records and allows end user to decide howthe record in the UCM should look like after the merge task is approved and committed. • Victim: the record that will be deleted (from master BC) • Survivor: the record that will be (from master BC) • Suggested: output from Surviving Engine (transient to the task) © Copyright 2010 Hitachi Consulting
  42. 42. Incomplete Records processingData Steward will analyze and re-process the Incomplete data through UCM Batchprocess. © Copyright 2010 Hitachi Consulting
  43. 43. UCM Survivorship RulesSurvivorship Rules UCM Merging Process UCM calculates Matching UCM process the Record is Record is sent Threshold score record based on updated based back to based on the the Matching on Survivorship boundary system defined Threshold Rules attributes  Survivorship Rules are used to automate the quality of the master customer data.  Once a record is determined to be merged, UCM will compare each attribute within a record and update the record accordingly  Data Steward will change the Survivorship rule weight age depends on source system’s and surviving field in Master record level.  There are three comparison methods used by Survivorship rules: • Recent – Incoming value will always survive • History – Existing value will always survive • Source – The value from the source will survive a.k.a., External Systems or Siebel. Remember that whether a record is auto merged by UCM or manually selected to be merged, the survivorship rules will apply. 43 © Copyright 2010 Hitachi Consulting
  44. 44. UCM Survivorship Rules Survivorship Rule Example - Source New incoming record from Siebel (primary source) Existing Record within UCM ( from Siebel ) Name Verizon Name Verizon Phone Number 4085467880 Phone Number 5105467880 Fax Number 4086548980 Fax Number 4086548980 Street Address 5649 Tasman Drive Street Address 5649 Tasman Drive City San Jose City San Jose State CA State CA Postal Code 93425 Postal Code 93425 Country USA Country USA Best version UCM record Name Verizon Phone Number 4085467880 Fax Number 4086548980 Street Address 5649 Tasman Drive City San Jose State CA Postal Code 93425 Country USA © Copyright 2010 Hitachi Consulting 44
  45. 45. UCM Survivorship Rules UCM Survivorship Rule set View © Copyright 2010 Hitachi Consulting
  46. 46. Enhanced Data Stewardship Capabilities © Copyright 2010 Hitachi Consulting
  47. 47. © Copyright 2010 Hitachi Consulting
  48. 48. For any Questions & ClarificationsTwitter : din2winEmail : Commercial in Confidence © Copyright 2009 Hitachi Consulting 48