MDM Mistakes & How to Avoid Them!


Published on

The Data Governance Annual Conference and International Data Quality Conference in San Diego was very good. I recommend this conference for business and IT persons responsible for data quality and data governenance. There will be a similar event in Orlando, December 2010. This is the presentation I delivered to a grateful audience.

Published in: Technology, Business

MDM Mistakes & How to Avoid Them!

  1. 1. MDM – Master Data Mistakes and How to Avoid Them! 05/19/2010 MDM-DQ University 06/07/2010 Data Governance Annual Conference & International Data Quality Conference Confidential & Proprietary • Copyright © 2009 The Nielsen Company
  2. 2. Abstract Data is an important asset to many companies and leveraging that data properly can result in operational and IT cost savings as well as drive business growth. Furthermore, managing strategic data assets is foundational to a service oriented architecture, which in turn facilitates business process management.. These statements make master data Alan White Enterprise Architect management an enticing proposition for many executives but to achieve Chief Technology Office these results, a proper examination and evaluation of the risks affecting The Nielsen Company such a decision must be performed. Phone: 813.366.4184 Mobile: 813.417.2946 When considering master data management, a proper due diligence effort should consider the business drivers, expected benefits, costs, resources, vendors, data profiling, integratability, infrastructure, social norms, and the new operating model. MDM is more than a single product or process, rather, it is an ecosystem of products, processes, people and information. When executed properly, a master data management initiative can provide both savings and revenue opportunities and fewer quality escapes. June 16, 2010 Page 2 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  3. 3. Master Data Mistakes… • Not leveraging your master data as an enterprise asset • Lack of master data management (MDM) education • Not instituting an Information Governance Board • Inability to identify and articulate your data quality needs • Failure assessing MDM needs and planning appropriately • Neglecting to put into place proper communication paths • Improper evaluation and selection process • Inability to identify an architecture and method for accessing data • Modeling for your MDM initiative as an afterthought • Failure to manage expectations on all facets of an MDM initiative • Trying to “boil the ocean” instead of attaining a “quick win” • Ignoring MDM best practices and principles • Not providing continuous business engagement • Planning without requirements for success June 16, 2010 Page 3 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  4. 4. Working definitions • Master Data – “Master data is the consistent and uniform set of identifiers and extended attributes that describe the core entities of the enterprise – and are used across multiple business processes.” – Gartner • Governance – “The way we make and set on decisions about managing a shared resource for the common good.” [1] • Master Data Management (MDM) – MDM comprises a set of processes and tools which allow the creation, management, and distribution of master data throughout the organization. • Data Governance – “Data governance is the political process of changing organizational behavior to enhance and protect data as a strategic enterprise asset.” [1] • MDM must start with data governance. June 16, 2010 Page 4 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  5. 5. Realize master data is a common asset Master Data, an Enterprise Asset • “First, your company must reach a collective understanding that master data in an enterprise is a common asset, and that it is not being effectively used to the greatest benefit of the organization.” [1] • “The specific approaches that can solve the specific problems of master and reference data management must be set within a strategy of the overall management of enterprise information.” •Source: Information Management Magazine June 16, 2010 Page 5 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  6. 6. Educate stakeholders on MDM! • Successful efforts start with a thorough understanding of MDM by all stakeholders throughout the organization. • Excellent resources are available for learning the business, technical, and organizational aspects of MDM. – See Resources • Learning should include MDM governance, data quality, methods-of-use, implementation styles, and information as a service. • Stakeholders will need to keep in mind that MDM is a strategic initiative with phased implementations. June 16, 2010 Page 6 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  7. 7. Foundation should be built on accurate data •Source: TCS •Source: Zoomix June 16, 2010 Page 7 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  8. 8. Put in place an Information Governance Board to… • Define the scope and aspects of master data that will be managed based on business goals and/or regulatory requirements • Decide who makes the decisions for master data, how the governing body is organized, and the processes to be followed • Distribute the decisions about what will and will not be done and what will and will not be encouraged when using master data • Assign responsibilities for implementing the processes, policies, and procedures to the shared resource • Monitor the use of master data and the master data itself in order to assess the effectiveness of the policies and procedures June 16, 2010 Page 8 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  9. 9. Information Governance Board in practice • Define Data Stewardship Activities – Set data quality (DQ) rules, validate and enforce them – Define data domain values for key attributes – Set up business rules – Set up security and privacy rules • Build standard and repeatable processes to govern data – The processes here are rules (not business rules) that outline how data is reconciled, or how a DQ rule is promoted – Ensure there is a clearly outlined escalation path for resolving data related problems • Write policy, standards, and data related requirements – Data quality rules – Naming conventions (in domain and logical models, but not physical models) – Business rules that are common over the enterprise – Security and privacy rules • Establish architecture and method for Master Data access – Access schema – SOA or manual request – File interchange formats •Source: [7] June 16, 2010 Page 9 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  10. 10. Understand your data quality needs • It is important to perform Within the same product class, the same structural and semantic profiling characteristic may be coded as many times as to truly understand your data required in order to meet internal needs quality needs and to properly estimate the effort required to 180GR improve the data quality 180 GR Across categories, the same attribute or = 180 G 0180 180 180G characteristic might have different names and 000180 its values may be different •Source: Nielsen Global Operations Country 1 Country 2 Packaging Packaging Pack Type • Survivorship • Verification Mineral Water Glass Bottle Glass Bottle • Consolidation • Standardization Juices Bottle Bottle • Matching • Enrichment Water Softeners Bottle Refill Bottle Refill • Classification • Recognition Laundry Detergents Plastic Bottle Plastic Bottle • Correlation • Versioning •Source: Nielsen Global Operations June 16, 2010 Page 10 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  11. 11. Articulate your data quality needs Metadata Classification Reconstruction Accurate Matching •Source: Zoomix June 16, 2010 Page 11 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  12. 12. Assess your MDM needs and plan properly • Project Drivers – Understand current state of the organization – Identify key business and technical drivers – Articulate the need for MDM • Stakeholder Management – Determine initial and ongoing stakeholders needed – Gain commitment across relevant business & technical areas • Project Scope – Understand the current landscape surrounding master data – Business processes – Organizational capabilities – Methods of use – Define what the target should be •Source: [1] June 16, 2010 Page 12 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  13. 13. Assessment & Planning Project Drivers • Operational Efficiencies – Lower IT costs – Reduced time to market – Faster implementation – Best demonstrated practices – Reduced operational footprint • Value Creation – New opportunities with existing customers – Designating high-value and low-value customers – Retaining customers – New information products (when data is your core competency) • Data Risk Management • Regulatory Compliance •Source: [1] June 16, 2010 Page 13 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  14. 14. Assessment & Planning Project Drivers Challenges Business Impacts • Lack of consistent information • Inability for IT to innovate quickly across transactional systems and cost effectively to support business mandates • Lack of automated processes or controls to validate and manage data • Downstream data errors are much more costly to fix than • Growth creates data errors fixed at their source fragmentation • Information re-work requires a • Customer mandates and lot of expertise and support regulatory compliance for standards based master data • Breakdowns in transactional synchronization with business processes units & partners •Source: Nielsen Global Operations June 16, 2010 Page 14 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  15. 15. Assessment & Planning Project Drivers UPC Item Identifier Brand Owner Recipe Flavor Packaging Material Europe 8257002274 EXTRUDED PROCTER & GAMBLE BARBEQUE CARDBOARD CRISPS productCode brandDescription parentDescription pccMajor variant US 1159 PRINGLES POTATO CHIPS PROCTER & GAMBLE CONFECTIONERY CAN/BARBEQUE CAN/BARBEQUE CO & SNACKS UPC Item Description FLVR SubPC MANUF Canada 5610010056 PRINGLES REGULAR BBQ 56 BBQ 0461 SNACK P&G GM FOODS-POTATO UPC Brand Owner Item Description Flavor Product Packaging Category Material Golden Record 5610010056 PROCTER & PRINGLES BARBEQUE SNACKS CARDBOARD GAMBLE POTATO CHIPS FOODS-POTATO For illustrative purposes only •Source: Nielsen Global Operations June 16, 2010 Page 15 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  16. 16. Assessment & Planning Stakeholder Management • Executive Sponsors – Chief Information Officer or proxy – Chief Information Security Officer or proxy – Chief Technology Officer or proxy – Lines of Business executive sponsors • Business/Operations – LOB resource owners – Business data stewards – Business analysts • IT Architecture and Operations – Strategist – Enterprise architect – Data architects – Solution architect(s) •Source: [1] June 16, 2010 Page 16 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  17. 17. Assessment & Planning Stakeholder Management • The MDM Executive Team should: – Provide oversight for the MDM project – Commission an MDM project team of business analysts, data stewards, architects, and data integrators to create the MDM roadmap – Set up a management and decision making structure for the MDM project team • The MDM Project Team should: – Define the project scope and the characteristics of your organization’s MDM governance practice – Collect information on master data quality, existing business processes, workflows, event notifications, data models, quality rules, & security controls – From a governance perspective, begin to answer the critical questions regarding the data to be stored, sources of data, current systems, data stores, and processes •Source: [1] June 16, 2010 Page 17 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  18. 18. Assessment & Planning Stakeholder Management Initiative Leader Operations Program Management Tech Build Lead Program Design Office Products & Governance Integrator Lead Brands Lead Leader Products Content Products Platform Operations App Dev Program Build Factory Locations Lead Lead Lead Program Manager Manager Lead Locations Lead Media Lead Production Media Lead HR Partner Quality Lead End State UAT Lead Consulting Planning Lead Finance Partner Partner •Source: Nielsen MDM Executive Team June 16, 2010 Page 18 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  19. 19. Assessment & Planning Project Scope EEU GC NA WEU MEA API •Zero footprint client •Synchronization LATAM •Data Governance •Workflow •Business Rules Local System •Data Quality Regional Hub •MDM as a Service Global System June 16, 2010 Page 19
  20. 20. Evaluate your options; including buy vs. build Engage Gather Research Create Distribute Leading Requirements Vendors Scorecard RFI Vendors Compile Select Compile Execute POC Deliver Qualitative Vendors for Quantitative and POT Results Results POC/POT Results June 16, 2010 Page 20 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  21. 21. Gather your requirements Accessibility Content • Services • Domain (Including but not limited to) – Searching capabilities – Business Enterprises – Cleansing, enriching, matching external and internal files – Products – Reporting – Location – Subscription based – Media - People; Titles, Domain Names, Lineups, … • Publish • Attributes – Real time and scheduled – Support of Global and Regional/Business’s attribute views; – Content following business rules and “Fit for Usage” • Relationships – Controls on replication of published data • Interface – Will support relationship between entities – A single integrated zero client foot print application • Hierarchy – Multilanguage – Multiple user and client defined hierarchy for an entity – Multiple collection points • Versioning • Availability – Keep historical version of entities – 24/7 with scheduled downtime windows • Audit trail Governance Security • Matching • Access, Ownership and Privacy – Users can configure all matching rules. – Maintained at field level; (licensing, client specific hierarchy) – Rules are domain and content type specific – External users only has access to “fit for use” data. • Content – Governance enabled at appropriate level for content types • Roles and Permission – Governed in real time (in all languages) – User are given CRUD and functionality access based upon – Direct impact on “fit for use” entity roles • Workflow Management – Integrated – Cater to specific domains and functions; • Knowledge Management – Integrated content knowledge base – Enable user to provide incremental updates June 16, 2010 Page 21 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  22. 22. What to ask for, look for, and prove – high level • Be sure you and the vendor agree on what out-of-the-box and configurable/modifiable means. • Often times OOB is misinterpreted to mean that the feature is available without any configuration or modification. June 16, 2010 Page 22 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  23. 23. What to ask for, look for, and prove – detailed June 16, 2010 Page 23 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  24. 24. Evaluating your options June 16, 2010 Page 24 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  25. 25. Determine your implementation style June 16, 2010 Page 25 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  26. 26. MDM approaches • Single-copy approach – Changes made directly to master data – Guarantees consistency Global Repository – Applications have to be modified • Multiple copies, single maintenance Media – Single master copy, changes sent to copies stored locally at source Locations – Applications can only change data not part Products UI of master data MDM Application – Reduces application changes – Learning curve for users • Continuous merge – Copies stored locally where applications Application can change master data Local Local – Local changes are sent to the master, Local where they are merged – Changes to master are sent to source systems and applied to copies – Minimal (maybe no) source system changes – Update conflicts may be difficult to reconcile June 16, 2010 Page 26 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  27. 27. Conceptual Sync Approach (Consolidation) Current Transition Target • Current State – Separate IMDB Instances Region Global • Consolidate IMDB system data into MDM Hub & Apply Global Characteristics • Retire legacy systems incrementally June 16, 2010 Page 27 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  28. 28. Conceptual Hybrid Approach (Federation) Global • No synchronization Global • Hybrid Architectural Style Products, (Transaction & Registry) begins Chars, & evolutionary approach Values with • Global hub stores global chars & keys to local local keys for federated access to Example Only local products/chars • Virtual consolidated view is Global ID Brand Parent Global Char System Key assembled dynamically 12345 Totinos General Mills xxx EIMDB 235 • Converge systems 34567 Triscuit Kraft xxx NIMDB 456 • Retire legacy systems 67546 Vault Coca Cola xxx ProdRef 897 incrementally 56473 Mach3 Gillette xxx CA IMDB 564 Europe North LATAM Local Chars America Local Chars Local Chars June 16, 2010 Page 28 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  29. 29. Conceptual Architecture Applications & Reports Data Quality Services Third party • Profile & Analyze data services • Standardize & Cleanse • Matching Process Manager Messaging FTP, HTTP, etc. Connectivity & Operability Application Integration Service Integration Information Master Data Mgmt LOB Systems Integration Services Services (MSci, Claritas, RMS, Media) Enterprise Data • Extract, Transform, Load • Workflow Warehouses • Abstract and Virtualize • Search • Federated Access • CRUD Mainframes RDBMS • Products • Locations • Media XML Files Web Files Excel Services June 16, 2010 Page 29 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  30. 30. Duopoly of Master & Reference Data Products Business Process User Interface Web Services Products^ Rules* Custom Chars* Hierarchies* Data Management Web Services Outputs Parser Parser 1 Execution Engine 2 ^ Reference Data Rules interface 1 Snapshot of filtered products * Master Data Rules management 2 Navigation of products Rules implementation (e.g. hierarchy) June 16, 2010 Page 30 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  31. 31. Model for your MDM initiative • Common Information Model – Object Oriented modeling – Reusable data types, inheritance, operations for validating data – Hierarchical – Nested data types with ability to declare behaviors on data – Relational – Manage referential integrity constraints • Canonical Model – Business rules and format specifications – Standard view of data for an organization – Mapping back to application view •Source: [6] • Operating Model – “Describe how your organization will govern, create, maintain, use, and analyze consistent, complete, contextual, and accurate data values for all stakeholders.” [4] – The most important set of models June 16, 2010 Page 31 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  32. 32. Manage expectations on all facets of MDM • MDM software is not the “silver bullet”; it is often times one of many components in a LOB system • MDM is usually a strategic initiative that involves a high degree of coordination across several Lines of Business • When estimating costs one should consider hardware, software, resources, training, consulting, travel, etc. • Skills should include systems integration, data quality, programming, data architecture, data stewardship, business process, etc. • People, processes, and politics are at the core of any MDM initiative! June 16, 2010 Page 32 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  33. 33. Think Big, Start Small; Don’t Boil the Ocean To provide Global Operations with a consistent business process and technology platform that enables global content and implements best demonstrated practices to govern, create, maintain, publish, and analyze data values for all stakeholders while providing the authoritative source of data assets in a flexible and scalable architecture capable of expanding into new markets and services aligning with Nielsen's business strategy June 16, 2010 Page 33 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  34. 34. Think Big, Start Small; Don’t Boil the Ocean Each track in in this phased implementation Track 1 - Consolidate Track 2 - Harmonize approach builds upon one another and provides incremental value to the business. Europe Europe Track 1: NA NA •Provides the foundation for global convergence by defining the global content schema (common LATAM LATAM information model). •Aligns with the global publication strategy. •Design common data structure •Reconcile Global Content Track 2: •Map source and target systems •Determine rules for products •Facilitates global content harmonization via global •Perform initial data load •Apply Global Characteristics characteristics and value administration. •Synchronize SOE with SOR •Synchronize SOR with SOE •Aligns with the global data integration strategy. Track 3: •Replaces the local user interfaces and business Track 3 - Centralize processes with one global interface using best demonstrated practices •Is part of the Global Operations convergence strategy NA €IMDB Incremental Deliverables LATAM Legend Track 1 •Develop global user interface Track 2 •Replace SOE with global SOE (starting with Europe) Track 3 June 16, 2010 Page 34 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  35. 35. Apply MDM best practices and principles • Executive sponsorship should be and remain actively involved • Business people must be involved and must collaborate with IT • Project management and organizational change management should be in place • Open communication across organization must be present • The operating model should drive processes • Processes should be enforced through automation • Processes should be built to support continuous improvement • Access to master data should occur at the MDM service interface layer • Data models should be extensible to allow changes as needed to meet requirements • Processes should be flexible for changing business process as the business dictates June 16, 2010 Page 35 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  36. 36. Engage the business with incremental success Phase User Interface Master Data Legacy Consumers June 16, 2010 Page 36 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  37. 37. Create decision and escalation paths • Identify parties and define their roles and responsibilities • Ensure that all parties have the information necessary to fulfill their responsibilities • Define the communication process, escalation process and decision making process June 16, 2010 Page 37 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  38. 38. Planning for success and on-time delivery • Strong leadership • Skilled resources • Team composition • Shared accountability • Achieve parallelism • Continuous development • Training as necessary • Business availability • No calendar or scope creep June 16, 2010 Page 38 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  39. 39. …and how to avoid them • Identify and leverage your master data assets • Educate stakeholders on MDM • Put in place an MDM governance board • Understand and articulate your data quality requirements • Involve data and enterprise architects in your MDM strategy • Create decision and escalation paths • Proper evaluation and selection process • Determine your implementation style • Model for your MDM initiative • Manage expectations on all facets of MDM • Think big, start small; don’t boil the ocean • Apply MDM best practices and principles • Engage the business with incremental success • Plan for success June 16, 2010 Page 39 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  40. 40. Questions June 16, 2010 Page 40 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  41. 41. References 1. Enterprise Master Data Management: An SOA Approach to Managing Core Information, Allen Dreibelbis et al, IBM Press 2008 2. Using Master Data in Business Intelligence, Colin White, BI Research, March 2007 Master Data in Business Intelligence 3. Seven Master Data Mgmt Best Practices, Hannah Smalltree, News Writer 05 Jul 2006 | 4. Modeling the MDM Blueprint Series, James Parnitzke, Applied Enterprise Architecture, 2009 | 5. Information service patterns, Part 4: Master Data Management Architecture Patterns, Allen Dreibelbis et al, 29 Mar 2007 | 6. Canonical Data Model: Design Challenge, Steve Hoberman, Information Management Magazine 01 Aug 2008 | 7. Information Governance Board Charter and Approach, Jay Noh, 2008, The Nielsen Company 8. Master Data Management (MDM) Hub Architecture, Roger Wolter, Microsoft Corporation, Apr 2007 | MSDN Library June 16, 2010 Page 41 Confidential & Proprietary Copyright © 2009 The Nielsen Company
  42. 42. Thank you June 16, 2010 Page 42 Confidential & Proprietary Copyright © 2009 The Nielsen Company Confidential & Proprietary • Copyright © 2007 The Nielsen Company