Business Process-Driven
MDM & BI
Strategy & Vision
by: Mark D Schoeppel
March 2016
MDM BI – Vision
Transaction
Systems
Business Process Management
Capture Consume
Master Data Management
Instantiate Provide
Monitor
Business Intelligence
Data
Warehouse
Big Data
Visualization
Goals:
• Enable Speed of Innovation for MDM BI activities across the
Enterprise, the Operational Units, and the Business Users
• Define Governance and Data Management Practices
throughout the Enterprise
• Improve the accuracy of Master Data for Unity and Legacy
Systems
• Build Enterprise Class Metrics and KPIs based on Business
Processes
Enterprise
• Data Stewards
• Data Governance
• BI Team
• Technology Team
Operational Unit
• MDM BI COEs
• BRM Teams
• Data Scientists
• Data Analysts
Business Users
• Data Stewards
• Reporting Stewards
Operational Unit
Business UsersEnterprise
Ideation
Transaction
and KPI
Consumption
Self
Service
Requirements
Data ProfilingGovernance
Visualization Testing
Publishing
Architecture
Standards
Methods
Technologies
Training
Solution Design
Data Ingestion
Analytic
Algorithms
Data
Exploration
Application
Development
Master Data
Business
Intelligence
Big Data
Data
Cleansing
MDM BI – Roles and Functions
Business Data Governance – Overview
Master Data
Management
DW & BI
Management
Data Quality
Management
Metadata
Management
Data Security
Management
Data
Architecture
Management
Data
Development
Misconception Reality
 It's an IT responsibility
Data Governance requires a partnership between Business,
Technology, and Operations
 One size fits all
The organization, processes, and technology must be tailored to
fit the culture and leverage existing governance structures and
technology
 It can succeed through a grass
roots bottoms up effort
Success requires executive advocacy and sponsorship
 It's about having the right tools
Data governance requires the integration of organization,
processes, and technology tools
 It can be an add on
responsibility that doesn't need
to be measured or rewarded
Data stewardship may require full time staff commitment. If the
role is not measured or rewarded, the result will be ineffective
governance action
 It’s a big bang implementation
Standing up data governance structures is an evolutionary
process that requires effective change management
Data governance is the orchestration of people, process, and
technology to enable the leveraging of data as an enterprise
asset through a well-defined organizational structure,
policies, rules, decision rights and accountabilities for
decision making and management of Master Data.
Data Governance – 180 Day Plan
•Create the appropriate review and escalation methods for managing data quality and integrity
•Enable the linkage between business process/data owners who “champion” data and metrics with the data
architects and data stewards who manage the transaction level detail
Governance
&
Stewardship
•Integrate roles across functions (e.g., data cleansing, data architects, data stewards, process/data owners)
•Understand the needs of the consumer of the data and connect appropriately
People/
Organization
•Define end to end, consistent processes across all data types, linking transaction level data with Management
Information
•Define proper controls to manage data quality and integrity on a sustainable basis
Process
•Identify Tools for cleansing, mapping, identifying anomalies, etc.
•Leverage data management tools and infrastructure that have rapid scalability and functionality
•Define consistent architecture that enables “one version of the truth”
Technology
MDM BI
Data Cleansing
MDM BI
Product
MDM BI
Customer
MDM BI
Vendor
MDM BI
Operational
MDM BI
Financial
MDM BI
Stat/Mgmt Rptg
EBPM - PTP
EBPM - PTD
EBPM - OTC
EBPM - RTR
EBPM - FTP
As requirements for
MDM and KPIs are
assembled,
PRIORITIZATION and
sequence can be
further refined
Outcomes
MDM BI – Discovery Approach
Enterprise
Data Model
Master Data
Models
Required
KPIs
MDM BI – Deployment Approach
Discovery Analyze / Define Design
Deliverable
• Conduct Discovery
Sessions
• Define Solution Scope
• Define Solution Concept
• Define General System
Concept
• Describe Potential
Impact
• Plan Project
• Analyze Guidance
Architecture
• Analyze Data
Architecture
• Create Data Schema
Map
• Assess Data Quality
• Analyze System
Architecture
• Analyze System
Requirements
• Design Guidance
Architecture
• Design Data
Architecture
• Design System
Architecture
Design Application
Specification
• Design Data Migration
• Design Human
Transition Support
Activity
• Discovery Summary
• Solution Scope
• Solution Concept
• System Concept
• Impact Summary
• Project Plan
• Organizational Model
• Guidance Model
• Data Model
• Data Schema Map
• Data Quality
Assessment
• System Interaction
Diagram
• System Requirements
Summary
• Organizational Model
• Guidance Model Data
Model
• System Interaction
Diagram
• Application Specification
Data Migration Plan
• Organization Change
Management Plan
• Training Plan
• Discovery & Analyze initiated via
common stakeholder interviews
• Design & Build executed on
Global Template
• SAP Deployments will review
efforts from previous phases but
largely be testing and refining
exercise
Build Test Deploy Sustain
MDM – Implementation Approaches
Consolidation Registry Coexistence Centralized
For Reporting, analysis, and
central reference
Mainly for real-time central
reference
For harmonization across
databases and for central
reference
Acts as system of record to
support transactional activity
Matches and physically stores a
consolidated view of master
data
Matches and links to create a
“skeleton” system of record
Matches and physically stores a
consolidated view of master
data
Matches and physically stores
the up-to-date consolidated
view of master data
Updated after the event and not
guaranteed up-to-date;
authoring remains distributed
Physically stores the Global ID,
links to data in source systems
and transformations
Updated after the event and not
guaranteed up-to-date;
authoring remains distributed
Supports transactional
applications directly – both new
and legacy – typically through
SOA interfaces
No publish and subscribe; not
used for transactions but could
be used for reference
Virtual consolidated view is
assembled dynamically and is
often read-only; authoring
remains distributed
Publishes the consolidated view;
not usually used for transactions
but could be used for reference
Central authoring of master data
Analytical Focus Operational Focus Operational Focus Operational Focus
System of Reference System of Reference System of Reference System of Record
Enterprise
Master Data
Model
Legacy
ERP
MDM People
MDM Processes
MDM Tools
Transformation
Certified Master
Data
2
3
4
1. Non-Certified Data from the
Legacy Systems is ingested and
transformed into the Enterprise
Master Data Model built during
the Discovery Phase
2. The Enterprise Data Model is
adjusted as necessary and
cleansed data is pushed back to
the Legacy ERP System
3. Certified Master Data is produced
and the required refinements to
processes and data architecture
are made to enable downstream
consumption
4. All of this is enabled by dedicated
MDM personnel, utilizing MDM
tools and processes
MDM – Pre-Cleanse & Deployment Process
1
Non-Certified
Data
BI – Ownership Structure
Sandbox (50%)
[user created content]
Shared (30%)
[user created and shared
content]
Production
(20%)
Gather
Data
Visualize
PublishConsume
Ideate
Business
Users
Require-
ments
Profile
Data
DesignDevelop
Test
IT
Sandbox Environment
• Business users author and use BI content with no
constraints or limitations. This is where data
exploration, discovery, and what-if analyses
happen.
• Tools and technologies: Microsoft Office
• IT involvement is strictly limited to infrastructure
and tools support plus monitoring to identify
usage patterns, commonalities, and opportunities
(using BI on BI) for potential production
hardening.
• Content produced here is used in individual tasks
and low-risk applications.
Shared Environment
• Business users share and collaborate on BI content
with their colleagues.
• Tools and technologies: SharePoint BI, Office 365
• IT steps up monitoring and now watches for red
flags (too much data, too many users, too critical
or risky applications) and opportunities (using BI
on BI) for production hardened BI Content.
• Content produced here is shared within
departments and workgroups. Low-risk, low-
criticality decisions can be made based on this
content.
Production Environment
• Business uses and authors BI content within the
limitations and constraints of the enterprise data
model, standards, policies, rules, guidelines, etc.
• Tools and technologies: EDW, Visual Studio,
SharePoint BI, Office 365
• Owned, run, and managed by IT.
IT Benefits
• Backlog Reduction – only heavily-used,
complex, or critical applications come to IT for
production-hardening.
• Requirements already defined; Project Lifecycle
is greatly reduced; Enhancements during
testing cycle minimized.
• Shadow IT is embraced as a competitive
advantage; however, using the strategic
technology stack defined by IT.
Business Benefits
• Business users are empowered to
create BI content on their own
schedule without any constraints
or limitations – at the speed of
business innovation.
• They modify the model and
visualizations through iteration until
the requirements are identified and
met.
Hadoop = Data Lake
• Land all data in Hadoop as-is from any source
• Enables Analytics Sandbox
• Enables MDM Pre-Processing
• Enables EDW Population with Relevant Data
• Enables Application Access via API Layer (including 3rd party
developers)
Actively Archive from EDW to Hadoop
• Little-Used Historic EDW Data resides in Hadoop (lower cost
storage)
• Define an archive strategy for various data types
Enable business analytics
• Identify tools, methods, and security requirements for
interaction with the distributed file system
• Introduce exploratory analytics without jeopardizing SLAs
• Introduce new machine learning, or data mining techniques
on years worth of data
Enable BU Innovation
• BU Teams continue to innovate with their business users
within the Enterprise Framework – ingestions driven by BU OR
Enterprise requirements
• Data Scientists and Analysts can access all Data for Analytics
• BU IT & Business Teams can access all Data for Visualizations
with proper security
• Enterprise Data Model, HDFS Standards, and Access Methods
extensible to manage localizations at the BU level and below
BI – Data Flows
Data Sources/Transports
Transaction Data
Customer Data
External Data
Industry Data
Sensor Data
DB
Files
REST
JMS
HTTP
SOAP
Hadoop
Compute +
storage
… … …
… … … …
… … … …
… … …
Compute +
storage
supporting technologies& packages
EDW
BI Tools & Applications
Query & Visualization Tools
JDBC/ODBC Compliant
Tools & Applications
Analytic & Reporting Tools
R
Mahout
Excel
Excel
PowerPoint
Power View
MDM
API Layer
BU1 BU2 BUn
BU1 BU2 BUn
BU1 BU2 BUn
Establish, Maintain, and
Periodically Review and
Recommend Changes to Data
Governance Policies, Standards,
Guidelines, and Procedures
The Team responsible to develop the strategy,
govern the tools selected to acquire and transform
relevant data into knowledge to drive business
decisions and actions to achieve desired results. In
addition, the resulting information has to be
tailored to – and distributed to – the appropriate
levels of management and operations in a timely
manner to be most effective. In some cases BI
Execution of Reporting and Analytics is performed
as well.
Provide Quality Assurance –
Oversight, Monitor, Report Results
to Data Governance Council
VP MDM & BI
Business
Governance
Leader
Data Stewards
by Domain
Data Stewards
by Business Unit
Data
Governance
Leader
Data Quality
Data
Architecture
Data
Conversions
Big Data
Architect
BI Governance
Leader
BI Leads
BI Visualization
Developers -
Enterprise
BI ETL
Developers -
Enterprise
BI Developers –
Business Units
Technology &
Tools Leader
Technology
SMEs
DBAs
System SMEs
• Develop and Deliver Data Governance
Program Educational, Awareness &
Mentoring Materials
• Assist in Defining Data Quality Metrics
for Periodic Release
• Support Data Quality Issue Analysis and
Remediation for “Strategic” Data
• Oversee Enterprise Data Governance
Program Development / Architect
Solution & Framework
• Administer the Program including
facilitate the Data Governance Council
meetings
• Provide the Agenda for the Data
Governance Council Meetings to the
Approved by Council Owner Pre-
Meeting
• Facilitate Data Governance Organization,
Tactical & Operational Stewards, the
Data Governance Council Involvement
• Conduct Audits to Ensure that Policies,
Procedures and Metrics are in Place for
Maintaining/Improving the Program
Functionally
Aligned Roles
Organizationally
Aligned Roles
Sample Organization

MDM & BI Strategy For Large Enterprises

  • 1.
    Business Process-Driven MDM &BI Strategy & Vision by: Mark D Schoeppel March 2016
  • 2.
    MDM BI –Vision Transaction Systems Business Process Management Capture Consume Master Data Management Instantiate Provide Monitor Business Intelligence Data Warehouse Big Data Visualization Goals: • Enable Speed of Innovation for MDM BI activities across the Enterprise, the Operational Units, and the Business Users • Define Governance and Data Management Practices throughout the Enterprise • Improve the accuracy of Master Data for Unity and Legacy Systems • Build Enterprise Class Metrics and KPIs based on Business Processes
  • 3.
    Enterprise • Data Stewards •Data Governance • BI Team • Technology Team Operational Unit • MDM BI COEs • BRM Teams • Data Scientists • Data Analysts Business Users • Data Stewards • Reporting Stewards Operational Unit Business UsersEnterprise Ideation Transaction and KPI Consumption Self Service Requirements Data ProfilingGovernance Visualization Testing Publishing Architecture Standards Methods Technologies Training Solution Design Data Ingestion Analytic Algorithms Data Exploration Application Development Master Data Business Intelligence Big Data Data Cleansing MDM BI – Roles and Functions
  • 4.
    Business Data Governance– Overview Master Data Management DW & BI Management Data Quality Management Metadata Management Data Security Management Data Architecture Management Data Development Misconception Reality  It's an IT responsibility Data Governance requires a partnership between Business, Technology, and Operations  One size fits all The organization, processes, and technology must be tailored to fit the culture and leverage existing governance structures and technology  It can succeed through a grass roots bottoms up effort Success requires executive advocacy and sponsorship  It's about having the right tools Data governance requires the integration of organization, processes, and technology tools  It can be an add on responsibility that doesn't need to be measured or rewarded Data stewardship may require full time staff commitment. If the role is not measured or rewarded, the result will be ineffective governance action  It’s a big bang implementation Standing up data governance structures is an evolutionary process that requires effective change management Data governance is the orchestration of people, process, and technology to enable the leveraging of data as an enterprise asset through a well-defined organizational structure, policies, rules, decision rights and accountabilities for decision making and management of Master Data.
  • 5.
    Data Governance –180 Day Plan •Create the appropriate review and escalation methods for managing data quality and integrity •Enable the linkage between business process/data owners who “champion” data and metrics with the data architects and data stewards who manage the transaction level detail Governance & Stewardship •Integrate roles across functions (e.g., data cleansing, data architects, data stewards, process/data owners) •Understand the needs of the consumer of the data and connect appropriately People/ Organization •Define end to end, consistent processes across all data types, linking transaction level data with Management Information •Define proper controls to manage data quality and integrity on a sustainable basis Process •Identify Tools for cleansing, mapping, identifying anomalies, etc. •Leverage data management tools and infrastructure that have rapid scalability and functionality •Define consistent architecture that enables “one version of the truth” Technology
  • 6.
    MDM BI Data Cleansing MDMBI Product MDM BI Customer MDM BI Vendor MDM BI Operational MDM BI Financial MDM BI Stat/Mgmt Rptg EBPM - PTP EBPM - PTD EBPM - OTC EBPM - RTR EBPM - FTP As requirements for MDM and KPIs are assembled, PRIORITIZATION and sequence can be further refined Outcomes MDM BI – Discovery Approach Enterprise Data Model Master Data Models Required KPIs
  • 7.
    MDM BI –Deployment Approach Discovery Analyze / Define Design Deliverable • Conduct Discovery Sessions • Define Solution Scope • Define Solution Concept • Define General System Concept • Describe Potential Impact • Plan Project • Analyze Guidance Architecture • Analyze Data Architecture • Create Data Schema Map • Assess Data Quality • Analyze System Architecture • Analyze System Requirements • Design Guidance Architecture • Design Data Architecture • Design System Architecture Design Application Specification • Design Data Migration • Design Human Transition Support Activity • Discovery Summary • Solution Scope • Solution Concept • System Concept • Impact Summary • Project Plan • Organizational Model • Guidance Model • Data Model • Data Schema Map • Data Quality Assessment • System Interaction Diagram • System Requirements Summary • Organizational Model • Guidance Model Data Model • System Interaction Diagram • Application Specification Data Migration Plan • Organization Change Management Plan • Training Plan • Discovery & Analyze initiated via common stakeholder interviews • Design & Build executed on Global Template • SAP Deployments will review efforts from previous phases but largely be testing and refining exercise Build Test Deploy Sustain
  • 8.
    MDM – ImplementationApproaches Consolidation Registry Coexistence Centralized For Reporting, analysis, and central reference Mainly for real-time central reference For harmonization across databases and for central reference Acts as system of record to support transactional activity Matches and physically stores a consolidated view of master data Matches and links to create a “skeleton” system of record Matches and physically stores a consolidated view of master data Matches and physically stores the up-to-date consolidated view of master data Updated after the event and not guaranteed up-to-date; authoring remains distributed Physically stores the Global ID, links to data in source systems and transformations Updated after the event and not guaranteed up-to-date; authoring remains distributed Supports transactional applications directly – both new and legacy – typically through SOA interfaces No publish and subscribe; not used for transactions but could be used for reference Virtual consolidated view is assembled dynamically and is often read-only; authoring remains distributed Publishes the consolidated view; not usually used for transactions but could be used for reference Central authoring of master data Analytical Focus Operational Focus Operational Focus Operational Focus System of Reference System of Reference System of Reference System of Record
  • 9.
    Enterprise Master Data Model Legacy ERP MDM People MDMProcesses MDM Tools Transformation Certified Master Data 2 3 4 1. Non-Certified Data from the Legacy Systems is ingested and transformed into the Enterprise Master Data Model built during the Discovery Phase 2. The Enterprise Data Model is adjusted as necessary and cleansed data is pushed back to the Legacy ERP System 3. Certified Master Data is produced and the required refinements to processes and data architecture are made to enable downstream consumption 4. All of this is enabled by dedicated MDM personnel, utilizing MDM tools and processes MDM – Pre-Cleanse & Deployment Process 1 Non-Certified Data
  • 10.
    BI – OwnershipStructure Sandbox (50%) [user created content] Shared (30%) [user created and shared content] Production (20%) Gather Data Visualize PublishConsume Ideate Business Users Require- ments Profile Data DesignDevelop Test IT Sandbox Environment • Business users author and use BI content with no constraints or limitations. This is where data exploration, discovery, and what-if analyses happen. • Tools and technologies: Microsoft Office • IT involvement is strictly limited to infrastructure and tools support plus monitoring to identify usage patterns, commonalities, and opportunities (using BI on BI) for potential production hardening. • Content produced here is used in individual tasks and low-risk applications. Shared Environment • Business users share and collaborate on BI content with their colleagues. • Tools and technologies: SharePoint BI, Office 365 • IT steps up monitoring and now watches for red flags (too much data, too many users, too critical or risky applications) and opportunities (using BI on BI) for production hardened BI Content. • Content produced here is shared within departments and workgroups. Low-risk, low- criticality decisions can be made based on this content. Production Environment • Business uses and authors BI content within the limitations and constraints of the enterprise data model, standards, policies, rules, guidelines, etc. • Tools and technologies: EDW, Visual Studio, SharePoint BI, Office 365 • Owned, run, and managed by IT. IT Benefits • Backlog Reduction – only heavily-used, complex, or critical applications come to IT for production-hardening. • Requirements already defined; Project Lifecycle is greatly reduced; Enhancements during testing cycle minimized. • Shadow IT is embraced as a competitive advantage; however, using the strategic technology stack defined by IT. Business Benefits • Business users are empowered to create BI content on their own schedule without any constraints or limitations – at the speed of business innovation. • They modify the model and visualizations through iteration until the requirements are identified and met.
  • 11.
    Hadoop = DataLake • Land all data in Hadoop as-is from any source • Enables Analytics Sandbox • Enables MDM Pre-Processing • Enables EDW Population with Relevant Data • Enables Application Access via API Layer (including 3rd party developers) Actively Archive from EDW to Hadoop • Little-Used Historic EDW Data resides in Hadoop (lower cost storage) • Define an archive strategy for various data types Enable business analytics • Identify tools, methods, and security requirements for interaction with the distributed file system • Introduce exploratory analytics without jeopardizing SLAs • Introduce new machine learning, or data mining techniques on years worth of data Enable BU Innovation • BU Teams continue to innovate with their business users within the Enterprise Framework – ingestions driven by BU OR Enterprise requirements • Data Scientists and Analysts can access all Data for Analytics • BU IT & Business Teams can access all Data for Visualizations with proper security • Enterprise Data Model, HDFS Standards, and Access Methods extensible to manage localizations at the BU level and below BI – Data Flows Data Sources/Transports Transaction Data Customer Data External Data Industry Data Sensor Data DB Files REST JMS HTTP SOAP Hadoop Compute + storage … … … … … … … … … … … … … … Compute + storage supporting technologies& packages EDW BI Tools & Applications Query & Visualization Tools JDBC/ODBC Compliant Tools & Applications Analytic & Reporting Tools R Mahout Excel Excel PowerPoint Power View MDM API Layer BU1 BU2 BUn BU1 BU2 BUn BU1 BU2 BUn
  • 12.
    Establish, Maintain, and PeriodicallyReview and Recommend Changes to Data Governance Policies, Standards, Guidelines, and Procedures The Team responsible to develop the strategy, govern the tools selected to acquire and transform relevant data into knowledge to drive business decisions and actions to achieve desired results. In addition, the resulting information has to be tailored to – and distributed to – the appropriate levels of management and operations in a timely manner to be most effective. In some cases BI Execution of Reporting and Analytics is performed as well. Provide Quality Assurance – Oversight, Monitor, Report Results to Data Governance Council VP MDM & BI Business Governance Leader Data Stewards by Domain Data Stewards by Business Unit Data Governance Leader Data Quality Data Architecture Data Conversions Big Data Architect BI Governance Leader BI Leads BI Visualization Developers - Enterprise BI ETL Developers - Enterprise BI Developers – Business Units Technology & Tools Leader Technology SMEs DBAs System SMEs • Develop and Deliver Data Governance Program Educational, Awareness & Mentoring Materials • Assist in Defining Data Quality Metrics for Periodic Release • Support Data Quality Issue Analysis and Remediation for “Strategic” Data • Oversee Enterprise Data Governance Program Development / Architect Solution & Framework • Administer the Program including facilitate the Data Governance Council meetings • Provide the Agenda for the Data Governance Council Meetings to the Approved by Council Owner Pre- Meeting • Facilitate Data Governance Organization, Tactical & Operational Stewards, the Data Governance Council Involvement • Conduct Audits to Ensure that Policies, Procedures and Metrics are in Place for Maintaining/Improving the Program Functionally Aligned Roles Organizationally Aligned Roles Sample Organization