Loshin operationalizingdatagovernance

Operationalizing Data Governance
through Data Quality Control
David Loshin
Knowledge Integrity, Inc.
www.knowledge-integrity.com
1© 2013 Knowledge Integrity, Inc.
(301)754-6350

Linking Organization to Practice
p  Mapping business value drivers to
data expectations and objectives,
granting oversight and
accountability, and verifying
performance of compliance with
corporate information policies
n  Processes prescribed for operations,
n  Procedures for day-to-day
observance
n  Oversight for verifying compliance
© 2013 Knowledge Integrity, Inc.
(301)754-6350
2
Accountability
Data Policies
Processes &
Best Practices
Information
Standards
Business
Policy
Roles and
responsibilities
Program management
Data policies and
standards
Business intelligence
Business terminology
Data quality
Auditability

Business Value and Data Dependence
(301)754-6350
3
Expenses
Risk
Management
Revenue Customer
Experience
Performance
p  Business policies,
corporate mission, and
strategic performance
objectives can be
translated into
dimensions of value
p  These criteria are used
for prioritizing effort in
relation to maximizing
value
p  Data governance helps
establish the
relationship between
value drivers and
information utility

Sources of Information Policy
(301)754-6350
4
Expenses
Risk
Management
Revenue Customer
Experience
Performance
•  Customer lifetime
value analysis
•  Voice of the
customer
•  Satisfaction surveys
•  Asset productivity
analysis
•  Human capital
performance
•  Regulations
•  Operational risk
•  Market factors
•  Customer acquisition
and retention
•  Investment
opportunities
•  Product performance
•  Spend analysis
•  Commodity risk
•  Cost management

Data Management Challenges
Data Use
and ReUse
Data
Requirements
Reinterpreting
Semantics
Measurement
Triage and
Remediation
Impact
Assessment
(301)754-6350
5

The Data Policy Lifecycle: Actualizing Governance
p  Refinement of business requirements for facets of information
utility
p  Specification for data quality measures and level of
acceptability
p  Determination of functional requirements to facilitate
continuous compliance
(301)754-6350
6
Determination
of need
Drafting a
Policy
Policy Review
& Approval
Design &
Development
Marketing Deployment

Information Policies and Data Governance
Integrated
Data
Governance
Manage User
Requirements
Data
Discovery
Shared
Semantics
Embedded
Validation
(301)754-6350
7
•  Business metadata management
•  Data quality discovery and assessment
•  Specifying data quality rules
•  Inspection, monitoring, measurement
•  Managing data lineage

Managing the Quality of Business Metadata
p  Many sources of entity concepts and business
terms may conflict with each other
p  The data governance framework must
facilitate the collection, documentation, and
harmonization of business terms
(301)754-6350
8
Policies
System Docs
Processes
Models
Standards
Applications
Business Rules
Profiling
Etc.
Entity Concepts
Business
Terms
Definition
Contextual
Meaning
… …
Definition
Contextual
Meaning
Definition
Contextual
Meaning
Definition
Contextual
Meaning

Data Discovery
p  Data Discovery enables these
types of questions to be
answered:
n  What data sets are available?
n  What entities are embedded?
n  What data elements are
available?
n  How is the data accessed?
n  What are the quality
constraints?
p  The results can be shared via
a platform for managing
semantic metadata
(301)754-6350
9
DataIntegration

Attribute
First d 4 6 y
Last f 6 2 h
Street d 4 7 n
City a 0 2 o
State
Value Count
A 12000
I 10000
L 7655
X 3208
N 120
M 8
Data Quality Assessment
p  Analysis of data sets, records, data elements, and data values
to
n  Identify potential anomalies
n  Determine business impacts
n  Evaluate dimensions for measurement of data quality
(301)754-6350
10
Analysis

Data Quality: Expectations, Rules, and Monitoring
p  Data quality rules can be used to monitor conformance to data
policies
p  Conformance can be measured, thresholded, and reported at
each handoff location in the processing stream
p  Specific failures can generate events as directed by Data
Quality Service Level Agreements
p  Static auditing: measurement applied to a “static” data set
n  Examples: SQL queries, data profiling tools
p  Inlined monitoring: measurement performed within a process
flow
n  Example: edit checks, dynamic monitors
p  All measurements are compared against acceptability
thresholds
p  Acceptability threshold is related to the degree of impact
1111© 2013 Knowledge Integrity, Inc.
(301)754-6350

Data Quality Rules: Measures and Thresholds
p  Provide specific
n  Measures
n  Methods of measurement
n  Units of measures
n  Levels of acceptability
(301)754-6350
12

(301)754-6350
1313
Data Quality Control
p  Controls measure
observance of data
expectations based on
information policies and
corresponding data rules
p  Those rules are refined
based on an analysis of the
data dependencies and
defined expectations
p  Controls are placed at
relevant locations within
the process stream
Producer
Process
Consumer
Process
As data is handed off
between process tasks,
controls validate accuracy,
completeness, consistency,
timeliness against defined
expectations

Instituting Inspection Using Data Quality Rules
p  Apply tools and techniques for measuring conformance to data
rules (using data profiling and data monitoring tools):
(301)754-6350
14

p  Data quality expectations are inspected and any emerging
issues are identified:
(301)754-6350
15

p  Different events can be triggered by a data failure, such as
notifications to data stewards:
(301)754-6350
16

p  Or logging the failure in a Data Quality Incident Management
System and score card:
(301)754-6350
17

p  Effectiveness demonstrated when:
n  Control events occur when data failure events take place,
n  The proper mitigation or remediation actions are performed,
n  The corrective actions to correct the problem and eliminate its root
cause are performed within a reasonable time frame, and
n  A control event for the same issue is never triggered further
downstream
p  Measurements can be aggregated over time into performance
metrics
(301)754-6350
18

Integrating Data Quality Reporting with Governance
(301)754-6350
19
Processing
Stage
Processing
Stage
Processing
Stage
Processing
Stage

Tools & Processes: Operationalizing Data Governance
p  Methods and tools for data discovery: profiling
data, statistical analysis of values, and model
evaluation
p  Metadata management through a central platform
for knowledge capture and communication
p  End-to-end visibility of lineage for structure,
semantics, and use across enterprise
p  Data Quality assessment
p  Integrated data quality control
p  Inspection, monitoring, and reporting
(301)754-6350
20

Questions and Open Discussion
p  www.knowledge-integrity.com
p  If you have questions,
comments, or suggestions,
please contact me
David Loshin
301-754-6350
loshin@knowledge-integrity.com
(301)754-6350
21
www.dataqualitybook.com
www.mdmbook.com

Loshin operationalizingdatagovernance

Recommended

Recommended

More Related Content

Similar to Loshin operationalizingdatagovernance

Similar to Loshin operationalizingdatagovernance (20)

More from Taldor Group

More from Taldor Group (12)

Loshin operationalizingdatagovernance