Operationalizing Data Governancethrough Data Quality ControlDavid LoshinKnowledge Integrity, Inc.www.knowledge-integrity.c...
Linking Organization to Practicep  Mapping business value drivers todata expectations and objectives,granting oversight a...
Business Value and Data Dependence© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63503ExpensesRiskMan...
Sources of Information Policy© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63504ExpensesRiskManageme...
Data Management ChallengesData Useand ReUseDataRequirementsReinterpretingSemanticsMeasurementTriage andRemediationImpactAs...
The Data Policy Lifecycle: Actualizing Governancep  Refinement of business requirements for facets of informationutilityp...
Information Policies and Data GovernanceIntegratedDataGovernanceManage UserRequirementsDataDiscoverySharedSemanticsEmbedde...
Managing the Quality of Business Metadatap  Many sources of entity concepts and businessterms may conflict with each othe...
Data Discoveryp  Data Discovery enables thesetypes of questions to beanswered:n  What data sets are available?n  What e...
AttributeFirst d 4 6 yLast f 6 2 hStreet d 4 7 nCity a 0 2 oStateValue CountA 12000I 10000L 7655X 3208N 120M 8Data Quality...
Data Quality: Expectations, Rules, and Monitoringp  Data quality rules can be used to monitor conformance to datapolicies...
Data Quality Rules: Measures and Thresholdsp  Provide specificn  Measuresn  Methods of measurementn  Units of measures...
© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63501313Data Quality Controlp  Controls measureobserv...
Instituting Inspection Using Data Quality Rulesp  Apply tools and techniques for measuring conformance to datarules (usin...
Instituting Inspection Using Data Quality Rulesp  Data quality expectations are inspected and any emergingissues are iden...
Instituting Inspection Using Data Quality Rulesp  Different events can be triggered by a data failure, such asnotificatio...
Instituting Inspection Using Data Quality Rulesp  Or logging the failure in a Data Quality Incident ManagementSystem and ...
Instituting Inspection Using Data Quality Rulesp  Effectiveness demonstrated when:n  Control events occur when data fail...
Integrating Data Quality Reporting with Governance© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-6350...
Tools & Processes: Operationalizing Data Governancep  Methods and tools for data discovery: profilingdata, statistical an...
Questions and Open Discussionp  www.knowledge-integrity.comp  If you have questions,comments, or suggestions,please cont...
Upcoming SlideShare
Loading in …5
×

Loshin operationalizingdatagovernance

274 views

Published on

  • Be the first to comment

Loshin operationalizingdatagovernance

  1. 1. Operationalizing Data Governancethrough Data Quality ControlDavid LoshinKnowledge Integrity, Inc.www.knowledge-integrity.com1© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-6350
  2. 2. Linking Organization to Practicep  Mapping business value drivers todata expectations and objectives,granting oversight andaccountability, and verifyingperformance of compliance withcorporate information policiesn  Processes prescribed for operations,n  Procedures for day-to-dayobservancen  Oversight for verifying compliance© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63502AccountabilityData PoliciesProcesses &Best PracticesInformationStandardsBusinessPolicyRoles andresponsibilitiesProgram managementData policies andstandardsBusiness intelligenceBusiness terminologyData qualityAuditability
  3. 3. Business Value and Data Dependence© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63503ExpensesRiskManagementRevenue CustomerExperiencePerformancep  Business policies,corporate mission, andstrategic performanceobjectives can betranslated intodimensions of valuep  These criteria are usedfor prioritizing effort inrelation to maximizingvaluep  Data governance helpsestablish therelationship betweenvalue drivers andinformation utility
  4. 4. Sources of Information Policy© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63504ExpensesRiskManagementRevenue CustomerExperiencePerformance•  Customer lifetimevalue analysis•  Voice of thecustomer•  Satisfaction surveys•  Asset productivityanalysis•  Human capitalperformance•  Regulations•  Operational risk•  Market factors•  Customer acquisitionand retention•  Investmentopportunities•  Product performance•  Spend analysis•  Commodity risk•  Cost management
  5. 5. Data Management ChallengesData Useand ReUseDataRequirementsReinterpretingSemanticsMeasurementTriage andRemediationImpactAssessment© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63505
  6. 6. The Data Policy Lifecycle: Actualizing Governancep  Refinement of business requirements for facets of informationutilityp  Specification for data quality measures and level ofacceptabilityp  Determination of functional requirements to facilitatecontinuous compliance© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63506Determinationof needDrafting aPolicyPolicy Review& ApprovalDesign &DevelopmentMarketing Deployment
  7. 7. Information Policies and Data GovernanceIntegratedDataGovernanceManage UserRequirementsDataDiscoverySharedSemanticsEmbeddedValidation© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63507•  Business metadata management•  Data quality discovery and assessment•  Specifying data quality rules•  Inspection, monitoring, measurement•  Managing data lineage
  8. 8. Managing the Quality of Business Metadatap  Many sources of entity concepts and businessterms may conflict with each otherp  The data governance framework mustfacilitate the collection, documentation, andharmonization of business terms© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63508PoliciesSystem DocsProcessesModelsStandardsApplicationsBusiness RulesProfilingEtc.Entity ConceptsBusinessTermsDefinitionContextualMeaning… …DefinitionContextualMeaningDefinitionContextualMeaningDefinitionContextualMeaning
  9. 9. Data Discoveryp  Data Discovery enables thesetypes of questions to beanswered:n  What data sets are available?n  What entities are embedded?n  What data elements areavailable?n  How is the data accessed?n  What are the qualityconstraints?p  The results can be shared viaa platform for managingsemantic metadata© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63509DataIntegration
  10. 10. AttributeFirst d 4 6 yLast f 6 2 hStreet d 4 7 nCity a 0 2 oStateValue CountA 12000I 10000L 7655X 3208N 120M 8Data Quality Assessmentp  Analysis of data sets, records, data elements, and data valueston  Identify potential anomaliesn  Determine business impactsn  Evaluate dimensions for measurement of data quality© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635010Analysis
  11. 11. Data Quality: Expectations, Rules, and Monitoringp  Data quality rules can be used to monitor conformance to datapoliciesp  Conformance can be measured, thresholded, and reported ateach handoff location in the processing streamp  Specific failures can generate events as directed by DataQuality Service Level Agreementsp  Static auditing: measurement applied to a “static” data setn  Examples: SQL queries, data profiling toolsp  Inlined monitoring: measurement performed within a processflown  Example: edit checks, dynamic monitorsp  All measurements are compared against acceptabilitythresholdsp  Acceptability threshold is related to the degree of impact1111© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-6350
  12. 12. Data Quality Rules: Measures and Thresholdsp  Provide specificn  Measuresn  Methods of measurementn  Units of measuresn  Levels of acceptability© 2012 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635012
  13. 13. © 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-63501313Data Quality Controlp  Controls measureobservance of dataexpectations based oninformation policies andcorresponding data rulesp  Those rules are refinedbased on an analysis of thedata dependencies anddefined expectationsp  Controls are placed atrelevant locations withinthe process streamProducerProcessConsumerProcessAs data is handed offbetween process tasks,controls validate accuracy,completeness, consistency,timeliness against definedexpectations
  14. 14. Instituting Inspection Using Data Quality Rulesp  Apply tools and techniques for measuring conformance to datarules (using data profiling and data monitoring tools):© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635014
  15. 15. Instituting Inspection Using Data Quality Rulesp  Data quality expectations are inspected and any emergingissues are identified:© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635015
  16. 16. Instituting Inspection Using Data Quality Rulesp  Different events can be triggered by a data failure, such asnotifications to data stewards:© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635016
  17. 17. Instituting Inspection Using Data Quality Rulesp  Or logging the failure in a Data Quality Incident ManagementSystem and score card:© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635017
  18. 18. Instituting Inspection Using Data Quality Rulesp  Effectiveness demonstrated when:n  Control events occur when data failure events take place,n  The proper mitigation or remediation actions are performed,n  The corrective actions to correct the problem and eliminate its rootcause are performed within a reasonable time frame, andn  A control event for the same issue is never triggered furtherdownstreamp  Measurements can be aggregated over time into performancemetrics© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635018
  19. 19. Integrating Data Quality Reporting with Governance© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635019ProcessingStageProcessingStageProcessingStageProcessingStage
  20. 20. Tools & Processes: Operationalizing Data Governancep  Methods and tools for data discovery: profilingdata, statistical analysis of values, and modelevaluationp  Metadata management through a central platformfor knowledge capture and communicationp  End-to-end visibility of lineage for structure,semantics, and use across enterprisep  Data Quality assessmentp  Integrated data quality controlp  Inspection, monitoring, and reporting© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635020
  21. 21. Questions and Open Discussionp  www.knowledge-integrity.comp  If you have questions,comments, or suggestions,please contact meDavid Loshin301-754-6350loshin@knowledge-integrity.com© 2013 Knowledge Integrity, Inc.www.knowledge-integrity.com(301)754-635021www.dataqualitybook.comwww.mdmbook.com

×