SlideShare a Scribd company logo
1 of 7
THE DEPENDENT
RELATIONSHIP OF
DATA QUALITY & DATA
GOVERNANCE
By Bill Sharp
DATA GOVERNANCE MISSION
ESSENTIALS
** No matter the driving initiative behind a data governance effort there
needs to be a three part process which includes:
1. Rule Design
2. Issue Resolution
3. Monitoring and Enforcement (of rules)
** FROM THE DATA GOVERNANCE INSTITUTE “BASICS”
HTTP://WWW.DATAGOVERNANCE.COM/ADG_DATA_GOVERNANCE_BASICS.HTML
RULE DESIGN
In my opinion there are two ways to develop governance rules.
1. Top Down
2. Bottom Up
The “top down” approach is one where rules are developed from the stand point of what data
should look like. These are idealistic rules in nature because they are heavily rooted in strict
adherence to business rules. In other words, they do not accommodate the legitimate
exceptions to rules. Also, these type of rules tend to turn less than perfect data into a violation
and produce voluminous exception reports.
The “bottom up” approach is one where rules are developed from extensive analysis of the
data, i.e. data profiling (a fundamental activity in data quality). This approach tends to be more
practical in that it deals directly with data related issues, rather than business rules, such as
common data entry and/or migration issues.
From my experience the “bottom up” approach tends to be more efficient and produce more
rules that improve the data over time. Another benefit of the “bottom up” approach is that data
profiling provides hard numbers on the occurrence of a specific violation. As a consequence,
profiling allows the governance committee to prioritize rule enforcement by exception volume
and create the most benefit for the effort.
ISSUE RESOLUTION
IR is one of those phases where it is not obvious what role data quality plays until
you use quality practices in the issue resolution process. Here are three ways to use
data quality in governance issue resolution:
1. Reviewing data profiles can provide hard facts around data value distributions
which can in turn provide direction on how to correct, or resolve, a data issue
2. Reviewing data quality scorecards can provide details around the volume of an
exception which can in turn provide direction on whether an issue resolution will
return a high yield on the remediation investment
3. Typically profiles and scorecards contain a source component or identifier. This
identifier can enable the governance committee to decide on who the
accountable parties are for developing the issue resolution
MONITORING AND
ENFORCEMENTSimply put … there is no rule monitoring or enforcement without data quality!
M&E is essentially a before and after process where the cycle in the
illustration below is continuously reviewed by the data governance
committee
Transform
rules into
scorecard
metrics
Examine
metric
exception
trends, volume
s and
instances
Build data
remediation
jobs to rectify
the exceptions
1. Transform data governance rules into scorecard metrics. This
enables monitoring and measuring data exceptions. At times this
can be straight forward because some rules are heavily data
intensive and can be traced to a data element. Other times this
process involves the decomposition of a business rule to it’s core
data element in order to transform.
2. Examining scorecard trends (is the data getting better or worse?),
volumes (how big is the issue?), and specific exception instances
(how is the rule being violated?). This is what data governance
committees should be talking about at almost every meeting!!
3. Building the data remediation processes is the post violation
enforcement of rules and a key component in the value the data
governance promises. Typically, this involves correcting the
exception to the defined and accepted value.
SUMMARY
o Data quality should play an active role in the core of any data governance
program
o Data profiling can provide direction on data issues, their volume
and, ultimately, the priority of data governance focus
o Data quality scorecards are the measuring and monitoring component of a
data governance program that provide clear evidence for governance
effectiveness
o Data quality remediation is the fundamental mechanism for turning data
governance issues into success stories
THANK YOU!
Address comments or
questions to
sharp@thedataqualitychronicle.
org

More Related Content

What's hot

DataEd Online: Let's Talk Metadata Strategies and Successes
DataEd Online: Let's Talk Metadata Strategies and SuccessesDataEd Online: Let's Talk Metadata Strategies and Successes
DataEd Online: Let's Talk Metadata Strategies and SuccessesDATAVERSITY
 
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance Chief
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance ChiefRWDG Slides: Corporate Data Governance - The CDO is the Data Governance Chief
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance ChiefDATAVERSITY
 
The Chief Data Officer Agenda: Metrics for Information and Data Management
The Chief Data Officer Agenda: Metrics for Information and Data ManagementThe Chief Data Officer Agenda: Metrics for Information and Data Management
The Chief Data Officer Agenda: Metrics for Information and Data ManagementDATAVERSITY
 
Real-World Data Governance: Data Governance Policy - Components and Content
Real-World Data Governance: Data Governance Policy - Components and ContentReal-World Data Governance: Data Governance Policy - Components and Content
Real-World Data Governance: Data Governance Policy - Components and ContentDATAVERSITY
 
RWDG Slides: Data Architecture Is Data Governance
RWDG Slides: Data Architecture Is Data GovernanceRWDG Slides: Data Architecture Is Data Governance
RWDG Slides: Data Architecture Is Data GovernanceDATAVERSITY
 
Data stewardship a primer
Data stewardship   a primerData stewardship   a primer
Data stewardship a primerGed Mirfin
 
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...DATAVERSITY
 
RWDG Webinar: A Data Governance Framework for Smart Data
RWDG Webinar: A Data Governance Framework for Smart DataRWDG Webinar: A Data Governance Framework for Smart Data
RWDG Webinar: A Data Governance Framework for Smart DataDATAVERSITY
 
RWDG Webinar: Achieving Data Quality Through Data Governance
RWDG Webinar: Achieving Data Quality Through Data GovernanceRWDG Webinar: Achieving Data Quality Through Data Governance
RWDG Webinar: Achieving Data Quality Through Data GovernanceDATAVERSITY
 
DataEd Slides: Data Governance Strategies
DataEd Slides: Data Governance StrategiesDataEd Slides: Data Governance Strategies
DataEd Slides: Data Governance StrategiesDATAVERSITY
 
Data Governance Strategies - With Great Power Comes Great Accountability
Data Governance Strategies - With Great Power Comes Great AccountabilityData Governance Strategies - With Great Power Comes Great Accountability
Data Governance Strategies - With Great Power Comes Great AccountabilityDATAVERSITY
 
TDWI checklist 2018 - Data Warehouse Infrastructure
TDWI checklist 2018 - Data Warehouse InfrastructureTDWI checklist 2018 - Data Warehouse Infrastructure
TDWI checklist 2018 - Data Warehouse InfrastructureJeannette Browning
 
Most Common Data Governance Challenges in the Digital Economy
Most Common Data Governance Challenges in the Digital EconomyMost Common Data Governance Challenges in the Digital Economy
Most Common Data Governance Challenges in the Digital EconomyRobyn Bollhorst
 
Real-World Data Governance Webinar: Data Governance Framework Components
Real-World Data Governance Webinar: Data Governance Framework ComponentsReal-World Data Governance Webinar: Data Governance Framework Components
Real-World Data Governance Webinar: Data Governance Framework ComponentsDATAVERSITY
 
The Five Pillars of Data Governance 2.0 Success
The Five Pillars of Data Governance 2.0 SuccessThe Five Pillars of Data Governance 2.0 Success
The Five Pillars of Data Governance 2.0 SuccessDATAVERSITY
 
RWDG Webinar: Mastering and Master Data Governance
RWDG Webinar: Mastering and Master Data GovernanceRWDG Webinar: Mastering and Master Data Governance
RWDG Webinar: Mastering and Master Data GovernanceDATAVERSITY
 
RWDG Webinar: Build Your Own Data Governance Tools
RWDG Webinar: Build Your Own Data Governance ToolsRWDG Webinar: Build Your Own Data Governance Tools
RWDG Webinar: Build Your Own Data Governance ToolsDATAVERSITY
 
Data Governance & Data Steward Certification
Data Governance & Data Steward CertificationData Governance & Data Steward Certification
Data Governance & Data Steward CertificationDATAVERSITY
 
2011 digital trends webinar presentation
2011 digital trends webinar presentation2011 digital trends webinar presentation
2011 digital trends webinar presentationEconsultancy
 
Emerging Data Quality Trends for Governing and Analyzing Big Data
Emerging Data Quality Trends for Governing and Analyzing Big DataEmerging Data Quality Trends for Governing and Analyzing Big Data
Emerging Data Quality Trends for Governing and Analyzing Big DataDATAVERSITY
 

What's hot (20)

DataEd Online: Let's Talk Metadata Strategies and Successes
DataEd Online: Let's Talk Metadata Strategies and SuccessesDataEd Online: Let's Talk Metadata Strategies and Successes
DataEd Online: Let's Talk Metadata Strategies and Successes
 
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance Chief
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance ChiefRWDG Slides: Corporate Data Governance - The CDO is the Data Governance Chief
RWDG Slides: Corporate Data Governance - The CDO is the Data Governance Chief
 
The Chief Data Officer Agenda: Metrics for Information and Data Management
The Chief Data Officer Agenda: Metrics for Information and Data ManagementThe Chief Data Officer Agenda: Metrics for Information and Data Management
The Chief Data Officer Agenda: Metrics for Information and Data Management
 
Real-World Data Governance: Data Governance Policy - Components and Content
Real-World Data Governance: Data Governance Policy - Components and ContentReal-World Data Governance: Data Governance Policy - Components and Content
Real-World Data Governance: Data Governance Policy - Components and Content
 
RWDG Slides: Data Architecture Is Data Governance
RWDG Slides: Data Architecture Is Data GovernanceRWDG Slides: Data Architecture Is Data Governance
RWDG Slides: Data Architecture Is Data Governance
 
Data stewardship a primer
Data stewardship   a primerData stewardship   a primer
Data stewardship a primer
 
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...
Real-World Data Governance: A Different Way of Defining Data Stewards & Stewa...
 
RWDG Webinar: A Data Governance Framework for Smart Data
RWDG Webinar: A Data Governance Framework for Smart DataRWDG Webinar: A Data Governance Framework for Smart Data
RWDG Webinar: A Data Governance Framework for Smart Data
 
RWDG Webinar: Achieving Data Quality Through Data Governance
RWDG Webinar: Achieving Data Quality Through Data GovernanceRWDG Webinar: Achieving Data Quality Through Data Governance
RWDG Webinar: Achieving Data Quality Through Data Governance
 
DataEd Slides: Data Governance Strategies
DataEd Slides: Data Governance StrategiesDataEd Slides: Data Governance Strategies
DataEd Slides: Data Governance Strategies
 
Data Governance Strategies - With Great Power Comes Great Accountability
Data Governance Strategies - With Great Power Comes Great AccountabilityData Governance Strategies - With Great Power Comes Great Accountability
Data Governance Strategies - With Great Power Comes Great Accountability
 
TDWI checklist 2018 - Data Warehouse Infrastructure
TDWI checklist 2018 - Data Warehouse InfrastructureTDWI checklist 2018 - Data Warehouse Infrastructure
TDWI checklist 2018 - Data Warehouse Infrastructure
 
Most Common Data Governance Challenges in the Digital Economy
Most Common Data Governance Challenges in the Digital EconomyMost Common Data Governance Challenges in the Digital Economy
Most Common Data Governance Challenges in the Digital Economy
 
Real-World Data Governance Webinar: Data Governance Framework Components
Real-World Data Governance Webinar: Data Governance Framework ComponentsReal-World Data Governance Webinar: Data Governance Framework Components
Real-World Data Governance Webinar: Data Governance Framework Components
 
The Five Pillars of Data Governance 2.0 Success
The Five Pillars of Data Governance 2.0 SuccessThe Five Pillars of Data Governance 2.0 Success
The Five Pillars of Data Governance 2.0 Success
 
RWDG Webinar: Mastering and Master Data Governance
RWDG Webinar: Mastering and Master Data GovernanceRWDG Webinar: Mastering and Master Data Governance
RWDG Webinar: Mastering and Master Data Governance
 
RWDG Webinar: Build Your Own Data Governance Tools
RWDG Webinar: Build Your Own Data Governance ToolsRWDG Webinar: Build Your Own Data Governance Tools
RWDG Webinar: Build Your Own Data Governance Tools
 
Data Governance & Data Steward Certification
Data Governance & Data Steward CertificationData Governance & Data Steward Certification
Data Governance & Data Steward Certification
 
2011 digital trends webinar presentation
2011 digital trends webinar presentation2011 digital trends webinar presentation
2011 digital trends webinar presentation
 
Emerging Data Quality Trends for Governing and Analyzing Big Data
Emerging Data Quality Trends for Governing and Analyzing Big DataEmerging Data Quality Trends for Governing and Analyzing Big Data
Emerging Data Quality Trends for Governing and Analyzing Big Data
 

Similar to The dependent relationship of data quality & data governance

5 Best Practices of Effective Data Quality Management
5 Best Practices of Effective Data Quality Management5 Best Practices of Effective Data Quality Management
5 Best Practices of Effective Data Quality ManagementData Entry India Outsource
 
Making Data Quality a Way of Life
Making Data Quality a Way of LifeMaking Data Quality a Way of Life
Making Data Quality a Way of LifeCognizant
 
Data quality management best practices
Data quality management best practicesData quality management best practices
Data quality management best practicesselinasimpson2201
 
Quality management best practices
Quality management best practicesQuality management best practices
Quality management best practicesselinasimpson2201
 
Data quality management system
Data quality management systemData quality management system
Data quality management systemselinasimpson361
 
Architecting the Framework for Compliance & Risk Management
Architecting the Framework for Compliance & Risk ManagementArchitecting the Framework for Compliance & Risk Management
Architecting the Framework for Compliance & Risk Managementjadams6
 
SDM Presentation V1.0
SDM Presentation V1.0SDM Presentation V1.0
SDM Presentation V1.0KirSinc
 
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...DATAVERSITY
 
Introduction to Data Governance
Introduction to Data GovernanceIntroduction to Data Governance
Introduction to Data GovernanceJohn Bao Vuu
 
Practical Guide to Data Governance Success
Practical Guide to Data Governance SuccessPractical Guide to Data Governance Success
Practical Guide to Data Governance SuccessAmple Insight Inc
 
Data governance, Information security strategy
Data governance, Information security strategyData governance, Information security strategy
Data governance, Information security strategyvasanthi4ever
 
chapter3-220725142737-bf613658.pdf
chapter3-220725142737-bf613658.pdfchapter3-220725142737-bf613658.pdf
chapter3-220725142737-bf613658.pdfMahmoudSOLIMAN380726
 
Chapter 3: Data Governance
Chapter 3: Data Governance Chapter 3: Data Governance
Chapter 3: Data Governance Ahmed Alorage
 
Data processing sunum-lesson 4-mis-dss
Data processing sunum-lesson 4-mis-dssData processing sunum-lesson 4-mis-dss
Data processing sunum-lesson 4-mis-dssUfuk Cebeci
 
Best Practices of Data Governance.pptx
Best Practices of Data Governance.pptxBest Practices of Data Governance.pptx
Best Practices of Data Governance.pptxpreludesyscloudmigra
 
Governance and Architecture in Data Integration
Governance and Architecture in Data IntegrationGovernance and Architecture in Data Integration
Governance and Architecture in Data IntegrationAnalytiX DS
 
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...AnalytixDataServices
 
The Growing Importance of Data Cleaning
The Growing Importance of Data CleaningThe Growing Importance of Data Cleaning
The Growing Importance of Data CleaningCarolineSmith912130
 
What is Data Observability.pdf
What is Data Observability.pdfWhat is Data Observability.pdf
What is Data Observability.pdf4dalert
 

Similar to The dependent relationship of data quality & data governance (20)

5 Best Practices of Effective Data Quality Management
5 Best Practices of Effective Data Quality Management5 Best Practices of Effective Data Quality Management
5 Best Practices of Effective Data Quality Management
 
Making Data Quality a Way of Life
Making Data Quality a Way of LifeMaking Data Quality a Way of Life
Making Data Quality a Way of Life
 
Data quality management best practices
Data quality management best practicesData quality management best practices
Data quality management best practices
 
Quality management best practices
Quality management best practicesQuality management best practices
Quality management best practices
 
Data quality management system
Data quality management systemData quality management system
Data quality management system
 
Architecting the Framework for Compliance & Risk Management
Architecting the Framework for Compliance & Risk ManagementArchitecting the Framework for Compliance & Risk Management
Architecting the Framework for Compliance & Risk Management
 
SDM Presentation V1.0
SDM Presentation V1.0SDM Presentation V1.0
SDM Presentation V1.0
 
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...
How Ally Financial Achieved Regulatory Compliance with the Data Management Ma...
 
Introduction to Data Governance
Introduction to Data GovernanceIntroduction to Data Governance
Introduction to Data Governance
 
Data quality management
Data quality managementData quality management
Data quality management
 
Practical Guide to Data Governance Success
Practical Guide to Data Governance SuccessPractical Guide to Data Governance Success
Practical Guide to Data Governance Success
 
Data governance, Information security strategy
Data governance, Information security strategyData governance, Information security strategy
Data governance, Information security strategy
 
chapter3-220725142737-bf613658.pdf
chapter3-220725142737-bf613658.pdfchapter3-220725142737-bf613658.pdf
chapter3-220725142737-bf613658.pdf
 
Chapter 3: Data Governance
Chapter 3: Data Governance Chapter 3: Data Governance
Chapter 3: Data Governance
 
Data processing sunum-lesson 4-mis-dss
Data processing sunum-lesson 4-mis-dssData processing sunum-lesson 4-mis-dss
Data processing sunum-lesson 4-mis-dss
 
Best Practices of Data Governance.pptx
Best Practices of Data Governance.pptxBest Practices of Data Governance.pptx
Best Practices of Data Governance.pptx
 
Governance and Architecture in Data Integration
Governance and Architecture in Data IntegrationGovernance and Architecture in Data Integration
Governance and Architecture in Data Integration
 
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
 
The Growing Importance of Data Cleaning
The Growing Importance of Data CleaningThe Growing Importance of Data Cleaning
The Growing Importance of Data Cleaning
 
What is Data Observability.pdf
What is Data Observability.pdfWhat is Data Observability.pdf
What is Data Observability.pdf
 

Recently uploaded

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGSujit Pal
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 

Recently uploaded (20)

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAG
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 

The dependent relationship of data quality & data governance

  • 1. THE DEPENDENT RELATIONSHIP OF DATA QUALITY & DATA GOVERNANCE By Bill Sharp
  • 2. DATA GOVERNANCE MISSION ESSENTIALS ** No matter the driving initiative behind a data governance effort there needs to be a three part process which includes: 1. Rule Design 2. Issue Resolution 3. Monitoring and Enforcement (of rules) ** FROM THE DATA GOVERNANCE INSTITUTE “BASICS” HTTP://WWW.DATAGOVERNANCE.COM/ADG_DATA_GOVERNANCE_BASICS.HTML
  • 3. RULE DESIGN In my opinion there are two ways to develop governance rules. 1. Top Down 2. Bottom Up The “top down” approach is one where rules are developed from the stand point of what data should look like. These are idealistic rules in nature because they are heavily rooted in strict adherence to business rules. In other words, they do not accommodate the legitimate exceptions to rules. Also, these type of rules tend to turn less than perfect data into a violation and produce voluminous exception reports. The “bottom up” approach is one where rules are developed from extensive analysis of the data, i.e. data profiling (a fundamental activity in data quality). This approach tends to be more practical in that it deals directly with data related issues, rather than business rules, such as common data entry and/or migration issues. From my experience the “bottom up” approach tends to be more efficient and produce more rules that improve the data over time. Another benefit of the “bottom up” approach is that data profiling provides hard numbers on the occurrence of a specific violation. As a consequence, profiling allows the governance committee to prioritize rule enforcement by exception volume and create the most benefit for the effort.
  • 4. ISSUE RESOLUTION IR is one of those phases where it is not obvious what role data quality plays until you use quality practices in the issue resolution process. Here are three ways to use data quality in governance issue resolution: 1. Reviewing data profiles can provide hard facts around data value distributions which can in turn provide direction on how to correct, or resolve, a data issue 2. Reviewing data quality scorecards can provide details around the volume of an exception which can in turn provide direction on whether an issue resolution will return a high yield on the remediation investment 3. Typically profiles and scorecards contain a source component or identifier. This identifier can enable the governance committee to decide on who the accountable parties are for developing the issue resolution
  • 5. MONITORING AND ENFORCEMENTSimply put … there is no rule monitoring or enforcement without data quality! M&E is essentially a before and after process where the cycle in the illustration below is continuously reviewed by the data governance committee Transform rules into scorecard metrics Examine metric exception trends, volume s and instances Build data remediation jobs to rectify the exceptions 1. Transform data governance rules into scorecard metrics. This enables monitoring and measuring data exceptions. At times this can be straight forward because some rules are heavily data intensive and can be traced to a data element. Other times this process involves the decomposition of a business rule to it’s core data element in order to transform. 2. Examining scorecard trends (is the data getting better or worse?), volumes (how big is the issue?), and specific exception instances (how is the rule being violated?). This is what data governance committees should be talking about at almost every meeting!! 3. Building the data remediation processes is the post violation enforcement of rules and a key component in the value the data governance promises. Typically, this involves correcting the exception to the defined and accepted value.
  • 6. SUMMARY o Data quality should play an active role in the core of any data governance program o Data profiling can provide direction on data issues, their volume and, ultimately, the priority of data governance focus o Data quality scorecards are the measuring and monitoring component of a data governance program that provide clear evidence for governance effectiveness o Data quality remediation is the fundamental mechanism for turning data governance issues into success stories
  • 7. THANK YOU! Address comments or questions to sharp@thedataqualitychronicle. org