An introduction to Data Quality Rule development
Data to Value Ltd.
42-44 Bishopsgate
London
EC2N 4AH
www.datatovalue.co.uk
What are DQ Rules?
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
 Automated data quality checks.
 Essential part of any coordinated
approach to Information Management.
 Pre or Post input.
 Many ways to group, classify & present
rules & results.
How are they used & what are benefits?
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
 Cost savings.
 Regulatory Compliance.
 Information Security.
 Risk reductions.
 Enhanced Customer Relationship Management (CRM)
US0378331005
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
Example of a basic DQ rule
ISIN – International Securities Identifier Number
12 Characters in
length
Alphanumeric
Begins with an ISO
country code
Contains a check
digit
May contain a
related code e.g.
CUSIP
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
Advanced DQ rules
 These can involve:
- Checking across data fields – e.g. if field
A is X, field B must be A, B or C.
- Matrix Lookups / transformations – e.g.
credit rating comparisons.
- Multiple transformation / validation
steps.
- Validation against external sources.
- More advanced logic – fuzzy matching
etc.
How to start developing rules
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
 Typical implementation cycle resembles:
- Discovery & analysis.
- Testing & enhancement
- Definition.
- Implementation.
- Review.
 What are alternatives to DIY?
- Data Quality rule repositories & bureaus.
- Standards bodies – e.g. ISO.
- Hire an expert consultancy.
Data to Value’s DQ Rule service
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk
 Full on/offsite rule research,
development & implementation
service.
 All rules documented in plain English
using a specified metadata standard
e.g. your internal Data Dictionary.
 Tailored presentation of results at your
required level of detail – summary
dashboards and/or in depth KPIs.
 Choice of Data to Value’s preferred
tools or your existing DQ analysis tools
(BI tools, profiling tools, reporting tools
etc.)
8 0 %
8 5 %
9 0 %
9 5 %
1 0 0 %
In t e g rity
C o m p le t e n e s s
U n iq u e n e s sC o n s is t e n c y
C o n f o rm ity
7 5 %
8 0 %
8 5 %
9 0 %
9 5 %
1 0 0 %
M a y Ju n e J u ly A u g u s t
O v e r a ll Q u a lit y C o m p le t e n e s s C o n s is t e n c y
C o n fo r m it y U n iq u e n e s s In te g r it y
-
1 0 ,0 0 0
2 0 ,0 0 0
3 0 ,0 0 0
4 0 ,0 0 0
5 0 ,0 0 0
6 0 ,0 0 0
7 0 ,0 0 0
8 0 ,0 0 0
9 0 ,0 0 0
1 0 0 ,0 0 0
M a y Ju n e J u ly A u g u s t D e p t A D e p t B D e p t C D e p t D
Data Quality Scorecard – Product Data May 2013
Commentary:
- Data coverage of priority field
A increased from 30% to 80%.
- Tier 1 incidents down 30%
- Data Dependency X now live.
- Data Maintenance
requirement reduced by 30%
over 3 months to date.
- 2 additional power users
trained in DQ tool.
- 2 additional Data Stewards
within Product Master Data Set.
Status =
Green
May DQ dimensions
Latest dependenciesData Maintenance activity
Monthly DQ dimensions
About us
Specialist Information Management Consultancy offering services within:
Over 40 years combined Information Management experience as Executives at blue chip organisations.
Experts at bridging understanding gap between 'the business' and technology within the Information Management space.
Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH
www.datatovalue.co.uk

Data Quality Rules introduction

  • 1.
    An introduction toData Quality Rule development Data to Value Ltd. 42-44 Bishopsgate London EC2N 4AH www.datatovalue.co.uk
  • 2.
    What are DQRules? Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk  Automated data quality checks.  Essential part of any coordinated approach to Information Management.  Pre or Post input.  Many ways to group, classify & present rules & results.
  • 3.
    How are theyused & what are benefits? Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk  Cost savings.  Regulatory Compliance.  Information Security.  Risk reductions.  Enhanced Customer Relationship Management (CRM)
  • 4.
    US0378331005 Data to ValueLtd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk Example of a basic DQ rule ISIN – International Securities Identifier Number 12 Characters in length Alphanumeric Begins with an ISO country code Contains a check digit May contain a related code e.g. CUSIP
  • 5.
    Data to ValueLtd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk Advanced DQ rules  These can involve: - Checking across data fields – e.g. if field A is X, field B must be A, B or C. - Matrix Lookups / transformations – e.g. credit rating comparisons. - Multiple transformation / validation steps. - Validation against external sources. - More advanced logic – fuzzy matching etc.
  • 6.
    How to startdeveloping rules Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk  Typical implementation cycle resembles: - Discovery & analysis. - Testing & enhancement - Definition. - Implementation. - Review.  What are alternatives to DIY? - Data Quality rule repositories & bureaus. - Standards bodies – e.g. ISO. - Hire an expert consultancy.
  • 7.
    Data to Value’sDQ Rule service Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk  Full on/offsite rule research, development & implementation service.  All rules documented in plain English using a specified metadata standard e.g. your internal Data Dictionary.  Tailored presentation of results at your required level of detail – summary dashboards and/or in depth KPIs.  Choice of Data to Value’s preferred tools or your existing DQ analysis tools (BI tools, profiling tools, reporting tools etc.) 8 0 % 8 5 % 9 0 % 9 5 % 1 0 0 % In t e g rity C o m p le t e n e s s U n iq u e n e s sC o n s is t e n c y C o n f o rm ity 7 5 % 8 0 % 8 5 % 9 0 % 9 5 % 1 0 0 % M a y Ju n e J u ly A u g u s t O v e r a ll Q u a lit y C o m p le t e n e s s C o n s is t e n c y C o n fo r m it y U n iq u e n e s s In te g r it y - 1 0 ,0 0 0 2 0 ,0 0 0 3 0 ,0 0 0 4 0 ,0 0 0 5 0 ,0 0 0 6 0 ,0 0 0 7 0 ,0 0 0 8 0 ,0 0 0 9 0 ,0 0 0 1 0 0 ,0 0 0 M a y Ju n e J u ly A u g u s t D e p t A D e p t B D e p t C D e p t D Data Quality Scorecard – Product Data May 2013 Commentary: - Data coverage of priority field A increased from 30% to 80%. - Tier 1 incidents down 30% - Data Dependency X now live. - Data Maintenance requirement reduced by 30% over 3 months to date. - 2 additional power users trained in DQ tool. - 2 additional Data Stewards within Product Master Data Set. Status = Green May DQ dimensions Latest dependenciesData Maintenance activity Monthly DQ dimensions
  • 8.
    About us Specialist InformationManagement Consultancy offering services within: Over 40 years combined Information Management experience as Executives at blue chip organisations. Experts at bridging understanding gap between 'the business' and technology within the Information Management space. Data to Value Ltd, 42-44 Bishopsgate, London EC2N 4AH www.datatovalue.co.uk