• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Data Quality
 

Data Quality

on

  • 844 views

A view of the importance of data quality and how to set about addressing this issue in your business

A view of the importance of data quality and how to set about addressing this issue in your business

Statistics

Views

Total Views
844
Views on SlideShare
835
Embed Views
9

Actions

Likes
0
Downloads
30
Comments
0

2 Embeds 9

http://www.linkedin.com 8
http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Data Quality Data Quality Presentation Transcript

    • “ Making your data a strategic asset” Data Quality - The Key to Successful Analytics and CRM Michael Collins BA(Hons), DipM, MCIM, FIDM Managing Consultant - Database Marketing Counsel Visiting University Lecturer in Database Marketing & CRM D ATABASE M ARKETING C OUNSEL
    • Our Agenda
      • The importance of understanding your core systems
      • How clean should your data be?
      • The impact of a data audit
      • Where does poor data come from?
      • Methodology for improvement
      • Getting employees to ‘live and breath’ data quality
    • Data as a corporate asset
      • Value to the business
      • Invest in the maintenance of an asset
      • Responsibility of all who use it, access it, are involved in its acquisition, storage or maintenance
      • Rules of management
      • Validation
      • Security
      • An appreciating asset – in everyone’s interest
      • Four kinds of quality issues
      • Common data-entry errors
      • Out of date – past its “use-by” date
      • Lack of consistency
      • Unreliable sources
      • Poor quality and integrity of data limits its value
    • Implications of Inaccurate Data
      • There is no substitute for acquiring accurate data - analysis tools can’t compensate for lack of data
      • The more “real time” contact we have with customers, suppliers or employees, the more accurate the data needs to be and the more devastating can be the results of inaccuracy
      • Quality will determine how much of a guide or ‘black & white’ analysis can be reached
    • Implications of Inaccurate Data
      • Skewed campaign planning
      • Improper selections for campaigns
      • Expensive product mistakes
      • Non-delivery of the message (esp. E-mail)
      • Reflection of your business
      • ‘ Junk mail/spam’ tag
      • £££££ Wasted
    • Implications of Inaccurate Data
      • Inaccuracy will
      • annoy customers,
      • suppliers and staff
    • Data Quality Quality Data 1 Profile Understanding 2 Audit Qualification 3 Integrate Consolidation 4 Enrich Improvement 5 Monitor Observation 6 Culture Compliance
    • Typical Framework Source A Source B Source C Sources Extract/Transform/Load Processes Operational CRM Campaign Management External Data BI & Visualisation Rules DATA QUALITY
    • Typical Framework Source A Source B Source C Sources Extract/Transform/Load Processes Operational CRM Campaign Management External Data BI & Visualisation Rules QUALITY DATA
    • Profiling Your Sources
      • Current business processes
      • Tactical activity
      • Enhancement from external sources
      • Business information vendors
      • Purchased lists
      • Marketing partners
    • Sources WARRANTY SURVEYS - Behavioural ENQUIRIES/HELP LINE SALES COMPLAINTS BRANCHES/CHANNELS ACCOUNTS OTHER TOUCH POINTS SMS Social Networking EXTERNAL DATABASE
    • Scoring the Sources
      • Score the data as part of your data strategy
      • Build a model that provides a level of confidence
      • Base the model on known factors
        • Source
        • Recency of update
        • Testing
      • Use the score to determine priorities for enhancement and to inform the business of the level of confidence
      • Strive to improve the level of confidence
    • Review Your Scores Business Services Company
    • Compare Your Scores Business Services Company – No. of Employees
    • Typical Framework Source A Source B Source C Sources Extract/Transform/Load Processes Operational CRM Campaign Management External Data BI & Visualisation Rules HIERARCHY
    • Data Quality Process
      • Data Audit -technology
      • What needs fixing
      • What needs summarizing
      • What derived data is required
      • Attrition - data use by date!
      • How do you fix and improve it
        • external enhancement, internal technology
      • Data business rules
      • What to do while you are fixing it!
      • Keeping it fixed – monitor and enhance!
    • Data Audit
      • Appraise the data
        • Technology for auditing the data
        • Do fields hold what they claim to hold?
        • Is it in a usable format
          • For operations?
          • For analytics?
        • How extensively populated are the fields?
      • Ascertain the age of the data – has it passed its ‘USE BY date’?
      • What needs to be done to make this data usable/valuable
    • Data Audit Technology
      • Tools to report on the quality of data - attention is drawn to those fields that require analysis.
      • Against each column name
      • Minimum Value and Maximum Value
      • Mean, Median, Mode
      • Minimum Length and Maximum Length
      • Mean and Mode Length
      • Defined Data Type and number/% records that conflict
      • % populated with valid characters (excluding spaces)
      • Number of unique values
    • Data Report – Logistics Company Example of some of the data irregularities identified – addresses in the name field, addresses and postcodes in the Town field, lower case characters, invalid postcodes etc What lies underneath?
    • Drill Down to Format Data Format No of Records Sample of Data XX## #XX 1203 AB12 3AB XX##X #XX 63 AB12A 3AB XX# #XX 2014 AB1 3AB XXXXX#XXX 1203 ABFDA1ABC Postcode Data Format No of Records Sample of Data ##### ###### 21003 01932 124689 #### ### #### 1095 0115 236 1236 ##### ###### XXXX### 2014 01892 226819 ext.354 XX XXX XXXX 54 Do Not Call Telephone Number
    • How to Fix what needs Fixing!
      • Internal processes
        • Data cleansing
        • Corrections
        • Use of address enhancement software
        • Use of touch-points
        • Use of people in the business who know
        • Source evaluation
    • Internal processes
      • Common functions of data cleansing technology
      • Find and Replace
      • Standardisation: compare values from the given column with a column in a compiled Knowledge Base e.g. list of Titles
    • Internal processes
      • Common functions of data cleansing technology
      • Find and Replace
      • Standardisation: compare values from the given column with a column in a compiled Knowledge Base e.g. Job Titles
      • Common functions of data cleansing technology
      • Find and Replace
      • Standardisation: compare values from the given column with a column in a compiled Knowledge Base e.g. Job Titles
      • Data split: Divide data in a single field into multiple fields e.g. Mr John Smith to be divided into three fields of Title, First Name and Surname
      • De-duplication and Merge/purge
      • Case conversion
      • Address technology – correction, replacement, batch and interactive
      Internal processes
    • External Processes
      • Bureau services
        • Name and address enhancement
        • Verification, insertion, correction
        • Data augmentation
        • Telephone services – calling to correct details
        • Dynamics – B2C
          • National Change of Address
          • Gone Away Suppression
          • Mortascreen Plus (Grey market)
          • Mortascreen
          • Bereavement register
        • Dynamics – B2B
          • Mergers & acquisitions
          • Job changes
          • Status change
          • Purchasing strategy (central/local)
          • Official name/colloquial name
          • Business demographics
    • External Data Example Company Name Postcode Business Demographics Sec tor Registration Code Advertising spend Job Function Job Title Turnover Product/Service 1. Business demographics: Enhancement /verification 2. PAF data (UK & Foreign) Address verification & formatting 3. Weather/Travel Info Exhibitions organiser 4. Advertising Monitoring Market share, expenditure comparison 5. Sector performance
    • Multi-source: The strength of “blended” data Source A Source B No. of Employees in the company
    • Data Business Rules
      • Business rules manage the validation process and the ongoing protection of data quality
      • Make your rules as stringent as you can to begin, then assess the volumes of rejects and adjust accordingly
      • Quarantine offenders
      • Impose rules on internal data acquisition
      • Ensure they are included in the brief for any external data capture resources
    • Example of Business Rules
    • Data Strategy
      • Data is volatile
      • A data strategy is required for keeping it up to date
      • Documented
      • Maintained
      • Reviewed
      • Internal & external data
    • Enterprise Data Maturity Model Local Global Local Collectively Local Global Global Global Undisciplined Reactive Proactive Governed Think Act Benefit High Low Risk Low High Data Governance Direct Marketing Database Marketing & Sales Force Automation
      • Data Warehousing
      • Enterprise
      • Project
      • Explorer
      • Marts
      ERP
      • CRM
      • Operational
      • Analytical
      • Collaborative
      Customer Data Integration Product Data Integration Master Data Management Business Process Management Business Intelligence Service Orientated Architecture
    • Remember the Real World Acquisition Retention Utilisation What is most What is most What is reliable? useful? 80/20 easily available/ done? Costs You cannot do it all overnight Any enhancement to the data must be driven by commercial benefit
    • Importance Accepted in the Business
      • Lip service to business as usual
      • Incentives or penalties
      • Demonstration of the implications of poor data and/or how it makes them more effective at their job
      • Ensure they know how important it is that they comply
      • Make it easy for them to adhere to the rules
      • Listen to them and address their problems – their view of poor quality data may be different to yours
      • Be prepared to change
        • Software amendments
        • Business processes
        • Forum or reporting channel for data issues
    • Touch Points
      • Encountering customers as part of regular business processes - the touch points
      • Opportunities for
        • Acquiring new data
        • Qualifying existing contacts
        • Verifying or updating existing data
        • Testing the relationship (Jenkinson 1995)
      • Consider all of these opportunities within your business processes
    • And finally….
      • Know your data
      • Document your data strategy
      • Score the data based on level of confidence
      • Determine internal & external solutions
      • Create rules to apply at all data collection points – don’t forget your external data capture bureaux, partners and sales channels
      • Regular review in the light of on-going data quality
      • Learn from your experience
      • Be prepared for change to processes and software
      • Achieve quality and maintain quality – get it right, keep it right!
    • Thank you [email_address]