Discussion of dimension of data quality in health sector

1. School of Health Systems and Public Health
Monitoring & Evaluation of HIV and AIDS Programs
Data Quality
Wednesday March 2, 2011
Win Brown USAID/South Africa
Slide 1 of 18

2. Objectives of the Session
• To Review and Discuss:
– A Data Quality approach to M&E
– Six important elements of data quality
– Practical applications
Slide 2 of 18

3. Why Data Quality?
• Program is “evidence-based”
• Data quality  Data use
• Accountability
Slide 3 of 18

4. Real World
In the real world, activities are
implemented in the field. These
activities are designed to produce
results that are quantifiable.
Data Management System
An information system represents
these activities by collecting the results
that were produced and mapping them
to a recording system.
Data Quality: How well the DMS represents the real world
Real
World
Data
Management
System
Data Quality
?
Slide 4 of 18

5. Validity
Valid data are considered accurate: They measure
what they are intended to measure.
Reliability
The data are measured and collected consistently; definitions
and methodologies are the same over time.
Completeness
Completely inclusive: the DMS represents the complete list of
eligible names and not a fraction of the list.
Precision
The data have sufficient detail; in this case the “accuracy” of
the data refers to the fineness of measurement units.
Timeliness
Data are up-to-date (current), and information is available on
time; the DMS produces reports under deadline.
Integrity
The data are protected from deliberate bias or manipulation for
political or personal reasons.
? = Dimensions of Data Quality
Slide 5 of 18

6. Good Data are Valid and Reliable
X X X
X X
X X
X
X X
XXX
XXXX
XXX XXX
XXXX
XXX
 Valid
 Reliable
≠ Valid
 Reliable
≠ Valid
≠ Reliable
Slide 6 of 18

7. What are:
– Valid data?
– Reliable data?
– Complete data?
– Precise data?
– Timely data?
– Data with integrity?
Slide 7 of 18

8. Framework for Enhancing
Data Quality
Data Management System
Data Management
Processes /
Procedures
Data Quality System
Data Quality
Processes /
Procedures
Auditable
System
Document!
Risk
Verification
Source Validity
Reliability
Completeness
Precision
Timeliness
Integrity
Paper Trail
that allows
verification of
the entire
DMS and the
data
produced
within it
Collection
Collation
Analysis
Reporting
Use
Slide 8 of 18

9. The South Africa Approach
• Data Quality Assessment
• Training
• Data Warehouse
• SASI Manual
• Standard M&E plan  DQ Plan
Slide 9 of 18

10. Data Quality Assessment
• PMTCT Data; District focus
• Trace and Verify
• Routine Data Quality
Assessment Tool (RDQA)
Slide 10 of 18

11. M&E Training?
M
•Routine data collection
•Data quality
•Results reporting
•Strategic planning
E
•Internal validity
•Operations research
•Instrument design
•Survey sampling
•Data analysis for data use
•Local training partners
•Participant follow-up
•User’s groups/networks
Slide 11 of 18

12. PEPFAR Reporting Issues
• Are PEPFAR’s results valid & reliable?
• How do you know?
• Are your patient numbers valid & reliable?
• How do you know?
Slide 12 of 18

13. 0% 10% 20% 30% 40% 50%
Percent reporting: “I understand statistics.”
100
50
1
# random
samples
drawn
Data and Statistics are Empowering
Slide 13 of 18

14. Data Warehouse
• Online results reporting system
• Standardized data capture
• Control of data quality
• Customized reporting tool
• Online indicator guidance
Slide 14 of 18

15. South Africa Strategic Information
Manual (SASI Manual)
• Operational manual
• Standard definitions for
PARTNERS
• Addresses common data quality
problems
Slide 15 of 18

16. Try Making a Data Quality Plan
• Component of the M&E plan
• Strategically think about data
quality
Slide 16 of 18

17. Measurement
With monitoring of progress in a clinic or in a
community, always try to hit the bull’s eye.
Paper Trail
Always document progress.
Data Use
Who is using the data?
Slide 17 of 18

18. Thank you
Slide 18 of 18