This document discusses verification processes for results-based financing (RBF) programs. It presents a pathway tool to examine quantity verification error rates, patient tracing error rates, and counter-verification error rates. The tool analyzes patterns at the facility level, indicator level, and verifier level. It also explores potential causes of errors, such as insufficient training, complex compliance criteria, or incentive amounts that encourage gaming. Addressing high error rates may involve investigating their underlying causes through facility visits, increasing sanctions, providing additional training or support, or adjusting the verification methodology. The overall goal is to expand knowledge of verification practices and improve the accuracy of operational data from RBF programs.

1. Verification of Results
Group Work
Petra Vergeer | Senior Health Specialist

2. Overview
1
• Operational data from RBF programs is of limited use
unless it is accurate.
• A cross-case analysis was carried out to expand knowledge
about verification processes and practices.
o Afghanistan,Argentina,Burundi, and Rwanda
• Introduction of verification “pathway tool”
o Initial consultation to obtain feedback to further develop tool

3. Verification Pathway Tool

4. Examine quantity verification error rates
1
Facility-level patterns
Indicator-level patterns
Generalized
across all
contracted
parties
Localized to specific
facilities (by
geographic area)
Localized to
specific types
of facilities
Indicators with
complex
compliance
criteria
Indicators that are
rewarded more
frequently and/or
have higher patient
volume
Indicators
rewarded at a
higher level
Examine
quantity
verification
error rates
Activities to
explore
Activities to
explore

5. Examine patient tracing error rates
Examine patient
tracing error
rates
Facility-level patterns
Indicator-level patterns
Generalized
across all
contracted
parties
Localized to specific
facilities (by
geographic area)
Localized to
specific types
of facilities
Indicators with
complex
compliance
criteria
Indicators that are
rewarded more
frequently and/or
have higher patient
volume
Indicators
rewarded at a
higher level
Activities to
explore
Activities to
explore

6. Examine counter-verification error rates
Verifier-level patterns
Indicator-level
patterns
Generalized
across all
verified
parties
Localized
to a
specific
verifier
Localized
to a
specific
type of
facility
Examine quantity
and quality counter-
verification error
rates
(Assume data are at level
of contracted party.)
Indicators with
complex
compliance
criteria
Indicators that are
rewarded more
frequently and/or
have higher patient
volume
Indicators
rewarded at a
higher level
Activities to
explore
Activities to
explore

7. Factors influencing verification: a
conceptual framework
Context
Verification Characteristics Impacton
accuracy,
cost,
sustainability
RBF Characteristics
RATIONALE FOR RBF
CONTRACT TYPE
USE OF RBF RESULTS
Improvinghealthoutcomes/HSS
Relational
Payment,improvingperformance
Financial accountability/Costcontrol
Classic
Transparency, Namingand Shaming
Monthly Annual
Yes
Large
Whole universe Risk-basedapproach
Internal
Verification Results and Their Use
FREQUENCY
ALLOWABLE ERROR MARGIN
SAMPLE SIZE
INSTITUTIONAL SETUP
ADVANCE WARNING No
Small
Third party
Learning, Error correction Cost recovery,
Sanction
PAYMENT FREQUENCY Monthly Annual
POLITICAL ENVIRONMENT GOVERNANCE CULTURE

8. Potential causes of generalized error rates:
verification
• Knowledge and tools
o Insufficient training for providers, data entry clerks
o Data collection tools insufficiently clear
o Explore validity of verification methods
• Allowable error margin
o Assess error margin
• Advance warning
• Institutionalsetup
o There is a conflict of interest betweenthe verifier and the contracted party
• Sample size
o Data are “noisy”
• Frequency
o Verification/counter-verification occurs too infrequently to deter gaming
• Verification results and use
o Incentives and sanctions are not sufficient to mitigate mis-reporting
Question: What are the causes in your country?
Back to
main slide

9. Potential causes of generalized error rates:
patient tracing
• Knowledge and tools
o Insufficient training for providers, data entry clerks
o Data collection tools insufficiently clear
o Explore validity of verification methods
• Allowable error margin
o Assess error margin
• Advance warning
• Institutionalsetup
o There is a conflict of interest betweenthe verifier and the contracted party
• Sample size
o Data are “noisy”
• Frequency
o Verification/counter-verification occurs too infrequently to deter gaming
• Verification results and use
o Incentives and sanctions are not sufficient to mitigate mis-reporting
Question: What are the causes in your country?
Back to
main slide

10. Potential causes of generalized error rates:
counter-verification
• Knowledge and tools
o Insufficient training for providers, data entry clerks
o Data collection tools insufficiently clear
o Explore validity of verification methods
• Allowable error margin
o Assess error margin
• Advance warning
• Institutionalsetup
o There is a conflict of interest betweenthe verifier and the contracted party
• Sample size
o Data are “noisy”
• Frequency
o Verification/counter-verification occurs too infrequently to deter gaming
• Verification results and use
o Incentives and sanctions are not sufficient to mitigate mis-reporting
Question: What are the causes in your country?
Back to
main slide

11. Potential causes of error rates localized to a
specific geographic area: verification
• Knowledge and tools
o Insufficient support for contracted parties
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
o Verification occurs too infrequentlyto deter gaming
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

12. Potential causes of error rates localized to a
specific geographic area: patient tracing
• Knowledge and tools
o Insufficient support for contracted parties
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
o Verification occurs too infrequentlyto deter gaming
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

13. Potential causes of error rates localized to a
specific verifier: counter-verification
• Knowledge and tools
o Insufficient support for verifiers
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
o Verification occurs too infrequentlyto deter gaming
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

14. Potential causes of error rates localized to a
specific type of facility: verification
• Knowledge and tools
• Allowable error margin
o Explore whether error margin is appropriate
• Advance warning
• Institutional setup
• Sample size
o Biased data due to insufficient sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

15. Potential causes of error rates localized to a
specific type of facility: patient tracing
• Knowledge and tools
• Allowable error margin
o Explore whether error margin is appropriate
• Advance warning
• Institutional setup
• Sample size
o Biased data due to insufficient sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

16. Potential causes of error rates localized to a
specific type of facility: counter-verification
• Knowledge and tools
• Allowable error margin
o Explore whether error margin is appropriate
• Advance warning
• Institutional setup
• Sample size
o Biased data due to insufficient sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

17. Potential causes of high error rates among
indicators with complex compliance criteria:
verification
• Knowledge and tools
o Confusing data collection tools
o Inadequate training for providers and data entry clerks
• Allowable error margin
o Indicator too complex for accurate reporting
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

18. Potential causes of high error rates among
indicators with complex compliance criteria:
patient tracing
• Knowledge and tools
o Confusing data collection tools
o Inadequate training for providers and data entry clerks
• Allowable error margin
o Indicator too complex for accurate reporting
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

19. Potential causes of high error rates among
indicators with complex compliance criteria:
counter-verification
• Knowledge and tools
o Confusing data collection tools
o Inadequate training for providers and data entry clerks
• Allowable error margin
o Indicator too complex for accurate reporting
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

20. Potential causes of high error rates among indicators that
are rewarded more frequently or have higher patient
volume: verification
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

21. Potential causes of high error rates among indicators that
are rewarded more frequently or have higher patient
volume: patient tracing
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

22. Potential causes of high error rates among indicators that
are rewarded more frequently or have higher patient
volume: counter-verification
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

23. Potential causes of high error rates among
indicators that are rewarded at a higher level:
verification
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
o Incentive amount encourages gaming
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

24. Potential causes of high error rates among
indicators that are rewarded at a higher level:
patient tracing
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
o Incentive amount encourages gaming
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

25. Potential causes of high error rates among
indicators that are rewarded at a higher level:
counter-verification
• Knowledge and tools
• Allowable error margin
• Advance warning
• Institutional setup
o Incentive amount encourages gaming
• Sample size
• Frequency
• Verification results and use
Question: What are the causes in your country?
Back to
main slide

26. Examples of options to address high error rates
• Visit small number of facilities to investigate underlying
causes of error rate patterns
• Increase sanction amount/proportion
• Conduct additional trainings for providers, data entry clerks
• Provide additional support on an ongoing basis for
providers, data entry clerks

27. Thank You