A presentation by Ha Thi Hong Nguyen, delivered during "Transforming Health Systems Through Results-Based Financing," an event held during the Third Global Symposium on Health Systems Research in Cape Town on September 30, 2014. This event was hosted by the Health Results Innovation Trust Fund at The World Bank, in partnership with the PBF Community of Practice in Africa.

1. The science of delivery
Use of administrative data in Health Result
Innovation Trust Fund (HRITF) portfolio
Ha Thi Hong Nguyen | Cape Town, 2014

2. What are administrative data?
• Data on payment to facilities based on verified
performance
– Can compare with reported data
– Typically only available in contracted facilities
• Data reported in the HMIS system
– Can be available for control facilities
• Individual patient records
– Can look at health outcomes and processes of care
– Rarely available in many HRITF countries
• HRITF mostly works with the first 2 categories, and
generally calls them operation data

3. The HRITF OP data portfolio
Country Start date Program areas Catchment population
Benin Mar 2012 8 districts 2.2 million (22%)
Burkina Faso* Dec 2011 3 districts 813 thousand (5%)
Burundi Mar 2010 Countrywide 9.8 million (100%)
Cameroon* Littoral: Apr 2011
3 other: Jul 2012
4 regions 2.8 million (13%)
Kenya* Dec 2011 1 sub-county 200 thousand (0.5%)
Nigeria* Dec 2011 3 LGAs 416 thousand (0.2%)
Zambia Apr 2012 11 districts 1.5 million (11%)
Zimbabwe Mar 2012 18 districts 4.2 million (30%)
Afghanistan April 2009 11 provinces 9.1 million (33%)
Laos Mar 2013 5 provinces 2.2 million (33%)
Sierra Leone Oct 2010 Countrywide 5.9 million (100%)
Total population is for 2012 (WDI)
Note several programs have expanded but OP data are not yet available
3
*Not include recently scaling up areas

4. Why operational data?
• To monitor programs’ progress as basis for further inquiry
and mid-course corrections
– Identifying high and low performing indicators
– Monitoring where money is spent
– Detecting outliers
– Comparing with control areas and watching for
unintended consequences
– Improving implementation design
• To promote transparency and hold providers accountable
for results
• To evaluate the impact of the program

5. Monitoring program progress to facilitate
further inquiries
80
70
60
50
40
30
20
10
0
4 1 2 3 4 1 2 3 4 1
2011 2012 2013 2014
Afghanistan
Benin
Nigeria
Zambia
Zimbabwe
%
Estimated coverage of institutional/SBA deliveries

6. Identifying high and low performance
Zambia: change between Q2 1012 and Q1 2014 in QOC components
Curative Care
100
80
60
40
20
0
ANC
FP
EPI
Delivery
Room
HIV
Community
Participation
HMIS
Management
Supply
General
Management
Q2 2012
Curative
Care
100
80
60
40
20
0
ANC
FP
EPI
Delivery
Room
HIV
HMIS
Supply
Manageme
nt
General
Manageme
nt
Community
Participatio
n
Q1 2014

7. Monitoring where money is spent on
Share of RBF payment for service delivery that went to health center and
lower level
0 20 40 60 80 100
Cameroon
Burundi
Zimbabwe
Benin
Burkina…
Nigeria
Zambia
Kenya
%

8. Monitoring where money is spent on
OP >5
11%
OP <=5
15%
Inst.
Deliver
ies
17%
Others
57%
Burundi
Zambia
Cameroon
Zimbabwe
OP
contact
6%
Inst.
Deliveri
es
35%
Others
19%
FP
40%
OP
contact
35%
Inst.
Deliverie
s
15%
Others
29%
FP
21%
OPC
21%
Hosp.
days
15%
VCT
12%
Others
52%
Figures reported are averages of all quarters to date
8
Three services absorbing largest share of payment

9. Detecting outliers
1600
1280
960
640
320
0
Nigeria: Performance on institutional deliveries by LGA
4 1 2 3 4 1 2 3 4 1
2011 2012 2013 2014
Adamawa
Ondo
Nasarawa

10. Assessing relative progress and watching out for
negative spillover
Afghanistan: number of SBA deliveries in
treatment and control facilities
Zimbabwe: Diarrhea cases among age 5+
(non-incentivized RBF indicator)
25000
20000
15000
10000
5000
0
Control
Treatment
2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1
2010 2011 2012 2013 2014
33
27
31
32
50
45
40
35
30
25
20
15
10
5
0
Mar-11
May-11
Jul-11
Sep-11
Nov-11
Jan-12
Mar-12
May-12
Jul-12
Sep-12
Nov-12
Jan-13
Mar-13
May-13
per 10,000 population
HMIS RBF
HMIS Non-RBF

11. Improving implementation design
Zimbabwe: switching to risk based evaluation based on comparing reported
and verified data
15
10
5
0
-5
-10
-15
Difference Between Declared and Verified 6 Month Totals
Within 5% Difference
HF1 HF2 HF3 HF4 HF5 HF6
Green Category:
• Verified on a quarterly basis
Amber Category
• Verified bi-monthly -
randomly selected 2 months
Red Category
• Verified on a monthly basis
• Also incorporates new
facilities
Difference above 5% but
below or equal to 10%
Difference above 10%
• Model based on three risk levels
• Comparison between declared and
verified values for 6-month totals

12. Promoting transparency and accountability
Burundi
Benin
Nigeria

13. Issues in working with operational data
• Quality of data
• Availability of data outside program (catchment
population)
• Capacity to design and manage a database
• Capacity to analyze data
• Standardized methods and assumption to calculate
coverage
• Practice of sharing data and using results for decision
making
• Integration with country HMIS