Profs. Shepard and Zeng have been leading projects for the Bank to develop methods for performing a cost-effectiveness analysis of Results-Based Financing (RBF) programs and applying them to maternal-child health (MCH) services in Zambia and Zimbabwe. Both countries’ RBF programs proved highly cost-effective. Methods and results should be informative to other RBF and MCH programs.

1. Cost-Effectiveness Analysis
of Results-Based Financing
in Zimbabwe and Zambia
Donald S Shepard, PhD
Wu Zeng, MD, PhD
Brandeis University, Waltham, MA
Nov 3, 2016, World Bank
Headquarters, Washington, DC

2. Schema for cost-effectiveness analysis
(CEA) for RBF programs
Cost Effects on coverage Effects on quality
Administrator’s cost
Other donors’ cost
Provider’s cost
(User’s cost)
Household survey
Facility survey
HMIS data
Quality score card
Incremental cost Incremental lives saved,
DALYs or QALYs
Incremental cost-effectiveness ratio
(ICER)
Inputs
Intermediate results
Component outcomes
Cost-effectiveness outcomes

3.  Costs
 Financial costs
 Government and donor
perspective
 Effectiveness—coverage
 Impact evaluation with
districts compared
 Lives Saved Tool (LiST)
software
 Literature and country data
 Effectiveness--quality
 Facility surveys
 Exit interviews
 Expert opinion (Delphi
panel)
Toolkit
Web:
http://documents.worldbank.org/curated/
en/2015/09/25069701/cost-
effectiveness-analysis-results-based-
financing-programs-toolkit

4. Evaluate cost-effectiveness by the ICER, the price of one unit of
good health. The lower the better!
Incremental cost-effectiveness ratio (ICER)
𝐼𝐶𝐸𝑅 =
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑖𝑛 𝑐𝑜𝑠𝑡𝑠
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑖𝑛 ℎ𝑒𝑎𝑙𝑡ℎ 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠
Numerator: Added costs of RBF (difference in costs between
RBF and control districts)
Denominator: Added effectiveness or health outcomes
(difference in health outcomes between RBF and controls),
often expressed as quality-adjusted life years, QALYs
Both quantity (coverage) and quality contribute

5. CEA of RBF in Zimbabwe

6. Annual operating costs of RBF program
$2.04
$0.41
$0.20
$0.12
$0.11
$0.09
$0.09
$0.04
$3.09
$2.04
$0.41
$0.10
$0.06
$0.05
$0.09
$0.04
$0.02
$2.82
$0.00 $0.50 $1.00 $1.50 $2.00 $2.50 $3.00 $3.50
RBF subsidy payments, $7,045,211
Staff costs, $1,434,096
General administration*, $699,311
Capacity building (meetings, workshops, training)*, $426,644
HQ support costs*, $372,845
Transport costs, $299,412
Capital items for Cordaid*, $298,745
Supplies / equipment for facilities*, $128,209
Total, $10,704,473
Per capita annual operating cost
Current per capita costs
Mature per capita costs
The number after each category on the left is
the current aggregate annual cost for the
intervention districts (population 3.46 million).
For categories marked with asterisks, half of the
current costs were considered start-up
expenses and would be reduced in a mature
program. The labels on the right are per capita
costs

7. Aggregate costs from Nat Pharm (US$)*
Group
Pre-period
(Jan 2011-
Mar 2012)
Post-period
(Apr 2012-
Jun 2014)
Spending
/year (Pre)
Spending/
year (Post) Net difference
Popu-
lation
Spending
/capita
Control 6,771,163 33,466,940 5,416,930 14,874,196 2,229,897 $6.67
Intervention 6,062,025 29,478,515 4,849,620 13,101,562 3,461,010 $3.79
Adjusted
Intervention 13,316,434 -1.63% 3,461,010 $3.85
Difference $0.06
*Source: Nat Pharm data base of drugs distributed representing 92,000 orders to 354 control and 359
intervention customers. Due to the substantial change in expenditure per year, we computed the
difference in differences based on the ratios of aggregate expenditure. We calculated the pre-period
ratio of intervention to control (0.8953). We computed the “adjusted intervention” by applying this ratio to
the control spending/year (post), and computed the net difference by comparing actual and adjusted
intervention values.

8. Financial net costs of current RBF per
capita per year (USD)
Cost components Cost Subtotal
Incremental RBF operational costs $3.09
Incremental costs at World Bank headquarters $0.10
Subtotal $3.19
Net costs of consumables from Nat Pharm -$0.06
Less Health Transition Fund payments to control
districts (no administrative costs included)
-$0.81
Subtotal -$0.87
Net cost $2.32

9.  Quantity of care
 Institutional delivery: 13.4%
 Postpartum care: 13.3%
 Quality of care
Impact of RBF in quantity and quality of
services
Quality indicator Baseline Endline DIDs P-
value
Relative
DIDsRBF Con-
trol
RBF Con
-trol
Vaccination 0.87 0.89 0.87 0.83 0.06** 0.009 6.90%
Institutional Delivery 0.73 0.75 0.75 0.68 0.10*** 0.001 12.90%
Ante-natal care 0.72 0.75 0.79 0.72 0.10*** 0.000 13.70%
Post-natal care 0.71 0.77 0.75 0.65 0.15*** 0.000 20.00%
Note: DID denotes difference-in-differences.

10.  Calculated lives and QALYs saved for selected services with RBF and projected from controls:
 Quantitative results
 Annual number of lives saved is 772, i.e., (72+1,471)/2,
 Equivalent to 18,288 QALYs gained
 With population of 3.46 million in RBF districts, RBF generates 528 QALYs/100,000 population/year
 Site visits suggested coaching strengthened RBF
Effectiveness (RBF vs. control): quality & quantity
Year RBF Control
Lives
saved
2012 9,705 9,705 0
2013 8,613 9,345 732
2014 8,136 8,875 739
Total 26,454 27,925 1471
Deaths in children under five and lives saved
Year RBF Control
Lives
saved
2012 416 416 0
2013 370 414 44
2014 365 393 28
Total 1,151 1,223 72
Maternal deaths and lives saved

11. Contributions of quality and quantity
Quantity
(Coverage
improvem
ent), 66%
Quality
improvement,
34%
Relative shares of quality and quantity
350
178
528
0
100
200
300
400
500
600
Quantity
(Coverage
improvement)
Quality
improvement
Overall
(effective
coverage
improvement)
QALYsgainedper100,000populationper
year
Projected health impacts

12. Incremental per capita costs: US $2.32
QALY impacts (per 100,000 population per year)
Quantity (coverage) alone: 350 QALYs
Quality and quantity (effective coverage): 528 QALYs
ICERs ($/QALY gained)
Quantity (coverage) impacts alone: $2.32/0.00350 = $663
Combined coverage and quality impacts: $2.32/0.00528 = $439
Cost-effectiveness results: RBF vs. control

13. CEA of RBF in Zambia

14. Study design

15. Incentivized services (indicators) and unit
prices
No Indicator
Unit Price
US$
1 Curative consultation 0·20
2 Institutional delivery by skilled birth attendant 6·40
3 Antenatal care (ANC) - prenatal and follow up visits 1·60
4 Postnatal care visit (PNC) 3·30
5 Full immunization of children under one year 2·30
6 Pregnant women receiving 3 doses of malaria intermittent preventive treatment (IPTp) 1·60
7 Family planning (FP) users of modern contraceptive methods 0·60
8 Pregnant women counselled and tested for HIV 1·80
9 HIV+ pregnant women given niverapine (NVP) and zidovudine (AZT) 2·00

16. Results
RBF vs INP RBF vs CON INP vs CON
HQ costs 0.22 0.33 0.10
Program costs 5.90 8.65 2.75
MSL costs 0.57 0.97 0.40
0.00
2.00
4.00
6.00
8.00
10.00
12.00
Incrementalcostpercapita($)
HQ costs
Program costs
MSL costs

17. Program costs (RBF + input financing) and
distribution of incentives
Curative
consultations
30.0%
Institutional
deliveries
14.0%ANC
1.4%PNC
6.6%
Full vaccination
6.3%
Third dose of IPTp
3.7%
Modern FP
methods
28.5%
Pregnant women
counselled and
tested for HIV
9.3%
Pregnant
women given
NVP and AZT
0.2%
Incentive
payment
51.4%
Consultancy
costs
16.3%
Trainings
6.9%
Meetings/
Workshops
2.2%
M&E
0.9%
Operational costs
7.6%
Equipment
14.6%

18. Coverage and quality of key maternal and
child health services at baseline and endline
Services
Baseline Endline DIDs
RBF INP CON RBF INP CON
RBF vs
INP
RBF vs
CON
INP vs
CON
Coverage of key maternal and child services
Ins Del 68·3% 56·4% 70·9% 80·8% 74·3% 71·2% -5·4% 12·2%** 17·6%***
ANC 97·5% 96·2% 96·3% 98·9% 99·0% 99·1% -1·4% -1·4% 0·0%
PNC 70·3% 56·0% 76·4% 82·4% 73·8% 80·7% -5·7% 7·8%* 13·5%***
BCG 95·6% 97·8% 97·6% 100·0% 99·5% 95·6% 2·7% 6·4%* 3·7%*
DPT 97·1% 95·2% 95·8% 98·6% 97·6% 91·8% -0·9% 5·5%* 6·4%*
HIB 82·5% 88·3% 81·8% 97·9% 88·7% 78·1% 15·0%*** 19·1%*** 4·1%
IPT 92·0% 92·4% 95·1% 98·0% 96·1% 98·1% 2·3% 3·0%** 0·7%
FP∆ 6·5% 9·9% 7·7% 34·0% 15·6% 15·7% 21·8%** 19·5%** -2·3%
Quality index of key maternal and child services
Ins Del 65·5% 66·8% 67·0% 73·5% 74·1% 71·9% 0·7% 3·1% 2·4%
ANC 66·9% 69·1% 68·6% 75·0% 77·2% 73·8% 0·0% 2·9% 2·8%
PNC 66·7% 68·4% 68·3% 74·1% 76·6% 73·4% -0·8% 2·3% 3·0%
Vaccination 78·7% 80·7% 81·7% 81·2% 80·0% 80·4% 3·2% 3·8% 0·6%
FP 77·7% 78·6% 80·6% 81·6% 77·6% 74·8% 4·9% 9·7% 4·8%

19. QALYs gained from the RBF program in
comparison with controls
RBF vs INP RBF vs CON INP vs CON
Mid-point (lower bound-upper bound) Mid-point (lower bound-upper bound) Mid-point (lower bound-upper bound)
Population
QALYs gained
(unadjusted for
quality)
QALYs gained
(adjusted for quality)
QALYs gained
(unadjusted for quality)
QALYs gained
(adjusted for quality)
QALYs gained
(unadjusted for
quality)
QALYs gained
(adjusted for quality)
Pregnant women 237 (216-302) 302 (237-345) 475(425-539) 604(539-626) 237(176-302) 302(237-345)
Children under 5 5 088(3 733-6 015) 6 300(4 826-7 323) 11 816(10 480-13 100) 14 574(13 195-15 953) 6728(5 171-8 131) 8 274(6 704-9 843)
Total 5 325 (3 948-6 317) 6 602(5 064-7 688) 12 291(10 905-13 639) 15 178(13 734-16 579) 6 966(5 347-8 433) 8 576(6 942-10 188)

20. Incremental cost-effectiveness ratios
Comparison
Cost/life saved (US$) Cost/QALY gained (US$)
Mid-point (lower bound- upper
bound)
Mid-point (lower bound-upper
bound)
RBF vs INP (unadjusted) 39 621 (33 388 - 53 381) 1 674 (1 411 - 2 258)
RBF vs INP (quality adjusted) 31 952 (27 514 - 41 657) 1 350 (1 163 - 1 761)
RBF vs CON (unadjusted) 25 553 (23 024 - 28 767) 1 079 (972 - 1 216)
RBF vs CON (quality adjusted) 20 689 (18 945 - 22 865) 874 (800 - 966)
INP vs CON (unadjusted) 14 786 (12 211 - 19 235) 624 (515 - 813)
INP vs CON (quality adjusted) 12 280 (10 110 - 14 837) 507 (427 - 626)

21. Discussion

22. Reference: Zimbabwe’s 2012 GDP/capita was $980
If ICER < GDP/capita, program highly cost-effective (WHO)
ICER of current RBF
Improved coverage alone: $663 or 0.68x GDP/capita, highly cost-
effective
Including quality gains: $439 or 0.45x GDP/capita, highly cost-
effective
Mature RBF program would lower cost by 9.0% to $2.11 per
capita
Discussion: Interpretation of RBF in Zimbabwe

23. Calculated average is 528 DALYs vs. mature program (704)
Potential increase for mature program: 33%
Discussion: program maturity (Zimbabwe
as an example)704
528
0
DALYs per100
population per
year
Phase I periodPhase I period Phase I period
Calculated averageEstimated phase in Mature program

24. Projected cost per capita $2.11
Projected impact is 704 QALYs per 100,000
population per year
Projected ICER is $300
i.e. $2.11 / (704 / 100,000) or 0.31 x GDP/capita
Projected ICER of mature RBF program

25. Reference: GDP/capita $1,759 (2013)
ICER of RBF
 Compared to Input-financing: $1,350 or 0.77 GDP/capita, highly cost-effective
 Compared to pure control: $874 or 0.50 GDP/capita, highly cost-effective
ICER of input financing
 Compared to pure control: $507 or 0.29 GDP/capita, highly cost-effective
Comparison with Zambia RBF

26. Reproductive health vouchers in Uganda (African Strategies
for Health, 2015)
$302 / QALY or 0.59 x GDP/capita ($510)
Simulated maternal community-based health insurance in
Uganda (African Strategies for Health, 2015)
$298 / QALY or 0.58 x GDP/capita ($510)
RBF is among the very highly cost-effective interventions
Comparison with other maternal-child health programs

27. 1. Use both penalties and rewards
Human nature: people work hard to avoid penalties
2. Establish a threshold and pay only for activities above it
e.g. antenatal care: pay only for incremental coverage over 90%
3. Pay for improvements over last year’s average as well as attainment
e.g. Last year’s average 4; this quarter 5; improvement 1
4. Pay a fixed dollar amount for remoteness
Current incentives as % of volume are too small
5. Combine RBF with more formative supervision and demand side
Helps providers learn to improve quantity and quality
Potential refinements to RBF

28. Ministry of Finance, Zimbabwe
Ministry of Health and Child Care,
Zimbabwe
World Bank, Zimbabwe
Cordaid, Zimbabwe
Acknowledgments
Ministry of Health, Zambia
World Bank, Zambia
World Bank Headquarters
Financial support
 World Bank Health Results
Innovation Trust Fund

29. Donald S Shepard, PhD
shepard@brandeis.edu
+1 781 736 3975
www.brandeis.edu/~shepard
Contacts
Wu Zeng, MD, PhD
wuzengcn@brandeis.edu
+1 781 736 3888
www.brandeis.edu