Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Performance-based financing of maternal and
child health: non-experimental evidence from
Cambodia and Burundi
Ellen Van de...
PBF of maternal and child health
• Capitation or salary based payments in primary care lead
to weak provider incentives
– ...
Introduction of PBF across Africa
Source: Meessen, 2013
Evidence scarce
• Basinga et al. (2011) evaluate PBF in Rwanda using RCT
– largest effect on institutional deliveries of a...
… but quickly growing
• World Bank is funding 42 PBF projects, all but 8 are
scheduled to produce an impact evaluation, ma...
Why quasi-experimental evaluation?
• Evidence is lacking on effectiveness of different designs,
distributional impact, int...
Why quasi-experimental evaluation?
• Evidence is lacking on effectiveness of different designs,
distributional impact, int...
Table : Performance-based financing schemes in Cambodia by Operational District (OD), 1999-2010
PBF in Cambodia
# of ODs 1...
PBF in Burundi
Table : Performance-based financing schemes in Burundi by province, 2007-2010
Provinces 2007 2008 2009 2010...
PBF in Burundi
Retrospective payment based on quantity and quality of
provided services, mainly maternal and child care.
O...
PBF subsidies in Burundi
0.00 5.00 10.00 15.00 20.00
Children 6 - 59 months receiving Vit A
Outpatient consultancy - new c...
Items for quality score
• Infrastructure & communication
• Outpatient consultations
• Maternal care
• Family planning
• Va...
Data
• Demographic Health Survey data
– Nationally representative household data
– Asks women retrospectively about births...
Data
• Demographic Health Survey data
– Nationally representative household data
– Asks women retrospectively about births...
Empirical strategy
• Difference-in-Differences
– Compare trends in outcomes in districts/provinces that got
PBF with those...
Empirical strategy
Outcome for child/pregnancy j in district d born at time t:
• OD and birth period FE
• 𝑃𝐵𝐹𝑑𝑡 =1 if a PB...
Empirical strategy
Outcome for child/pregnancy j in district d born at time t:
• OD and birth period FE
• 𝑃𝐵𝐹𝑑𝑡 =1 if a PB...
Empirical strategy
• Difference-in-Differences
– Compare trends in outcomes in districts/provinces that got
PBF with those...
Burundi
Cambodia
0
.2.4.6.
1995
2000
2005
2010
Delivery in public facility
.3.4.5.6.7.8.
1995
2000
2005
2010
Vaccinations
.2.4.6.8...
Empirical strategy – weaken PTA
• Control for rich battery of observable characteristics
• Cambodia:
– select only control...
Cambodia
0 .2 .4 .6 .8
Propensity Score
Full sample
0 .2 .4 .6 .8
Propensity Score
Matched controls
Controls Treated
Empirical strategy – heterogeneity of
effects
• Heterogeneity of effects across different implementation models
and across...
Empirical strategy – heterogeneity of
effects
• Heterogeneity of effects across different implementation models
and across...
Empirical strategy – heterogeneity of
effects
• Heterogeneity of effects across different implementation models
and across...
Burundi – overall effects of PBF
outcome marginal
effect
baseline
mean
>1 antenatal care visit
no sig
effect
0.96
1st trim...
Cambodia – overall effects of PBF
outcome Marginal effect Baseline mean
≥2 antenatal visits no sig effect 0.25
institution...
Cambodia – delivery in public facility
0.075**
0
0.05
0.1
0.15
0.2
0.25
0.3
Percentagepoints
full sample
25% compared to
c...
Cambodia – delivery in public facility
0.075**
0.045*
0
0.05
0.1
0.15
0.2
0.25
0.3
Percentagepoints
full sample
any facili...
Cambodia – delivery in public facility
0.075**
0.045*
0.024
0.134***
0
0.05
0.1
0.15
0.2
0.25
0.3
Percentagepoints
full sa...
Cambodia – delivery in public facility
0.075**
0.045*
0.024
0.134***
0.26***
0.07**
0
0.05
0.1
0.15
0.2
0.25
0.3
Percentag...
Cambodia – delivery in public facility
OUT: NGO full autonomy to hire
and fire and set incentives
IN: NGO required to oper...
Cambodia – delivery in public facility
IN:
- management within MoH
- threat of not paying out
Internal:
- management withi...
Cambodia
• Effect on institutional deliveries
- partly driven by switch from private to public
- only among non-poor
- qua...
Key points – Which services?
• PBF has increased institutional deliveries by 10% (Burundi) - 25%
(Cambodia)
– Effects gene...
Key points – How to incentivize?
• Variation across subperiods in Cambodia suggest that PBF
has most impact if
– Contracto...
Key points – Quality?
• Quality (Burundi): some effect on BP measurement and tetanus
shots, large increase in quality scor...
What next?
• How to best measure and incentivize quality?
• How to improve distributional impact?
• What is broader health...
References
• Van de Poel E, Flores G, Ir P, O’Donnell O. Impact of
performance based financing in a low resource setting: ...
Additional slides
PBF in Cambodia
Scheme Period # ODs
to
2010
OD
management
responsibility
Payment explicitly
linked to
performance
targets?...
PBF subsidies in Burundi
0.00 5.00 10.00 15.00 20.00
Children 6 - 59 months receiving Vit A
Outpatient consultancy - new c...
Spending the PBF subsidies
• Additional funding through PBF, 40% of total average
facility budget
• Money spend by facilit...
Methods Burundi
• 3 repeated cross-sections
• panel data on quality scores from 75 health care facilities
Difference in di...
Upcoming SlideShare
Loading in …5
×

Performance-based financing of maternal and child health: non-experimental evidence from Cambodia and Burundi

Ellen Van de Poel presents the findings of two studies that evaluate the impact of Performance-Based Financing (PBF) in Burundi and Cambodia. Both studies exploit the geographic expansion of PBF to estimate its effect on the utilization of maternal and child health services using data from the Demographic Health Surveys.

  • Be the first to comment

  • Be the first to like this

Performance-based financing of maternal and child health: non-experimental evidence from Cambodia and Burundi

  1. 1. Performance-based financing of maternal and child health: non-experimental evidence from Cambodia and Burundi Ellen Van de Poel Institute of Health Policy and Management Erasmus University Rotterdam Netherlands
  2. 2. PBF of maternal and child health • Capitation or salary based payments in primary care lead to weak provider incentives – Low quality of care – Low utilization patterns – High absenteism • PBF - move to financing that rewards performance – Linking payments and results – Financial autonomy for health centers – Verification cycle
  3. 3. Introduction of PBF across Africa Source: Meessen, 2013
  4. 4. Evidence scarce • Basinga et al. (2011) evaluate PBF in Rwanda using RCT – largest effect on institutional deliveries of around 20%, no effect on ANC and vaccination • Bushan et al. (2007) evaluate randomized pilot scheme in Cambodia and find large effects on some, but not all outcomes • Internal validity of both studies somewhat compromised
  5. 5. … but quickly growing • World Bank is funding 42 PBF projects, all but 8 are scheduled to produce an impact evaluation, many using RCT • Recent iHEA conference: 35 sessions on PBF in LMICs!
  6. 6. Why quasi-experimental evaluation? • Evidence is lacking on effectiveness of different designs, distributional impact, interactions of PBF with demand side schemes • Difficult to obtain such evidence through RCTs – Too complex design – Very large sample size needed • Unclear whether results from RCTs will hold up when programs are scaled up nationwide
  7. 7. Why quasi-experimental evaluation? • Evidence is lacking on effectiveness of different designs, distributional impact, interactions of PBF with demand side schemes • Difficult to obtain such evidence through RCTs – Too complex design – Very large sample size needed • Unclear whether results from RCTs will hold up when programs are scaled up nationwide • This presentation: evidence from retrospective studies. Exploit the gradual rollout of at-scale programs to evaluate their impact
  8. 8. Table : Performance-based financing schemes in Cambodia by Operational District (OD), 1999-2010 PBF in Cambodia # of ODs 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 1 2 1 2 4 3 3 1 1 1 8 1 2 2 45 no performance based financing
  9. 9. PBF in Burundi Table : Performance-based financing schemes in Burundi by province, 2007-2010 Provinces 2007 2008 2009 2010 Bubanza Bujumbura-mairie Bujumbura-rural Bururi Cankuzo Cibitoke Citega Karuzi Kayanza Kirundo Makamba Muramvya Muyinga Mwaro Ngozi Rutana Ruyigi
  10. 10. PBF in Burundi Retrospective payment based on quantity and quality of provided services, mainly maternal and child care. Or more formally: 𝑃𝐵𝐹 𝑠𝑢𝑏𝑠𝑖𝑑𝑦𝑖𝑡 = 𝑗=1 𝐽 𝑃𝑗 𝑁𝑖𝑗𝑡 ∗ 𝑄𝑖𝑡 with 1 ≤ 𝑄𝑖𝑡 ≤ 1.25 𝑃𝑗 = subsidy received by facility per health care service j 𝑁𝑖𝑗𝑡 = # services j delivered in facility i in period t 𝑄𝑖𝑡 = quality bonus which health care facility i receives in period t
  11. 11. PBF subsidies in Burundi 0.00 5.00 10.00 15.00 20.00 Children 6 - 59 months receiving Vit A Outpatient consultancy - new case Antenatal care: new and standard visits Diagnosis and treatment of STD In patient bed day Pregnant woman fully immunized Small surgery intervention Latrine newly constructed Child treated after birth HIV mother Family planning: referral of tubal ligation and… HIV mother treated Patient referred to hospital and feedback obtained Pregnant woman counseled and tested for HIV Person voluntary counseled and tested for HIV Bed net distributed Child under 1 completely immunized HIV case diagnosed and referred Family planning: new and re-attendants, oral &… HIV mother referred to hospital Institutional delivery by qualified staff Family planning: implant or IUD Patient diagnosed with TB (3 sputum checks) TB patient correctly treated during 6 months Subsidy in US dollar
  12. 12. Items for quality score • Infrastructure & communication • Outpatient consultations • Maternal care • Family planning • Vaccinations • Laboratory services • Drug availability • Medical consumables availability
  13. 13. Data • Demographic Health Survey data – Nationally representative household data – Asks women retrospectively about births in 5 years preceding the survey – Cambodia 2000, 2005 & 2010 – Burundi 2010 • Outcomes: ANC, delivery, vaccinations – Burundi: quality indicators in Burundi – Cambodia: mortaliy
  14. 14. Data • Demographic Health Survey data – Nationally representative household data – Asks women retrospectively about births in 5 years preceding the survey – Cambodia 2000, 2005 & 2010 – Burundi 2010 • Outcomes: ANC, delivery, vaccinations – Burundi: quality indicators in Burundi – Cambodia: mortaliy • Burundi: repeated cross sectional household survey data (2006, 2008, 2010) and facility panel data collected by Cordaid (but only selected provinces)
  15. 15. Empirical strategy • Difference-in-Differences – Compare trends in outcomes in districts/provinces that got PBF with those that did not
  16. 16. Empirical strategy Outcome for child/pregnancy j in district d born at time t: • OD and birth period FE • 𝑃𝐵𝐹𝑑𝑡 =1 if a PBF scheme in operation on OD d at time t • 𝑿𝒊𝒅𝒕 are ind/ birth/ mother/hh specific controls • Standard errors adjusted for clustering on OD level 𝑦𝑖𝑑𝑡 = 𝛽𝑃𝐵𝐹𝑑𝑡 + 𝑿𝒊𝒅𝒕 𝛀 + 𝑂𝐷 𝑑 + 𝜏 𝑡 + 𝜀𝑖𝑑𝑡
  17. 17. Empirical strategy Outcome for child/pregnancy j in district d born at time t: • OD and birth period FE • 𝑃𝐵𝐹𝑑𝑡 =1 if a PBF scheme in operation on OD d at time t • 𝑿𝒊𝒅𝒕 are ind/ birth/ mother/hh specific controls • Standard errors adjusted for clustering on OD level Cambodia: add indicators for voucher schemes, Health Equity Funds, Government schemes +𝜆1 𝑉𝑜𝑢𝑐ℎ𝑒𝑟𝑑𝑡 + 𝜆2 𝐻𝐸𝐹𝑑𝑡 + 𝜆3 𝑆𝑈𝐵𝑂 𝑑𝑡 𝑑 + 𝑦𝑖𝑑𝑡 = 𝛽𝑃𝐵𝐹𝑑𝑡 + 𝑿𝒊𝒅𝒕 𝛀 + 𝑂𝐷 𝑑 + 𝜏 𝑡 + 𝜀𝑖𝑑𝑡
  18. 18. Empirical strategy • Difference-in-Differences – Compare trends in outcomes in districts/provinces that got PBF with those that did not – Parallel trends assumption (PTA): trend in control district is a good counterfactual • Asses plausibility of PTA – Compare baseline characteristics across both groups – Compare pre-PBF trends in outcomes across both groups
  19. 19. Burundi
  20. 20. Cambodia 0 .2.4.6. 1995 2000 2005 2010 Delivery in public facility .3.4.5.6.7.8. 1995 2000 2005 2010 Vaccinations .2.4.6.8 1 . 1995 2000 2005 2010 birth year Antenatal care ODs in which PBF introduced by 2010 ODs still without PBF by 2010
  21. 21. Empirical strategy – weaken PTA • Control for rich battery of observable characteristics • Cambodia: – select only controls with similar pre-treatment trends in outcomes (PSM) – weigh the controls on the basis of similarity in pre-treatment trends (IPW) • Probit of treated on outcomes and time trend in pre- treatment period (4 years) -> propensity score • Nearest neighbor matching without replacement • Weighting controls by inverse of pscore
  22. 22. Cambodia 0 .2 .4 .6 .8 Propensity Score Full sample 0 .2 .4 .6 .8 Propensity Score Matched controls Controls Treated
  23. 23. Empirical strategy – heterogeneity of effects • Heterogeneity of effects across different implementation models and across patients’ socioeconomic status • Burundi: – pilot versus scaling-up phase – poor/non-poor
  24. 24. Empirical strategy – heterogeneity of effects • Heterogeneity of effects across different implementation models and across patients’ socioeconomic status • Burundi: – pilot versus scaling-up phase – poor/non-poor • Cambodia: – different PBF models (varying in the degree of management authority of the contractor, and the credibility of the link between pay and performance) – poor/non-poor – urban-rural – PBF in combination with demand side subsidies (vouchers)
  25. 25. Empirical strategy – heterogeneity of effects • Heterogeneity of effects across different implementation models and across patients’ socioeconomic status 𝑦𝑖𝑑𝑡 = 𝛿𝑃𝐵𝐹𝑑𝑡 + 𝜸 𝟏 𝑷𝑩𝑭 𝒅𝒕 × 𝑽𝒐𝒖𝒄𝒉𝒆𝒓 𝒅𝒕 + 𝜸 𝟐 𝑷𝑩𝑭 𝒅𝒕 × 𝑷𝒐𝒐𝒓𝒊𝒅𝒕 + 𝜸 𝟑 𝑷𝑩𝑭 𝒅𝒕 × 𝑼𝒓𝒃𝒂𝒏 𝒅𝒕 + 𝜃1 𝑉𝑜𝑢𝑐ℎ𝑒𝑟𝑑𝑡 + 𝜃2 𝐻𝐸𝐹𝑑𝑡 + 𝜃3 𝑆𝑈𝐵𝑂 𝑑𝑡 + 𝑿𝒊𝒅𝒕 𝚿 + 𝑂𝐷 𝑑𝑡 + 𝜏 𝑡 + 𝑢𝑖𝑑𝑡
  26. 26. Burundi – overall effects of PBF outcome marginal effect baseline mean >1 antenatal care visit no sig effect 0.96 1st trimester antenatal visit no sig effect 0.23 BP measured during pregnancy 0.06* 0.53 only among non-poor ≥1 anti-tetanus vaccination 0.100* 0.64 institutional delivery 0.051* 0.46 only among non-poor child fully vaccinated 0.044* 0.29 only among poor quality score 17.24** 38.68 reported satisfaction with quality no sig effect 0.9 Effects of pilot typically much larger than scale-up
  27. 27. Cambodia – overall effects of PBF outcome Marginal effect Baseline mean ≥2 antenatal visits no sig effect 0.25 institutional delivery 0.075** 0.04 (0.30) child fully vaccinated no sig effect 0.35 neonatal mortality no sig effect 0.04
  28. 28. Cambodia – delivery in public facility 0.075** 0 0.05 0.1 0.15 0.2 0.25 0.3 Percentagepoints full sample 25% compared to counterfactual
  29. 29. Cambodia – delivery in public facility 0.075** 0.045* 0 0.05 0.1 0.15 0.2 0.25 0.3 Percentagepoints full sample any facility Any facility
  30. 30. Cambodia – delivery in public facility 0.075** 0.045* 0.024 0.134*** 0 0.05 0.1 0.15 0.2 0.25 0.3 Percentagepoints full sample any facility poor non-poor Any facility
  31. 31. Cambodia – delivery in public facility 0.075** 0.045* 0.024 0.134*** 0.26*** 0.07** 0 0.05 0.1 0.15 0.2 0.25 0.3 Percentagepoints full sample any facility poor non-poor with vouchers without vouchers Any facility
  32. 32. Cambodia – delivery in public facility OUT: NGO full autonomy to hire and fire and set incentives IN: NGO required to operate within MoH rules
  33. 33. Cambodia – delivery in public facility IN: - management within MoH - threat of not paying out Internal: - management within NGO - FFS+paying bonuses (e.g. for absence of informal payments)
  34. 34. Cambodia • Effect on institutional deliveries - partly driven by switch from private to public - only among non-poor - quadrupled when implemented with vouchers - strongest in schemes where contractor had clear management authority
  35. 35. Key points – Which services? • PBF has increased institutional deliveries by 10% (Burundi) - 25% (Cambodia) – Effects generally not pro-poor • PBF not had significant impact on ANC and vaccinations – ANC: high marginal cost compared to small monetary incentive – Vaccinations heavily targeted by vertical programs (GAVI in Cambodia)
  36. 36. Key points – How to incentivize? • Variation across subperiods in Cambodia suggest that PBF has most impact if – Contractor has management authority – Finance is explicitly & credibly linked to verifiable performance targets – Incentives are large enough
  37. 37. Key points – Quality? • Quality (Burundi): some effect on BP measurement and tetanus shots, large increase in quality score, not perceived as such by patients • But, no effects on neonatal mortality in any of the periods in Cambodia might suggest quality is not sufficiently high to raise health outcomes? – None of the other PBF evaluations (including Rwanda’s RCT has established mortality effects)
  38. 38. What next? • How to best measure and incentivize quality? • How to improve distributional impact? • What is broader health system impact of PBF schemes?
  39. 39. References • Van de Poel E, Flores G, Ir P, O’Donnell O. Impact of performance based financing in a low resource setting: A decade of experience in Cambodia. Forthcoming in Health Economics. • Bonfrer I, Soeters R, Van de Poel E, Basenya O, Longin G, Van de Looij F, Van Doorslaer E. The achievements and challenges of performance based financing in Burundi. Health Affairs 2014. • Bonfrer I, Van de Poel E, Van Doorslaer E. The effects of performance incentives on the utilization and quality of maternal and child care in Burundi. Social Science and Medicine 2014.
  40. 40. Additional slides
  41. 41. PBF in Cambodia Scheme Period # ODs to 2010 OD management responsibility Payment explicitly linked to performance targets? Phase I Pilot 1999- 2003 5 OUT: NGO IN: NGO within MoH rules At discretion of NGO Phase II IN 2004- 2008 11 NGO within MoH At discretion of NGO Internal 2005- 2010 8 MoH with advisors Yes Phase III SOA 2009-now 22 Autonomous within MoH At discretion of facility head FFS 2007-now 10 MoH Pay-for-procedure not targets (fee-for- service)
  42. 42. PBF subsidies in Burundi 0.00 5.00 10.00 15.00 20.00 Children 6 - 59 months receiving Vit A Outpatient consultancy - new case Antenatal care: new and standard visits Diagnosis and treatment of STD In patient bed day Pregnant woman fully immunized Small surgery intervention Latrine newly constructed Child treated after birth HIV mother Family planning: referral of tubal ligation and… HIV mother treated Patient referred to hospital and feedback obtained Pregnant woman counseled and tested for HIV Person voluntary counseled and tested for HIV Bed net distributed Child under 1 completely immunized HIV case diagnosed and referred Family planning: new and re-attendants, oral &… HIV mother referred to hospital Institutional delivery by qualified staff Family planning: implant or IUD Patient diagnosed with TB (3 sputum checks) TB patient correctly treated during 6 months Subsidy in US dollar
  43. 43. Spending the PBF subsidies • Additional funding through PBF, 40% of total average facility budget • Money spend by facility managers based on business plan max. 60% to increase staff salaries other expenditures often to increase quality
  44. 44. Methods Burundi • 3 repeated cross-sections • panel data on quality scores from 75 health care facilities Difference in differences analyses in regression framework: • Effect of phase I of PBF (controls are untreated): 𝑌2008,𝐼 − 𝑌2006,𝐼 − 𝑌2008,𝐼𝐼 − 𝑌2006,𝐼𝐼 • Effect of phase II of PBF (controls are already treated): 𝑌2010,𝐼𝐼 − 𝑌2008,𝐼𝐼 − 𝑌2010,𝐼 − 𝑌2008,𝐼

×