Scaling Up Performance Based
           Financing in
             Rwanda
            2004-2008


      Rwanda PBF team:
Pa...
Outline
2



         Reconstruction and Innovations
    1.

         Scaling Up Performance Based
    2.
         Financi...
Outline
3



        Reconstruction and Innovations
    

        Scaling Up Performance Based
    
        Financing
  ...
The post-colonial times ..
    Modern health introduced in Rwanda free of charge to users and

    funded through direct ...
Innovation II: Scaling Up of Community
Health Insurance




   Source: Cellule d'appui aux mutuelles de sante. Ministry of...
Innovation III: Scaling Up of
    Performance Based Financing




Phase -0 (white shaded): the three PBF pilot projects
Ph...
Outline
7



        Reconstruction and Innovations
    

        Scaling Up Performance Based
    
        Financing
  ...
Chronology of Performance-Based
Financing in Rwanda
The PBF pilot experiments (2002-2005)
Three pilot schemes:
 Butare (since 2002)

 Cyangugu (since 2003)

 BTC (since 20...
Scaling up: 2005-2008
    2004: Evaluation of Butare and Cyangugu pilots





    2005: Institutionalization:





    2...
Evaluation: Cyagungu and Butare PBF
               2004
                     Use of Assisted Deliveries over time
        ...
Institutionalization: integration in
        country budget
    Since 2005, government pays outputs through recurrent


 ...
Institutionalization: the HIVAIDS
     money
    One national approach, one institutional set-up, same unit


    costs a...
National PBF Model : 2005-2008
National PBF model : 2005-2008

    The national model for health centers is based on contracts


    between different l...
PBF Payment at Facility Level

    Payments for performance are based on the quantity of outputs


    achieved (through ...
General health indicators and PBF prices
                                                                                 ...
HIV/AIDS indicators and PBF prices
(1$ = 545 RWF)
                                                             Amount Amou...
Process evaluation

    Meeting of the Steering committee : quantity and


    quality assessment
    Counter verificatio...
Administrative & management
coordination
    PBF admin system with internet based data entry and

    retrieval facilitat...
•ICT management tools: www.pbfrwanda.org.rw




              •INSERT GRAPHIC TO ADD MAP
              •MAP IS 6.17” TALL
How many persons to do that?

    MOH central PBF Unit (CAAC): 1 coordinator and two


    full-time staffs;
    A key ro...
Outline
23



         Reconstruction and Innovations
     

         Scaling Up Performance Based
     
         Financ...
Results
      Increases in the Volume of Services
 1.

      Increase of the Quality of Services
 2.

      Increase of st...
Outline
25



         Reconstruction and Innovations
     

         Scaling Up Performance Based
     
         Financ...
Increase in Volume of Services (after 27
 months)
PBF Indicator         January 2006 average/          March 2008         ...
Results for Family Planning Users at the
end of the Month                               Family Planning, Modern Methods, U...
FP Injections and oral methods at Health
Centers % Increase in Prevalence over 24
months; through December 2007
  •January...
Other improvements
    Over 16 months of PBF, the Quality increased on


    average by 7% across these 13 services.
    ...
Improvements confirmed by survey
based data
Assisted delivery – Modern contraceptive
     use




Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et in...
Decrease of fertility
                             Fertility Rate 2005-2008

              6.2
              6.1
         ...
Dramatic Increase of Coverage of
Insecticide Treated Nets
                         ITNs coverage 2005-2007
             Pr...
Decrease of malaria incidence

                                               Malaria out patient                         ...
Malaria deaths decreased

                                             Malaria death

                                   ...
Infant and Under 5 mortality rate




                                    36
Outline
37



         Reconstruction and Innovations
     

         Scaling Up Performance Based
     
         Financ...
Reforms 2000-2008
    Autonomization





    Performance Based Budgeting





    Decentralization,

Reform I: Autonomization
    Based on Bamako Initiative


    Health centers and hospitals fully autonomous : 60%


    ...
Reform II: Performance Based
Transfers
    “IMIHIGO”: contract between the President of

    the Republic and the distric...
Reform III: Decentralization
    Administrative and fiscal

    decentralization gives                                   ...
Outline
42



         Reconstruction and Innovations
     

         Scaling Up Performance Based
     
         Financ...
Study Rationale
43

         No examples of rigorously evaluated bonus
     
         payment schemes to public sector he...
Hypotheses
44
     For both general health services and HIV/AIDS
       services, we test whether PBC:
      Increases th...
Evaluation Design
         Make use of expansion of PBC schemes over time
45   

         The rollout took place at the D...
More money vs. More incentives

    Incentive based payments increase the total


    amount of money available for healt...
Baseline Survey: Sampling Strategy
47




                                          47
Lessons Learned: PBF

    Start with easy things and then go progressively to complexity.


        Health centers before...
Lessons Learned: PBF

  Institutionalization is the key phase:


Importance of institutional contracts
        Critical r...
Lessons learned
    Fiscal decentralization can help increase resources for health

    facilities if well designed.. To ...
IMPACT EVALUATION OF
        PERFORMANCE BASED
            FINANCING
                for
    GENERAL HEALTH AND HIV/AIDS S...
Roll-out plan
52
          Phase 0 districts (white) are those districts in which PBF
     
          was piloted
       ...
Rollout plan for PBC in General Health
   Sets   Phase I                                Phase II
    1     Kibungo       a...
Rollout plan for PBC in HIV/AIDS services
Imm ediately           Phase I                              Phase II
These place...
Program and Evaluation Roll-Out Plan
 55




                                                                            T...
Sampling Issues

    Law of large numbers does not apply here…


    Proposed solution:


        Propensity scores matc...
Analysis Plan
57
     All analyses will be clustered at the district level
      Compare the average outcomes of faciliti...
Difference in differences models
         To test the robustness of the analysis
     
58

         Control sample (both ...
SECTION 4:
Impact evaluation
 Implementation



 SECTION 4.1 :
 Baseline surveys

                    59
Baseline Survey: Sampling Strategy
60




                                          60
The baseline has 4 surveys
    December 2005-March 2006:


        General Health facility survey (166 centers)
    

  ...
General Health Centers Survey:
     Content
62


         General characteristics
     

         Human resources module:...
Baseline Health Facility: Utilization
                         NUM BER OF CONSULTATIONS IN AUGUST 2005
                   ...
General Health Household Level:
     Sampling Method
64



         Identified sectors and cells served by each of the 164...
Baseline Field Sampling: GH HH

                                       HEALTH FACILITY




       CELL 1                 C...
Baseline General Household Sample
    2159 HH, 10,880 individuals


    Average HH size is 5.71 individuals,


    75% o...
Baseline General Household Content

    Socio-economic information


    Anemia finger prick test: children 12-71 months ...
Baseline Household Data: Education
68
  Sample: ages 6 and up              P HASE I (Intervention)     P HASE II (Control)...
Baseline Household Data: Assets
                                         P HASE I                P HASE II
Variable       ...
Baseline Household Data:
   Activities of Daily Living (21+ years)

Adults, mothers and pregnant women          Phase I   ...
Baseline Household Data: Prenatal Care
71


                                           P hase I                       P ha...
Baseline Household Data:
     Child Immunization
72


                                   P hase I                         ...
Baseline Household Data:
   Child Health Care Utilization (<6 years)
                             P hase I                ...
Baseline Household Data:
Child Biomarkers (<6 years)
                  27.82%
30.00%


25.00%   22.16%
                   ...
Validity of Sample

    Require two different validations:


        Validate the sampling for the evaluation design
    ...
Validity of Sample and Data
76
         Evaluation design
     

             Of 110 key characteristics and output varia...
SECTION 4.2 :
Program implementation and
       monitoring



                             77
Timeline of Activities
                                              2005           2006              2007      2008
     ...
Monitoring Program Roll-out
    Regular participation in the PBF technical committee

    meetings by the impact evaluati...
Monitoring Program Roll-out
    Additional data collection effort focused on

    monitoring PBF roll-out at facility lev...
Monitoring Program Roll-out
    Key points for GH analysis


        Baseline: Collected prior to training or first payme...
SECTION 4.3 :
Follow up surveys




                    82
Follow-up surveys
    February-September 2008


    3 surveys:


        Combined health facilities survey for General H...
Follow-up Field Sampling: GH HF
    Return to 166 facilities


        Some GH facilities began offering HIV/AIDS service...
Follow-up Field Sampling: GH HH

    Objective:


        Return to the same households to create panel data set
    

 ...
Follow-up Field Sampling: GH HH

    Return to same households:


        In total 2159 were suppose to be surveyed in th...
Follow-up Field Sampling: GH HH

    External reasons:


        Migration result of 1) avoiding Gacaca, 2) employment in...
Follow-up Field Sampling: GH HH
    WHAT DOES THIS MEAN FOR ANALYSIS?


        Restrict to only matched households
    ...
Assisted delivery – Modern contraceptive
     use




Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et in...
P roportionnal Malaria morbidity in Health
                 C entres vs Health Utilization R ate
80
                      ...
Infant and Under 5 mortality rate




                                    91
Trend of Maternal Mortality ratio




                                                            92
Source : Demographic ...
Section 5:
Current and next steps




                         93
Current and Next Steps

    Reformat, clean data bases to create panel data


    Initial GH HF results: January 2009


...
QUESTIONS?
SUGGESTIONS?




               95
Performance Based Financing

                                  Rwanda : Increase in utilization of services
              ...
Upcoming SlideShare
Loading in …5
×

Scaling Up Performance Based Financing in Rwanda 2004-2008

1,519 views

Published on

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,519
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
66
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Scaling Up Performance Based Financing in Rwanda 2004-2008

  1. 1. Scaling Up Performance Based Financing in Rwanda 2004-2008 Rwanda PBF team: Paulin Basinga, Ghyuri Fritsche, Bruno Meessen, Laurent Musango, Louis Rusa, Claude Sekabaraga, Agnes Soucat et al 1
  2. 2. Outline 2 Reconstruction and Innovations 1. Scaling Up Performance Based 2. Financing Results 3. Reforms 4. Impact Evaluation 5. 2
  3. 3. Outline 3 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  3
  4. 4. The post-colonial times .. Modern health introduced in Rwanda free of charge to users and  funded through direct public subsidy : infrastructure, equipments, personnel etc 1980s : shortages and rationing, dilapidation of health services  1992: community participation for financing and management of  health care (Bamako Initiative). 1994 : Genocide  1995-97: Reconstruction after the genocide : emergency  situations, NGOs, services free of charge. 1998-Willingness to come back to development and government  leadership: drug revolving funds re-established and cost sharing reintroduced
  5. 5. Innovation II: Scaling Up of Community Health Insurance Source: Cellule d'appui aux mutuelles de sante. Ministry of Health / Rwanda 5
  6. 6. Innovation III: Scaling Up of Performance Based Financing Phase -0 (white shaded): the three PBF pilot projects Phase-1 (pink shaded): districts in which PBF was started in Jan 2006 Phase-2 (red shaded): the seven ‘control districts’ in which PBF was implemented in April 2008.
  7. 7. Outline 7 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  7
  8. 8. Chronology of Performance-Based Financing in Rwanda
  9. 9. The PBF pilot experiments (2002-2005) Three pilot schemes:  Butare (since 2002)  Cyangugu (since 2003)  BTC (since 2005) Led at provincial level by International NGOs.  Priority health interventions: child immunisation, ANC,  assisted deliveries, family planning, curative care. A fee-for service at health center level  9
  10. 10. Scaling up: 2005-2008 2004: Evaluation of Butare and Cyangugu pilots  2005: Institutionalization:  2006: Scaling Up to 23 districts with 7 controls  2008: All districts 
  11. 11. Evaluation: Cyagungu and Butare PBF 2004 Use of Assisted Deliveries over time Performance Based Contracts Control 30 25 % of births 20 2001 15 2002 2004 10 5 0 Butare Cyangugu Gikongoro Kibungo
  12. 12. Institutionalization: integration in country budget Since 2005, government pays outputs through recurrent  budget (PBF budget line): 2005 US$ 800,000 for 4 districts  2006 US$ 5,000,000 for the country  Funds flow quarterly from Treasury directly to health  facilities’ Bank Account on the basis of results of previous quarter Since 2007, budget line item for PBF scheme for the  District Steering Committee activities based at District level
  13. 13. Institutionalization: the HIVAIDS money One national approach, one institutional set-up, same unit  costs and same admin system facilitates alignment: Global Fund pays for HIV indicators into their supported sites . Payment through same Bank Account: e.g MSH and ICAP  – USG contractor-, FHI and BTC Careful assessment of incentives through HIV monies in  PBF: protecting PHC services by linking payments of HIV and PHC monies to levels of quality of general services. Unit Fee * Quantity * % Quality = Payment;
  14. 14. National PBF Model : 2005-2008
  15. 15. National PBF model : 2005-2008 The national model for health centers is based on contracts  between different levels:  Steering Committee (comité de pilotage) with representation of health authorities  Three layers of contracts: Contract between CAAC and comité de pilotage  Contract between comité de pilotage and health facility  Contract between Health facility and individual health workers  In-depth verification activities :  Done by one focal point per administrative district for quantitative  evaluation Done by hospital for qualitative evaluation  Separation of functions  15
  16. 16. PBF Payment at Facility Level Payments for performance are based on the quantity of outputs  achieved (through case-based remuneration) conditional on the quality of services rendered. The outputs (quantity) are measured monthly  The quality is measured quarterly through the use of an elaborate  supervisory checklist. (13 services) PBF Payment HC = Quantity * % Quality The formula :  ‘PBF Payment HC’ is the consolidated quarterly health center invoice (for either general  or HIV), ‘Quantity’ stands for the quarterly provisory health center invoice (the sum of all  indicators multiplied with their unit fees), ‘% Quality’ stands for the consolidated score—expressed as a percentage—obtained  from the quarterly quality supervisory checklist 16
  17. 17. General health indicators and PBF prices Amount paid per Nu Amount paid per case (US$) m INDICATEUR case (Rwf) 1 Number of New cases 100 $ 0.18 2 Number of New cases received at the prenatal care (first visits) 50 $ 0.09 3 Number of Women who received 4 prenatal consultations 200 $ 0.37 Number of women completed the 2 or 3 or 4 or 5 Tetanus 4 vaccines 250 $ 0.46 Number of Women who received the 2nd dose of Intermittent 5 Preventive Treatment of malaria 250 $ 0.46 Number of at risk pregnancies Referred before 9 months of 6 pregnancy 1000 $ 1.83 Number of child aged 12-59 months seen at the curative care 7 service for growth monitoring 100 $ 0.18 Family planning P new users (DIU, Pills, injections, implants) 100 $ 0.18 8 Family planning : number of users: DIU, Pills, injections, 9 implants 1000 $ 1.83 10 Fully Vaccinated Child 500 $ 0.92 11 Institutional Deliveries at the health center 2500 $ 4.59 12 Emergency referrals to the Hospital for obstetric care 2500 $ 4.59 13 Malnourished children referred 1000 $ 1.83 17 14 Others Emergency referrals 1000 $ 1.83
  18. 18. HIV/AIDS indicators and PBF prices (1$ = 545 RWF) Amount Amount paid Nu paid per per case (US$) m INDICATORS case (Rwf) 1Number of clients tested for HIV at the VCT center 500 $ 0.92 Number of couples/partners tested during the reporting 2 month 2500 $ 4.59 Number of HIV+ pregnant women on ART treatment 3 during labor 2500 $ 4.59 4Number of infants born topatients who received CD4 Number of HIV positives HIV+ mothers tested 5000 $ 9.17 5 test 2500 $ 4.59 Number of HIV + patients traited with cotrimoxazole 6 each month 250 $ 0.46 7Number of new adults HIV+ on ART treatment 2500 $ 4.59 8Number of new infants HIV+ on ART treatment 3750 $ 6.88 9Number of HIV+ women on contraception 1500 $ 2.75 10Total number of HIV+ patients tested for tuberculosis 1500 $ 2.75 18
  19. 19. Process evaluation Meeting of the Steering committee : quantity and  quality assessment Counter verification of the patients in the  community : looking for the phantom patients Counter verification of the quality score by hospital  team: randomly selected site Comparison of PBF data with HMIS data  Peers evaluation (Hospital PBF)  19
  20. 20. Administrative & management coordination PBF admin system with internet based data entry and  retrieval facilitate decentralized management and future decentralized payments (by districts); Semi-automated payment module, linked to central  database, witch allow for ease of payments by MOF (Ministry of finances) and others (MSH; BTC; FHI and GF); Central database allows for following trends and  forecast accurately financial risk;
  21. 21. •ICT management tools: www.pbfrwanda.org.rw •INSERT GRAPHIC TO ADD MAP •MAP IS 6.17” TALL
  22. 22. How many persons to do that? MOH central PBF Unit (CAAC): 1 coordinator and two  full-time staffs; A key role for partners (members of the CAAC and on  the field) An Extended team approach has been put in place to  cover 23 districts, and includes PBF focal points from the MOH, eight NGOs and a bilateral agency as a coordination structure
  23. 23. Outline 23 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  23
  24. 24. Results Increases in the Volume of Services 1. Increase of the Quality of Services 2. Increase of staff productivity 3. Provider Enthusiasm and Motivation 4.
  25. 25. Outline 25 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  25
  26. 26. Increase in Volume of Services (after 27 months) PBF Indicator January 2006 average/ March 2008 Percentage increase month/ average/month/ (linear/log R2) health center health center ( 258 health centers on (286 health centers on average) average) Institutional 21 37.5 78% Deliveries (log 0.75) New Curative 985 1,489 51% Consultations (log 0.19) ANC: second dose 21 52.5 150% of Tetanus Toxid (log 0.63) Family Planning 15.5 47.9 209% new users (linear 0.88) Family Planning 175.2 711.6 306% users at the end of (linear 0.98) the month
  27. 27. Results for Family Planning Users at the end of the Month Family Planning, Modern Methods, Users at the End of the Month Average Per Health Center per Month 700 640 600 Average number per month 500 R2 = 0.9784 400 300 175 200 100 0 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 2006 2007
  28. 28. FP Injections and oral methods at Health Centers % Increase in Prevalence over 24 months; through December 2007 •January 2006
  29. 29. Other improvements Over 16 months of PBF, the Quality increased on  average by 7% across these 13 services. A sharp increase in staff productivity.  Whilst all providers appreciate the additional bonuses  that they earn through PBF, most also see clear advantages in the better services they provide, and take clear pride and ownership of these activities which originate ‘from within’ as opposed to being dictated from above.
  30. 30. Improvements confirmed by survey based data
  31. 31. Assisted delivery – Modern contraceptive use Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007 31 Meilleures pratiques en SM au Rwanda
  32. 32. Decrease of fertility Fertility Rate 2005-2008 6.2 6.1 6 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 2005 2007 Source : Rwanda DHS 2005-2007
  33. 33. Dramatic Increase of Coverage of Insecticide Treated Nets ITNs coverage 2005-2007 Proportion of Children less than 5 sleeping under a bed net 80% 70% 60% 50% 40% 30% 20% 10% 0% 2005 2007 Source : Rwanda DHS 2005-2007
  34. 34. Decrease of malaria incidence Malaria out patient Non Malaria out patient   Sheet 1 Sheet 1 Age group Age group 5 years and above 5 years and above 220K Under 5 years Under 5 years 25K 200K 180K 20K Sum of Conf# outpatient malaria 160K Sum of Non malaria OPD 140K 15K 120K 100K 10K 80K 60K 5K 40K 20K 0K 0K 2001 2002 2003 2004 2005 2006 2007 2001 2002 2003 2004 2005 2006 2007 The trend of sum of Conf# outpatient malaria for Year G#C. Color shows details The trend of sum of Non malaria OPD for Year G#C. Color shows details about about Age group . The data is filtered on Country , which keeps Rwanda. The Age group . The data is filtered on Country , which keeps Rwanda. The view is view is filtered on Age group , which keeps 5 years and above and Under 5 filtered on Age group , which keeps 5 years and above and Under 5 years. years.
  35. 35. Malaria deaths decreased Malaria death  Non-Malaria Death  Sheet 1 Sheet 1 1000 Age group Age group 300 5 years and above 5 years and above Under 5 years Under 5 years 900 250 800 alaria death (clinical + conf) 700 200 eath 600 on alaria D Sum of N M 500 150 Sum of M 400 100 300 200 50 100 0 0 2001 2002 2003 2004 2005 2006 2007 2001 2002 2003 2004 2005 2006 2007 The trend of sum of Malaria death (clinical + conf) for Year G#C. Color shows The trend of sum of Non Malaria Death for Year G#C. Color shows details about details about Age group . The data is filtered on Country , which keeps Rwanda. Age group . The data is filtered on Country , which keeps Rwanda. The view is The view is filtered on Age group , which keeps 5 years and above and Under 5 filtered on Age group , which keeps 5 years and above and Under 5 years. years.
  36. 36. Infant and Under 5 mortality rate 36
  37. 37. Outline 37 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  37
  38. 38. Reforms 2000-2008 Autonomization  Performance Based Budgeting  Decentralization, 
  39. 39. Reform I: Autonomization Based on Bamako Initiative  Health centers and hospitals fully autonomous : 60%  healh centers public autonomous, 40% faith- based private not for profit Facilities are financially autonomous: Commercial bank  account, revenue from user payments and payments from community insurance Subsidized by the government: Needs based block grant  for wages, Performance Based Grant for recurrent costs, and specific financing from public health programs (vaccines, contraceptives, TB drugs and ARV etc)
  40. 40. Reform II: Performance Based Transfers “IMIHIGO”: contract between the President of  the Republic and the district mayors Key health indicators integrated in the contract (in  2007: ITNs, Mutuelles, FP, safe deliveries, hygiene..) Strong political commitment to results  Quartely review with Prime Minister, President  attending twice a year
  41. 41. Reform III: Decentralization Administrative and fiscal  decentralization gives Fiscal and Financial Decentralization flexibility to local 80,000,000,000 governments by providing them with needs and performance based block 60,000,000,000 grants Amount in RWF Decentralization of wages  Transfers to Districts 40,000,000,000 CDF sent as a block grant to Transfers to Provinces facilities 20,000,000,000 Facilities have the  authority to hire and fire Facilities receive  0 Disbursed 2002 Disbursed 2003 Disbursed 2004 Disbursed 2005 Projected 2007 Budget 2006 blockgrant from government “People follow the  Ye ar money”
  42. 42. Outline 42 Reconstruction and Innovations  Scaling Up Performance Based  Financing Results  Reforms  Impact Evaluation  42
  43. 43. Study Rationale 43 No examples of rigorously evaluated bonus  payment schemes to public sector health care providers in developing countries No distinction between the incentive effect and  the effect of an increase in resources for the health facilities No unbundling of extrinsic and intrinsic-altruistic-  motivation Link between worker motivation programs and  quality of care 43
  44. 44. Hypotheses 44 For both general health services and HIV/AIDS services, we test whether PBC:  Increases the quantity of contracted health services delivered  Improves the quality of contracted health services provided  Does not decrease the quantity or quality of non- contracted services provided,  Decreases average household out-of-pocket expenditures per service delivered  Improves the health status of the population 44
  45. 45. Evaluation Design Make use of expansion of PBC schemes over time 45  The rollout took place at the District level; random assignment at the  district level Treatment and control facilities were allocated as follows:  Identified districts without PBC in health centers in 2005  Group the districts in “similar sets” based on characteristics:  rainfall  population density  livelihoods  Flip a coin to assign districts within each “similar” to treatment  and control groups. 45
  46. 46. More money vs. More incentives Incentive based payments increase the total  amount of money available for health center, which can also affect services Phase II area receive equivalent amounts of  transfers average of what Phase I receives  Not linked to production of services  Money to be allocated by the health center  Preliminary finding: most of it goes to salaries  46
  47. 47. Baseline Survey: Sampling Strategy 47 47
  48. 48. Lessons Learned: PBF Start with easy things and then go progressively to complexity.  Health centers before hospitals  Simple quality indicators  Need for strong leadership and political will from authorities  Need for strong implementation oriented coordination structures  and large pool of trainers
  49. 49. Lessons Learned: PBF Institutionalization is the key phase:  Importance of institutional contracts Critical role of validation institution  PBF was used as a lever for reform:  Allowed to raise reuneration n assoication with  performance Progressively shifted management of humn resources  Assoicated decentralization with rapid results 
  50. 50. Lessons learned Fiscal decentralization can help increase resources for health  facilities if well designed.. To serve purpose of service delivery.. autonomy of provision is essential… Results Based Financing is a powerful mechanism to achieve the  twin objective of increased performance and increased retention of qualified service providers Combining public subsidy and private funding leads to increased  remuneration and better adequacy with needs Delinking healh workers from the central wage bill and civil  service is possible..and health workers like it .. Rwanda is back on track to reach the MDGs including MDG5 
  51. 51. IMPACT EVALUATION OF PERFORMANCE BASED FINANCING for GENERAL HEALTH AND HIV/AIDS SERVICES in RWANDA A collaboration between the Rwanda Ministry of Health, CNLS, and SPH, the INSP in Mexico, UC Berkeley and the World Bank 51
  52. 52. Roll-out plan 52 Phase 0 districts (white) are those districts in which PBF  was piloted Cyangugu = Nyamasheke + Rusizi districts  Butare = Huye + Gisagara districts  BTC = Rulindo + Muhanga + Ruhango + Bugesera + Kigali ville  Phase 1 districts (yellow) are districts in which PBF is being  implemented in 2006, following the ‘roll-out plan’ Phase 2 districts (green) are districts in which PBF is not  yet phased in; these are the so-called ‘Phase 2’ or ‘control districts’ following the roll-out plan. According to plan, PBF will be introduced in these districts by 2008. 52
  53. 53. Rollout plan for PBC in General Health Sets Phase I Phase II 1 Kibungo all Kirehe all 2 Nyanza rem aining Kam onyi all 3 Gakenke rem aining Rulindo rem aining Byum ba rem aining 4 Rwam agana rem aining Kayonza west, all Gatsibo west, all 5 Gatsibo east, all Nyagatare east,all Kayonza east, all 6 Rutsiro all, south west Kibuye west, all Nyam asheke rem aining 7 Ngororero all Kibuye east, all Gasiza 8 Rutsiro all, except south west Kabaya all 9 Nyaruguru rem aining Gikongoro all 10 Burera all Ruhengeri all Nyagatare west, all 53
  54. 54. Rollout plan for PBC in HIV/AIDS services Imm ediately Phase I Phase II These places already HIV / A IDS PBC to be introduced HIV / A IDS PBC to be introduced have PBC in health A FTE R or SIMULA TE OUSLY A FTE R or SIMULA TE OUSLY centers for non- with PBC for general services (ie. with PBC for general services (ie. HIV / A IDS services N OT before) N OT before) Rulindo Kibungo Kirehe Kam onyi Nyanza Kam onyi Gisozi Gakenke Gisagara Rulindo Butare Byum ba Bugesera Rwam agana Kayonza Gasabo Gatsibo Nyagatare Nyarugenge Nyam asheke Kibuye Cyangugu Ngororero Gasiza Rutsiro Kabaya Nyaruguru Gikongoro Burera Ruhengeri 54
  55. 55. Program and Evaluation Roll-Out Plan 55 Timeline Jan-06 Mar-06 Jun-06 2007 Feb-08 Apr-08 May-08 Jul-08 Treatment Start of intervention Program Implementation Control Start of intervention FACILITY GH Household HIV Household Impact Evaluation SURVEYS Baseline FOLLOW-UP 55
  56. 56. Sampling Issues Law of large numbers does not apply here…  Proposed solution:  Propensity scores matching of communities in treatment and  comparison based on observable characteristics Over-sample “similar” communities in Phase I & Phase II  It turned out  Couldn’t find enough characteristics to predict assignment to  Phase I Took a leap of faith and did simple stratified sampling  56
  57. 57. Analysis Plan 57 All analyses will be clustered at the district level  Compare the average outcomes of facilities and individuals in the treatment group to those in the control group 24 months after the intervention began.  Use of multivariate regression (or non-parametric matching) : control confounding factors  Test for differential individual impacts by: Gender, poverty level  Parental background (If infant : maternal education,  HH wealth) 57
  58. 58. Difference in differences models To test the robustness of the analysis  58 Control sample (both observed and unobserved) heterogeneity  A two-way fixed effect linear regression:  ∑ yit = α PBC it + Bk X itk + γ i + δ t + ε it k Where: yit is an outcome variable for facilitiy i in period t or for individual i who lives in the catchment area of facility i, PBC it is an indicator of whether facility i is being paid by PBC in period t, X itk are time varying control variables, i is a facility fixed effect, δ t is a year fixed effect, ε it is an error term. 58
  59. 59. SECTION 4: Impact evaluation Implementation SECTION 4.1 : Baseline surveys 59
  60. 60. Baseline Survey: Sampling Strategy 60 60
  61. 61. The baseline has 4 surveys December 2005-March 2006:  General Health facility survey (166 centers)  Phase 1 : 80 facilities  Phase 2 : 86 facilities  General Health household survey (2,016 HH)  August – November 2006:  HIV/AIDS facility survey (64 centers)  HIV/AIDS household survey (1994 HH)  HIV/AIDS study for another presentation  61
  62. 62. General Health Centers Survey: Content 62 General characteristics  Human resources module: Skills, experience and  motivations of the staff Services and pricing  Equipment and resources  Vignettes: Pre-natal care, child care, adult care  VCT, PMTCT, AIDS detection services Exit interviews: Pre-natal care, child care, adult  care, VCT, PMTCT 62
  63. 63. Baseline Health Facility: Utilization NUM BER OF CONSULTATIONS IN AUGUST 2005 Phas e I Phas e II Num be r Num be r Se rvice s Provide d of Obs . cs td of Obs . cs td T-Stat Curative child care 79 307.165 314.925 81 222.247 162.384 -2.151 Curative adult care 79 581.608 799.890 81 461.543 632.451 -1.055 Child grow th monitoring 74 129.770 227.201 70 152.100 357.436 0.450 Prenatal care 77 77.065 45.741 77 76.013 62.327 -0.119 Institutional delivery 74 17.041 17.940 74 13.230 12.221 -1.510 Home delivery 49 1.408 5.330 49 1.776 7.811 0.272 TB treatment 61 2.393 11.960 62 0.403 0.877 -1.307 Malaria treatment 79 305.557 238.645 80 234.838 211.342 -1.979 VCT 62 127.048 153.189 57 96.860 185.501 -0.971 63
  64. 64. General Health Household Level: Sampling Method 64 Identified sectors and cells served by each of the 164  health facilities in the sample, Randomly selected four cells from the catchment  area, For each cell, obtained number of zones (10-15 hh)  Randomly selected three zones in each cell  Obtained household lists for each of the zones  Randomly selected one household for each zone  Produced random sample of 12 households per health  facility, with a final sample size of 2,016 households.64
  65. 65. Baseline Field Sampling: GH HH HEALTH FACILITY CELL 1 CELL 2 CELL 3 CELL 4 HH 1 HH 2 HH 3 HH 4 HH 5 HH 6 HH 7 HH 8 HH 9 HH 10 HH 11 HH 12 65
  66. 66. Baseline General Household Sample 2159 HH, 10,880 individuals  Average HH size is 5.71 individuals,  75% of the sample is under the age of 30 years old.  (Sampling strategy) Not a nationally representative sample:   Sample of rural households with children < 6 years old 66
  67. 67. Baseline General Household Content Socio-economic information  Anemia finger prick test: children 12-71 months old  Malaria dip stick test: children under age 6  Anthropometrics: <6 years old  Mental health: mothers, pregnant women, adults over  age 20 Sexual history and preventative behavior knowledge  Pre-natal care utilization and results  Parents or caretakers were asked for information  regarding child (<5 years) health status
  68. 68. Baseline Household Data: Education 68  Sample: ages 6 and up P HASE I (Intervention) P HASE II (Control) Variable Nr. Obs Mean Std. Error Nr. Obs Mean Std. Error T -stat Ever attended 3104 79.68% 0.011 4042 80.88% 0.010 -0.825 No schooling 2458 10.26% 0.007 3252 10.88% 0.007 -0.606 At least some P rimary 2458 82.93% 0.010 3252 79.97% 0.010 2.195 ** At least some Secondary 2458 6.46% 0.008 3252 8.42% 0.007 -1.841 * Attended school in last12 months 2391 41.13% 0.015 3181 41.34% 0.012 -0.112 Able to read Kinyarwanda 2532 63.59% 0.015 3359 66.37% 0.014 -1.375 68
  69. 69. Baseline Household Data: Assets P HASE I P HASE II Variable Nr. Obs Mean Std. Error Nr. Obs Mean Std. Error T -stat Complete sofa set 921 2.18% 0.005 1147 3.08% 0.007 -1.073 Radio 921 49.63% 0.023 1147 50.27% 0.017 -0.221 Radio-cassette or music system 921 4.29% 0.009 1147 5.89% 0.008 -1.274 Telephone Mobile 921 1.07% 0.005 1147 3.50% 0.007 -2.778 *** Mosquito nets 921 22.53% 0.023 1147 28.00% 0.027 -1.542 Sewing machine 921 0.76% 0.003 1147 1.18% 0.004 -0.883 A bed 921 62.82% 0.023 1147 58.71% 0.022 1.290 W ardrobe 921 4.53% 0.009 1147 6.07% 0.010 -1.155 Metalic library 921 0.57% 0.002 1147 1.28% 0.004 -1.645 T able 921 63.47% 0.022 1147 63.40% 0.021 0.021 Chair 921 85.48% 0.018 1147 84.82% 0.019 0.255 A bicycle 921 16.37% 0.020 1147 17.26% 0.027 -0.262 69
  70. 70. Baseline Household Data: Activities of Daily Living (21+ years) Adults, mothers and pregnant women Phase I Phase II Variable Obs Mean std Obs Mean std tstat Total Sample Accepts to perform ADL 1705 79.71% 0.011 2081 81.28% 0.012 -0.988 Nr of seconds for 1 sit-to-stand 1376 8.815 0.796 1680 8.807 0.680 0.008 Nr of seconds for 5 sit-to-stand 1373 16.991 0.406 1676 16.161 0.437 1.392 Nr of sit-to-stand in previous 1329 4.932 0.019 1619 4.904 0.015 1.114 Squat for 30 seconds, seconds 1373 29.194 0.228 1676 28.879 0.231 0.970 Balance on right foot, seconds 1374 29.074 0.223 1676 28.278 0.240 2.429 ** Balance on left foot, seconds 1373 28.893 0.238 1674 28.190 0.248 2.046 ** 70
  71. 71. Baseline Household Data: Prenatal Care 71 P hase I P hase II Variable Nr.Obs. Mean St d.Error Nr.Obs Mean St d.Error T -st at Of all births since Jan. 2005 In facility birth 1238 32.05% 0.026 1462 34.87% 0.025 -0.777 Of most recent pregnancy Times received PNC 723 2.781 0.046 900 2.687 0.052 1.368 Injection to prevent tetanus 728 74.52% 0.023 900 72.56% 0.019 0.654 71
  72. 72. Baseline Household Data: Child Immunization 72 P hase I P hase II Variable Nr.Obs. Mean St d.Error Nr.Obs Mean St d.Error T -st at 12-23 months old fully_immunized 262 75.36% 0.03 295 77.04% 0.04 -0.33 12-71 months old fully_immunized 1154 63.77% 0.03 1387 65.28% 0.02 -0.43 72
  73. 73. Baseline Household Data: Child Health Care Utilization (<6 years) P hase I P hase II Variable Nr.Obs. M ean St d.Error Nr.Obs M ean St d.Error T -st at days_sick 1438 1.96 0.12 1717 2.59 0.13 -3.54 *** receive_care 597 25.21% 0.02 803 25.58% 0.02 -0.13 times_receive_care 143 1.58 0.10 218 1.44 0.08 1.12 cost_fees 135 246.20 43.80 213 285.71 42.82 -0.65 cost_supplies 132 93.05 52.04 208 101.43 28.25 -0.14 cost_medicine 133 287.77 64.66 207 475.54 76.57 -1.87 * cost_medicine_nopres162 73.48 19.46 283 109.43 30.67 -0.99 cost_lab 134 77.13 21.38 208 78.92 21.43 -0.06 cost_other 134 21.35 8.28 204 45.29 19.51 -1.13 73
  74. 74. Baseline Household Data: Child Biomarkers (<6 years) 27.82% 30.00% 25.00% 22.16% 19.53% 18.67% 20.00% Phase I 15.00% Phase II 10.00% 7.51% 7.33% 5.00% 0.00% Anemic Malaria Fever and Malaria 74
  75. 75. Validity of Sample Require two different validations:  Validate the sampling for the evaluation design  Diff in means tests between Phase I and Phase II to determine  if intervention and comparison groups balanced at baseline Validate the quality of data  Compare descriptive stats to other sources of national data  (i.e.: 2005 & 2007 DHS, MOH data) 75
  76. 76. Validity of Sample and Data 76 Evaluation design  Of 110 key characteristics and output variables of HF, the  sample is balanced on 104 of the indicators. Of 80 key HH output variables, the sample is balanced on 73  of the variables. Majority of the indicators which differ between Phase I and Phase II are  results from patient exit interview, which is not a random sample. Quality of data  HH Results comparable to the 2005 DHS, MOH data  76
  77. 77. SECTION 4.2 : Program implementation and monitoring 77
  78. 78. Timeline of Activities 2005 2006 2007 2008 12 1 2 3 4 5 6 7 8 9 10 11 12 123456789 General Health Facility BASELINE DATA General Health Household COLLECTION HIV/ AIDS Facility HIV/ AIDS Household PBF TRAINING PHASE I DISTRICTS PHASE I DISTRICTS: OUTPUT PBF PAYMENTS PHASE II DISTRICTS: INPUT MONITORING DATA COLLECTION Health Facility FOLLOW UP DATA General Health Household COLLECTION HIV/ AIDS Household PHASE I DISTRICTS PBF TRAINING PHASE II DISTRICTS 78
  79. 79. Monitoring Program Roll-out Regular participation in the PBF technical committee  meetings by the impact evaluation team members Monitor threat to internal validity of sample  from: Political pressure to expand PBF into Phase II districts  before 2008 Many facility directors and providers in Phase II districts  heard of PBF through colleagues or media so attempt to imitate treatment Ensure exposure to PBC for enough time  Avoid contamination: Training of phase 2 started after  data collection for HF in May 2008 79
  80. 80. Monitoring Program Roll-out Additional data collection effort focused on  monitoring PBF roll-out at facility level Date received training  Relationship with Comite de Pilotage: Number of  audits conducted Amounts received at facility due to PBF  Allocation of PBF to salaries and other  Monitoring helped to ensure the evaluation team  understood the actual roll-out 80
  81. 81. Monitoring Program Roll-out Key points for GH analysis  Baseline: Collected prior to training or first payments.  Found little evidence of imitation of treatment in Phase  II Follow-up: The MOH initiated a revised training  course for ALL districts in 2008. Phase I districts received March-April 2008, and Phase II  districts received in May 2008 May look at indicators up to March 2008 for all health  facilities as health facility data collection didn’t end until July 2008 81
  82. 82. SECTION 4.3 : Follow up surveys 82
  83. 83. Follow-up surveys February-September 2008  3 surveys:  Combined health facilities survey for General Health -  HIV/AIDS Household survey for General Health (panel data)  Household survey for HIV/AIDS (panel data)  83
  84. 84. Follow-up Field Sampling: GH HF Return to 166 facilities  Some GH facilities began offering HIV/AIDS services  (VCT, PMTCT and/or ARV) between 2006-2008 Identified in the field and used the HIV/AIDS HF  questionnaire; all GH HF questions still asked but in different format 94 (56.63%) GH 2006 & 2008  60 (36.14%) GH 2006; HIV/AIDS 2008  12 (7.23%) incomplete information  84
  85. 85. Follow-up Field Sampling: GH HH Objective:  Return to the same households to create panel data set  (2006-2008) Print baseline roster (names, codes) and keep consistent  across waves Account for household members who left and new  arrivals from 2006-2008 85
  86. 86. Follow-up Field Sampling: GH HH Return to same households:  In total 2159 were suppose to be surveyed in the  catchment area of 167 facilities. 1888 (87%) were interviewed in 2006 and 2008  267 (12%) were replaced  4 (0.2%) were not found and not replaced  % of replacement by region:  South (24%), North (14%), East (4%) and West (13 %)  86
  87. 87. Follow-up Field Sampling: GH HH External reasons:  Migration result of 1) avoiding Gacaca, 2) employment in Kigali,  3) famine in South during the last 2 years Decentralization in 2006 renamed some areas in study sample;  impossible to locate based on baseline location Internal reasons:  For some health facilities, the baseline HH team didn’t follow  sampling procedure Given a cell to survey by the SPH team but difficult to reach  Used the same cell information but surveyed households in another  area Health facilities with 10-13 hh replaced  87
  88. 88. Follow-up Field Sampling: GH HH WHAT DOES THIS MEAN FOR ANALYSIS?  Restrict to only matched households  1,888 households  88
  89. 89. Assisted delivery – Modern contraceptive use Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007 89 Meilleures pratiques en SM au Rwanda
  90. 90. P roportionnal Malaria morbidity in Health C entres vs Health Utilization R ate 80 75 73,5 71,1 70,3 70 67,4 60 50,4 50 44,4 40 37,8 37,9 30 29,9 28,4 27,4 25 20 15 10 0 2001 2002 2003 2004 2005 2006 2007 Malaria morbidity Health utilis ation rate 90
  91. 91. Infant and Under 5 mortality rate 91
  92. 92. Trend of Maternal Mortality ratio 92 Source : Demographic and Heath survey 1992, 2000 et 2005.
  93. 93. Section 5: Current and next steps 93
  94. 94. Current and Next Steps Reformat, clean data bases to create panel data  Initial GH HF results: January 2009  Initial GH HH results: March 2009  Plan dissemination workshop in Kigali to discuss  initial results with key stakeholders and TWG
  95. 95. QUESTIONS? SUGGESTIONS? 95
  96. 96. Performance Based Financing Rwanda : Increase in utilization of services 2006-2008 (average of 206 health centers) 350 300 250 200 % 1 50 1 00 50 0 Planning Institutional Planning New New Curative Consultations Tetanus Deliveries Family Toxid Total Family Health Interventions

×