Your SlideShare is downloading. ×
Understanding the Effect of the GAVI Initiative on Reported Vaccination Coverage Levels
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Understanding the Effect of the GAVI Initiative on Reported Vaccination Coverage Levels


Published on

Steve Lim …

Steve Lim

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Skilled birth attendance (SBA) coverage improved from 33% in 1990 to 48% in 2005Maternal mortality ratio declined from around 500 per 100,000 live births in 1990 to around 260 in 2005Neonatal mortality rate decreased from 54 per 1,000 live births in 1990 to 29 in 2005
  • ASHA = Accredited Social Health Activist1400 INR = ~31 dollars, 1000 INR = ~22 USD, 500 INR= ~11 USD, 200 INR = ~4.4 USD600 INR = ~13 dollars
  • Consistent results across three analytical approachesTranslation: For every 10 women receiving JSY, 1 additional woman will complete 3 ANC visits, 4 to 5 additional women will give birth in-facility, 3 to 4 women will give birth in a facility or with a skilled attendant
  • Less consistent results across three analytical approaches for mortality impacts. Individual level analysis suggest reduction of around 3 to 6 perinatal deaths per 1,000 births for every woman receiving JSY; similar magnitude for still birth or neonatal mortality. District-level results suggest larger effects but are not significantly different from zero change for perinatal, stillbirth/neonatal mortality. We also see no effect on maternal mortality. Two possible explanations:Small numbers problem in the district-level numbers. District analysis removes bias associated with selective uptake of JSY that the individual-level analyses are not controlling for. Given the consistent results across the three analytical approaches for intervention coverage that is not prone to small numbers problems we think the former is more likely but cannot say definitively. Maternal mortality a big problem – need an alternative approach to measure impact
  • Much larger effects for in-facility birth/SBA in high-focus states, compared to north-east states compared to other states
  • Examining differences by high-focus states vs other states shows smaller effects in high-focus states compared to other states. Two possible explanations:Different targeting of women in high-focus compared to non-high-focus states. In high-focus states all women are eligible for JSY, whereas in non-high-focus states only women living below the poverty line are eligible. As a result, the former group may include lower risk women where the benefit of the program may be smaller. Another possible explanation is that high-focus states have lower quality of obstetric care or are less likely to be able to cope with the increased workloads that have resulted from the large increases of in-facility deliveries due to JSY.
  • Transcript

    • 1. Accountability, transparency and corruption in global health: the critical role of health metrics and evaluation
      Stephen S Lim
      Assistant Professor of Global Health
    • 2. 2
      Two examples
      Tracking immunization coverage
      Conditional cash transfers to women for delivering in a health facility
    • 3. 3
      Two examples
      Tracking immunization coverage
      Conditional cash transfers to women for delivering in a health facility
    • 4. Development Assistance (Billions US$) for Health by Institution, 1990, 2007
    • 5. 5
      Global goals, funders and initiatives
      1978: Alma-Ata: Primary Health Care and Health for All
      1984: Universal childhood immunization by 1990
      2000: Millennium Development Goals
      2003: 3 by 5 – 3 million people on antiretroviral drugs by 2005
      2008: Malaria Elimination
      Private philanthropic organizations
      2000: Bill and Melinda Gates Foundation
      2006: Warren Buffet pledged additional US$30 to BMGF
      Global Health Initiatives
      2000: Global Alliance for Vaccines and Immunizations (GAVI)
      2002: Global Fund for Aids, Tuberculosis and Malaria (GFATM)
      2003: US President’s Emergency Plan for AIDS Relief (PEPFAR)
      2005: US President’s Malaria Initiative (PMI)
      2007: International Health Partnership (IHP+)
    • 6. 6
      National health sector reforms and programs
      2001: Thailand Universal Coverage
      30 Baht Scheme
      2003/04: Mexico Health Sector Reforms
      System of Social Protection in Health including Seguro Popular
      2005: Indian National Rural Health Mission
      Conditional cash transfers for women to give birth in health facilities
    • 7. Critical role of health metrics and evaluation
      High-quality measurement of health indicators and evaluation of programs is central to issues of transparency and accountability
      Are resources being used effectively?
      Have initiatives and reforms led to improvements in health system delivery and population health?
      Mounting pressure from funders, civil society, etc to document this
      Economic crisis has led to rising fears that development assistance budgets will be cut
    • 8. 8
      Two examples
      Tracking immunization coverage
      Conditional cash transfers to women for delivering in a health facility
    • 9. 9
      Tracking childhood immunization coverage
      Substantial resources have been invested in delivering immunization services
      Basic vaccines, e.g. three-dose diptheria, pertussis and tetanus (DTP3) as well as new vaccines, e.g. HiB, rotavirus
      Global initiatives
      1984: Universal Childhood Immunization (UCI) by 1990 initiative, defined as 80% immunization coverage
      1999: Global Alliance on Vaccines and Immunizations (GAVI)
      Monitoring the extent of immunization delivery is critical for evaluating how effective these investments and initiatives have been
    • 10. 10
      Tracking childhood immunization coverage
      GAVI’s Immunization Services Support (ISS) is the funding that aims to increase coverage of basic vaccines such as three-dose diptheria, tetanus and pertussis (DTP3) vaccination.
      ISS payments are performance-based with funds disbursed in proportion to the number of additional children targeted or reported to receive DTP3.
      Number of additional children receiving DTP3 is based on official reports from countries to WHO and UNICEF.
    • 11. 11
      Tracking childhood immunization coverage
      Two main questions:
      What is the trend in the fraction of children receiving three-dose diptheria, tetanus and pertussis vaccination (DTP3 coverage) over the period 1986 to 2006?
      Do target-oriented initiatives such as universal childhood immunization (UCI) and results-based financing initiatives such as GAVI’s Immunization Services Support (ISS) lead to over-reporting of DTP3 immunization coverage?
    • 12. 12
      Data sources
      Micro-data from standardized multi-country surveys
      DHS, MICS, CDC
      Crude coverage: three DTP vaccinations by maternal recall or card documented, irrespective of vaccine schedule
      Estimated for each birth cohort (up to 5 years prior to the survey)
      225 surveys
      Survey reports and WHO/UNICEF database:
      78 multi-country surveys
      142 country-specific surveys with sample size reported
      145 country-specific surveys without sample size reported
      Administrative data estimates based on health service provider registries
      Reported routinely to WHO and UNICEF since 1990
      Officially reported estimates since 1980
    • 13. 13
      Quick review of Immunization Services Support (ISS)
      Performance-based payment
      Number of additional children reported by countries to have received DTP3
      Reports largely based on administrative data
      Baseline is the year prior to approval of the proposal
      US$20 is paid once per additional child
      Data quality audit (DQA) of administrative data system before reward payments commence (from Year 3)
    • 14. 14
    • 15. 15
    • 16. 16
    • 17. 17
    • 18. 18
      Estimating missing survey coverage
      Two purposes:
      Generate plausible estimates over time to allow monitoring of indicators
      Reduce compositional bias in in causal inference that can stem from missing data
      We use validated statistical approaches that are
      Characterizes uncertainty
    • 19. 19
    • 20. 20
      Global trends in DTP3 coverage
      Survey-based global coverage of DTP3 (black) with 95% uncertainty estimates compared to countries’ officially reported (red) and WHO and UNICEF estimates (blue), 1986 to 2006.
    • 21. 21
      Regional trends
    • 22. 22
      Does ISS lead to over-reporting?
      Statistical analysis of over-reporting (officially reported coverage minus survey coverage) by years since the GAVI ISS baseline
    • 23. 23
      GAVI Immunization Services Support (ISS)
      Number of additional children vaccinated in 51 countries receiving ISS funding up to the year 2006 :
      Based on official reports: 13.9 million
      Survey-based: 7.3 (5.5 to 9.2) million
      ISS payments
      Based on official reports: US$289 million
      Survey-based: US$148 million
      Over-reporting is not uniform
      4 countries that reported increases, number of additional children did not increase
      6 overestimated by > 4x
      10 overestimated by > 2x but ≤ 4x
      23 overestimated by > 1x but ≤ 2x
      8 countries underestimated
    • 24. 24
      At the global level, survey-based immunization coverage has increased continuously and gradually over the last 20 years
      Reflects time and investment needed to expand health services
      Improvements more pronounced in some regions (e.g. Central, West sub-Saharan Africa) and countries (e.g. Cameroon) during recent time period
      Targets and payments for performance such as GAVI’s ISS can incentivize improvements but also lead to over-reporting
      Over-reporting likely reflects pressures throughout the reporting system to meet targets
      Monitoring and evaluation systems need to be based on independent, rigorous, empirical measurements that are robust to these effects
    • 25. 25
      Two examples
      Tracking immunization coverage
      Conditional cash transfers to women for delivering in a health facility
    • 26. JananiSurakshaYojana – “Safe motherhood scheme”
      Launched in 2005; 100% centrally funded
      Goal: reduce maternal and neonatal mortality
      Works by incentivizing women to deliver in a health facility
      Implemented through Accredited Social Health Activists (ASHAs)
      ASHAs also receive a cash benefit
      Budget allocation of US$342 million in 2009-10
      Accredited Social Health Activist (ASHA)
      Madhya Pradesh, India
      Photo: Department for International Development, UKAID
    • 27. National guidelines, Eligibility
      In 10 high-focus states
      All pregnant women delivering in government facility or accredited private institutions
      Other states & home deliveries
      Below the Poverty Line
      >19 years of age
      First two live births
      Targeted to women from scheduled caste or tribe
    • 28. National guidelines, Cash payments
      1 U.S. dollar ~ 45 Indian Rupees
    • 29. Questions
      What is the level of implementation of JSY at district and state-levels?
      Is JSY reaching its intended beneficiaries?
      Does receipt of financial assistance under JSY lead to increased antenatal care and in-facility delivery and reduced perinatal, neonatal and maternal mortality?
    • 30. Data
      India District-level Household Surveys (DLHS)
      DLHS-2: ~1,000 households from 593 districts, 2002 to 2004
      DLHS-3: 1,000 to 1,500 households from 611 districts, late 2007 to early 2009
      Ever-married women aged 15 to 44, for most recent pregnancy
      Antenatal care (no. of visits)
      Delivery care (type of provider, location)
      Outcome (live birth, still birth, spontaneous or induced abortion)
      Survival of the child in the case of a live birth
      Receipt of financial assistance under JSY (DLHS-3)
      Individual and household characteristics, e.g. asset-based wealth, caste, education, location of residence and distance to facility
    • 31. 31
      Births receiving JSY, 2007/08
      In-facility birth coverage, 2001 to 2003
    • 32. JSY uptake by socioeconomic indicators, national-level
    • 33. 33
      In-facility birth coverage, 2007/08
      In-facility birth coverage, 2001-2003
    • 34. Evaluating impact of JSY on coverage and outcomes
      Exact matching
      Match births receiving JSY to those not receiving JSY in DLHS-3
      Matching covariates: urban/rural residence, BPL card ownership, wealth quintile, caste, education, parity, and maternal age
      Logistic regression on matched data allows more precise control for confounders
      Logistic regression, comparing births receiving JSY to births that did not receive JSY in DLHS-3 and all births in DLHS-2
      District-level differences-in-differences
      Compare districts by level of JSY uptake, controlling for baseline differences (DLHS-2)
      580 district aggregates from DLHS-2 to DLHS-3.
    • 35. Outcomes
      Intervention coverage
      Antenatal care with at least three visits
      In-facility birth
      Skilled birth attendance (in-facility birth or birth outside of a facility with a skilled attendant)
      Perinatal death (stillbirth or death up to and including 7 days after a live birth)
      Neonatal death (death up to and including 1 month after a live birth)
      Maternal mortality* (death of women aged 15 to 49 during pregnancy or up to 6 weeks after birth or termination)
      * In district-level analysis only
    • 36. Potential confounders
      Controlled for:
      maternal age;
      number of live births;
      birth interval;
      single or multiple birth;
      maternal education;
      household wealth based on asset ownership;
      religion; and
      location of residence with respect to distance to the nearest health facility
      Varied using district, state-level fixed and random effects
    • 37. JSY and intervention coverage, national level
      Change in probability of receiving intervention: JSY vs no JSY
    • 38. Impact on mortality, national level
      Change in probability of death: JSY vs no JSY
    • 39. Variation by State: Intervention coverage
    • 40. Variation by State: Mortality
    • 41. Implications
      Varied uptake of JSY across states; not reaching the very poor
      Increases in ANC coverage and intra-partum care coverage
      Likely reductions in perinatal and stillbirth/neonatal mortality
      But potential quality of care issues in high-focus states indicates
      Alternative monitoring approach needed for maternal mortality
      Continued monitoring and evaluating the program is critical
    • 42. 42
      Substantial resources are being directed towards improving population health
      Need to track in a valid, reliable and comparable way health indicators and evaluate the impact of programs
      Ensure that increased resources for health are being utilized intended purpose and are making a difference to the health of populations
      Increasing relevance during a time of global financial crisis
      Independent and empirically-based monitoring of health indicators and evaluation of programs