• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Examples from Prospective Impact Evaluations

Examples from Prospective Impact Evaluations



Presentation by Arianna Legovini, at the Overlooked Links in the Results Chain presentation.

Presentation by Arianna Legovini, at the Overlooked Links in the Results Chain presentation.



Total Views
Views on SlideShare
Embed Views



4 Embeds 366

http://ieg.worldbankgroup.org 357
http://cqauthorops.worldbank.org 4
http://ieg.worldbank.org 3
url_unknown 2



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Examples from Prospective Impact Evaluations Examples from Prospective Impact Evaluations Presentation Transcript

    • Examples from Prospective Impact Evaluations
      Arianna Legovini
      Head, Development Impact Evaluation Initiative (DIME)
      World Bank
    • Results chains is our working hypothesis but demand response is uncertain
    • Questions
    • Impact Evaluation
      Tracks mean outcomes over time in
      the treatment group relative to
      the control group
      what DID happen with
      what WOULD HAVE happened (counterfactual)
      Identifies causal output-outcome link
      separately from the effect of other time-varying factors
    • Babies & Bath Water
      Uganda Community-Based Nutrition
      Failed project (HU rating)
      Project ran into financial difficulties
      Parliament negative reaction
      Intervention stopped
      Strong impact evaluation results
      Children in treatment scored half a standard deviation better than children in the control
      Presidency asked to take a second look at the evaluation: saving the baby?
    • Doing BETTER
    • Does Financial Literacy Work?
      Financial literacy found not effective in changing behavior
      Large financial literacy pilot in Brazil shows strong results! High school students have:
      higher financial proficiency -- 7% increase
      financial autonomy -- 6% increase
      intention to save -- 4% increase
      students who save some of their income -- 6 percentage points increase
      Why the difference? What are we measuring?
      Small researcher led initiatives (half day workshops) vs
      Large coalition of financial sector actors working with the education sector since 2006 to develop state of the art curriculum
      What works?
    • What are the returns to reform? Minas Gerais, Brazil
      Large reform simplifying registration procedures
      29 one stop shops
      3 levels of government and 7 separate agencies
      98% of small firms still informal
      How to capitalize on the reform?
      Experiment with various hypotheses
      Firm don’t know
      Registration and formality too costly
      Randomly assign communication, incentives and punitive enforcement to firms to understand
      If communication and incentives are enough or
      Tax regime and accounting requirements need to be changed
      Make it work!
    • What are the returns to infrastructure?
      It depends…
      Rate of return to irrigation depends on:
      Rate of technology adoption among farmers
      Use of chlorine for drinking water
      Institutional arrangements that have water repayment rates to pay for operations & maintenance
      Ethiopia, Mozambique:
      experimenting with technology adoption and payment systems
      To secure high returns to irrigation!
      Increase returns to investment!
    • Supply chains in Zambia
      Zambia suffered from high stock-out rates of essential drugs (up to 44%)
      Randomized pilot compared two supply chains to deliver drugs to facilities
      The system that seals packages at central facilities and ships them to them reduces stock out to less than 10%
      Scaled up projections (on malaria alone):
      Cure 312,014 cases and 8,433 severe cases of malaria per yr
      Avert 16,600 Under 5 and 2,200 adult deaths by 2015
      Reduce child and adult malaria mortality by 21% and 25%
      Compare alternatives, improve results!
    • HIV Prevention in Senegal :Is peer-to-peer better than traditional social mobilization?
      National Aids control agency decided to replace social mobilization with peer-to-peer mentoring
      Peer-to-peer was rolled out to a random set of health districts
      IE found that the approaches are complementary
      Peer mentoring doubles who get tested, take pre-test counseling and collect test results
      Social mobilization is more effective addressing partners of HIV-positive individuals who get tested
      Policy impact
      Senegal now implements calls for community-based organizations to deliver prevention based on combining social mobilization and peer-to-peer counseling
      The new approach doubles the number of people attended
    • contacts
      Check results at