Examples from Prospective Impact Evaluations


Published on

Presentation by Arianna Legovini, at the Overlooked Links in the Results Chain presentation.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Examples from Prospective Impact Evaluations

  1. 1. Examples from Prospective Impact Evaluations<br />Arianna Legovini<br />Head, Development Impact Evaluation Initiative (DIME)<br />World Bank<br />
  2. 2. Results chains is our working hypothesis but demand response is uncertain<br />2<br />
  3. 3. Questions<br />3<br />
  4. 4. Impact Evaluation<br />Tracks mean outcomes over time in <br />the treatment group relative to <br />the control group<br />Compares <br />what DID happen with <br />what WOULD HAVE happened (counterfactual)<br />Identifies causal output-outcome link <br />separately from the effect of other time-varying factors<br />
  5. 5. Babies & Bath Water<br />Uganda Community-Based Nutrition<br />Failed project (HU rating)<br />Project ran into financial difficulties<br />Parliament negative reaction<br />Intervention stopped<br />…but…<br />Strong impact evaluation results<br />Children in treatment scored half a standard deviation better than children in the control<br />Presidency asked to take a second look at the evaluation: saving the baby?<br />
  6. 6. Doing BETTER<br />6<br />
  7. 7. Does Financial Literacy Work?<br />Financial literacy found not effective in changing behavior<br />Large financial literacy pilot in Brazil shows strong results! High school students have:<br />higher financial proficiency -- 7% increase <br />financial autonomy -- 6% increase<br />intention to save -- 4% increase<br />students who save some of their income -- 6 percentage points increase <br />Why the difference? What are we measuring?<br />Small researcher led initiatives (half day workshops) vs<br />Large coalition of financial sector actors working with the education sector since 2006 to develop state of the art curriculum<br />What works?<br />
  8. 8. What are the returns to reform? Minas Gerais, Brazil<br />Large reform simplifying registration procedures <br />29 one stop shops<br />3 levels of government and 7 separate agencies<br />98% of small firms still informal<br />How to capitalize on the reform?<br />Experiment with various hypotheses<br />Firm don’t know<br />Registration and formality too costly<br />Randomly assign communication, incentives and punitive enforcement to firms to understand<br />If communication and incentives are enough or<br />Tax regime and accounting requirements need to be changed<br />Make it work!<br />
  9. 9. What are the returns to infrastructure?<br />It depends…<br />Rate of return to irrigation depends on:<br />Rate of technology adoption among farmers<br />Use of chlorine for drinking water<br />Institutional arrangements that have water repayment rates to pay for operations & maintenance<br />Ethiopia, Mozambique: <br />experimenting with technology adoption and payment systems<br />To secure high returns to irrigation!<br />Increase returns to investment!<br />
  10. 10. Supply chains in Zambia<br />Zambia suffered from high stock-out rates of essential drugs (up to 44%)<br />Randomized pilot compared two supply chains to deliver drugs to facilities<br />The system that seals packages at central facilities and ships them to them reduces stock out to less than 10%<br />Scaled up projections (on malaria alone):<br />Cure 312,014 cases and 8,433 severe cases of malaria per yr<br />Avert 16,600 Under 5 and 2,200 adult deaths by 2015<br />Reduce child and adult malaria mortality by 21% and 25%<br />Compare alternatives, improve results!<br />
  11. 11. HIV Prevention in Senegal :Is peer-to-peer better than traditional social mobilization?<br />National Aids control agency decided to replace social mobilization with peer-to-peer mentoring<br />Peer-to-peer was rolled out to a random set of health districts<br />IE found that the approaches are complementary<br />Peer mentoring doubles who get tested, take pre-test counseling and collect test results<br />Social mobilization is more effective addressing partners of HIV-positive individuals who get tested<br />Policy impact<br />Senegal now implements calls for community-based organizations to deliver prevention based on combining social mobilization and peer-to-peer counseling<br />The new approach doubles the number of people attended<br />
  12. 12. contacts<br />Check results at<br />www.worldbank.org/dime<br />alegovini@worldbank.org<br />