Maidan Summit 2012 Dr. Jyotsna Puri, 3ie


Published on

Sport for Development- Making a difference at the Policy Level.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 3ie was set up in 2008 after the report ‘When will we ever learn’? Was put out by the Center for Global development. The organization’s main mission is to increase development effectiveness through the better use of evidence in developing countries. Our main work is in encouraging impact evaluations and systematic reviews. 3ie provides mainly technical, management and financial support, provided by our donors to high quality impact evaluations and systematic reviews. We disburse approximately XX dollars annually. We have three offices with our headquarters in Delhi and two offices in London and another in Washington DC.
  • Policy Window grants: India: $500k, breakthrough, early child marriage. China: 2 awards total $1 Million; Intestinal parasites.  Uganda: Phase 1:  $60k   (total to be determined, expected approx US$300k): Staff recruitment and retention in post conflict areas. Morocco: US$330k; youth training.   Policy preparation grants: Fiji, ongoing - poverty reduction strategy. South Africa, completed: School feeding programme and ECD Ecuador, 2 proposals submitted: Philippines, on-going with the department of education. Sri Lanka, award in process – to be determined.   Others in play Argentina (CIPPEC) Kyrgyzstan (doubtful) FINO (India) NEPAD, e-Africa programme IFRC, Haiti Pakistan, Planning Commission Colombia, SINERGIA  Also under discussion a 'Philippines Policy Window' of 10-12 studies financed by AusAID.
  • Also heterogeneity; What can be scaled up? What can be/should be replicated? Policy impact and relevance.
  • Maidan Summit 2012 Dr. Jyotsna Puri, 3ie

    1. 1. Using impact evaluations in sport: An art and a science Dr. Jyotsna Puri Head of Evaluation Deputy Executive Director, 3ie
    2. 2. PROGRESA
    3. 3. So what was different with Progesa?• Why was it different?• Why did policy makers take note?• ……• It was an IMPACT EVALUATION!• ….• What is an IMPACT EVALUATION??
    4. 4. What do we need to measure impact?Providing sportsprograms todisadvantaged youth Before AfterProject (treatment) 92comparison
    5. 5. Before versus after single difference comparison Before versus after = 92 – 40 = 52 Before After Project (treatment) 40 92 comparison The project led a larger number of girls and boys to higher levels of health.But this could have happened in the absence of the program?!
    6. 6. Post-treatment comparison Single difference = 92 – 84 = 8 Before After Project (treatment) 92 comparison 84But we don’t know if they were similar before…
    7. 7. What do we need to measure impact? Before AfterProjectComparison
    8. 8. Basic Setup of a Randomized Evaluation Potential Participants Evaluation Sample Random Assignment Treatment Group Comparison Group Participants No-Shows Based on Orr (1999)
    10. 10. Evidence on the impact ofsport on development hasbeen scarce.
    11. 11. An impact evaluation examining the impact of sportDid the program cause anincrease in developmentindicators amongst programparticipants?Were the effects on boys andgirls the same?What are thenecessary/sufficient conditionsfor success?Can this be extrapolated toother circumstances/contexts?
    12. 12. 2010 study on sports - ConclusionsDoes sport contribute topersonal development and wellbeing of disadvantaged youth?•Unclear and non-significantimpact on self-esteem and self-efficacy.•Peer leaders (South Africa: yes,Magic Bus: unclear)•HIV/AIDS, gender beliefs?
    13. 13. Evaluating Sports and development : A need for better thinking • Ill defined program theory. • Small sample sizes • Survey samples powered for heterogeneous impacts (e.g. girls vs. boys, starting levels of self-esteem) • Before vs. after but comparisons not chosen well.
    14. 14. Improving Lives through ImpactEvaluation
    15. 15. 3ie Improv ing the supply side an the dem d and side ofeviden ce basedevelo d pment s
    16. 16. 3ie sub-grant windows• Funds ‘IMPACT’ evaluations and provides technical expertise;• Necessary attributes: - Counterfactual analysis - Policy relevant, LIMCs - Good program theory - Capacity development - High technical quality - Use mixed methods - Policy impact and potential for scaling up
    17. 17. Policy impact!?!
    18. 18. We take sophisticated techniques
    19. 19. Which make nice academic studies
    20. 20. Which get published in top journals
    21. 21. Bridging the gap: Bringing facts to policy makers and program managers• School based deworming: cuts absenteeism by one- fourth and costs only 50 cents/child/year.• Conditional cash transfer programs: reduce poverty and increase level of secondary school enrollment.• Financial performance incentives: can improve use and quality of health services.
    22. 22. Policy WindowPolicy window grantsPreparation grants: Sri Lanka, Philippines, Fiji, Ecuador, South Africa, NEPAD, Uganda etc.
    23. 23. 3ie: State of the art expertise• More than 100 impact evaluations.• Systematic reviews• Policy impact• More than 50 countries• Other: – Staff expertise – Experts network – Training – Monitoring – In-house research
    24. 24. Impact Evaluations– Efficacy and design– Effectiveness of interventions.– Cost effectiveness of interventions– Trade-offs and unintended consequences.
    25. 25. Thank you. #jo_puriWWW.3IEIMPACT.ORG FACEBOOK/TWITTER