Presentation by Phillip Davies, Head of 3ie London, delivered at side event at IFAD Governing Council, 12 February 2013.

1. Measuring impact:
Understanding pathways to rural transformation
A side event of the IFAD Governing Council 2013

2. Measuring impact:
Understanding pathways to rural transformation
The Importanceof Measuring the Impact of
The Importance of Measuring the Impact
of Development Interventions
Development Interventions
Philip Davies PhD
Head of London Office of 3ie
[International Initiative for Impact Evaluation]

3. The Importance of Measuring the Impact of
Development Interventions
Why Evaluate Impact ?
• Effectiveness - ensure that interventions have a positive impact
• Attribution – ensure that the outcomes are the results of (are caused by) the
intervention
• Efficiency – use scarce public resources to maximum effect (VfM)
• Implementation – ensure that interventions are/can be implemented
successfully and effectively.
• Service Orientation – ensure we meet citizens’ needs/expectations
• Adaptation – allow policy makers to know when a change of
policy/different intervention is required
• Accountability - transparency of what is done and why
• Democracy - enhance the democratic process

4. The Importance of Measuring the Impact of
Development Interventions
What Is To Be Evaluated?
• Intervention effectiveness - what works?
• Implementation effectiveness – how/why it works?
• Diversity of effectiveness across different groups –
what works for whom and when?
• Experiential effectiveness - users’ views
• Resource effectiveness - at what cost/benefit?

5. The Importance of Measuring the Impact of
Development Interventions
The Need to Measure Impact Properly
It is difficult to gauge whether
agricultural interventions improve
the nutritional status of children.
Many studies were conducted on
small samples. Several animal
husbandry and home gardening
studies found no change in the
nutritional status of the children Source: Massett, E. et al, 2011, A Systematic Review of
Agricultural Interventions That Aim to Improve The
involved. More importantly, the Nutritional Status of Children, London, EPPI-Centre.
studies reviewed were statistically
unable to identify impact if any
existed.

6. The Importance of Measuring the Impact of
Development Interventions
Key Features of Good Impact Evaluation
• Ask a clear and answerable question
• Have a clear and explicit theory of change
• Have an appropriate comparison/counterfactual
• Have adequate sample sizes, and sufficient statistical power
• Have meaningful outcomes, and data on those outcomes
• Choose the most appropriate evaluation design
• Ensure that the evaluation in undertaken properly
• Check for internal and external validity
• Check for adequacy of reporting and analysis

7. The Importance of Measuring the Impact of
Development Interventions
Types of Impact Evaluation Design
• Randomised Controlled Experiments
[Allocate the intervention to groups on a random basis]
• Matched Comparison Designs
[Carefully matched those who get the intervention with those who do not]
• Regression Discontinuity Designs
[Allocate the intervention on a clearly define basis – check for
differences post-intervention]
• Difference-in-Difference Designs
[Accept initial differences between intervention and control group –
check on the size of the difference post-intervention]

8. The Importance of Measuring the Impact of
Development Interventions
Types of Evaluation for Different Policy Questions

9. The Importance of Measuring the Impact of
Development Interventions
Thank you
Philip Davies
Email: pdavies@3ieimpact.org
+44 (0)207 958 8350
Visit www.3ieimpact.org

10. The Importance of Measuring the Impact of
Development Interventions
Thank you
Philip Davies
Email: pdavies@3ieimpact.org
+44 (0)207 958 8350
Visit www.3ieimpact.org