This document discusses evaluating development projects. It defines impact evaluation as assessing intended and unintended changes brought about by a project, requiring analysis of outcomes with and without the intervention. An impact evaluation plan should include background context, objectives and scope, a theory of change, key evaluation questions based on OECD criteria like relevance, effectiveness, efficiency, impact and sustainability. Data collection methods include reviewing documents and budgets, interviews, and establishing good or bad case studies. The document also discusses experimental and control groups in randomized controlled trials to link changes to the intervention, cost analysis, theories of change, and definitions of impact evaluation.

1. Evaluating a Development Project
What is and what is not?
M. Katundu
Moshi Cooperative University
January, 2023

2. Definition
 IE seek to assess the changes
brought about by a project or
programme, whether these
changes are intended or
unintended.
 The proper analysis of impact
requires a counterfactual of what
those outcomes would have been
in the absence of the
intervention.1

3. impact evaluation plan
include:
1. Background and context (provides
information on the background of
the intervention to be evaluated)
2. The purpose, objectives and scope
of the evaluation
3. Theory of change (ToC)
4. Key evaluation questions the
evaluation aims to answer
5. Proposed designs and
methodologies
Key Impact Evaluation Questions based on the OECD-DAC evaluation criteria
Relevance •To what extent did the intended impacts match the stated priorities of the organisation and intended
participants?
Effectiveness •Did the intervention produce the intended impacts in the short, medium and long term? If so, for whom,
to what extent and in what circumstances?
•What helped or hindered the intervention to achieve these impacts?
•What variations were there in the quality of implementation in different sites?
•To what extent are differences in impact explained by variations in implementation?
•Did implementation change over time as the intervention evolved?
•How did the intervention work in conjunction with other interventions to achieve outcomes?
Efficiency •What resources and strategies have been utilized to produce these results?
Impact •What unintended impacts, positive and negative, did the intervention produce?
Sustainability •Are impacts likely to be sustainable?
•Have impacts been sustained

4. Process
Review project documents (proposal, reports etc
Review the project theory of change
Review the project budget
Conduct Interviews with project team, internal and external project
partners
Pay attention to:
 The Counterfactual
 Lessons learned
 Establish cases both good or bad if any

5. Efficiency/Cost analysis
Including an analysis of a programme's costs in an impact
evaluation provides another essential piece of information
policymakers need. Unfortunately, too many impact
evaluations omit this essential component.

6. Impact

7. The effect of leaflet adverts on preference for coca-cola
Experimental
group
Control
group
30%
drink coke
30%
drink coke
Introduce
leaflet
Nothing
60%
Drink coke
39%
Drink coke
O1 O2
O3 O4

8. So what is a theory of change?
• is a map, diagram or written description of how the activities you take
part in will create the change you want to see in the world and
ultimately deliver on your long term goals.
• It works through the inputs you need (from funding, human
resources and equipment) and what main activities you need to do in
order to achieve the completed products or services which your
charity provides.

9. theory of change

12. Randomized controlled trial (RCT)
Randomisation:
subjects randomly assigned to control &
experimental groups.
each subject has equal chance of being assigned
to either group.
Manipulation/Intervention:
something (experiment) is done to selected
group(s) of subjects in study & not to other(s)
Control:
one or more control group(s) are compared with
experimental group (s)
If changes noted are more in
experimental than control group they
are linked to experiment

13. Asanteni