Evaluation processes

1. Evaluation
.

2. 2
A time-bound assessment that systematically and objectively
assesses the relevance, performance, success and effectiveness
of ongoing and completed programmes and projects.
It is a value judgement of the set objectives in the project.
Definition of Evaluation?

3. Definition of Evaluation?
• It can be done during the implementation of the project
keeping in view the improvement of performance and
strategy. It can also be done after the completion of the
project in order to derive the outcome;
• Thus evaluation can be formative and summative;
• When the cook tastes the soup, that’s formative; when the
guests taste the soup, that is summative evaluation.
3

4. Evaluation
Purpose
• The purpose of an evaluation is the reason why you are
doing it. It goes beyond what you want to know to why
you want to know it.

5. Evaluation
Examples of an evaluation purpose could be:
• To provide the organization with information needed to
make decisions about the future of the project.
• To assess whether the organization/project is having
the planned impact in order to decide whether or not to
replicate the model elsewhere.
• To assess the programme in terms of effectiveness,
impact on the target group, efficiency and sustainability
in order to improve its functioning.

6. 6
Programme Evaluation
 Ex Ante (Before)
• Can the programme achieve the results set out in the plan?
 Mid Term (Mid way)
• How are we doing and what changes are needed?
 Ex Post (After)
• What did we achieve vs plan?

7. 7
Project Evaluation
• Project evaluation is a structured process
comprising:
 Collection of relevant information
 Assessment of this information against agreed criteria
 Reporting of the evaluation results
 Incorporation of these results into future planning

8. What intended beneficiaries needed?
8

9. What intended beneficiaries asked for?
9

10. What the NGOs promised …
10

11. What intended beneficiaries felt…
11

12. What actually was delivered…
12

13. A Case for discussion
• A Rural water supply project was initiated in 1990 on
community participatory basis. Project was to be completed
in 1994 with cost Rs. 100 million but it actually completed in
1993 with cost Rs. 80 million.
• Was project a success story? OR
• Do you need some additional information to make judgment?
13

14. A Case for discussion
• Project in question, was to provide safe drinking water to
1 Lac families living in 25 villages by 2008 but it was
revealed it was accessible to only 25,000 families by that
year;
• Was the project a success story?
14

15. A Case for discussion
• Project in question, was claimed to reduce incidence of
water borne diseases by 40% (1990=70%) by 2004;
• But in actual incidences increased to 90% by 2004
15

16. A Case for discussion
• A survey was carried out in 2005 to see how many water
supply schemes were still in operation and how many had
ceased to exist;
• Only 10% were found operational;
• Is project a success story?
16

17. The Result Chain
17

18. The Power of Measuring Results
• If you do not measure results, you cannot tell success or
failure
• If you can not see success, you cannot reward it
• If you cannot reward success, you are probably rewarding
failure
• If you cannot see success, you cannot learn from it
• If you cannot recognize failure, you cannot correct it
• If you can demonstrate results, you can win public
support
18

19. 19
Monitoring vs Evaluation
Monitoring Evaluation
Who? Internal management
responsibility – all levels
Usually incorporates external inputs
When? Ongoing Periodic – mid-term, completion, ex-
post
Why? Check progress, take remedial
action, update plans
Learn broad lessons applicable to
other programs/projects, policy
review, etc
Focus on Inputs, activities, outputs Results, purpose, overall objective

20. Monitoring vs Evaluation
20
Monitoring
• Routine collection of
information
• Tracking implementation
progress
• Measuring efficiency
“Is the project doing things
right ?”
Evaluation
• Analyzing information
• Ex-post assessment of
effectiveness and impact
• Confirming project expectations
• Measuring impacts
“Has the project done the right
things?”

21. 21
Reporting of results to:
• Sponsors
• Decision Makers
• Stakeholders
• Partner Organisations
• Beneficiaries
Why Evaluate?

22. 22
 Relevance
 Efficiency
 Effectiveness
 Impact
 Sustainability
Evaluation and Indicators

23. Relevance
• Relevance is concerned with assessing whether the project was
the right development intervention chosen to address a specific
need of the people;
• For example, a project for constructing a road can be judged for its
relevance by asking whether its objectives were rightly framed in
accordance to the state's development plans
23

24. Efficiency
• Efficiency deals with the relationship between project inputs and
outputs;
• a project may be called an efficient intervention, if it uses minimum
inputs at least available cost to obtain the desired result;
• Often there is a trade-off involved with quality of the inputs versus
the costs involved;
• least-cost methods often compromise on quality. It involves careful
decision making on part of the project executioners
24

25. Effectiveness
• Measuring the effectiveness of the project refers to how well it fulfilled
its objectives, by taking into account the effect of the project on the
beneficiaries. When assessing project effectiveness, the following
questions need to be asked:
• How have the conditions of the intended beneficiaries group changed
since the beginning of the development intervention?
• To what extent was the change in the conditions caused by factors
external to the project.
25

26. Impact
• Impact is a measure of all significant effects of the development
intervention, positive or negative, expected or unforeseen, on its
beneficiaries and other affected parties;
• For most projects, it gets increasingly difficult to attribute broad
effects to specific causes. This is primarily due to the fact that
various number of factors that can lead to that particular effect;
• Hence in order to conduct an impact evaluation, which aims at
measuring the effect of an intervention, an estimation of a
counterfactual situation is required. This can be achieved by
comparing the two identical situations - one which has been
subjected to the intervention and one which has not
26

27. Sustainability
• A project may fulfill all the above criteria, but if its positive impacts are
not long-lasting, the project loses its worthiness;
• Sustainability deals with asking questions such as the availability of
sufficient resources to maintain project results. It is often the most
rigorous evaluation measure as it is concerned with how well a project
is environmentally and financially sustainable;
• For example, sustainability of a road project could be measured by the
likelihood of the road being maintained and its perceived usefulness in
the future, keeping in mind forthcoming developments surrounding the
constructed road.
27

28. 28
Evaluation and Indicators
Needs
Objectives Inputs Operations Outputs
Objectives
Impact
Relevance
Sustainability
Effectiveness
Efficiency
Socio-economic problems
Programme
Evaluation

29. 29
One for the road!
It is not necessarily a crime to be running late…………
……..It is always a crime not to know!
You can only manage the work that remains to be done!
Getting people to think ahead
is 50% of the benefit of project planning
A successful project is one that where outputs are delivered on time,
within budget, of the required quality…….
…….and no one died of a heart attack