This document describes an evaluation of an education project in India that was funded by donations to rebuild and improve schools after the 2004 tsunami. The evaluation objectives were to show the project's impact on education access, quality and environment. It also aimed to assess sustainability after funding ended. Due to limitations, the evaluation used a methodology called "Most Significant Change," collecting stories from community volunteers about the most impactful changes, such as empowering child clubs to improve water and sanitation. The most significant change found was increased empowerment of children. However, communication of findings was limited due to the donor not publishing the report. Lessons included using stories to communicate success and ensuring stakeholders are engaged in evaluating and communicating results.

1. The Most Significant Change
ProLearn Project in India
Inka Píbilová
Czech Evaluation Society Conference in Prague, 28 May 2013

2. A „tsunami relief“ project
 2004 first tsunami relief in Andra Pradesh India funded by
individual donors of ADRA Czech Republic
 2007 needs assessment with Education Office among 20
schools
 2008 – 2009 infrastructure development in 8 schools to
increase and retain the number of children
– Classrooms construction
– Teacher training
– Learning aids
– Community sensitisation
– Children participation

3. Evaluation Objectives
• Show evidence of the project impact on access to
education, increased education quality and child-friendly
learning environment, thus achieving learning outcomes
and higher literacy.
• Show evidence of attitudinal changes of community,
school staff, children and government representatives in
education.
• Assess sustainability of the “Pro Learn” project after
ADRA withdrew support.
• Draw lessons learnt and recommendations for
improvements in future projects with respect to
planning, implementation, monitoring and evaluation.

4. Evaluation Methodology - Factors
1. Logical matrix did not correspond to reality
2. Audit of the construction done, client interested in
impacts and sustainability.
3. Donor did not have experience with evaluations
and did not have a specific method in mind.
4. Lack of funding to conduct the evaluation.
5. Lack of baseline data.
6. Field mission restricted to 3 weeks

5. TermsofReference–Objectives,
scope,stakeholders,questions,
budget,schedule,outputs,use.
Desk study
Interviews
The Most
Significant
Change
Community
conversations
Observations
Preliminaryfindings&conclusions
Final
debriefing
of project team
and
community
volunteers
Communication with the Project Partners
Draft evaluation
report
Finalevaluationreport
Inception
phase
1 month
Field
research
1 months
Reporting
phase
2 months
Initialbriefingandinception
Evaluator
Selection
Evaluation Methodology

6. http://www.mande.co.uk/docs/MSCGuide.pdf

7. http://www.mande.co.uk/docs/MSCGuide.pdf

8. http://www.mande.co.uk/docs/MSCGuide.pdf

9. How was the technique applied
MSC Training of
community
volunteers
Collection of
the MSC
stories by
volunteers
Selection of the
MSC with
community
volunteers and
project team
MSC drawings by
children

10. How we selected the „MSC story“

11. What were the findings
• Renovated or constructed school buildings and games for
children were appreciated, but...

12. What were the findings
The Most Significant Change was the encreased
empowerment of children in established Child Clubs, who
were able to secure access to potable water or clean
toilets from the local authorities within 3-6 months.
What were the findings

13. Communication of findings
• Fiel mission closed after evaluation finished - key
communication had to be done at the end of the evaluation.
• The local community was to a big extent illiterate - community
volunteers were expected to debrief the respective
communities.
• Headquarters in India unavailable for debriefing – done by
Skype, donor organisation debriefed back in Europe.
• Donor organisation decided to delay the communication and
finally did not published the evaluation report.
• Though a short debriefing was held with District Education
Office, despite the original plan findings were not officially
communicated to school management, teachers and students.
• Stories had already been used in local media.

14. Lessons Learnt
• The method can be used for evaluating impacts of projects
related to behavioral changes as an additional data collection
method, even in case of lack of resources.
• Secure and train reliable volunteers / interviewers.
• The atmosphere of trust and openess is key (interpreting)
• Verify the stories in the field.
• Ensure availability of key stakeholders in the selection of the
Most Significant Change story.
• The stories help in communicating successes of the project,
therefore communication incl. budget needs to be carefully
planned (to be a part of the evaluation mission or a subsequent
project atcivity)

15. Links
Detailed information on the method, including studies, evaluation
reports etc. :
• http://mande.co.uk/special-issues/most-significant-change-
msc/
• http://groups.yahoo.com/group/MostSignificantChanges
Guide:
• http://www.mande.co.uk/docs/MSCGuide.pdf
Trainings currently available in Europe:
• www.socialimpactconsulting.com.au
• www.traceydelaney.net/

16. Any questions?
Inka Píbilová
inka@evaluace.com