The most significant change as a part of a project evaluation in india
The Most Significant ChangeProLearn Project in IndiaInka PíbilováCzech Evaluation Society Conference in Prague, 28 May 2013
A „tsunami relief“ project 2004 first tsunami relief in Andra Pradesh India funded byindividual donors of ADRA Czech Republic 2007 needs assessment with Education Office among 20schools 2008 – 2009 infrastructure development in 8 schools toincrease and retain the number of children– Classrooms construction– Teacher training– Learning aids– Community sensitisation– Children participation
Evaluation Objectives• Show evidence of the project impact on access toeducation, increased education quality and child-friendlylearning environment, thus achieving learning outcomesand higher literacy.• Show evidence of attitudinal changes of community,school staff, children and government representatives ineducation.• Assess sustainability of the “Pro Learn” project afterADRA withdrew support.• Draw lessons learnt and recommendations forimprovements in future projects with respect toplanning, implementation, monitoring and evaluation.
Evaluation Methodology - Factors1. Logical matrix did not correspond to reality2. Audit of the construction done, client interested inimpacts and sustainability.3. Donor did not have experience with evaluationsand did not have a specific method in mind.4. Lack of funding to conduct the evaluation.5. Lack of baseline data.6. Field mission restricted to 3 weeks
TermsofReference–Objectives,scope,stakeholders,questions,budget,schedule,outputs,use.Desk studyInterviewsThe MostSignificantChangeCommunityconversationsObservationsPreliminaryfindings&conclusionsFinaldebriefingof project teamandcommunityvolunteersCommunication with the Project PartnersDraft evaluationreportFinalevaluationreportInceptionphase1 monthFieldresearch1 monthsReportingphase2 monthsInitialbriefingandinceptionEvaluatorSelectionEvaluation Methodology
What were the findings• Renovated or constructed school buildings and games forchildren were appreciated, but...
What were the findingsThe Most Significant Change was the encreasedempowerment of children in established Child Clubs, whowere able to secure access to potable water or cleantoilets from the local authorities within 3-6 months.What were the findings
Communication of findings• Fiel mission closed after evaluation finished - keycommunication had to be done at the end of the evaluation.• The local community was to a big extent illiterate - communityvolunteers were expected to debrief the respectivecommunities.• Headquarters in India unavailable for debriefing – done bySkype, donor organisation debriefed back in Europe.• Donor organisation decided to delay the communication andfinally did not published the evaluation report.• Though a short debriefing was held with District EducationOffice, despite the original plan findings were not officiallycommunicated to school management, teachers and students.• Stories had already been used in local media.
Lessons Learnt• The method can be used for evaluating impacts of projectsrelated to behavioral changes as an additional data collectionmethod, even in case of lack of resources.• Secure and train reliable volunteers / interviewers.• The atmosphere of trust and openess is key (interpreting)• Verify the stories in the field.• Ensure availability of key stakeholders in the selection of theMost Significant Change story.• The stories help in communicating successes of the project,therefore communication incl. budget needs to be carefullyplanned (to be a part of the evaluation mission or a subsequentproject atcivity)
LinksDetailed information on the method, including studies, evaluationreports etc. :• http://mande.co.uk/special-issues/most-significant-change-msc/• http://groups.yahoo.com/group/MostSignificantChangesGuide:• http://www.mande.co.uk/docs/MSCGuide.pdfTrainings currently available in Europe:• www.socialimpactconsulting.com.au• www.traceydelaney.net/