Information note Most Significant Change  - Ruberto/Pradhan
Upcoming SlideShare
Loading in...5
×
 

Information note Most Significant Change - Ruberto/Pradhan

on

  • 979 views

Information Note prepared for the Bangkok KM/KS write-shop (IFAD) about Scampis MSC experience

Information Note prepared for the Bangkok KM/KS write-shop (IFAD) about Scampis MSC experience

Statistics

Views

Total Views
979
Views on SlideShare
537
Embed Views
442

Actions

Likes
0
Downloads
8
Comments
0

29 Embeds 442

http://scampisblog.blogspot.it 145
http://www.scampisblog.blogspot.it 66
http://www.scampisblog.blogspot.com 58
http://scampisblog.blogspot.com 52
http://scampisblog.blogspot.co.nz 18
http://www.scampisblog.blogspot.co.nz 17
http://www.scampisblog.blogspot.nl 8
http://www.scampisblog.blogspot.co.uk 8
http://www.scampisblog.blogspot.com.au 8
http://www.blogger.com 7
http://scampisblog.blogspot.co.uk 6
http://scampisblog.blogspot.in 6
http://scampisblog.blogspot.be 4
http://scampisblog.blogspot.nl 4
http://scampisblog.blogspot.ru 4
http://www.scampisblog.blogspot.in 4
http://www.scampisblog.blogspot.fr 3
http://scampisblog.blogspot.fr 3
http://scampisblog.blogspot.ca 3
http://scampisblog.blogspot.de 3
http://www.scampisblog.blogspot.com.ar 3
http://scampisblog.blogspot.com.ar 2
http://www.scampisblog.blogspot.ca 2
http://scampisblog.blogspot.com.au 2
http://scampisblog.blogspot.com.br 2
http://scampisblog.blogspot.com.es 1
http://www.scampisblog.blogspot.ch 1
http://scampisblog.blogspot.sg 1
http://www.scampisblog.blogspot.be 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Information note Most Significant Change  - Ruberto/Pradhan Information note Most Significant Change - Ruberto/Pradhan Document Transcript

  • Information note:“RBM KS in IFAD RuralDevelopment Context”Bangkok, 3-7 December  Cecilia Ruberto,M&E consultant, IFAD Hq Policy and TechnicalAdvisory division, Rome, Italy. Mail:c.ruberto@ifad.org andceciliaruberto@gmail.com Skype: ceci1982lia  Nrusingh Charan Pradhan,Project Coordinator, InternationalDevelopment Enterprises India, Plot No. LocalShopping Centre, Sector-12, Dwarka, NewDelhi-110075 (India)nrusingh.p@ide-india.orgncp.bbsr@gmail.com
  • ContentsAbstract ................................................................................................................................................ 21.What is Most significant Change? ...................................................................................................... 22. The SCAMPIS MSC technique ............................................................................................................ 3 Stories collections ............................................................................................................................. 3 The analysis implemented at various levels ...................................................................................... 4 Lessons Learned ................................................................................................................................ 53. Experience of Most Significant Change (MSC) Methodology under the SCAMPIS Project in Odisha. 5 Staff Training & Kids Training ............................................................................................................ 5 Kids/ Youth Training .......................................................................................................................... 6 Stories collection: methods and results ............................................................................................ 6 Second level analysis: methods and results (how it has been done, main results and lessons learned about the methodology ....................................................................................................... 7 Overall Lessons learned MSC Scampis India ..................................................................................... 7Web references .................................................................................................................................... 8Bibliography .......................................................................................................................................... 8List of AcronymsSCAMPIS Scaling up of Micro Irrigation SystemsMIS Micro Irrigation SystemsLOF Liquid Organic FertilizersSTP Surface Treadle PumpBK Bucket KitDK Drum KitIFAD International Fund for Agriculture DevelopmentMSC Most Significant ChangeNGO Non Government OrganisationRBM Resource Based ManagementKM Knowledge managementKS Knowledge SharingM&E Monitoring & EvaluationOTELP Odisha Tribal Empowerment and Livelihoods programme BKK Information Note – Scampis MSC experience 1
  • AbstractIn 2012 the Scampis project1 introduced an innovative M&E methodology to complement thequantitative data collection. The project, implemented in India, Guatemala and Madagascarexperimented in the first 2 countries the Most Significant Change methodology.The main objective in using MSC was to maximize the possibilities to learn from the project, from thebeneficiaries and the staff involved and to discover hidden factors/dynamics that are the cause ofthe project success or not. Factors invisible to quantitative, pre-established M&E tools.Many are the lessons learned, problems solved and problems still open in the adaptation andapplication of the MSC method, but overall we can say that in both India and Guatemala it has beenappreciated very much and also from the Hq. point of view it has been very important to solve sensemaking issues and create connection among different (and far) stakeholders.The experience has been documented in an informal blog: www.scampisblog.blogspot.com . Somework should still be done to capitalize the experience, for sure the same learning from it for furtherinnovation and use in new context is part of the capitalization process, as something will not bereally part of our background until we do not experiment it practically. For the advantage of theBangkok write-shop event a synthetic overview of what has been the Scampis MSC experience withthe exemplification of the India case study is offered. The Guatemala experience is available here.1.What is Most significant Change? The most significant change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded (collection of significant change (SC) stories) and in analysing the data. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole. And it contributes in strengthening organization learning and communication through the analysis process, when the staff comes together to discuss about the perceived MSC or the not perceived (Answering to the question: “what went wrong?”) Why collecting stories? • People tell stories naturally - indigenous • Stories can deal with complexity and context • People remember stories • Stories can carry hard messages /undiscussables • But stories not known for accuracy/truth Why the analysis process • Analysis for discussions/communication • For learning: elaborate information and transform it in practical lessons • For improving our work/our organization work: increase efficetiveness in our activities • To create bridges among far places and different cultures/experiences • To understand numbers • To discover new important element difficoultly perceived by normal M&E tools (especially if qualitative)1 http://www.ifad.org/english/water/scampis/ BKK Information Note – Scampis MSC experience 2
  • Why MSC is relevant for us? To complete qualitatively the quantitative M&E analysis • To improve planning (having a better understanding of what is going on in the field) • To help explain a point to a farmer (can be used as KM material) • To recruit new participants • To help explain a point to another member of staff2. The SCAMPIS MSC techniqueIn our case, the SCAMPIS project, the technique was extremely relevant to complete theparticipatory M&E process started in 2010 2. The methodology was adopted by SCAMPIS inGuatemala and India, developed in a participatory way with IFAD Scampis M&E responsible. The toolconsisted in the implementation of 4 steps: • Stories collection • Field level analysis and selection / second field level analysis and selection • National level Analysis (Hq) (only analysis no more selection) • Cross-countries analysisThe methodology have been reshaped and modified to better match: specific needs and context,limited availability of resources (money, time).The main goal for Scampis MSC were: i) understanding M&E numbers, ii) knowledge and learn about(unexpected) farmers perception of the project/technology, iii) fill the gaps between field reality andhigher top management or donor, iv) involve all staff in a coherent process of discussion andreflection of the work conducted.Stories collections3The methodology has been reshaped from the original MSC (Davis and Dart) in order to adapt tospecific goal and context.Collected by kids (age between 13-16 years) who will contemporaneously, as real reporters, film,photograph and ask questions to the project beneficiaries. A training for the kids was provided at thebeginning of the day to break the ice, practice the use of the technologies and understand thereason why of the activity as well the questions for the adults.Why Kids or young people?? They are curious, they can easily learn how to use the technology, theywill let the interviewed feel comfortable (elder or senior people are not advisable as they wouldinfluence to much the interview). They can at the best preserve the “ethic” of the MSC storiescollection that is: DO NOT INFLUENCE THE ANSWERS, LET COME OUT THE REAL STORY OF THEINTERVIEWED. 4 Finally the see how important they are for their community!2 Video on Scampis M&E and MSC http://www.youtube.com/watch?v=LbjJ4If9yUI3 Guatemala photos here: http://www.youtube.com/playlist?list=PLA547A54E873204EB&feature=addto ; and videos:https://picasaweb.google.com/CeciliaRuberto/ScampisGuatemala20124 Additional information on how kids have been trained and involved in the stories collection, the central role they played,opportunities, pitfalls and lessons learned will be shared separately. BKK Information Note – Scampis MSC experience 3
  • Figure 1 Stories collection India Figure 2 Stories collection GuatemalaThe analysis implemented at various levelsThe analysis at various level was conducted using brain storming modality reflecting on each storyform 4 different point of view: A. Who is the interviewed and what I understood of his specific life experience (general) B. MSC perceived and shared C. Difficulties and risks perceived D. New ideas (hidden stories)After each analysis an overview of the findings have been discussed within the group. During thefirst stories analysis, the staff selected the “best”10 MSC stories, and during the second analysisamong this 10 stories 6 have been chosen as most representatives. The videos were translated andsubtitled.5 After the national analysis a cross-country analysis have been implemented (Guatemalanational staff analysed the Indian video and vice versa). Finally the various analysis plus the stories collected have been collected at IFAD Hq. level for final evaluation and connection to M&E data. Below a representation of the main learning from MSC.Figure 3 Scampis MSC - findings5 Guatemala MSC video playlist: http://www.youtube.com/playlist?list=PLmkP-_uSr1KqUMlmTRPu6crS5FDBsWAGx IndiaMSC video playlist: http://www.youtube.com/playlist?list=PL166815510C378424 BKK Information Note – Scampis MSC experience 4
  • Lessons LearnedLimits of the analysis process were mostly due to time constrains and end of the activities in thefield, no longer staff employed and involved in the project): - Cross-country analysis only at national level and not local level (which would have been extremely useful). - IFAD Hq analysis has not been implemented yet (not sure if will be done..) - Storage of information: even if the MSC community suggested various different tools for appropriate storage and elaboration of the info accumulated (text, photos, videos) there has not been time to learn to use them, so has been used a simple excel programme.The strength of the tool adopted showed strengthens on the major areas of monitoring, evaluation,learning and communicating - Organization development/learning: improve dialogue, trust and democratic discussion. Moreover it can fill the gaps between top management and field reality - Project implementation: Activities implementation & Strategy adaptation: if expected results are not satisfied, the staff will understand it quick and can modify the approach - M&E: Understanding and contextualize quantitative M&E (it is compatible) - Effectiveness: Involve project beneficiaries in improving project effectiveness. - Communication: Improve internal and external communication - Sustainability: it can be easily implemented and managed by communities and staff (low costs). - Evaluation: It can be used for ex-post evaluation3. Experience of Most Significant Change (MSC) Methodology underthe SCAMPIS Project in Odisha.6Staff Training & Kids TrainingThe project facilitators like 2 Senior Marketing Officers, 5 Business Associates, Project Coordinatorand Chief General Manager participated in a two day state level training programme on“Introduction of the new concept of Most Significant Change (MSC). The trainings was attended byall the stakeholders namely, NGOs, Government agencies under the SCAMPIS programme in May-2012 at BhubaneswarThe Scampis IFAD M&E responsible implemented atraining to develop skills and knowledge on theconcept during the training. Basically we learntabout the technique which was completely new andinnovative in nature. The process of adoption waspurely participatory where the local youths/kids andusers would be the main stakeholders in executingthe process in the operational project villages.During training programme, the IFAD officials Figure 4 Youths training, Indiadiscussed with all the participants on M&E and MSCsimilarities and differences, the principles of MSC, importance of getting the stories and also did the6 See authors references in the bibliography section. Additional info on the Scampis India MSC activities andresults: http://www.scampisblog.blogspot.com/p/scampis-india.html .Training event info: http://scampisblog.blogspot.com/2012/11/most-significant-change-training.html BKK Information Note – Scampis MSC experience 5
  • practical exercise on story collection, practical exercise in analysing stories and the role of variousstakeholders.Accordingly we identified two blocks namely Mohana & Raigad of Gajapati district & three blocksPottangi, Nandpur & Dasmanthpur of Koraput district for execution of the MSC tools.Kids/ Youth TrainingAfter completion of the training we proceeded to the identified operational blocks and in the firststep we organised the village meetings to identify user farmers and youth/kids who could be trainedto conduct the interviews. Their identification & training was critical as they were to take the lead inexecuting Most Significant Change (MSC) activities. Subsequently 5 training programmes for thevillage youths/kids was conducted by the trained Senior Marketing Officers, Business Associates ofthe project and also trained staff of local NGOs in 5 blocks. There are few questionnaires developedby the facilitators and put up during interaction in order to support the narration of the story of theusers and understand the individual Most Significant Change: I) Could you tell me your name? II) Where are you from? III) How many are in your family? How many sons and daughters do you have? IV) When did you start using the SCAMPIS technology? Where did you see it for the first time? What did you think of it at the beginning? What technology do you use? V) Do you cultivate and take care of the plot alone? Someone helps you? Who? VI) What did you harvested since you started/ How much? VII) Do you used to cultivate vegetables before having technologies? If answer yes- Do you see any difference? If answer No-Why? VIII) Is the main source of income for your family? Have you sold part of the vegetables produced with MIS? What did you do with the amount obtained from the selling? IX) How was before the project and now, which are the most significant change you perceive? Why those changes are important? The team conducted interactions with the user farmers and shared about the MSC and its purpose.As a result many farmers volunteered to participate and share their leanings, degree of mobilisation,success in adopting the technologies and the benefits they accrued. Based on this, plan for the kidswas made.The youths/Kids were trained in different aspects including conducting interviews, making videos,taking still photography, developing story writing & also in observing and documenting the process.At the end of training, each youth/Kid was asked if they had understood their roles & responsibilitieswell & any doubts were clarified. In all 24 youth were trained and out of these 18 were boys & 6were girls. Those Youths/Kids learned about:  Interaction with project beneficiaries  Videography  Questionnaires  Still Photography  Observation  Documentation of success stories  Field Visits  Analysis of the stories  Focus Group DiscussionStories collection: methods and resultsHow it has been done, how many interviews and the results of the interviews. Highlight difficultiesand ideas for improving it. BKK Information Note – Scampis MSC experience 6
  • After the youth were trained they started conducting the interviews. A total of 24 kids- 18 boys and6 girls were trained and they conducted the interview.Based on the farmer feedback and numbers distributed across the identified locations, theyouths/kids were spread out. 24 kids covered 13 villages in Koraput and Gajapati district from 18-25May’ 2012 and they met with and interviewed 25 user farmers (Male-23 & Female-2). This wasfollowed by organising a one day farmers meeting & training for the youths/Kids in each of theselected villages.Feedback from the participants (MSC Facilitators) on MSC: “I was very much excited about thetraining & facilitating interviews with the user farmers. I learnt many things out of the training andwas enthusiastic how to carry out my responsibilities like making of videos, taking still photographsand interaction with the users etc. After training, I started to think myself how I will do it…what I willsay etc. But when I actually interacted with the farmers I was very surprised to hear them talk somuch. I am personally inspired by the task entrusted to me carried out up to my satisfaction. Thistraining helped me to learn more about M&E. It helped me in understanding the communities andimpact of the development interventions. I was also delighted to hear with the farmers who havebeen benefited out of the project their increased income and how they feel a sense of social dignity.Once all the interviews were completed and videos recorded, these were sent for translation intoEnglish. In the meantime, during IDEI’s monthly SCAMPIS staff meeting, the team invited peoplefrom other NGOs, Odisha Tribal Empowerment and Livelihoods Programme (OTELP), BusinessAssociates (IDEI) and Senior Marketing Officers of SCAMPIS (IDEI) who had attended state leveltraining at Bhubaneswar to analyze all stories and to select 10 best stories (1st level) on MSCcollected by the youths/Kids. This exercise was successfully completed and 10 stories of the total 25were selected for 2nd level analysis.Second level analysis: methods and results (how it has been done, mainresults and lessons learned about the methodologySoon after the 2nd level analysis was undertaken. In the second level analysis senior representativesfrom NGOs, Odisha Tribal Empowerment and Livelihoods Programme (OTELP), Senior MarketingOfficers and Project Coordinator of SCAMPIS were invited to analyze selected 10 stories and toselect best 6 stories on MSC which would be again analyzed and to select best 3 stories at state levelby the next level representatives of IDEI, and IFAD Delhi team.Keeping the guidelines in mind, the second level analysis involved watching the videos together (allrepresentatives present). Everybody was handed over the MSC analysis sheet and each filled theircomments on it. At the end of watching all the videos, six from the ten were selected and six analysissheets (one for each video) was finalized and submitted.Everybody liked the idea of involving the local youth, since it is there habitation they understand thelocal environment better and they would not miss the nuances so this reduces the loss ofinformation.Overall Lessons learned MSC Scampis IndiaPitfalls, opportunities, potentiality of the MSC tool for monitoring, for evaluation (at various level) forstrengthening the organization, limitations of the tool.The overall experience of participating in the MSC training and conducting it was very encouragingfor all involved. First key learning, which we now look as an opportunity is engaging with the localyouth. At first when this was shared we were bit sceptical. We had not involved the local youth in an BKK Information Note – Scampis MSC experience 7
  • evaluation process before like this so the anticipation. However, as we met with the local youth andshared with them about the training and the following work they need to undertake, needless to saythey were very excited and were keen learners.Secondly, MSC is a good technique that has helped us capture the qualitative aspects/ impact of thework. It has helped us go beyond numbers and also actually helped us better analyse the numbersand make cross analysis. Example- if a certain number was achieved (as reported) and in the MSCfarmers shared how they used the income it validated the impact. It helped us get a better insightinto the ‘real’ impact of our work.While it has many strengths, however I feel that it would be useful to always use it as in addition to atool that measures quantitative value of programme success.Web referencesScampis blog www.scampisblog.blogspot.comYoutube (video material) www.youtube.com/ceciliarubertoPicasA https://picasaweb.google.com/CeciliaRubertoScampis IFAD webpage http://www.ifad.org/english/water/scampis/M&E news (Rick Davies’ Blog) http://mande.co.uk/special-issues/most-significant-change-msc/BibliographyThe “Most Significant Change”(MSC) Technique – A Guide to Its Use by Rick Davies and Jess Dart(April 2005). Funded by CARE International, UKThe Most Significant Change Technique (MSC)Power Point document - Dr Jessica Dart, Clear HorizonThe Scampis “Most Significant Change”(MSC) Technique (May 2012) – Cecilia Ruberto, IFADMSC Experience in India, Information Note, November 2012. • Sri Nrusingh Charan Pradhan, Designation: Branch Manager, IDE-India, Mailing Address: A/7, Nilakantha Nagar, Nayapalli, Unit-8, Bhubaneswar-751012, Odisha, India, Email Id: nrusingh.p@ide-india.org • Sri Debendra Kumar Samal, Senior Marketing Officer, IDE-India, mail: debendra@ide- india.org • Sri Tapan Kumar Pattanayak, Chief General Manager, IDE-India, mail: tapan@ide-india.org BKK Information Note – Scampis MSC experience 8