Introduction to Participatory Monitoring-Evaluation


Published on

A simplified trainer’s guide

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Introduction to Participatory Monitoring-Evaluation

  2. 2. Published by West Africa Rural Foundation (WARF) N° 10075, Sacre- Cœur III- VDN - C.P. 13 DAKAR-Fann, SENEGAL Tel: (221) 33 865 00 60 - Fax: (221) 33 860 66 89 - E-mail: - Web site: Many thanks to: All IFAD funded projects in West and Central Africa for their contribution, WARF staff and consultants © FRAO/WARF - Dakar, September 2012 Design: This publication was made possible by grant from IFAD for the Programme of Support for Monitoring and Evaluation Systems of IFAD projects in West and Central Africa. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of IFAD.4 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  3. 3. TABLE OF CONTENTSACRONYMS AND ABBREVIATIONS 06FOREWORD 07SESSION 1: Introduction to the workshop 08SESSION 2: Brief overview of the participatory approach 12SESSION 3: Introduction to participatory monitoring and evaluation (PME) 15SESSION 4: Importance of PME for IFAD projects 18SESSION 5: Stages of the PME process 22SESSION 6: Setting up a pme system 24SESSION 7: Identifying actors and analyzing their needs 26SESSION 8: Definition of PME objectives 29SESSION 9: Choice of PME indicators 31SESSION 10: Choice of information gathering methods 35SESSION 11: Information gathering and analysis 37SESSION 12: Implementing changes 45SESSION 13: Management and consolidation of the PME system 47SESSION 14: Monitoring and evaluation of the PME implementation 49 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 5
  4. 4. ACRONYMS AND ABBREVIATIONSCAS Community Assessment SheetIFAD International Fund for Agricultural DevelopmentWARF West Africa Rural FoundationIED Afrique Innovation, Environment and Development in AfricaMFI Microfinancing institutionNHDI National Human Development InitiativeOVI Objectively verifiable indicatorPRA Participatory Rural AppraisalMSRE Micro and Small Rural EnterprisesOP Organisation of ProducersIAP Information and Awareness-raising ProvidersPROMER Rural Micro-Enterprises Promotion ProjectPME Participatory Monitoring and EvaluationSFPO Success, Failures, Potentials, ObstaclesMES Monitoring and Evaluation SystemRIMS Results and Impact Management SystemPMU Project Management Unit6 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  5. 5. FOREWORDT his simplified guide is addressed to facilitators who pursued the trainers’ training course held in Toubab Dialao, Senegal in April 2008. These trainers will later be required to train officials in charge of Monitoring and Evaluation of IFAD projects in Participatory Monitoring and Evaluation (PME).These skills should enable the latter to assist beneficiaries of their interventions to set up a ParticipatoryMonitoring and Evaluation system. Thus, the grassroots communities are the final beneficiaries of thiscapacity building process. For this reason, the content of this guide has been simplified; with emphasisessentially placed on the aspects that are of practical relevance for the establishment of the PME systemat local level. Of course, facilitators are strongly encouraged to improve on it by drawing on their ownexperience or through documentary research.The design of the training content and guide are based on two fundamental assumptions.The first is that the selected trainers sufficiently master participatory research tools (PRA, ParticipatoryDiagnosis) to be able to adapt and/or improve the proposed tools. The second assumption is that thetrainers have a certain experience in educational activities. The document provides trainers with thebasic elements required to prepare the staff of the concerned projects to assist the communities to setup a PME system.The document is organized in such a way as to provide (pedagogical) guidelines on the conduct of train-ing sessions and content on the PME system’s implementation process. Thus, each session is organizedaround two sheets.The first sheet (referred to, by convention, as learning exercise) proposes a pedagogical approach forthe conduct of the session. To effectively apply these pedagogical indications it must be ensured that allthe necessary logistics (including the availability of rooms for group work) are in place. The pedagogicalproposals are indicative. The facilitators can modify them to ensure that they adapt to the context ofeach session. They can also be replaced by other techniques considered more appropriate. Each ses-sion bears a number.The second sheet (referred to as content sheet) provides a content summary to which the facilitatormay refer to provide additional information. Of course, these sheets are not exhaustive. Facilitators aretherefore strongly urged to enrich them with their own research and additional documentation in theirpossession. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 7
  6. 6. SESSION 1 INTRODUCTION TO THE WORKSHOP LEARNING EXERCISE 1 PRINCIPLES AND PEDAGOGICAL PROTOCOL1. Objectives of the session • Inform participants about the context of the workshop • Identity participants • Determine the workshop objectives • Define the workshop’s management principles • Define the working rules (group contract)2. Proposed technique: Interaction, brainstorming3. Material and equipment required:Required materials and equipment: Padex paper, padex board, markers of different colours, wallsellotape, index cards of different colours4. Total duration of session: 60 mn5. Educational progression • Prior to participants’ arrival and installation, the facilitator should check the state of preparation of the room, the learning facilitation system and the equipment installed the day before. • The room should be arranged in U-form or in staggered rows depending on the space available • The tables should be mobile for the organization of group work • The order in which participants will intervene is determined before the start of the session Expression of expectations: • Provide each participant with three index cards on which he/she is required to state what he/she expects from this workshop (ex: Achievements to be consolidated, constraints/difficulties to be removed, needs to be met). • The cards are then collected, analyzed and classified according to category. • These expectations are analyzed on the basis of the objectives predefined by the organizers. • The workshop agenda is presented to participants, discussed and adopted • The draft group contract is presented to participants, amended and validated (Content card 1a)6. Information provided by the facilitatorThe facilitator introduces the learning from experience approach which will guide the teaching facilitationapproach (CONTENT CARD1b). The session’s work is summarised.8 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  7. 7. SESSION 1 INTRODUCTION TO THE WORKSHOP CONTENT CARD 1A1. Group contract • Switch off or set mobile phones on vibration mode • Answer telephone calls outside the room • Ask for the floor before speaking • Listen to one another • Avoid repeating what others have already said • Be brief to facilitate effective time management • Limit going in and out of the room2. Workshop principle • Learning based on experience • Allow participants to express their expertise • Diversify the interventions • Progress together on the basis of consensus3. Organization of discussion sessions • Exchanges and discussions on salient points • Diversify teaching techniques • Production of ideas or experience acquisition to be listed on an index card • Plenary to present the outcome and collective analysis • Designation of a rapporteur for the day4. Management of group work • Individual reflection based on past experience • Pooling • Write the ideas discussed on the index card • An idea per card • Write legibly, preferably in block letters PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 9
  8. 8. SESSION 1 INTRODUCTION TO THE WORKSHOP CONTENT CARD 1BStage 1: Sharing the participant’s experience (or a similar experience)The facilitator can use different tools or techniques such as case studies, simulation or role playing,group work, showing a documentary film, brainstorming during which participants talk about their expe-riences, etc. There is however need to underline that regardless of the technique used by the facilitator,prior preparation is essential.This can be very time-consuming. For example, if the facilitator wants to use case studies, he shouldgather and organize the information long before the training. He can use case studies already preparedbut should examine them thoroughly himself in order to be able to answer questions that participantscould ask him. In case he wants to introduce a role playing exercise, he should prepare his scenariobeforehand with a clear definition of the different roles, the definition of adapted profiles to play the roles,etc.Stage 2: Reflection, conceptualizationAfter presenting the experience/experiment, the facilitator invites participants to share their views aboutthis experience. To properly organize the discussions, the facilitator prepares a certain number of keyquestions beforehand? For example: What are the key observations? How do they explain them? Whatare the lessons learnt? etc. All the ideas are written, as and when required, on the board or cards to bedisplayed on a notice board prepared beforehand.Stage 3: Conclusions, generalizationDuring the third stage, the facilitator establishes the link between the analyses made by participantsand the concepts or theory he wants to introduce. He will seize this opportunity to introduce any otheridea or concept which had not been mentioned by participants. It will be discovered that most of theconcepts which the facilitator wants to introduce during this session would have been mentioned by par-ticipants. This stage thus provides the facilitator with the opportunity to show participants that they al-ready have a real-life experience and an experience with these concepts; since the facilitator’s role isto help them express their knowledge. In short, help them to discover the knowledge hidden in each ofthem. This approach has another advantage because it helps to build the participant’s confidence, sincethere is little space for theoretical presentations.10 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  9. 9. SESSION 1 INTRODUCTION TO THE WORKSHOP CONTENT CARD 1CApplicationThe last stage consists of the practical application of these new concepts. This application is generallymade after the training, when the participant returns to his/her working environment. It is therefore ex-tremely important to evaluate the training application. This evaluation is generally done by the trainee’sstructure of origin. But the introduction of a practical application stage in the training provides participantswith the opportunity to test some of the concepts introduced during the training. Experience Application Reflection General Conclusion PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 11
  10. 10. BRIEF OVERVIEW OF THE SESSION 2 PARTICIPATORY APPROACH WORKSHEET 21. Objectives of the session • Ensure that all participants have basic knowledge of participatory research principles, techniques and tools • Provide participants with additional information, if necessary2. Proposed technique: Show a film on « Question of Differences» : Gender and PRA or another film(the film is among the distributed resources; the technique can therefore be applied by trainers).3. Material and equipment required: Television, Cameoscope VHS, portable computer, padex paper,padex board, DVD/CD or pen drive.4. Duration of session: 60 mn5. Educational progression • Before showing the film, the facilitator informs participants that they should take note of the different parts (the principles, tools, recommendations, etc.) • Show the first part of the film on the principles of communication and introduction to PRA. • After the film show, open a discussion in plenary to ask participants to share the essential points they have identified. It will be appropriate to proceded by theme by theme to facilitate the discussions and the organization of information. • Ask one of the participants to write the ideas on the padex board as they are given. • When the discussion focuses on tools, ask participants whether they are all familiar with these tools and/or whether they know others (to be noted on the table). • At the end of the discussion, the facilitator can provide documentation sources to participants re- quiring additional training. To that end, it will be useful to prepare a small documentation for distribu- tion.6. Information provided by the facilitatorIn case the facilitator has no film to show, he may use another teaching method.For example, he can organize a brainstorming session around the following key questions: - What is participation? - What are its main principles? - What conditions are absolutely necessary for its implementation?. - Which participatory tools do you know? - What are the constraints and difficulties linked to the implementation of such participation? NB : This session is not meant to provide in-depth training on participatory research; since being well-versed in the participatory approach principles, techniques and tools is among the criteria on which the selection of trainers is based. The purpose of the session is thus to recall these principles, techniques and tools.12 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  11. 11. BRIEF OVERVIEW OF THE SESSION 2 PARTICIPATORY APPROACH CONTENT CARD 2 ON PARTICIPATIONWhat is participation?Participation is an active process in which the initiative is taken or «jointly taken» by the people, guidedby their own ideas and using means and processes over which they have no effective control(Guèye,1999).Participation therefore means taking an active and decisive part in decisions concerning what shouldbe done, how it is done, profit sharing, monitoring and evaluation.On the effectiveness of participation in development projectsDifferent projects apply the principles of participation in different ways. Some involve the populationsfully during the identification of problems and the planning of actions; but they quickly return to their oldpractices during the implementation stage.Other projects define their priorities and choose actions without the population’s participation and try to« catch-up » by entrusting a few responsibilities to beneficiaries.Others limit themselves with involving the populations in the monitoring and evaluation stage.Of course, none of these three situations is to be encouraged because the beneficiaries’ participationshould be pursued at all stages of a programme’s life cycle. Question: How do you perceive the effectiveness of participation in your project ? Project’s internalfactors The project’s internal that encourage the constraints and limits Poor Average High populations participation restricting the popula- tions participation Planning Implementation Monitoring and Evaluation PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 13
  12. 12. BRIEF OVERVIEW OF THE SESSION 2 PARTICIPATORY APPROACH WORKSHEET 2BConditions of participationTo ensure effective participation, the following three conditions should be simultaneously met: • All groups, particularly the so-called vulnerable ones, should be present • Such participation should not be limited to physical presence during decision-making; it should rather translate into an effective influence on the decisions and the monitoring of their implementation • It should not be limited to a specific stage of the project cycle but should be effective at all stages.Some principles to be respected? • The adoption of an interdisciplinary approach to better take into account the diversity of situations and perspectives in the analysis of local situations. However, interdisciplinarity supposes the inte- gration, in the team, of resource persons from the local community. • Making optimum use of local knowledge and capacities in the analysis of problems and the choice of solutions. The institution of PME provides a good opportunity for the application of this principle since the beneficiaries become key actors of the system while the facilitator/presenter plays the fa- cilitating role. • The adoption of an iterative approach, which makes it possible to better link the reflection and the action and immediately apply, in the field, ideas generated through collective reflection and action research. • The adoption, by project agents, of attitudes and behaviours that recognize the populations’ central position in the analysis, planning, implementation, monitoring-evaluation process and their prepared- ness to establish a relationship of partnership and mutual learning with the populations. • Emphasis to be placed on visualization tools which facilitate communication and everyone’s partic- ipation including those who can neither read or write.14 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  13. 13. INTRODUCTION TO PARTICIPATORY SESSION 3 MONITORING AND EVALUATION (PME) WORKSHEET 31. Objectives of the session • Define monitoring, evaluation and participatory monitoring and evaluation notions • Identify the main characteristics of participatory monitoring and evaluation • Analyze the differences between PME and conventional monitoring and evaluation2. Proposed technique: Group work3. Material and equipment required: Kraft paper, markers in sufficientnumbers,cards of different colours4. Duration of the session: 70 mn5. Educational progression • Form groups of 5 to 7 people • Give 5 cards of the same colour to each group • Ask the following focal question: «What are the main characteristics of participatory monitoring and evaluation? » • Give 10 mn to groups to identify the main characteristics of participatory monitoring and evaluation • Prepare an area (kraft paper pinned on the wall, boards, etc.). • Ask the rapporteurs of different groups to present the results of their reflection • Classify the cards by regrouping those relating to the same idea in order to obtain a group of key points (ask a participant to do this classification) • Make a summary of the main characteristics of participatory monitoring and evaluation • Get participants to identify the similarities and differences in the definitions and propose a summary definition (5 mn). • The definition retained is written on padex paper and displayed in the room6. Additional information provided by the facilitator:It is at this stage? Which one? that the facilitator may, if necessary, provide additional information, butif the proposed definitions take into account most of the elements contained in CONTENT CARD N° 3,it is recommended that emphasis be laid on the quality of the proposed definitions. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 15
  14. 14. INTRODUCTION TO PARTICIPATORY SESSION 3 MONITORING AND EVALUATION (PME) CONTENT CARD 3Introduction to PME: Definition of conceptsBasic Concepts (a few definitions drawn from the training)Monitoring: • Thorough and regular appraisal of resources, accomplishments and results of the intervention. Ap- proach consisting in ensuring that the planned activities have been carried out • Systematic and ongoing process for the collection of information on a programme or an activity to observe the changes that occurred and propose measures to guide the decisions to be takenEvaluation: • Specific judgement on the programme process, • Assessment of the level of accomplishment of results at the end of the project, • Judgement on the value of an intervention with reference to criteria and standards, • Reflection on the match between actions undertaken and initial objectives • Action of making judgement such as the progress towards meeting the objectives (effectiveness) of a programme or an activity, comparing results and costs (efficiency), or the sustainable changes gen- erated (impact). The initial objective may be formative to help the programme improve its perform- ance. However, other evaluations may lead to more radical decisions: Continue or stop (summative evaluation)Participatory monitoring-evaluation • An ongoing and regular process which actively involves stakeholders in all the stages of collecting, analyzing and using information on an intervention with a view to assessing the processes and results and making recommendations (provide information about decision-making). • It is a process to support the implementation of a development project/programme by grassroots communities and stakeholders which strengthens appropriation, mutual responsibility, transparency and knowledge of interrelations between the results, implementation factors and the environment. Differences between conventional monitoring-evaluation and PME CONVENTIONAL MONITORING PARTICIPATORY MONITORING -EVALUATION (CME) -EVALUATION (PME) Generally the projects or The beneficiaries with projector Who designs the system? programmes programme agents and other actors Indicators often selected More often criteria than indicators When to select indicators? at the start of the project and which are reviewed as time goes on generally remains unchanged Generally the project The populations participate Who analyzes the information? agents actively in the analysis What types of information? Essentially quantitative Pivotal role of the qualitative analysis Taking the different perspectives The diversity of view points is Poor, standardization into account highlighted Poor, standardized indicators Degree of flexibility Iterative process and collection tools Source : Guèye, B16 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  15. 15. INTRODUCTION TO PARTICIPATORY SESSION 3 MONITORING AND EVALUATION (PME) WORKSHEET 3Importance of the PME • The PME contributes to improving transparency in the management of supported programmes sup- ported through the beneficiaries’ direct and ongoing control over decisions concerning resource al- location and the definition of priorities. • At the same time, it contributes to creating a reflex for the accountability reflex and the feeling of re- sponsibility towards beneficiaries. • It strengthens the relevance of programmes and the performance of actions initiated thanks to the populations’ enhanced participation in monitoring their implementation. • It strengthens the feeling of ownership of actions initiated among beneficiaries, and can thus en- courage them to be more involved. • The improvement of transparency in decision-making has led to strengthened confidence between beneficiaries and agents in charge of implementing the programmes • It strengthens the populations’ negotiation, planning and organizing capacities. The effective implementation of PME considerably improves the performance of programmes and the creation of longer lasting impacts. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 17
  16. 16. IMPORTANCE OF PME FOR SESSION 4 IFAD PROJECTS WORKSHEET 41. Objectives of the sessionShow the importance of Participatory Monitoring-Evaluation for IFAD projects2. Proposed technique: Group work, plenary3. Material and equipment required:Padex paper and board, sellotape, markers, video-projector, portable computers,Well-lit room, rooms for group work4. Duration of session: 90 mn5. Educational progression • The facilitator introduces group work. • Divide participants into 2 groups. • Ask the focal question below to which participants should respond: «What improvements could the PME bring about in the monitoring-evaluation system being implemented by IFAD projects»? • Give 40 minutes to the groups. • Ask the different groups to present the results of their reflections. • Open the discussions by asking speakers to lay emphasis on the contributions that may enrich the groups’ proposals.6. Information provided by the facilitatorThe facilitator draws the key lessons based on CONTENT CARD N° 4:18 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  17. 17. IMPORTANCE OF PME FOR SESSION 4 IFAD PROJECTS CONTENT CARD 41. ObjectivesSatisfy the needs relative to core functions in the life of a project • A sound planning of activities • Monitoring results and • Evaluating the project’s impact.These three functions structure the system’s architecture: • Planning • Monitoring results • Evaluating the impactThe discharge of these functions facilitates the project’s supervision, mid-term review and final evalua-tion.The key actors of MES at the Branch and Local levels.At branch levelTwo MSE actors are present at this level: Professional organizations and chambers of commerce.The branch represents the PMU and, in that capacity, constitutes a level of the project’s operationaloversight.To achieve this, it is actively involved in the design and implementation of the MES through: • the validation of teaching materials and methods of the MES; • the collection, consolidation of MES data; • the use of MES products for monitoring (Trend charts); • data transmission at the PMU level; • support to MES actors at branch level;The same activities will be carried out by the SAFIR with respect to component B.PO and Chambers of commerceProfessional organizations (PO) and chambers of commerce of which the MSRE are members, will pro-vide information about the support received and transmitted to MSRE within the framework of the proj-ect.At local levelAt this level, two key actors are present, support service providers and the MSRE which are the project’smain targetsSupport service providersThe project has two categories of providers: information and awareness providers (IAP), specialist/gen-eralist non-financial service providers. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 19
  18. 18. IMPORTANCE OF PME FOR SESSION 4 IFAD PROJECTS CONTENT CARD 4These service providers are involved in the collection, analysis and use of data concerning the servicesthey provide to MSRE; the said data is then forwarded to the head of PMU monitoring-evaluation throughthe branches.The MSRE (micro, very small and small Enterprises), as well as AGR and PIE in the rural areas),These MSRE express their needs for support received from the various providers and MFI; they provideinformation concerning such support through the various providers. At this level, the type of beneficiaries(enterprise) makes it fundamental to collect process and use data to ensure better management in orderto maximize the benefits.Logical FrameworkThe logical framework is the main source for indicators of the monitoring-evaluation system. It helps todefine the project objectives more clearly, and determine the expected causal connections - the « pro-gramme logic » - between the various elements of the chain of events that should lead to the achieve-ment of results: providing resources, implementation terms, outputs, results and impact.In this logical framework, the objectively verifiable indicators are the key tools for monitoring the progressregistered in the achievement of objectives. The hierarchy of indicators follows that of objectives: Theresults achieved by the project and the anticipated effects can be accessed through some indicators,while other indicators are interested in assessing specific objectives.The monitoring mechanisms specify the data sources and the publication systems which will be usedto verify the state of each indicator. They should show what has been achieved in terms of input, results,specific objectives and overall objective. The monitoring mechanisms and information system shouldprovide evidence that the objectives have been achieved.The guidelines for a results and impact management system (RIMS) require the monitoring-evaluationsystem of IFAD-funded projects to fill in a certain number of key indicators; however, these are not ex-clusive, since a wider range of indicators (OVI of the logical framework) continue to be filled by the MES,a range in which the RIMS first and second level performance indicators and impact indicators take animportant position.The identification report presents the logical framework with indicators per level of objectives (overallobjective, specific objectives) as well as results indicators per component.20 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  19. 19. IMPORTANCE OF PME FOR SESSION 4 IFAD PROJECTS CONTENT CARD 4Why a Participatory Monitoring and Evaluation System (PME) ?The purpose of the participatory approach in a monitoring-evaluation system is to guarantee: • that all project actors, particularly the direct beneficiaries, take ownership of the results to ensure that there are more effects and impacts • the involvement of these actors (MSRE, providers of non-financial support services, privileged part- ners) as much as other actors in the collection and analysis of results. • the facilitation of interaction, the development of synergy and consistency in the implementation of activities of the entire IFAD programme. It will therefore take into account the level of IFAD actors, projects and programmes in Senegal. • take into account the specificity of results and their indicators for the different categories of actors • enable direct beneficiaries to generate and participate in collecting data and information on the progress of the project and the achievement of results.Implementing the participatory approach in the monitoring-evaluation systemConsidering the need to collect data on the indicators, actors of the system can be classified into twomajor groups: • illiterates (the majority of RME managers, individual initiative developers) ; • literates (other actors)The collection system should therefore be adapted to the specific or different situation by taking the il-literate target into account, notably at the MSRE level, without changing the data quality.There is thus need to provide illiterate beneficiaries with the appropriate collection materials to enablethem to monitor and evaluate their activities and results.Regardless of its simplicity, this system has to be buttressed by a results collection and analysis capacitybuilding process. This capacity building process will be continuous and iterative in order to adapt betterto the monitoring-evaluation system. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 21
  20. 20. SESSION 5 STAGES OF THE PME PROCESS WORKSHEET 51. Objectives of the sessionIdentify the main stages for the setting up of a results-based PME system.2. Proposed technique: group work, role playing, plenary3. Material and equipment required: Padex paper and board, sellotape, markers, video-projector,portable computers, well-lit room, rooms for group work4. Duration of session: 90 mn5. Learning approachIntroductionIntroduce the theme of the session: • Divide participants into groups of 5 to 6 people, and ensure that the composition of groups alternates to avoid having the same people in the same groups all the time • Give a set of 10 cards to each group • Explain to the groups that they are required to identify the main stages which, in their views, are needed to set a participatory monitoring-evaluation system. They have a maximum of 10 cards. The number of stages should not be more than 10. • They have 15 minutes to discuss and identify the main PME stages • Prepare a space (kraft paper pinned on the wall, ZOPP boards, etc.). • At the end of the allowed time, ask the rapporteurs of different groups to present the cards on the PME stages • Choose a voluntary participant to classify the cards by regrouping those relating to the same stage in order to obtain a group of key points • Start a discussion to enable participants to draw a picture of the main stages • The main stages are written on a padex paper6. Information provided by the facilitatorAt this stage, the facilitator may, if necessary, provide additional information. On the basis of CONTENTCARD N° 5, he should check whether there are any missing key stages. If any, he can again provokethe participants’ reflection to get them to identify the missing stage(s).22 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  21. 21. SESSION 5 STAGES OF THE PME PROCESS WORKSHEET 5 STAGES OF THE PME PROCESS 1 Decide to set up the system 2 7 Implement actions Identify the actors for change Management Define the Expectations and consolidation and Objectives 3 of PME system Collect and 6 analyze information Identify the Criteria and Indicators 4 5 Chose the information collection methods and tools Bara Guèye: Introduction to Participatory Monitoring and Evaluation PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 23
  22. 22. SESSION 6 SETTING UP A PME SYSTEM WORKSHEET 61. Objectives of the session • Provide participants with information system, suggestions about the setting up of a PME • Share analysis tools in the process for the setting up of a monitoring-evaluation system • Provide participants with additional methodological guidance, if necessary2. Proposed technique: brainstorming3. Material and equipment required: Cards, padex paper and boards, markers, sellotape,4. Duration of session: 60 mn5. Educational progression • Organize brainstorming based on the following focal questions: In the specific context of your project or organisation, what reasons or factors motivated the decision to set up a PME system? What are the constraints and opportunities generated by the setting up of PME in your projects • The answers are noted on the index cards and then organized and discussed • The facilitator then introduces CONTENT CARD N° 7 • He asks representatives of different projects participating in the workshop to fill aide-mémoire card 7 (if a project has several representatives, they form a group). • At the end of this exercise, the groups present their results. • These results are discussed6. Information provided by the facilitatorAt the end of the discussions, the facilitator invites participants to make suggestions and draw conclu-sions by referring to CONTENT CARD N° 6.24 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  23. 23. SESSION 6 SETTING UP A PME SYSTEM CONTENT CARD 6 Stage 1: DECIDE TO SET UP THE MONITORING SYSTEM ANALYSIS WORKSHEET ON THE CONDITIONS OF PME IN IFAD PROJECTSWhat institutional conditions/preconditions factors factors Depending on theare required to successfully implement the Favourable Unfavourable reply, whatPME? measures should be taken?The PME is perceived by the project as an or-ganizational needWithin the concerned IFAD project, there is adecentralized decision-making system whichallows communities to take into account theresults emanating from the application of thePMEAll members of the project have a clear readingof the change of attitude which the PMEinvolves and are ready for these changesParticipation is perceived as a democratic andnon extractive processGroups who need the system most wereidentifiedThere is an institutional context that facilitatescommunity participationThere are good facilitators within the project,who can assist the communitiesExistence, within the community, of people(presenters, community facilitators) whocan possibly facilitate the process PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 25
  24. 24. IDENTIFYING ACTORS AND SESSION 7 ANALYZING THEIR NEEDS WORKSHEET 71. Objectives of the session• Provide participants with the necessary tools and skills to identify actors2. Proposed technique: Role playing, brainstorming, presentation, group work,3. Material and equipment required: Kraft paper, markers in sufficient number, cards of differentcolours4. Duration of session: 90mn5. Educational progression • Choose 2 to 3 IFAD projects whose representatives in the training will act as resource persons. It is recommended to select (if possible) projects intervening in different sectors or having different components. • Divide the rest of the group equitably between the selected projects (group work) • The exercise consists in identifying key actors who have an influence on the project or are affected by it: The focal question « who are the different actors affected by the project or with direct or indirect influence on its functioning ? » (Brainstorming) duration • Give 10 minutes to draw up the list of actors • At this stage, the facilitator introduces the actors’ analytical grid (see CONTENT CARD N°7) • After presenting the tool, the groups are requested to pursue the exercise by filing the grid on the basis of the list of identified actors. Duration: 20 minutes • At the end of the allocated time, ask each group rapporteur to present the results of its group • Open a discussion on the different presentations. Duration: the remaining 50 minutes? • The facilitator makes recommendations /gives advice on how to use and take advantage of the grid within the framework of their work.When will the simulation announced in the proposed techniques be made?6. Information provided by the facilitatorBased on CONTENT CARD N° 7, the facilitator informs participants of the existence of other authors’analysis tools. He may thus request participants to propose tools which may be listed on padex paper.The facilitator may request those making proposals to prepare summary sheets for the presentation onthe use of these tools.These sheets could be improved, if necessary, and distributed to participantsThe facilitator can also present the power-interest grid explaining that it is complementary to the actors’analysis grid presented earlier on. In general, the power-interest grid is applied first to classify actorsaccording to their interest and power. The actors’ analysis grid can then be applied to better understandthe nature of these interests and powers.26 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  25. 25. IDENTIFYING ACTORS AND SESSION 7 ANALYZING THEIR NEEDS CONTENT CARD 7 Stage 2: IDENTIFICATION AND ANALYSIS OF ACTORSWho are the actors?The actors are those who have an influence on or are affected by the project or programme concernedby the participatory monitoring-evaluation.The identification and analysis of actors are an important phase in the establishment of the PME sys-tem.The PME is set up to meet the concerns of these actors, particularly those directly affected by the inter-vention of the project or programme (direct and indirect beneficiaries).Some actors are more visible because of the positions they occupy and the roles they play in the com-munity or project. But there are also less visible actors who generally belong to so-called vulnerablegroups and are most affected by the activities undertaken. These groups, which constitute the primarybeneficiaries of the project, should play a central role in the design and management of the PME sys-tem.It is therefore important to have an appropriate approach and tools to identify them and examine theirinterests and what they expect from the project, the type of influence they can exert on the project ac-tivities and the arrangements to be made.How to conduct the actors’ analysis?There are several tools for that purpose. They include:The power-interest grid:It is used to make a simple mapping of actors by taking into consideration the interest that each of themcould have for the PME system to be set up, as well as the influences (positive or negative) that he/shecould have on the system. To apply it, one should first, identify all the project actors and second, preparea typology by placing each actor in one of the 4 spaces of the grid, corresponding to its interest and theimportance of his/her influence. Of course such a classification should be justified. At the end of theclassification, one should examine the actions to be undertaken for each of the 4 categories of actorsidentified for the success of the PME system to be set up.The actors’ analysis grid:It facilitates the identification of the different actors, their interest for the system, their influence on thesystem, and the actions to be taken to improve their participation. As can be observed, this grid makesit possible to produce the same type of information as those generated by the power-interest grid, theonly difference being that the former is used to classify actors.The two grids are also complementary since the actors’ analysis grid can be used as a back-up for theorganization of information generated by the reflection which accompanies the preparation of the power-interest grid. But, for reasons of simplicity, one can choose to use only one of them. One should notlose sight of the fact that this process is, first and foremost, meant for the local populations and that, asa result, one should avoid using a variety of tools which would complicate the process even more. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 27
  26. 26. IDENTIFYING ACTORS AND SESSION 7 ANALYZING THEIR NEEDS CONTENT CARD 7 POWER-INTEREST GRID HIGH Keep them informed about the Important actors! conduct of the programme Do everything to maintain this group in the process Generally key beneficiaries POWER People or groups living in the community but Step up their level of information not directly affected by the and training Programme. If they are potential Need for negotiating capacity beneficiaries, they should, if possi- ble, be associated in some actions in order to prepare themLOW INTEREST Low HIGH THE ACTORS’ ANALYSIS GRID What negative What interest do What positive How to step up influence can this group of influence can this the participation Actors this group have actors have in group have on the of this group on the PME the PME? PME system ? of actors? system?28 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  27. 27. SESSION 8 DEFINITION OF PME OBJECTIVES WORKSHEET 81. Objectives of the sessionImprove the participants’ knowledge of PME to strengthen their capacity to assist the populations inidentifying PME objectives2. Proposed technique: Case study, role playing3. Material and equipment required: Kraft paper, markers in sufficient number, cards of differentcolours4. Duration of session: 90 mn5. Educational progression • Choose 2 IFAD projects whose representatives in the training will simulate the role of direct beneficiaries. It is recommended to choose (if possible) projects intervening in different sectors or having different components. • Train as many groups as selected projects, if the number of participants allows it. • Explain that the exercise consists in putting oneself in the shoes of beneficiaries to identify the monitoring-evaluation objectives in the selected project. Other members of the group should ensure that the objective is clearly stated. Ask the following focal question « What are we trying to know or pursue through our participatory monitoring-evaluation system? » • Ask each group to formulate a maximum of two (2) objectives • Give each group a padex paper to list the objectives • At the end of the allotted time, ask each group rapporteur to present the objectives set for the IFAD project he/she represents • Open a discussion on the different presentations. Focal questions to guide the discussions: - Are the objectives well formulated? - Are they effectively PME objectives? - If not, what other reformulations can be proposed?6. Information provided by the facilitator Drawing inspiration from Content Card N° 8, the facilitatorprovides the necessary additional information about the formulation of objectives. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 29
  28. 28. SESSION 8 DEFINITION OF PME OBJECTIVES CONTENT CARD 8 Stage 2: DEFINITION OF THE OBJECTIVES OF PME SYSTEM TO BE SET UP Define the monitoring objective.For simplicity, the objective of PME specifies what the key beneficiaries seek to monitor and evaluate.Who defines the PME objectives?Within the framework of PME, the objectives are those of key beneficiaries. The project is meant toassist the latter to identify the objectives they intend to assign to their PME system. The facilitator shouldbuild on the analysis previously conducted by actors to identify all persons or groups who should par-ticipate in the identification of objectives.A few rules • Since it is a PME system essentially meant for the communities, one should avoid being dogmatic in the formulation of these objectives • The populations should be provided with the opportunity to present their ideas (objective) clearly, regardless of the method they use. For example, the populations may formulate their PME objectives as follows: - we would like to know whether women participate effectively in the meetings held by our group- ing; - we would like to see whether the decisions taken concerning the management of financial re- sources allocated to the project are transparent, - we would like to see whether the quality of services provided by the project effectively meets our expectations • Ensure that the populations do not select many PME objectives. The more objectives there are, the more indicators one has to pursue. This makes the system more complex and more difficult to man- age • When the PME is launched for the first time, one should be limited to one or two simple objectives; and as the populations understand the system, new objectives can be added • For example, in view of the current situation of the project, is the monitoring of service quality the most important? Or the participatory monitoring-evaluation of the different actors’ participation? Or the PME of the achievement of project objectives?30 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  29. 29. SESSION 9 CHOICE OF PME INDICATORS WORKSHEET 91. Objectives of the session • Enable participants to understand what a monitoring-evaluation indicator is • Define the main characteristics of the indicator • Describe the process for the participatory definition of indicators with target groups2. Proposed technique: Brainstorming, Group work3. Material and equipment required: Padex, padex paper, markers, sheets,video projector, computer4. Total duration of session: 90 mn5. Educational progression • Ask participants to form pairs • Give 3 sheets to each group and ask them to write a key word describing the indicator, on each sheet • In plenary, ask each group to designate a rapporteur to present their results; • The sheets are displayed on the board or on a wide Kraft paper as they are presented; • The facilitator asks a participant to classify the sheets in homogenous clusters; • The list of key words is written on a padex paper • Building on content card n° 9, the facilitator completes the results produced, where relevant After this, the facilitator requests the working groups to come together around the PME objectives defined during the preceding session (session 8) • Ask the groups to define indicators in relation to the objectives defined • Ask the groups to present their results • Ask participants to discuss the results: question to guide the discussion: are the indicators clearly defined? • Write down the different comments on paper • Then ask in plenary: What characteristics are common to the different criteria defined? • The answers are written on padex paper6. Information provided by the facilitatorThe facilitator draws a conclusion on the main characteristics of the indicatorWhat is an indicator? • A revealer, • A marker, • A set of measures • A describer, • A simplifier of a more complex realityWhat is it used for?To measure the progress made during a given period of time in relation to a monitoring objective definedbeforehand. Since each indicator provides information concerning only one part of the objective to bemonitored, it is naturally necessary to have several indicators for each objective. For example, if themembers of a grouping participate regularly in meetings, the different indicators should be identified. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 31
  30. 30. SESSION 9 CHOICE OF PME INDICATORS CONTENT CARD 9 Stage 4: CHOICE OF MONITORING INDICATORSWhat are the types of indicators • Resource indicators concern the budget allocated at every level of the intervention, the human resources mobilized, etc. The measurement of these indicators provides information about the efficiency of the project’s intervention. Example • Grants • Annual Budget • Borrowings • Labour • Input costs • Etc. • Activities indicators: to achieve the project objectives, a certain number of activities are implemented. The monitoring system may focus on the level of achievement of these activities • Meetings • Training • Visits • Etc. • Process indicators: They relate to the decision-making process and makes it possible to determine, for example, the inclusion and/or participation of actors, the regularity of meetings, the regularity of reports, etc. • Existence of reporting rules • Degree of women’s participation in decisions • Frequency of debriefing meetings • Etc. • Product indicators: When an activity is carried out, it generally generates a tangible or non tangible product. For example, at the end of a training, the number of people trained (product) can be assessed. The product is the immediate and tangible result of the combination of a resource (expenditure) and an activity (training): product: number of trainees • Trainees • Number of text books produced • Number of credit received • Etc. • The results indicators reflect behaviour. For example, a few months after the training, one can assess how many participants have applied what they learnt. Or if a project distributes equipment (product) one can assess how many beneficiaries use this equipment.Results indicators also assess the scaling down of women’s working hours following the introduction ofa new activity or equipment. • Number of people who applied the training received • Use of credit obtained • Number of people who use the text books • Etc.32 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  31. 31. SESSION 9 CHOICE OF PME INDICATORS CONTENT CARD 9 Stage 4: CHOICE OF MONITORING INDICATORS • Impact indicators assess the long-term consequences of an intervention, beyond the immediate effects. However, it is very difficult to evaluate an impact and link its cause to a given project.Actually, the impacts are generally the product of several unrelated interventions in time and space. Thisexplains why, in PME, emphasis Is particularly laid on the other levels and, in particular, on the effects.A few principles • Avoid being dogmatic: the criteria can also serve as a basis • Different groups may have different criteria or monitoring indicators • Since the process is iterative, the number and type of indicators may be readjusted as time goes on • At the beginning, choose a limited number of relevant indicators to foster learning • It is essential to have a certain balance between the different levels of criteria/indicators (associate criteria related to the activities and others related to the process);Example Our objective Indicators We would like to know • Number of meetings held whether women participate • Number of participants effectively in the meetings • Number of women held by our grouping • Number of decisions taken • Types of services offered • Total number of beneficiaries We would like to see • Number of female beneficiaries whether the quality of • Number of districts or villages services provided by the • Number of women beneficiaries per project effectively meets district/village our expectations • Number of people expressing satisfaction • Number of unsatisfied people • Etc. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 33
  32. 32. SESSION 9 CHOICE OF PME INDICATORS CONTENT CARD 9 Stage 4: CHOICE OF MONITORING INDICATORSWhat characteristics are common to these different indicators?They are • Simple: Each indicator refers to only one thing and is free of ambiguity • Measurable: These indicators are formulated in such a way that they would be measurable: Thus the changes made can be measured from one participation monitoring-evaluation session to another • Achievable/accessible: The populations may have access to information required to assess the changes • Effectively reflect the objective (relevancy): the indicator should measure what it is supposed to measure and not another reality • Temporally measurable: When a monitoring periodicity is defined, it should be ensured that it helps observe the expected developments in the indicator measurement.It is worth noting that the measurement notion does not necessarily refer to numbers and quantities.Less tangible indicators like perception can be measured using the appropriate measurement tools. Be careful to: Make a distinction between the criterion and the indicatorThe criterion refers to general values accepted by the community and serving as a beacon thatguides the community’s view: ex, women’s participation in decision-making, the poor’s accessto services. Very often, in the PME exercises, the populations mention the criteria first. The fa-cilitator, through probing questions, should be able to get them to identify the indicators. Forexample, if the group says it wants to evaluate women’s participation, the facilitator can ask itsmembers what measures they intend to use for that (number of women participating in meet-ings, number of women in the office, number of participations at meetings, etc.) and which referto indicators. A few steps for the participatory definition of indicators: • Bring together all the groups whose participation in the monitoring exercise in necessary • Identify what the success of the initiative means for each group, and this results in criteria • On the basis of the criteria, define indicators either in plenary, or in small groups • Define the units of analysis and the sampling procedure • The indicators are likely to change, depending on the progress of the project, the context, partners34 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  33. 33. CHOICE OF INFORMATION SESSION 10 GATHERING METHODS WORKSHEET 101. Objectives of the sessionFamiliarize participants with the PME’s results-based participatory data collection tools and techniques2. Proposed technique: Brainstorming, group work, presentation, case study3. Material and equipment required: Kraft paper, padex paper, markers in sufficient number, cards ofdifferent colours4. Duration: 65 mn5. Educational progression • Brainstorming on participatory monitoring evaluation data analysis tools at the projects’ level (15mn) «What data analysis tools do you use in projects? » • If necessary, the list of tools is completed by the trainer/ facilitator (5 mn) • The participants are divided into small groups of 3 to 4 people and the facilitator assigns to each group 1 or 2 tools with which participants are familiar • The conditions of use of the listed tools are analyzed (favourable conditions, limits) 30mn • The groups present the results of their reflection (15 mn)6. Information provided by the facilitator • Insist on the fact that in a participatory process, data collection and analysis are done simultane- ously • Based on CONTENT CARD N° 10, the facilitator provides the additional information required Before closing this session, he announces the presentation of the video on the use of the Evaluation Form by the Community as an example of an appropriate PME tool for the monitoring of qualitative indicators. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 35
  34. 34. CHOICE OF INFORMATION SESSION 10 GATHERING METHODS CONTENT CARD 10 Stage 5: CHOICE OF INFORMATION GATHERING METHODS AND TOOLSSeveral participatory tools can be used within the PME framework. However, the choice of a specifictool depends on the system that has been put in place. In fact, the PME application scale, the types ofactors involved, the targeted objectives and the PME indicators influence the choice of tools. However,regardless of the types of tools chosen, an important principle, namely the demand for PME simplicityshould be taken into consideration. In this perspective, a few rules should be complied with: • The tools should be chosen in close collaboration with the local actors entrusted with managing the system to be set up; • Limit the number of tools as much as possible. The greater their number, the more complex the PME system is; • To facilitate the involvement of everyone, and make the PME tool more user-friendly, there is need to promote visualization, as far as possible. It would be a mistake to believe that visualization is only meant to facilitate access to information for illiterates. Today, visualization has become a key dimen- sion of modern communication, and it is increasingly preferred to facilitate learning; • As regards visualization, illustrations should be chosen by /with the actors themselves (avoid having the consultant decide the forms of illustrations) ; • Need to translate the key concepts in local languagesProposed tools: The form for evaluation by the Community Assesment Forum (CAF) Advantages of the FEC • It is a tool that makes it possible to take into account, in an integrated manner, the different PME stages • Its application is based on the participation of all the groups • Its application provides IFAD project managers with immediate feedback on how beneficiaries perceive the quality of services and on measures to be taken • Actions to be implemented result from a collective reflection that takes the views of all actors into consideration • It is simple and accessible to all because it is based on visualization • The information is generated through focus groups, which facilitates the participation of groups with difficulties to express themselves during mixed group discussions Source : Guèye, B (INDh, 2008) CONTENT CARD N° 11 describes the Community Assessment Form (FEC) application process36 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  35. 35. INFORMATION GATHERING SESSION 11 AND ANALYSIS WORKSHEET 111. Objectives of the sessionProvide participants with the ability required for data collection2. Proposed technique: Video projection, group work3. Material and equipment required: video on the use of the form for evaluation by the community,Kraft paper, markers in sufficient number, cards of different colours4. Duration of session: 90 mn5. Educational progression • Introduce the video application context • Before starting the projection, ask participants to note the different stages of the Community Assessment Form (FEC) application (meaning of the acronym) • After the presentation, open a discussion session for a summary of the main stages; • The facilitator then proposes to participants to apply the Community Assessment Form (FEC) in their evaluation of the application of the participatory approach within IFAD projects • To that end, it is proposed that participants be divided into 2 groups representing field workers of IFAD projects (group of actors 1) and the populations beneficiaries of IFAD projects (group of actors 2). Choose an IFAD project • The two groups meet first to identify the indicators that will enable them to assess the effectiveness and quality of the application of the participatory approach by IFAD projects • Limit the number of criteria to no more than 5 (it is an educational exercise) • Follow the stages described in the card for the completion of the exercise • Following the presentation of results, the methodological observations are noted (difficulties, questions, etc.).6. Facilitator’s information input • On the basis of CONTENT CARD N° 11, the facilitator tries to provide answers to the questions asked. He also provides all other necessary information on the use of the form, • He should however explain that the Community Assessment Form (FEC) was the tool preferred for the training, but that other tools exist. He gives examples of monitoring tools contained in CONTENT CARD 11. He may also ask participants to share other tools they know or use. The facilitator lists them on a padex paper PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 37
  36. 36. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11 Stage 6: INFORMATION GATHERING AND ANALYSIS Information gathering Presentation of the process on the use of the Community Assessment Form (FEC)FEC application processSession for acquaintance, mutual understanding, presentation of the objectives of the visit and the pro-ceedings of the session. During the presentation session, insist on the importance of the local actors’commitment as an emerging approach to strengthen the performance of public programmes; but espe-cially on the need for local actors to acquire the capacities required to ensure monitoring and evaluation.Such a mechanism should be positively perceived, because the way in which beneficiaries perceive theservices provided by the programme gives signals to officials and grassroots representatives that wouldenable them to improve the quality and performance of their services on a daily basis.Identify the key stakeholder groups. These groups benefit from the actions of the programme, those incharge of providing these services, or those in charge of controlling the implementation, etc. In reality,this identification has already been done (see session 8);Focus groups: division of team members into homogeneous sub-groups whose number is equal to thatof the stakeholder groups. The criteria for the composition of these focus groups may vary from onePME system to another; it may be a criteria related to gender, residence, types of services received,etc.Ensure the proper distribution of skills, profiles and sex. A facilitator will be attached to each group toassist the latter to evaluate the indicators to be identified. There is need to ensure optimal time man-agement (moderation, reporting, in particular).Organize a brainstorming session regrouping all stakeholder groups to discuss and choose the mostrelevant monitoring-evaluation indicators. Avoid having too many indicators. In some cases, each stake-holder group chooses its indicators separately. But the advantage of having the same criteria lies in thefact that this facilitates comparisons; the evaluations made by through the differences relative to thesame aspects;Before initiating the following stage, and in the event that the process is applied for the first time, the fa-cilitator assists the group to design the monitoring-evaluation medium. The latter is composed of a gridadjusted for the qualitative type indicators. The grid facilitates the measurement of the levels of satis-faction in relation to services received/rendered, actors’ involvement, management, etc. Below is a modelof the assessment form:38 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  37. 37. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11 Project Village or Zone: Nature of the monitored component: Stakeholder group: COMMENTS: jot down INDICATOR the reasons justifying the scores given 1.Dissatis- 2. Poorly 3.Moderately - 5.Very satisfied 4. Satisfied fied satisfied satisfied Ooooooo Ooooooo Ooooooooooo Oooooooooo Ooooooooooo INDICATOR (16%) (14%) (24%) (22%) (24%) INDICATOR INDICATOR INDICATOR INDICATOREtc. • On the basis of the selected indicators, the different groups work separately to carry out the evaluation with the help of the evaluation form. In each group, explain clearly the principle of the exercise and the results expected: - copy the evaluation grid on padex paper; - Explain the grid filling principle and, in particular, the fact that participants vote individually (each participant should give a score for each criteria). It is even preferable for each participant to record his/her vote directly on the grid; - Regardless of the score given, the reasons for such a choice should be well argued and as one goes on, scored in the « comments » column by the facilitator; -When all the individual scores are recorded next to the corresponding criterion, these scores should be converted into percentage. The most significant observations are recorded in the cor- responding box (comment) • After collecting the scores given for all the evaluation criteria, start a discussion to collect all additional observations on the final results of the exercise. • Interface between actors. This is an important stage. It consists in regrouping the different stakeholder groups to compare the respective analyses. The exercise is not meant to build a consensus but to understand the factors that explain the differences in perception. It can be structured around key questions like: What are the differences/similarities observed in the perceptions and analyses by the different groups? How does the extended group interpret these differences and/or similarities ? What are the causes of these differences? What are the changes to be made? All this information shall be carefully noted on padex paper. The facilitator shall then help the group to organize this information into key thrust areas or themes. • Negotiation: On the basis of the results of the interface session, the actors identify the different actions to be implemented in order to improve the situation. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 39
  38. 38. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11Move from evaluation to monitoringTo simplify things, monitoring can be considered as a regular succession of evaluation sessions con-cerning the same objectives and indicators. Thus, the monitoring process entails repeating the sessionsat regular intervals to assess the different indicators that have been identified. After each session, theresults will be recorded in the monitoring table (see model below). However, after recording the resultsin the monitoring table, the facilitator should get the group to compare these results with those obtainedduring the preceding monitoring session. The grid for the analysis of changes occurring between ses-sions may be used to gather the various information emanating from this analysis.Monitoring-evaluation of quantitative indicatorsA diverse range of simple tools may be used to monitor quantitative indicators. However, given the spe-cific situation of each participatory monitoring-evaluation system, the creativity and innovation of localactors are required to develop suitable tools. This also has the advantage of facilitating the appropriationof tools by the populations. For this reason, it is always strongly recommended to facilitators workingwith the communities- after identifying the indicators- to invite the actors of the system to create simpletools to measure these indicators by taking into account some principles discussed earlier, namely sim-plicity and the use of visualization to represent the indicators and quantities. The facilitator should avoidproposing his own tools from the onset. He should first invite the populations to explore their own cre-ativity potential before showing them the tool(s) at his disposal. Actually, it might not even be necessaryfor him to present his own tools if the populations succeed in developing a simple and efficient tool tomeasure the selected indicators.40 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  39. 39. INFORMATION GATHERINGSESSION 11 AND ANALYSIS CONTENT CARD 11 A few examples of PME tools Example 1: Labour utilization monitoring formType of beneficiary Period Period Period Initiative developing RME Number of jobs created Number of apprentices Training received Composante BNumber of loans received Amount of loans Amount repaid Composante C PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 41
  40. 40. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11 Example 2: Programme CES 2, Burkina Faso. This tool may be adjusted to serve as a monitoring-evaluation medium. To that end, instead of daily intervals, one can adopt intervals that are more in line with the selected indicators (monthly, quarterly monitoring, etc.). Farming Activity Days in the Duration of Labor input Labor input Labor input Exaternal Cash expenses week work man woman child labor input on hired labor Day 1 1 symbol Day 1 1 symbol Day 1 1 symbol Day 1 1 symbol Day 1 1 symbol Day 1 1 symbol Day 1 1 symbol LEGEND Full Half Day of the Local X W workday workday week currencySource: Hien, Fidèle et Ouédraogo, Ali42 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  41. 41. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11Example 3: The evaluation wheel used to assess local governance in Fissel (Senegal). The use of thistool at regular intervals can show the Rural Council and other participants of the process, how the as-sessment of citizens can change following actions taken after each PME session.This tool can be used within the framework of any IFAD project to monitor and evaluate other qualitativeindicators. DOXINU CONSEY RIIRAL 10 10 10 Limu jëf yi Caytub alal ji Japandik tënku 10 kaayi dogal yi 10 Jaapandik jumtu kaayi tétëlin Jaapandik tasu Xeeti kom-kom yi kaayi xabaar yi 10 10 Dayoob jumtukaay yi Bookiinu ndaw 10 ci dogal yi Limu way Limu ndaje yi japalé yi Téewayu 10 ndaw yi ci 10 ndaje yi Limu dogal yi 10 10 10Source : Guèye, B (2005) PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 43
  42. 42. INFORMATION GATHERING SESSION 11 AND ANALYSIS CONTENT CARD 11 Monitoring table model Monitoring session of:............. Monitoring session of:………. Monitoring session of:…… dissatis- Poorly Modera- satisfied Very dissa- Poorly Mode- Satis- Very Dis- Poor- Mode- Satis- Very Criterion fied satis- tely satis- tisfied satis- rately fied satis- satis- ly rately fied satis- fied satisfied fied fied satis- fied fied satis- satisfied fied fied fied 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 XX X XX XXx Xxx Xxx XXx XXx Xx XX XX XX xxx xxx xxx Indicator XX X 0% 15% 15% X X 30% 30% 30% 10% 35% 20% 15% Indicator Indicator Indicator Indicator Grid for the analysis of changes occurring between two monitoring sessions Indicators Compared to the preceding Factors explaining Factors explaining What actions need monitoring session, did the the positive the negative to be taken? evaluation of the indicator change in the change in the reveal (1) a positive change, evaluationof the evaluationof the (2) a negative change or criterion criterion (3) No change at all? Indicators Indicators Indicators Indicators Indicators44 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  43. 43. SESSION 12 IMPLEMENTING CHANGES WORKSHEET 121. Objectives • Provide information about the process for the activation of changes • Discuss with participants about tools and techniques for the introduction of changes2. Proposed technique: Group work, role playing3. Material and equipment required: Padex, padex paper, wall sellotape, markers,4. Duration of session: 30mn5. Educational progression • Maintain the same groups as those of the preceding session • Ask them to pursue the exercise by identifying and planning, on the basis of the evaluation results, the actions to be undertaken with a view to strengthening the resolve to take into consideration the participatory approach within IFAD projects • Introduce the tool (planning table below) which will be used to organize information: Who? What? How? When? How many? Who pays? • Each group draws its table, fills it and puts it on padex paper • The results of the work are presented and discussed in plenary • The facilitator provides other clarifications or further details, if necessary (adopting a name)6. Additional information provided by the facilitator:The facilitator draws inspiration from the substance of CONTENT CARD N° 12 to conclude the session. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 45
  44. 44. SESSION 12 IMPLEMENTING CHANGES CONTENT CARD 12IMPLEMENTING CHANGESReview the different actions proposed in the cards during preceding stages on the evaluation of indica-tors. The facilitator should however ensure that the actions scheduled fulfil a certain number criteria: • They should be realistic and achievable within reasonable time • The capacities and means required to achieve them should be available • Actors are firmly committed to implement them. The table below can be used to plan the implementation of actions How much and What? By whom? How? When? who pays ?46 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  45. 45. MANAGEMENT AND CONSOLIDATION SESSION 13 OF THE PME SYSTEM WORKSHEET 131. Objectives of the session • Identify the key elements of the results-based management of the PME system • Determine the measures and actions to be taken to ensure the sustainability of the system2. Pedagogical approach: Brainstorming, Group work3. Duration: 60 mn4. Material and equipment required: Kraft paper, markers in sufficient number, cards of differentcolours5. Educational progression • Ask participants to divide themselves into groups of 6 to 8 persons. • Explain the purpose of the exercise: It consist in identifying the key elements to be taken into con- sideration in the management of a system. • Give 5 Bristol cards to each group and ask them to write the results by putting only one idea on each card • At the end of the allocated time, ask the group rapporteurs to reconstitute the results of their pro- ceedings, • As they go along, the cards are affixed on a board in an appropriate space prepared beforehand • Request a voluntary participant to organize the different cards by putting together those that talk about the same idea • After this work, write on a padex paper, the key elements of PME management as they emerge from the group work6. Information provided by the facilitatorOn the basis of CONTENT CARD N° 13, the facilitator provides any other additional information andclarifications required. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 47
  46. 46. MANAGEMENT AND CONSOLIDATION SESSION 13 OF THE PME SYSTEM CONTENT CARD 13 MANAGEMENT AND CONSOLIDATION OF THE PME SYSTEMThe management of a PME system consists in:1. Mobilizing the actors; • Inform the populations about the different activities related to the PME system • Raise their awareness about the importance of the system and the need for their participation • Organize sessions to reconstitute the results of monitoring sessions and collect their feedback • Ensure the regular participation of all the groups, particularly the vulnerable groups • Etc.2. Facilitate the PME sessions (Facilitation, Appropriation and Sustainability of the system) : • Plan and organize the different PME sessions • Prepare the pedagogical materials • Organize the different focus groups • Facilitate the monitoring and evaluation sessions • Organize information emanating from monitoring sessions • Prepare reports • Disseminate information to the different actors3. Resource management (media and other resources). • Manage the PME medium. It is suggested that the PME grid be reproduced on a solid and detach- able medium (Wooden table, for example) and place it in an accessible place (office of the monitoring committee) To enable everyone to be regularly informed • Manage all the reports and documents produced within the framework of the system3. Follow-up the implementation of recommendations from the PME sessions; • Be in contact with the project and other actors to ensure the implementation of recommendations from the PME sessions • Evaluate the implementation of the PME system itself To play these different roles effectively, there is need to institute a simple, flexible but efficient orga- nizational mechanism. Generally, a body in charge of managing and coordinating the PME system is set up for that purpose. Regardless of the appellation (monitoring committee, PME system coordi- nation committee, etc.), the institution of this body should fulfil a certain number of criteria : • It should be representative of key stakeholders concerned by the programme on which the PME is focussed • It should have a reasonable size (between 5 and 8 people) to avoid cumbersomeness • The profile of members should be in line with the roles assigned to them and mentioned earlier. They should have complementary profiles: Some members have mobilization and/or facilitation skills, others are more skilled in management and logistic organization, while others are more comfortable in the monitoring of activities, etc. • Even though project representatives can be included in the organ responsible for coordinating the process, it is important to keep in mind that it is above all, a PME system.The main beneficiaries should be at the heart of the mechanism and should discharge the different mo-bilization, leadership and facilitation functions.Of course, the role played by the project staff at the beginning of the process may be relatively moresignificant, but should be progressively reduced as and when the beneficiaries acquire the necessaryskills.48 PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012
  47. 47. MONITORING AND EVALUATION SESSION 14 OF THE PME IMPLEMENTATION EDUCATIONAL EXERCISE 141. Objectives of the sessionDefine a simple approach to monitor and evaluate the implementation of the PME system2. Proposed technique: Individual work but in case of a big group (more than 15 people) the partici-pants can be arranged in groups of two or three.3. Material and equipment required: Kraft paper, markers in sufficient numbers, cards of differentcolours4. Duration: 45 minutes5. Pedagogical progression • Give 3 cards to each participant • Ask each participant to propose 3 monitoring indicators for the PME system based on the following focal question: What are the indicators that can provide us with information about the functioning and quality of our PME system? • Give them 5 minutes to fill the cards. The rule is known to all: a card, a criterion or indicator • When all the participants have finished filling their cards, ask a volunteer to collect and display them [haphazardly) on the board or on the space provided for that purpose • Then ask the volunteer to regroup the different cards according to the key ideas • Write these ideas on a Padex paper6. Information provided by the facilitator • Using Content Card N° 14, the facilitator may come up with other criteria. He may propose them but lay emphasis on the fact that the criteria defined by the main actors themselves are more important. For the record, the facilitator should be prepared to direct and possibly provide some points for the discussion on the PME success or failure indicators. • The community evaluation sheet may also be used in the evaluation of the results-based PME implementation. PARTICIPATORY MONITORING AND EVALUATION (PME) ­ 2012 49