Build Your NGO: Monitoring & Evaluation

12,012 views

Published on

The presentation attached is designed for grassroots NGOs wanting to learn more about monitoring and evaluation.

The presentation is a mini 'how to', in addition to providing an overview of strategic planning

To learn more or with any direct questions, please visit our website: www.thepariproject.com

Published in: Technology, Business
3 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total views
12,012
On SlideShare
0
From Embeds
0
Number of Embeds
622
Actions
Shares
0
Downloads
577
Comments
3
Likes
6
Embeds 0
No embeds

No notes for slide

Build Your NGO: Monitoring & Evaluation

  1. 1. Monitoring & Evaluation
  2. 2. Agenda• Introduction • M&E• Basic concepts – Introduction to M&E Definitions of M&E systems Why do M&E? – How to measure• Project Planning • What indicators to use Theory of change • How to collect data Result chain • What targets to set Assumptions and risks – Putting it all together
  3. 3. Definitions• Monitoring Routine, systematic collection of data relating to project performance Provides regular feedback and early indications of progress (or lack of progress)• Evaluation Episodic and objective assessment of project Often done at mid-point and at end of project Can determine whether objectives were achieved, how efficient the project was, and what the project’s impact was
  4. 4. Why do M&E?• Knowledge of impact M&E tells you whether your projects are meeting objectives and having results• Improved decision-making Information generated through M&E helps you make better decisions about projects• Greater accountability and transparency M&E data allows donors and others to assess how well and efficiently you are using resources
  5. 5. Theory of Change• What is the theory of change for your project?• A chain of so thats that lead to the change you hope for• Example: we teach children how to wash their hands So that they will understand the importance of washing their hands AND will have the skills to wash their hands properly So that they will wash their hands regularly So that they will be less susceptible to certain illnesses So that the incidence of certain illnesses will decrease So that child mortality will decrease
  6. 6. Result Chain Def: A picture of the chain of “so that”relationships that lead to the desired outcomesSource: University of Wisconsin Extension, Developing a Logic Model
  7. 7. Result Chain – Simplest Form Purpose/ Goal/ Short- Long-Inputs Activities Outputs term term objective objective
  8. 8. Result Chain – More DetailedInputs Activities Outputs Purpose/ Objectives Goal / Long-Term Objective • What you invest • What you do •What you deliver • What the short- and • What the ultimate impact • Staff • Specific actions you undertake •Products and/or services you medium-term results are is • Volunteers to achieve particular outputs deliver (~1 yr) • Situations, conditions, behavio • Long-term impact of the • Time •You should be able to ur changed as a consequence project (+ 5 years) guarantee these of the project (~3-5 yrs) • Project contributes to goal, but • Money •Examples: • Outside your control likely doesn’t achieve it • Research base •Conduct workshops • Learning: • Conditions: • Materials •Deliver services • Changes in • Changes in • Equipment •Develop awareness, knowledge, attitud social, economic, civic, environ • Technology es, skills, opinions, aspirations, mental conditions • Partners products/curriculum/resources motivations •Train • Action: •Provide counseling •Assess • Changes in behaviour, practice, policies, d •Facilitate ecision-making, •Partner
  9. 9. Result Chain Example: HIV Prevention for Sex WorkersInputs Activities Outputs Purpose/Objectives Goal/Long-Term Objective • Staff - 3 trainers • Make schedule of • Conduct 1 workshop • Learning: • HIV prevalence among • Volunteers – national workshops every 2 months • Participants are aware of Cambodian sex workers network of 8 volunteers • Coordinate with local • Target Cambodian sex the importance of HIV declines • Time – 20 hours/month volunteers to recruit workers prevention • Money – space workshop participants • Participants have rental, snacks • Plan workshop curriculum knowledge of means of purchase, transportation • Assemble workshop transmission, methods of reimbursement materials prevention • Research base – • Book workshop facilities • Participants have knowledge of negotiation skills STDs, knowledge of • Action: training techniques • Participants practice safer • Materials – workshop sex materials
  10. 10. Assumptions and Risks• How do you plan for events, conditions, and decisions beyond your direct control?• Assumptions positive statements about what will go right• Risks negative statements about what might go wrongSource: AusAid, AusGuideline 3.3 The Logical Framework Approach, 2005; NORAD, The Logical FrameworkApproach (LFA): Handbook for objectives oriented planning (4th ed.), 1999.
  11. 11. Assumptions and Risks Purpose/Short- Goal/Long-termInputs Activities Outputs term objective objective Assumptions Assumptions Assumptions Assumptions Decreasing control Decreasing control Increasing significance of risks Increasing significance of risks
  12. 12. Models and M&E• A result chain says what results you expect to achieve• M&E tells you if you are achieving the results you expected to achieve
  13. 13. Good M&E Systems• Dynamic encourage `learning by doing’• Participative and gender sensitive seek to overcome barriers of gender, age, power, and culture• Reflective encourage creating regular space and time for analysing information and reflecting on the theories of change• Evolving adapting and changing in order to keep them as light and simple as possible while providing `real time’ information Source: AusAid, Guidance on M&E for Civil Society Programs, December 2008
  14. 14. How to Measure: Indicators• Indicators tell you how you will recognize success• Indicators are a unit of measurement
  15. 15. Quantitative Indicators• Need a balance between quantitative and qualitative indicators• Quantitative: Can be measured in numbers number of water pumps installed amount of rice harvested percentage of children immunized HIV prevalence rate student-teacher ratio frequency of attending class
  16. 16. Qualitative Indicators• Qualitative: reflect people’s judgments, opinions, perceptions, and attitudes perception of well-being appropriateness of intervention quality of engagement level of commitment sense of empowerment
  17. 17. Indicators to Use1) What questions do you want to answer? (i.e. your KeyPerformance Questions)?2) What data can provide the evidence you need to answeryour KPQs? Is the data high-quality? Is it participative?3) What data can you collect? Do the benefits justify thecosts?
  18. 18. Indicators to Use1) What questions do you want to answer? (i.e. your KeyPerformance Questions)?• This depends on Your audience – Management? Donors? Beneficiaries? The purpose of measurement – To assess impact? Efficiency? Equity? Conflicts can arise between M&E for accountability and M&E for learning
  19. 19. Indicators to UseLevel in Key Performance Questions - examplesresult chainInputs • What interventions and resources are needed? • Are resources being used efficiently?Activities • What are we doing? Are we doing it right? • Have planned activities been completed on time and within the budget?Outputs • Are we implementing the project as planned? • What direct tangible products or services has the project/programme delivered?Purpose/ • Are interventions working or making a difference?objectives • What changes have occurred as a result of the outputs? Are these changes contributing towards the project purpose and desired impact? • Has the project achieved the changes for which it can realistically be held accountable?Goal/ long-term • Are we intervening on a large enough scale?objective • To what extent has the project contributed towards its longer term goals? Why or why not? • What unanticipated positive or negative consequences did the project have? Why did they arise?Sources: International Federation of Red Cross and Red Crescent Societies, Monitoring and Evaluation in a Nutshell,2007; Global Fund, Monitoring and Evaluation Toolkit, 2009
  20. 20. Indicators to Use2) What data can provide the evidence you need to answeryour KPQs? Is the data high-quality? Is it participative?
  21. 21. Indicators to UseLevel in What to monitor & evaluate/ Key Key Performanceresult Performance Questions Indicators - exampleschainInputs • Are resources being used efficiently? • Number of staff hours spent • Money spentActivities • Have planned activities been completed on time and within • Percent of milestones achieved on time the budget? • Variance between budget and actual • What unplanned activities have been completed?Outputs • What direct tangible products or services has the project • Number of workshops held delivered? • Number of sex workers trainedPurpose/ • What changes have occurred as a result of the outputs? To • Percent of workshop participantsobjectives what extent are these likely to contribute towards the demonstrating a high level of project purpose and desired impact? knowledge of HIV transmission and • Has the project achieved the changes for which it can prevention realistically be held accountable? • Percent of workshop participants reporting using condoms “frequently” or “always”Goal/ long- • To what extent has the project contributed towards its • HIV prevalence rate among Cambodianterm objective longer term goals? Why or why not? sex workers • What unanticiplated positive or negative consequences did the project have? Why did they arise?
  22. 22. Indicators to UseEnsure that high-quality data will be collected. It must be• Accurate• Reliable• Precise• Complete• Timely• With Integrity• Respectful of confidentiality Source: Global Fund, Monitoring and Evaluation Toolkit, 2009
  23. 23. Indicators to Use3) What data can you reasonably collect?
  24. 24. Indicators to Use Result Chain Purpose/Short- Goal/Long-termInputs Activities Outputs term objective objective Increasing difficulty of measurement Decreasing frequency of measurement
  25. 25. Indicators to Use Hierarchy of Effects Social-economic-environmentalParticipation Reactions Learning Actions Improvements •Number and •Degree of •Changes in •Changes in •Changes in frequency of satisfaction knowledge, behaviours society people with attitudes, ski and reached programme lls, aspiratio practices •Intensity of •Level of ns contact interest •Feelings toward activities, educational methods Increasing difficulty of measurement Source: Bennett and Rockwell, 1995, Targeting Outcomes of Programs
  26. 26. Indicators to Use• Do the benefits justify the costs?• Look for a balance between rigour and realism Source: Results-Based Management Tools at CIDA: A How-to Guide, http://www.acdi-cida.gc.ca
  27. 27. How to Collect DataIdentify for each Performance Indicator: Definition ExamplesData source(s) People, organisations, Beneficiaries, partners, government documents providing data about documents the indicatorData collection How data will be collected Analysis of documents, surveys,methods/tools interviews, focus groups, observation, case study, tests, testimonials, expert panelFormula/ scale / How the data will be captured Different scalesassessmentmethodFrequency and How often data will be collected Single time, continuously,duration and for how long weekly/monthly/annually, baseline/mid-term/finalResponsible Who will collect the data Staff, outside evaluatorsperson(s)
  28. 28. Indicators to Use• Indicators ≠ Targets• Indicators are units of measurement E.g. HIV prevalence rate among Cambodian sex workers• Targets specify a particular value for an indicator to be accomplished by a specific date E.g. HIV prevalence rate among Cambodian sex workers is reduced by 50% by 2020• Targets are essential: they are a concrete expression of your goals and objectives
  29. 29. Targets to Set• You can only set targets once you have baseline data• Baseline data show the situation before you start the project
  30. 30. Targets to Set• Targets can be set as Absolute targets (e.g. increase by 5) Proportional or percentage targets (e.g. increase by 5%) Relative to benchmarks (e.g. be within the top three schools in our area) Relative to costs or budgets (e.g. increase or reduce by 5% same level of budget).Sources: Advanced Performance Institute, How to Design Key Performance Indicators, 2010; Results-BasedManagement Tools at CIDA: A How-to Guide, http://www.acdi-cida.gc.ca
  31. 31. Targets to Set• Targets should be SMART Specific, Measurable, Attainable, Relevant, Time bound Ambitious but realistic
  32. 32. Targets to Set• Targets should account for Trends and past performance Predictable variation in performance (e.g. seasonal cycles) National targets, best practice benchmarks, etc Result chain (i.e. do not set outcome targets before you have set input targets) Time lag between intervention and effect Dependence on others (e.g. partners, government actors)
  33. 33. Targets to Set• Indicator - # of workshops held• Add target group (for whom?) - # of workshops for sex workers• Add quality (what? how well?) - # of workshops for sex workers addressing HIV prevention• Add quantity (how much?) - # of workshops for sex workers addressing HIV prevention increased from 10 to 12• Add time (by when?) - # of workshops for sex workers addressing HIV prevention increased from 10 to 12 in 2012• Add place (where?) -# of workshops for sex workers addressing HIV prevention increased to 6 in Phnom Penh and 6 in the provinces in 2012Source: NORAD, The Logical Framework Approach (LFA): Handbook for objectives oriented planning (4thed.), 1999.
  34. 34. Putting it All Together… Key Performance Questions Key Performance Baseline Target Means of Indicators VerificationInput Are resources being used # staff hours/workshop 90 in 2011 50 in 2012 Staff time sheets efficiently?Activities Have planned activities been % variance between 2012 2012 Records kept by completed on time and within budgeted and actual budget budget accountant the budget? spendingOutputs What direct tangible products or # workshops held 10/year in 12/year in Participant lists services has the 2011 2012 project/programme delivered? # sex workers reached 300/year in 480/year 2011 in 2012Purpose/ What changes have occurred as % of workshop participants 50% before 90% after QuestionnairesObjectives a result of the outputs and to demonstrating a high level the the before and after what extent are these likely to of knowledge of HIV workshop workshop workshop contribute towards the transmission and prevention project/programme purpose and desired impact? % of workshop participants reporting using condoms Has the project/programme “frequently” or “always” 5% before 20% after Surveys before achieved the changes for which it the 1 month workshop, after 1 can realistically be held workshop 10% after month, and after 6 accountable? 6 months monthsGoal/ To what extent has the HIV prevalence rate among X% n 2011 Y% 2020 UNAIDS statisticsLong-term project/programme contributed Cambodian sex workersobjective towards its longer term goals?
  35. 35. Want to Keep Going? Put Pari’s M&E Services to work for your organization!• We can help you set up an M&E system from scratch or assess your current system and help you improve it• We’ll asses your team’s capacity and make an inventory of current monitoring, evaluation and reporting activities• We’ll work with your team to map out your programmes/projects, select indicators, and create tools and schedules for tracking data and engaging in evaluation• We’ll help you pilot the new system, gather baseline data and set targets• We’ll work alongside your team to maintain the system Email us at allie@thepariproject.com to learn more

×