Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

6 M&E - Monitoring and Evaluation of Aid Projects

2,768 views

Published on

A series of course modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.

This is part 6 of 11, beginning with 2 modules on leadership and conflict resolution, then 9 modules on project cycle management.

This module has 3 handouts and presenter notes as separate documents.
Sample Proposal: http://www.slideshare.net/Makewa/6-watsan-training-sample-proposal-09
Slides as a handout: http://www.slideshare.net/Makewa/6-me-handout
Presenter notes: http://www.slideshare.net/Makewa/6-module-6-presenter-notes

Published in: Business, Technology
  • Be the first to comment

6 M&E - Monitoring and Evaluation of Aid Projects

  1. 1. Development Project Planning 6 Monitoring and Evaluation In the Project Cycle
  2. 2. Ground Rules <ul><li>Suggested Rules: </li></ul><ul><li>Come on time </li></ul><ul><li>No Phones </li></ul><ul><li>No come and go </li></ul><ul><li>No chat </li></ul>
  3. 3. M&E and the Project Cycle <ul><li>What is M&E </li></ul><ul><li>Monitoring </li></ul><ul><ul><li>Types of monitoring </li></ul></ul><ul><ul><li>Working with Indicators </li></ul></ul><ul><ul><li>Data sources </li></ul></ul><ul><li>Evaluation </li></ul><ul><li>Choosing Indicators </li></ul>
  4. 4. Why ? <ul><li>The purpose of monitoring and evaluation is to measure program effectiveness </li></ul>
  5. 5. Uses Of M & E <ul><li>Monitoring and evaluation helps: </li></ul><ul><li>make decisions on implementation </li></ul><ul><li>ensure the most effective and efficient use of resources </li></ul><ul><li>determine if the project is on track and make any needed corrections </li></ul><ul><li>evaluate the impact of the project </li></ul>
  6. 6. What is Monitoring ? <ul><li>Monitoring is collecting routine data to measure the progress of a project or activity. </li></ul><ul><li>It is used to track proje c t performance over time, to make informed decisions about the effectiveness of projects and the efficient use of resources. </li></ul><ul><li>Monitoring is also called process evaluation because it looks at the implementation process and asks: </li></ul><ul><ul><li>How well has the project been implemented? </li></ul></ul><ul><ul><li>How much does implementation vary from site to site? </li></ul></ul><ul><ul><li>Did the project benefit the intended people? At what cost? </li></ul></ul>
  7. 7. What is Evaluation ? <ul><li>Evaluation measures how well the program activities have achieved the objectives, </li></ul><ul><li>and </li></ul><ul><li>how much the changes in outcomes are due to the project. </li></ul><ul><li>The difference in the outcome between having the project or not having the project is known as its “impact,” and measuring this difference is referred to as “impact evaluation.” </li></ul>
  8. 8. Some Other Words <ul><li>Impact: any effect caused by project activity </li></ul><ul><li>including human health and safety, plants, animals, soil, air, water, climate, landscape, structures, cultural heritage or socio-economic conditions . </li></ul><ul><li>Metric: a unit of measure </li></ul><ul><li>Benchmark: a chosen level of an indicator </li></ul>
  9. 9. Some Other Words <ul><li>Objective: factual, real; can be measured </li></ul><ul><li>scientific, repeatable result </li></ul><ul><li>e.g. how many people have TB? </li></ul><ul><li>S u b j e c t i v e : from one person’s view </li></ul><ul><li>Can be different for different people </li></ul><ul><li>e.g. are we a democratic country? </li></ul>
  10. 10. M&E Plan <ul><li>The project proposal must include an M&E plan </li></ul><ul><li>M&E requires resources – time, staff, money </li></ul><ul><li>The indicators must relate to the project aim </li></ul><ul><li>The data must be reliable </li></ul><ul><li>Managers must be willing to use and learn from the results, and follow up </li></ul><ul><li>Dissemination – share information and lessons learned </li></ul>
  11. 11. M&E Plans should include.. <ul><li>assumptions regarding context, activities, and goals </li></ul><ul><li>relationships between activities, targets, and outcomes </li></ul><ul><li>description of measures and operational definitions (indicators and metrics), with baseline values, monitoring schedule, data sources, and M&E resource estimates </li></ul><ul><li>partnerships and collaborations required to achieve results </li></ul><ul><li>specific attention to periodic evaluation , with resources allocated at least midterm and at project end. </li></ul>
  12. 12. M&E <ul><li>M&E looks at progress against the INDICATORS in the logframe </li></ul><ul><li>So the first step for good M&E is choice of indicators </li></ul>
  13. 13. What is an Indicator ? <ul><li>a variable … </li></ul><ul><li>that measures one aspect of a project </li></ul><ul><li>that is directly related to the program’s objectives. </li></ul>
  14. 14. What is an Indicator ? <ul><li>An indicator is </li></ul><ul><li>a variable whose value changes from the baseline level (at the time the program began) to a new value after the program and its activities have made their impact felt. </li></ul><ul><li>Then the variable, or indicator, is calculated again. </li></ul>
  15. 15. What is an Indicator ? <ul><li>Secondly, an indicator is a measurement. </li></ul><ul><li>It measures the value of the change in meaningful units that can be compared to past and future units. </li></ul><ul><li>This is usually expressed as a percentage or a number. </li></ul>
  16. 16. What is an Indicator ? <ul><li>Finally, an indicator focuses on a single aspect of a program or project. </li></ul><ul><li>This aspect may be an input, an output or an overarching objective, but it should be narrowly defined in a way that captures this one aspect as precisely as possible. </li></ul>
  17. 17. How many Indicators? <ul><li>an appropriate set of indicators will include at least one for each significant element of the project (i.e. at least one per box in a logframe) </li></ul><ul><li>a reasonable guideline recommends one or two indicators per result, at least one indicator for each activity, but no more than 10-15 indicators per area of significant program focus. </li></ul>
  18. 18. A good indicator should: <ul><li>produce the same results when used repeatedly to measure the same condition or event; </li></ul><ul><li>measure only the condition or event it is intended to measure; </li></ul><ul><li>show changes in the state or condition over time; </li></ul><ul><li>have reasonable measurement costs; and </li></ul><ul><li>be defined in clear and unambiguous terms. </li></ul>
  19. 19. Good Indicators <ul><li>Valid </li></ul><ul><li>Reliable </li></ul><ul><li>Precise </li></ul><ul><li>Independent </li></ul><ul><li>Timely </li></ul><ul><li>Comparable </li></ul>
  20. 20. Good Indicators Validity <ul><li>The indicator measures what it is intended to measure </li></ul>
  21. 21. Good Indicators Reliability <ul><li>The indicator minimises measurement error </li></ul><ul><li>repeatable </li></ul><ul><li>Types of measurement error </li></ul><ul><li>Sampling Error - caused by observing a sample instead of the whole population </li></ul><ul><li>Non-Sampling Error – all other errors </li></ul><ul><li>Subjective Measurement - bias </li></ul>
  22. 22. Good Indicators Precise Definition <ul><li>Is defined in clear terms </li></ul><ul><ul><li>What you measure and how </li></ul></ul>
  23. 23. Good Indicators Independence <ul><li>Not directional – can be positive OR negative </li></ul><ul><li>One dimensional – up and down a line </li></ul><ul><li>Describe a discrete result at a single point in time </li></ul>
  24. 24. Good Indicators Timeliness <ul><li>Provides a measurement over the periods of time that matter (e.g. the project life) </li></ul><ul><li>with data available for all appropriate intervals </li></ul>
  25. 25. Good Indicators Comparability <ul><li>Compares with other similar situations </li></ul><ul><li>Assists in understanding results across different population groups and project approaches </li></ul>
  26. 26. Additional Factors Influencing Indicator Selection <ul><li>Data availability </li></ul><ul><li>Resources </li></ul><ul><li>Program needs </li></ul><ul><li>Donor requirements </li></ul>
  27. 27. <ul><li>Take a break </li></ul>
  28. 28. Types of Indicators <ul><li>Quantitative - an actual number of some output </li></ul><ul><li>Qualitative - descriptive observations that can supplement the numbers and percentages provided by quantitative indicators. </li></ul><ul><li>They add to quantitative indicators a richness of information about the context in which the program has been operating. </li></ul><ul><li>Examples include “availability of a clear, strategic organisational mission statement” and “existence of a multi-year procurement plan for each product offered.” </li></ul><ul><li>“ For a quantitative indicator you would collect numbers , and for a qualitative indicator you would collect facts or opinions .” </li></ul>
  29. 29. Types of Indicators <ul><li>Confusion exists in what are qualitative and quantitative indicators. </li></ul><ul><li>It is clear that quantitative indicators measure changes that can be counted. It is not clear what is a qualitative indicator. </li></ul><ul><li>some say qualitative indicators relate to the quality of the change being measured (DFID, 1995) - e.g. women’s political representation: a quantitative indicator could be the percentage of parliament seats occupied by women, while a qualitative indicator would describe the quality of women’s political participation. </li></ul><ul><li>some say qualitative indicators describe a subjective opinion on an issue or project impact. </li></ul>
  30. 30. Qualitative is Quantitative? <ul><li>Most do not define indicators as qualitative or quantitative - they assume that all indicators are by definition quantitative. </li></ul><ul><li>For example: </li></ul><ul><li>A qualitative indicator could be “most village women feel they have a voice at meetings” </li></ul><ul><li>[ but how do you know this? ] </li></ul><ul><li>So this could be quantitative: </li></ul><ul><li>“ the percentage of women surveyed who say they have a voice at meetings” </li></ul>
  31. 31. Indicators of What ? Utilisation Shows if a service is being used, e.g. • Number of people attending a nutrition course • Number of people requesting VCT • Applications received for a training course Utililisation Shows what a service has done or provided, e.g. • Number of brochures produced • Number of condoms distributed • Number of home care visits • Number of clients counselled and tested Activity or Output Shows what goes in to providing a service - the resources used, e.g. • Amount spent on travel per week • Home care supplies purchased per month • Wages, allowances and incentives paid • Production costs for brochures and posters Inputs
  32. 32. Indicators of What ? Performance Shows how well something was done, e.g. • Number of people reporting they are ‘satisfied’ with a training workshop • Number of reported cases of STIs • Proportion of VCT clients returning to collect their HIV test results • Number of orphans supported in the community Performance Coverage Shows what proportion of people / groups in need receive a service, e.g. • Proportion of all orphans receiving visits • Proportion of schools with an AIDS awareness club • Proportion of commercial farms with peer education programmes Coverage
  33. 33. Exercise <ul><li>Water and Sanitation Project </li></ul><ul><li>Read the Concept Note </li></ul><ul><li>Using the Logframe, choose the Indicators </li></ul>
  34. 34. Setting up an Indicator <ul><li>You must identify exactly how a given concept or behaviour will be measured – the Metric. </li></ul><ul><li>The Metric is the precise calculation or formula on which the indicator is based. </li></ul><ul><li>Calculation of the metric establishes the indicator’s objective value at a point in time. </li></ul><ul><li>Even if the factor itself is subjective or qualitative, (eg attitudes of a target population), the indicator metric calculates its value at a given time objectively </li></ul><ul><li>This can be called “Operationalising” an Indicator </li></ul>
  35. 35. Setting up an Indicator <ul><li>You need to be careful about exactly how you define the metric </li></ul><ul><li>e.g. “the percentage of HIV+ mothers who have prepartum AZT therapy” </li></ul><ul><li>is it – ‘% of those births attended by the health care system’ or ‘% of all births’ </li></ul><ul><li>is it – ‘% of recorded diagnosed HIV+ women’ or ‘% of all HIV+ women’ </li></ul>
  36. 36. S e t t i n g u p a n I n d i c a t o r <ul><li>In many cases, indicators need to have definitions of the terms used. </li></ul><ul><li>For instance, let’s look at the indicator: ‘number of antenatal care (ANC) providers trained’. If this indicator is used by a program, definitions need to be included. </li></ul><ul><li>Providers would need to be defined, e.g </li></ul><ul><li>‘ any worker providing direct clinical services to clients seeking ANC at a public health facility’. </li></ul><ul><li>For this indicator then, providers would not include those working in private facilities. </li></ul><ul><li>Trained would also need to be defined, perhaps as ‘ those staff who attended every day of a five-day training course and passed the final exam with a score of at least 85%’. </li></ul>
  37. 37. Thankyou <ul><li>Please keep papers for next session </li></ul>
  38. 38. Produced by Tony Hobbs Health Unlimited, Ratanakiri, Cambodia www.healthunlimited.org With the support of Australian Volunteers International www.australianvolunteers.com © 2009 HU. Use with Acknowledgement

×