7 M&E: Indicators


Published on

A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 7 of 11.
There are two handouts to go with this module, Population Indicators, and a Logframe with blanks. http://www.slideshare.net/Makewa/population-indicators-handout and http://www.slideshare.net/Makewa/exercise-watsan-logframe-with-blanks

Published in: Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Watsan Proposal from last session Print evaluation plan
  • Focussing on the PROCESS
  • Focussing on the PROCESS Start at the bottom and work up We can look at monitoring the same way … (next slide)
  • Prerequisites: baseline studies, effectiveness questionnaires interviews, efficiency largely quantitative – Cost Benefit Analysis Think creatively about sources of data and methods of collection Participatory impact monitoring: return to combination of monitoring and evaluation but not so donor-led Cost: 2-3% for agric/rural dev project 5% complex or urban projects
  • Focussing on the IMPACTS At evaluation level a social cost-benefit analysis as well as cost-benefit analysis is useful Roche’s model assumes sustainability – generic indicators Rights, Livelihoods, Knowledge, Resources
  • Focussing on the IMPACTS At evaluation level a social cost-benefit analysis as well as cost-benefit analysis is useful Roche’s model assumes sustainability – generic indicators Rights, Livelihoods, Knowledge, Resources
  • No detail – go on to next 3 slides
  • This is often expressed as.. (go to next slide)
  • (check the availability of baseline information)
  • Accurate – not just correct, but reflecting the intended output
  • Watsan Proposal from last session Print evaluation plan
  • 7 M&E: Indicators

    1. 1. Development Project Planning 5 Monitoring and Evaluation Indicators
    2. 2. This Session <ul><li>Recap – about M&E and Indicators </li></ul><ul><li>M&E Planning </li></ul><ul><li>Some Indicators </li></ul><ul><ul><li>choosing </li></ul></ul><ul><ul><li>setting up </li></ul></ul>
    3. 3. Monitoring <ul><li>Performance Measurement </li></ul><ul><ul><li>of the Process </li></ul></ul><ul><ul><li>against the sections of the Logical Framework </li></ul></ul><ul><ul><li>routine </li></ul></ul><ul><ul><li>Inputs </li></ul></ul><ul><ul><li>Outputs </li></ul></ul><ul><ul><li>Outcomes ( Results Based Management ) </li></ul></ul>
    4. 4. What Indicators Show Utilisation Shows if a service is being used, e.g. • Number of people attending a nutrition course Utilisation Coverage Shows what proportion of people / groups in need receive a service, e.g. • Proportion of schools with an AIDS awareness club Coverage Performance shows how well something was done, e.g. • Number of reported cases of STIs Performance Shows what a service has done or provided, e.g. • Number of condoms distributed Activity or Output Shows what goes in to providing a service - the resources used, e.g. • Amount spent on travel per week Inputs
    5. 5. Monitoring (Performance Measurement) <ul><li>Indicators show success at different levels </li></ul><ul><ul><li>Strategic </li></ul></ul><ul><ul><ul><li>Goals, Objectives </li></ul></ul></ul><ul><ul><li>Sustainability </li></ul></ul><ul><ul><ul><li>after project complete </li></ul></ul></ul><ul><ul><li>Effectiveness </li></ul></ul><ul><ul><ul><li>on the objectives </li></ul></ul></ul><ul><ul><li>Efficiency </li></ul></ul><ul><ul><ul><li>inputs against outputs </li></ul></ul></ul>Quality, Quantity, Time
    6. 6. Monitoring Criteria <ul><ul><li>Relevance to goal/purpose and in-country needs </li></ul></ul><ul><ul><li>Efficiency in providing inputs and converting to outputs </li></ul></ul><ul><ul><li>Effectiveness – has production of outputs achieved objectives? </li></ul></ul><ul><ul><li>Impact – is purpose making anticipated level of contribution to high-level goals </li></ul></ul><ul><ul><li>Sustainability – the benefits will continue, without future negative impact </li></ul></ul>
    7. 7. Exercise 5 mins <ul><li>“ Look back to our proposal for a watsan project in Ratanakiri.” </li></ul><ul><li>Come up with one indicator for ‘pump maintenance’ AT EACH LEVEL </li></ul><ul><ul><li>Activity (efficiency) </li></ul></ul><ul><ul><li>Objectives (effectiveness) </li></ul></ul><ul><ul><li>Strategic - Goals (impact) </li></ul></ul><ul><ul><li>Sustainability – (continues after project) </li></ul></ul>
    8. 8. Data Collection Flow in Projects
    9. 9. What should you monitor and how? <ul><li>You should monitor information that will help you to track what you are doing and to measure the success of your work. </li></ul><ul><li>So you should review your objectives and indicators and decide what information needs to be collected to allow the indicators to be measured. </li></ul><ul><li>You should only collect information that you will use. </li></ul>
    10. 10. Evaluation <ul><li>“episodic” – happens at certain times, not routine </li></ul><ul><li>Looks at the Impact (Impact Assessment) </li></ul>
    11. 11. Evaluation: Impact Assessment <ul><li>Approach </li></ul><ul><ul><li>Impact from beneficiaries’ point of view </li></ul></ul><ul><ul><li>What do they think is significant? </li></ul></ul><ul><ul><li>To whom is it important? </li></ul></ul><ul><ul><li>Ex-ante and Ex-post </li></ul></ul><ul><li>Criteria </li></ul><ul><ul><li>Efficiency – relate inputs to outputs </li></ul></ul><ul><ul><li>Effectiveness- extent to which achieved objectives </li></ul></ul><ul><ul><li>Consistency- methods/approaches with objectives </li></ul></ul><ul><ul><li>Impact – change to lives/environment </li></ul></ul>
    12. 12. Audit <ul><li>Mainly look at financial operations and statements </li></ul><ul><ul><li>compliance with legal and contractual obligations </li></ul></ul><ul><ul><li>external, imposed </li></ul></ul><ul><li>Can have “Performance Audits” </li></ul><ul><ul><li>efficiency and good management </li></ul></ul><ul><ul><li>Data Quality audits </li></ul></ul>
    13. 13. What is an Indicator ? <ul><li>a variable … </li></ul><ul><li>that measures one aspect of a project </li></ul><ul><li>that is directly related to the program’s objectives. </li></ul>
    14. 14. Break Please – back here ON TIME !
    15. 15. A good indicator should: <ul><li>produce the same results when used repeatedly to measure the same condition or event; </li></ul><ul><li>measure only the condition or event it is intended to measure; </li></ul><ul><li>show changes in the state or condition over time; </li></ul><ul><li>have reasonable measurement costs; and </li></ul><ul><li>be defined in clear and unambiguous terms. </li></ul>
    16. 16. Good Indicator - is… <ul><li>Valid - measures what it is intended to </li></ul><ul><li>Reliable - accurate, repeatable </li></ul><ul><li>Precise - measures intended condition only </li></ul><ul><li>Independent - gives a line of possible results </li></ul><ul><li>Timely - available when needed </li></ul><ul><li>Comparable - with other similar situations </li></ul><ul><li>Varies - shows changes in the metric over time; </li></ul><ul><li>Costed - reasonable measurement costs; and </li></ul><ul><li>Defined - in clear and unambiguous terms. </li></ul>
    17. 17. Other Factors In Indicator Choice <ul><li>Data availability </li></ul><ul><li>Resources </li></ul><ul><li>Program needs </li></ul><ul><li>Donor requirements </li></ul>
    18. 18. Indicators: An Example <ul><li>Objective: Polluted water put into the Bassac is reduced </li></ul><ul><li>Select the indicator: Concentration of heavy metals ( Pb, Cd, Hg ) </li></ul><ul><li>Define the targets: </li></ul><ul><ul><li>the quantity: Co ncentration of heavy metal compounds ( Pb, Cd, Hg ) is r educed by 75 % from year 2008 levels … </li></ul></ul><ul><ul><li>the quality: ... to meet the limits for irrigation water . .. </li></ul></ul><ul><ul><li>the target group: ... used by the farmers of Phan village, ... </li></ul></ul><ul><ul><li>the place : ... in Buphon District, ... </li></ul></ul><ul><ul><li>the time: ... 2 years after the project has started . </li></ul></ul>
    19. 19. Reporting Times ● Health services statistics ● Health facility surveys ● Program monitoring Continuously Input/Process ● Surveillance ● Population-based surveys ● Special studies 2 to 5 years Impact ● Population-based surveys ● Health facility surveys ● Special studies 1 to 3 years Outcome ● Health services statistics ● Health facility surveys ● Program monitoring Quarterly, semi-annually, or annually Output Examples of Data Collection Methods Frequency of Reporting Level of Indicator
    20. 20. Means of Verification <ul><li>M&E plans should include multiple data sources. </li></ul><ul><li>they should include indicators that use data from each sector, and </li></ul><ul><li>include data from the program facilities, population measures, and special surveys. </li></ul><ul><li>The M&E plan should clearly state the sources for collecting data and how often the data should be collected. </li></ul>
    21. 21. Means of Verification <ul><li>Data source must be: </li></ul><ul><ul><li>Available </li></ul></ul><ul><ul><ul><li>time </li></ul></ul></ul><ul><ul><ul><li>cost </li></ul></ul></ul><ul><ul><li>Accurate </li></ul></ul><ul><ul><ul><li>Showing what you want to show </li></ul></ul></ul><ul><ul><ul><li>Show what is really there </li></ul></ul></ul>
    22. 22. Exercise <ul><li>Look at our watsan logframe </li></ul><ul><li>Fill in the MoV column </li></ul>
    23. 23. Data Systems <ul><li>Tools: </li></ul><ul><ul><li>Progress reports </li></ul></ul><ul><ul><li>Team meetings, team briefing reports </li></ul></ul><ul><ul><li>HIS records </li></ul></ul>
    24. 24. Evaluation Plan <ul><li>A part of the project documents </li></ul><ul><li>Use to plan staff needs and budget </li></ul>
    25. 25. Contents of a Typical Evaluation Plan (1) <ul><li>Brief project description </li></ul><ul><li>Objectives of the project </li></ul><ul><li>Objectives of the evaluation </li></ul>
    26. 26. Contents of a Typical Evaluation Plan (2) <ul><li>Evaluation methodology – type of evaluation, indicators for each objective, data collection methods, sample size, methods of sampling / selecting participants </li></ul><ul><li>Resources needed, timetable, and budget – material, human, financial, transportation and logistics </li></ul><ul><li>Planned use of results – for example, improve the project mid-course, plan future projects, guide decision-making </li></ul>
    27. 27. Thankyou GrKuN
    28. 28. Produced by Tony Hobbs Health Unlimited, Ratanakiri, Cambodia www.healthunlimited.org With the support of Australian Volunteers International www.australianvolunteers.com © 2009 HU. Use with Acknowledgement