Program Monitoring and Evaluation Colorado State University,USA  14 September 2002 Milan K.Dinda Ph.D
M I L A N - CSU Why the Need for M&E?
MILAN-CSU Why M&E?
MILAN-CSU Defining the Terms
MILAN-CSU What is Monitoring?   <ul><li>A  routine process  used to determine the extent to which  a program has been effe...
MILAN-CSU What is Evaluation?   <ul><li>An  episodic process  designed to determine the value or worth of a specific proje...
MILAN-CSU M and E in a Nutshell   Monitoring  Evaluation   Routine, daily assessment of on-going activities and progress  ...
MILAN-CSU Three Types of Evaluation   Process Evaluation   <ul><li>Are activities actually implemented? (Content) </li></u...
MILAN-CSU Three Types of Evaluation   Outcome Evaluation   <ul><li>Examines short-term effects (based on program objective...
MILAN-CSU Three Types of Evaluation   Impact Evaluation   <ul><li>Assesses the longer-term results of a program  </li></ul...
MILAN-CSU Systems Approach  to M&E
MILAN-CSU Systems Analysis to M&E   Input  Output Process  Impact  Outcome
MILAN-CSU Systems Analysis to M&E   Inputs  Output Process  Impact  Outcome <ul><li>Inputs </li></ul><ul><li>Resources inv...
MILAN-CSU Systems Analysis to M&E   Inputs  Output Process  Impact  Outcome <ul><li>Process  </li></ul><ul><li>activities ...
I  C  O  M  P Systems Analysis to M&E   Inputs  Output Process  Impact  Outcome <ul><li>Outputs  </li></ul><ul><li>immedia...
MILAN-CSU Systems Analysis to M&E   Inputs  Output Process  Impact  Outcome <ul><li>Outcomes (Effects) </li></ul><ul><li>c...
MILAN-CSU Systems Analysis to M&E   Inputs  Output Process  Impact  Outcome <ul><li>Impact   </li></ul><ul><li>changes in ...
MILAN-CSU Logical Framework and Systems Approach to M&E
MILAN-CSU What is a Logical Framework? Management tool for strategic planning and project management Systematically and lo...
MILAN-CSU What is a Logical Framework? Hierarchy of Aims Goal (Impact):  the ultimate end of the program to which the spec...
MILAN-CSU Logframe and Systems Analysis
MILAN-CSU M & E Framework for HIV/AIDS Program
MILAN-CSU M&E Framework for HIV/AIDS Program
MILAN-CSU M & E Process
MILAN-CSU Process <ul><li>Define the Purpose of M & E </li></ul><ul><li>Define the Scope of M & E  </li></ul><ul><li>Devel...
MILAN-CSU Define the Purpose of M & E <ul><li>Three W’s </li></ul><ul><li>What:   What interventions to be monitored and e...
MILAN-CSU Define the Scope of M & E <ul><li>Important: Review first your program goals and objectives </li></ul><ul><li>In...
MILAN-CSU Develop Indicators  <ul><li>Marker of performance -  to track progress in the performance of the HIV/AIDS interv...
MILAN-CSU Develop Indicators  <ul><li>Characteristics of Good A Good Indicator  </li></ul><ul><li>Valid:   It is applicabl...
MILAN-CSU Develop Indicators
MILAN-CSU Determine Data Sources/Methods  <ul><li>Program-based: Direct Interventions  </li></ul><ul><li>- routine program...
MILAN-CSU Develop an Implementation Plan  <ul><li>Tip: Review your Logframe (if you have) </li></ul><ul><li>Who is doing w...
MILAN-CSU Disseminate and Utilize Results  <ul><li>Utilize in Decision Making  </li></ul><ul><li>Discuss with program staf...
MILAN-CSU Principles of M & E
MILAN-CSU Principles (BLEND)  <ul><li>Bias-free  </li></ul><ul><li>Logical and Clear  </li></ul><ul><li>Educational  </li>...
MILAN-CSU Pitfalls to Avoid
MILAN-CSU Avoid Pitfalls  <ul><li>Too scientific  </li></ul><ul><li>Perfecting the design  </li></ul><ul><li>Ignoring Stor...
Upcoming SlideShare
Loading in …5
×

M%20and%20 E Milan

797 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
797
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

M%20and%20 E Milan

  1. 1. Program Monitoring and Evaluation Colorado State University,USA 14 September 2002 Milan K.Dinda Ph.D
  2. 2. M I L A N - CSU Why the Need for M&E?
  3. 3. MILAN-CSU Why M&E?
  4. 4. MILAN-CSU Defining the Terms
  5. 5. MILAN-CSU What is Monitoring? <ul><li>A routine process used to determine the extent to which a program has been effectively implemented at different levels, in time and at what cost </li></ul><ul><li>Basically an internal activity </li></ul><ul><li>Carried out regularly, for example, monthly,quarterly, half-yearly or annually. </li></ul>
  6. 6. MILAN-CSU What is Evaluation? <ul><li>An episodic process designed to determine the value or worth of a specific project or program. </li></ul><ul><li>Could be both internal and external </li></ul><ul><li>Links particular outputs or outcome directly to an intervention </li></ul>
  7. 7. MILAN-CSU M and E in a Nutshell Monitoring Evaluation Routine, daily assessment of on-going activities and progress Looks at what is being done Tracks changes over time in inputs, processes and outputs Episodic assessment of achievements Examines what has been achieved Measures outcomes/ effects and impact cased by interventions
  8. 8. MILAN-CSU Three Types of Evaluation Process Evaluation <ul><li>Are activities actually implemented? (Content) </li></ul><ul><li>Does the program reach its intended clientele? (Scope and Coverage) </li></ul><ul><li>Are activities being implemented as planned with necessary inputs? (Timing) </li></ul>
  9. 9. MILAN-CSU Three Types of Evaluation Outcome Evaluation <ul><li>Examines short-term effects (based on program objectives) </li></ul><ul><li>With HIV/AIDS interventions, immediate effects or outcomes are often related </li></ul><ul><li>to behavior changes and underlying changes in knowledge, attitudes and beliefs (e.g., use of condom) </li></ul><ul><li>Usually looks at “before and after situation” </li></ul>
  10. 10. MILAN-CSU Three Types of Evaluation Impact Evaluation <ul><li>Assesses the longer-term results of a program </li></ul><ul><li>Takes a long time, are expensive and complex </li></ul><ul><li>Example: Reduction in the new infection of HIV/AIDS </li></ul>
  11. 11. MILAN-CSU Systems Approach to M&E
  12. 12. MILAN-CSU Systems Analysis to M&E Input Output Process Impact Outcome
  13. 13. MILAN-CSU Systems Analysis to M&E Inputs Output Process Impact Outcome <ul><li>Inputs </li></ul><ul><li>Resources invested in a program (supplies, people, </li></ul><ul><li>5 M’s: (Wo)Man, Machines, Money, Methods, Materials </li></ul>
  14. 14. MILAN-CSU Systems Analysis to M&E Inputs Output Process Impact Outcome <ul><li>Process </li></ul><ul><li>activities carried out to achieve objectives </li></ul><ul><li>shows what is done and how well it is done, for example, how volunteers are recruited and trained to become peer counselors </li></ul>
  15. 15. I C O M P Systems Analysis to M&E Inputs Output Process Impact Outcome <ul><li>Outputs </li></ul><ul><li>immediate results of activities achieved (deliverables of the program) </li></ul><ul><li>number of workshops conducted; MSMs reached by BCC materials; STI, VCT care; knowledge of HIV transmission </li></ul>
  16. 16. MILAN-CSU Systems Analysis to M&E Inputs Output Process Impact Outcome <ul><li>Outcomes (Effects) </li></ul><ul><li>changes observed among clients of the program </li></ul><ul><li>changes in behavior or skills (e.g., increased usage of condoms among truck drivers; exchange of clean needles); reduction in STI trend; increase social support </li></ul>
  17. 17. MILAN-CSU Systems Analysis to M&E Inputs Output Process Impact Outcome <ul><li>Impact </li></ul><ul><li>changes in the longer term that occur in the community level as a result of a given program </li></ul><ul><li>usually observed in community level (e.g., reduction in HIV/AIDS related mortality; decreased prevalence; coping capacity of the community) </li></ul>
  18. 18. MILAN-CSU Logical Framework and Systems Approach to M&E
  19. 19. MILAN-CSU What is a Logical Framework? Management tool for strategic planning and project management Systematically and logically sets out a program’s objectives, outputs and activities .
  20. 20. MILAN-CSU What is a Logical Framework? Hierarchy of Aims Goal (Impact): the ultimate end of the program to which the specific project will contribute (reduction of prevalence of HIV/AIDS among sex workers in Chiang Mai) Purpose (Outcome/Effect): what the project is expected to achieve once it is completed within the allocated time (provision of counseling and clinic services to sex workers) Outputs: products or deliverables of activities (trained peer counselors; clinical services provided) Activities (Inputs; Process): actions to be undertaken and the resources available to produce the outputs (upgrading of clinics)
  21. 21. MILAN-CSU Logframe and Systems Analysis
  22. 22. MILAN-CSU M & E Framework for HIV/AIDS Program
  23. 23. MILAN-CSU M&E Framework for HIV/AIDS Program
  24. 24. MILAN-CSU M & E Process
  25. 25. MILAN-CSU Process <ul><li>Define the Purpose of M & E </li></ul><ul><li>Define the Scope of M & E </li></ul><ul><li>Develop Indicators </li></ul><ul><li>Determine Data Sources and Methods </li></ul><ul><li>Develop an Implementation Plan </li></ul><ul><li>Disseminate and Utilize Results </li></ul>
  26. 26. MILAN-CSU Define the Purpose of M & E <ul><li>Three W’s </li></ul><ul><li>What: What interventions to be monitored and evaluated? </li></ul><ul><li>Why: Is it a routine process or a short-term process for decision making? Is it to examine results at output, outcome or impact level? </li></ul><ul><li>Who: Who will use the results (e.g., NGO, program manager, CBD staff , community, donor) </li></ul>
  27. 27. MILAN-CSU Define the Scope of M & E <ul><li>Important: Review first your program goals and objectives </li></ul><ul><li>Intervention Level (Coverage): Program level, community level </li></ul><ul><li>Causality Links: Cause and Effect of what? </li></ul><ul><li>Timetable : How long will the M & E be undertaken? </li></ul>
  28. 28. MILAN-CSU Develop Indicators <ul><li>Marker of performance - to track progress in the performance of the HIV/AIDS interventions </li></ul><ul><li>Operational measures - “if you can measure it you can manage it” </li></ul><ul><li>Key Standards: Quality (binary - yes/no measure, categorical); Quantity (rate, absolute number, average, proportion), Cost and Timing </li></ul>
  29. 29. MILAN-CSU Develop Indicators <ul><li>Characteristics of Good A Good Indicator </li></ul><ul><li>Valid: It is applicable to the interventions being measured (percentage of condom usage) </li></ul><ul><li>Reliable: Bias-free and consistent </li></ul><ul><li>Specific: Identifies specific areas to be examined (users during the last 3 months in District 1) </li></ul><ul><li>Sensitive: Reflects changes in the situation (Increase in the number of condom users) </li></ul><ul><li>Operational: It is measurable or quantifiable (number, percentage) </li></ul>
  30. 30. MILAN-CSU Develop Indicators
  31. 31. MILAN-CSU Determine Data Sources/Methods <ul><li>Program-based: Direct Interventions </li></ul><ul><li>- routine program statistics, client surveys, FGDs, facility-based surveys, in-depth interviews, provider survey, program documents and reports, direct observations </li></ul><ul><li>Community-based: Indirect Interventions </li></ul><ul><li>- community-based surveys, FGDs, qualitative surveys, population census </li></ul><ul><li>Context-based </li></ul><ul><li>- parliamentary records, public libraries, media reports </li></ul>
  32. 32. MILAN-CSU Develop an Implementation Plan <ul><li>Tip: Review your Logframe (if you have) </li></ul><ul><li>Who is doing what? </li></ul><ul><li>When is it being done? </li></ul><ul><li>How much time does it take? </li></ul><ul><li>How much does it cost? </li></ul>
  33. 33. MILAN-CSU Disseminate and Utilize Results <ul><li>Utilize in Decision Making </li></ul><ul><li>Discuss with program staff </li></ul><ul><li>Advocacy </li></ul><ul><li>Share to donors and other interested parties </li></ul><ul><li>Share to stakeholders and communities </li></ul>
  34. 34. MILAN-CSU Principles of M & E
  35. 35. MILAN-CSU Principles (BLEND) <ul><li>Bias-free </li></ul><ul><li>Logical and Clear </li></ul><ul><li>Educational </li></ul><ul><li>Non-threatening </li></ul><ul><li>Diversity of views </li></ul>
  36. 36. MILAN-CSU Pitfalls to Avoid
  37. 37. MILAN-CSU Avoid Pitfalls <ul><li>Too scientific </li></ul><ul><li>Perfecting the design </li></ul><ul><li>Ignoring Stories </li></ul><ul><li>Probing only Success </li></ul><ul><li>Getting Done With It </li></ul>

×