Monitoring and EvaluationMinistry of InformationBy Rishi MaharajSenior Freedom of Information Officer (M&E)
The Power of Measuring ResultsIf you do not measure results, you can not tell success from failureIf you can not see success, you can not reward itIf you can not reward success, you are probably rewarding failureIf you can not see success, you can not learn from itIf you can not recognize failure, you can not correct itIf you can demonstrate results, you can win public supportAdapted from Osborne & Gaebler, 1992
DefinitionResults-Based Monitoring(what we will call “monitoring”) is a continuous process of collecting and analyzing information to compare how well a project, program or policy is being implemented against expected results
A New Emphasis on Both Implementation and Results-Based Monitoring Traditional monitoring focuses on implementation monitoringThis involves tracking inputs ($$, resources, strategies), activities (what actually took place) and outputs (the products or services produced)This approach focuses on monitoring how well a project, program or policy is being implementedOften used to assess compliance with work plans and budget
A New Emphasis on Both Implementation and Results-Based Monitoring Results-based monitoring involves the regular collection of information on how effectively government (or any organization) is performingResults-based monitoring demonstrates whether a project, program, or policy is achieving its stated goals
Indicator50%40%20 %30%Year12345  Clarifies program objectivesMonitoringLink inputs and activities to results to be achieved
Translates results to be achieved into performance indicators
Periodically collect data on the indicators and compare actual results with targetRegular collection and reporting of information to track whether actual results are being achieved as plannedReports progress and alerts management to problems in implementationResults-Based MonitoringGoal(Impacts)Long-term, widespread improvement in society
Intermediate effects of outputs on clientsOutcomesProducts and services producedOutputsTasks personnel undertake to transform inputs to outputsActivitiesInputsFinancial, human, and material resourcesResultsImplementationBinnendijk, 2000
Results-Based Monitoring: Adult LiteracyHigher income levels; increase access to higher skill jobsGoal(Impacts)Increased literacy skill; more employment opportunitiesOutcomesNumber of adults completing literacy coursesOutputsActivitiesLiteracy training coursesInputsFacilities, trainers, materialsDefinitionResults-Based EvaluationAn assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact and sustainability.  The intent is to incorporate lessons learned into the decision-making process.
Evaluation Addresses
WithprojectIndicator50%15 %40%With outproject30%Year12345Analyses why intended results were or were not achievedEvaluation  Explores unintended results
Assesses causal contributions of activities to results and validate hypothesis
  Examines implementation processesAnalytical efforts to answer specific questions about performance of a program activities. Oriented to answering WHY? And HOW?Provides lessons learned and recommendations for improvementAttribution GapLong-term Goal (Impact)OutcomesOutputsActivitiesInputsTraditional M&EResults-based M&EResults Building BlocsPLANING FOR RESULTSEffects or behavior changes resulting from program outputsProducts and services to be used to simulate the achievement of resultsUtilization  of resources to generate products and servicesResources committed to program activitiesLong-term, widespread improvement in societyResultsImplementation
Examples of Results ChainLong-Term Goal (Impact)OutcomesOutputsTeachers trained
Text Books providedIncrease literacy ratesIncreased student completion ratesEducationIncreased use of health clinicsDoctors hired
Health workers  trainedImproved maternal mortalityHealthIncreased population is food secure Increased % of vulnerable population using saftey net programs Increased coverage of crop-insurance programsSocial Protection and labor
Why an Emphasis on Outcomes?Makes explicit the intended objectives of government action    (“Know where you are going  before you get moving”)Outcomes are what produce benefitsThey tell you when you have been successful or not

Monitoring And Evaluation Presentation

  • 1.
    Monitoring and EvaluationMinistryof InformationBy Rishi MaharajSenior Freedom of Information Officer (M&E)
  • 2.
    The Power ofMeasuring ResultsIf you do not measure results, you can not tell success from failureIf you can not see success, you can not reward itIf you can not reward success, you are probably rewarding failureIf you can not see success, you can not learn from itIf you can not recognize failure, you can not correct itIf you can demonstrate results, you can win public supportAdapted from Osborne & Gaebler, 1992
  • 3.
    DefinitionResults-Based Monitoring(what wewill call “monitoring”) is a continuous process of collecting and analyzing information to compare how well a project, program or policy is being implemented against expected results
  • 4.
    A New Emphasison Both Implementation and Results-Based Monitoring Traditional monitoring focuses on implementation monitoringThis involves tracking inputs ($$, resources, strategies), activities (what actually took place) and outputs (the products or services produced)This approach focuses on monitoring how well a project, program or policy is being implementedOften used to assess compliance with work plans and budget
  • 5.
    A New Emphasison Both Implementation and Results-Based Monitoring Results-based monitoring involves the regular collection of information on how effectively government (or any organization) is performingResults-based monitoring demonstrates whether a project, program, or policy is achieving its stated goals
  • 6.
    Indicator50%40%20 %30%Year12345 Clarifies program objectivesMonitoringLink inputs and activities to results to be achieved
  • 7.
    Translates results tobe achieved into performance indicators
  • 8.
    Periodically collect dataon the indicators and compare actual results with targetRegular collection and reporting of information to track whether actual results are being achieved as plannedReports progress and alerts management to problems in implementationResults-Based MonitoringGoal(Impacts)Long-term, widespread improvement in society
  • 9.
    Intermediate effects ofoutputs on clientsOutcomesProducts and services producedOutputsTasks personnel undertake to transform inputs to outputsActivitiesInputsFinancial, human, and material resourcesResultsImplementationBinnendijk, 2000
  • 10.
    Results-Based Monitoring: AdultLiteracyHigher income levels; increase access to higher skill jobsGoal(Impacts)Increased literacy skill; more employment opportunitiesOutcomesNumber of adults completing literacy coursesOutputsActivitiesLiteracy training coursesInputsFacilities, trainers, materialsDefinitionResults-Based EvaluationAn assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact and sustainability. The intent is to incorporate lessons learned into the decision-making process.
  • 11.
  • 12.
    WithprojectIndicator50%15 %40%With outproject30%Year12345Analyseswhy intended results were or were not achievedEvaluation Explores unintended results
  • 13.
    Assesses causal contributionsof activities to results and validate hypothesis
  • 14.
    Examinesimplementation processesAnalytical efforts to answer specific questions about performance of a program activities. Oriented to answering WHY? And HOW?Provides lessons learned and recommendations for improvementAttribution GapLong-term Goal (Impact)OutcomesOutputsActivitiesInputsTraditional M&EResults-based M&EResults Building BlocsPLANING FOR RESULTSEffects or behavior changes resulting from program outputsProducts and services to be used to simulate the achievement of resultsUtilization of resources to generate products and servicesResources committed to program activitiesLong-term, widespread improvement in societyResultsImplementation
  • 15.
    Examples of ResultsChainLong-Term Goal (Impact)OutcomesOutputsTeachers trained
  • 16.
    Text Books providedIncreaseliteracy ratesIncreased student completion ratesEducationIncreased use of health clinicsDoctors hired
  • 17.
    Health workers trainedImproved maternal mortalityHealthIncreased population is food secure Increased % of vulnerable population using saftey net programs Increased coverage of crop-insurance programsSocial Protection and labor
  • 18.
    Why an Emphasison Outcomes?Makes explicit the intended objectives of government action (“Know where you are going before you get moving”)Outcomes are what produce benefitsThey tell you when you have been successful or not
  • 19.
    Selecting Key PerformanceIndicators to Monitor OutcomesOutcome indictors are not the same as outcomesEach outcome needs to be translated into one or more indicatorsAn outcome indicator identifies a specific numerical measurement that indicates progress toward achieving an outcomeUrban Institute 1999
  • 20.
    An Outcome IndicatorAnswersthe question:“How will we know achievement when we see it?”
  • 21.
    Why is Results-BasedM&E Important? Improving internal management
  • 22.
    Focusing interventions/reallocating inputsto achieve results. Programs focused are better able to demonstrate results over time.
  • 23.
    Better reporting: Programshave effective M&E system develop results-oriented reporting
  • 24.
  • 25.
    Capturing lessons-learned: Showing general program progress & sharing practices and innovations.
  • 26.
    Secure political &public support: “If you can demonstrate results, you can win public support“
  • 27.
  • 28.
    Preoccupation with resultsis a global issue : demonstrate results against money spent
  • 29.
    Results are moreimportant than processes: provide evidence that the program is producing longer term benefits. IMPORTANT TO REMEMBERWe Budget to OutputsButWe Manage to Outcomes