MONITORING AND EVALUATION
UNIT IV
The purpose of covering the topic
To equip students with the knowledge that can can enable them to:
ď‚§ Design an M&E system for a nutrition Programme
ď‚§ Conduct M&E
ď‚§ Effectively use a M&E data base to monitor progress in
Programme implementation
M&E cont.
• The M&E session covers the following areas:
– Purpose of M&E in programming.
– Components of an M&E plan /framework for
a nutrition Programme
– Conducting M&E activities
– M&E indicators
– Evaluation of nutrition programmes
Monitoring and Evaluation
• Monitoring: what is it?
It is the ongoing collection and review of
information on project implementation,
coverage and utilization that is reported
and acted on, on on-going basis.
M&E cont.
• Why do monitoring?
– To improve intervention programmes by identifying
aspects that are working as planned and those that
need correction.
– To modify Programme as per identified need.
– To ensure that all needy cases are reached:
General food ration reaches all targeted persons
By maximising on inclusion of deserving cases and
exclusion of undeserving cases in special targeting I.e
severely malnourished
– To track ( and demonstrate) results at the program or
population level.
Monitoring cont.
• Thus monitoring is used to determine how well a
Programme is being implemented
– At different levels
– At what cost
– Also tracks the changes occurring due to
interventions being implemented
(positive or negative
M&E cont
Evaluation: What is it?
• A process of data collection designed to assess
the effectiveness of the project in attaining its
originally stated objectives and the extent to which
observed changes are attributable to the project.
• Done at the end of the project but could be planned
at strategic periods during during the life of the
projects;inform of reviews e.g. mid- term reviews,
or biennial reviews.
Evaluation can use rigorous study designs e.g.,
experimental design or quasi-experimental-involving
control groups.
M&E
• M&E happens only after the decision to
implement a certain intervention has been made
• Both monitoring and evaluation need
– clearly stated goals
– clearly stated objectives.
Components of M&E
• Usually four components are considered
– Inputs
– Processes
– Outputs
– Outcomes
Components
• Inputs are the set of resources dedicated to a
Programme:
• They include human and financial resources,
physical facilities, equipment and operational
policies that enable services to be delivered,I.e.,
– Personnel
– Facilities
– Space: storage, room for admitting children onTFP, for
conducting SFP and space for guardians.
– Equipment
– Supplies
Components
• Process refers to the set of activities in which
Programme inputs are utilized in pursuit of the results
from the Programme
• process refer to multiple activities that are carried out to
achieve the objectives of the projects
– Service delivery operations (food distribution)
– Management oriented activities
– Training, IES, research, etc.
Components
• Outputs are the results obtained at the
Programme level through execution of activities
using its resources (inputs):
Outputs cont.
• Outputs
– Could be staff performing better as a result of
having been trained
– As a result of clear policy environment
– Staff better informed about policies- better in
taking decisions.
Components
• Outputs classified into three levels:
• Functional outputs: number of nutrition IEC talks,
food preparation demonstration, people trained
• Service outputs:e.g, number benefiting/accessing
TFP,SFP, quality of service, acceptability/ image
• Service utilization: number using the
service(collecting rations, visiting ANC clinics for
supplements)
Outcomes
• Outcome: the set of results expected to occur at
the population level due to Programme activities
and generation of Programme outputs. The
intermediate effects are often behavioral and
result directly from project outputs. They may be
necessary to achieve a desired impact.
Outcomes cont.
• Outcomes may be divided into two components:
Intermediate outcomes and long-term outcomes.
• Immediate outcomes:are set of results at the
population level that are closely and clearly linked to the
Programme activities.
• Long-term outcomes: refer to set of results at the
population level that are long- term in nature and are
produced through action of immediate outcomes.
Outcomes cont.
• There is generally a considerable time lag 5-10
years between inception and change in long
range outcomes.Good health, reduced mortality
rates.
Components
• Outcomes- two levels
• Intermediate outcomes
Occur at population level and are closely associated with
Programme inputs(drop in prevalence of severe cases of
malnutrition, anemia).
• Long range outcomes
Refer to results at population level that are long term in
nature and come through action of intermediate
outcomes (drop in overall prevalence of malnutrition,
deaths, better health)
Outcomes cont.
• Six key outcomes for nutrition
– Exclusive breastfeeding
– Appropriate complementary feeding,
– Adequate iron intake
– Adequate vitamin A intake
– Adequate iodine intake
– Adequate nutritional care during illness and
severe malnutrition
Elements of M&E
Indicators
• Indicators are variables that measure the
different aspects of a given Programme.
– The inputs, processes, outputs and outcomes.
• An indicator can be assigned a numeric value, a
percentage, a mean value, a ranking, an
absolute number of yes/ no score e.g., presence
verses absence.
Indicators cont
• Selection of indicators
Indicators must be selected to provide evidence
that defines the extent to which project
interventions are successful in achieving the set
objectives.
Indicators cont.
Criteria for selection of indicators
• Validity: It measures what it is intended to
• Sensitivity: its changes reflect desired changes
• Reliable: It produces the same results when repeated
• Uni-dimensional: it measures only one phenomenon
• Operational: it is measurable
• Objective: it is not subject to Measurer’s biases
• Practical: its data collection is reasonably feasible
• Comparability (from time to time/ place to place)
Monitoring and Evaluation Plan
What is a Monitoring and Evaluation Plan?
ď‚§ A monitoring and evaluation (M&E) plan is a
document that helps to track and assess the
results of the interventions throughout the life of
a program.
ď‚§ It is a living document that should be referred to
and updated on a regular basis.
 While the specifics of each program’s M&E plan
will look different, they should all follow the same
basic structure and include the same key
elements.
• An M&E plan will include some documents that
may have been created during the program
planning process, and some that will need to be
created new.
• For example, elements such as the logical
framework and monitoring indicators may have
already been developed.
• The M&E plan takes those documents and
develops a further plan for their implementation.
Why develop a Monitoring and Evaluation Plan?
• It is important to develop an M&E plan before beginning
any monitoring activities so that there is a clear plan for
what questions about the program need to be answered.
• It will help program staff decide how they are going to
collect data to track indicators, how monitoring data will
be analyzed, and how the results of data collection will
be disseminated both to the donor and internally among
staff members for program improvement.
• M&E data alone is not useful until
someone puts it to use!
• An M&E plan will help make sure data is
being used efficiently to make programs as
effective as possible and to be able to
report on results at the end of the program.
Steps in developing an M&E Plan
Step 1: Identify Program Goals and Objectives
• The first step to creating an M&E plan is to
identify the program goals and objectives.
• If the program already has a logical
framework then the program goals are
most likely already defined.
• However, if not, the M&E plan is a great
place to start.
Step 2: Define Indicators
• Once the program’s goals and objectives
are defined, it is time to define indicators
for tracking progress towards achieving
those goals.
• Program indicators should be a mix of
those that measure process, or what is
being done in the program, and those that
measure outcomes.
• Process indicators track the progress of the
program.
• They help to answer the question, “Are activities
being implemented as planned?” Some
examples of process indicators are:
• Number of trainings held with health providers
• Number of outreach activities conducted at
youth-friendly locations
• Outcome indicators track how successful
program activities have been at achieving
program objectives.
• They help to answer the question, “Have
program activities made a difference?”
• Some examples of outcome indicators are:
• Percent of women consuming orange
fleshed sweet potatos
Step 3: Define Data Collection Methods and TImeline
• After creating monitoring indicators, it is time to
decide on methods for gathering data and how
often various data will be recorded to track
indicators.
• This should be a conversation between program
staff, stakeholders, and donors.
• These methods will have important implications
for what data collection methods will be used
and how the results will be reported.
• The source of monitoring data depends
largely on what each indicator is trying to
measure.
• The program will likely need multiple data
sources to answer all of the programming
questions.
• once it is determined how data will be collected, it is also
necessary to decide how often it will be collected.
• This will be affected by donor requirements, available
resources, and the timeline of the intervention.
• Some data will be continuously gathered by the program
(such as the number of trainings), but these will be
recorded every six months or once a year, depending on
the M&E plan.
• Other types of data depend on outside sources, such as
clinic and DHS data.
• After all of these questions have been
answered, a table like the one below can
be made to include in the M&E plan.
• This table can be printed out and all staff
working on the program can refer to it so
that everyone knows what data is needed
and when.
Indicator Data source Timing
Number of trainings held
with health providers
raining attendance
sheets
Every 6 months
Number of outreach
activities conducted at
outreach locations
Activity sheet Every 6 months
Number of Vitamin A
vitamin A Capsules
distributed at outreach
locations
Vitamin A distribution
sheets
Every 6 months
Step 4: Identify M&E Roles and Responsibilities
• The next element of the M&E plan is a section
on roles and responsibilities.
• It is important to decide from the early planning
stages who is responsible for collecting the data
for each indicator.
• This will probably be a mix of M&E staff,
research staff, and program staff.
• Everyone will need to work together to get data
collected accurately and in a timely fashion.
• .
• Data management roles should be
decided with input from all team members
so everyone is on the same page and
knows which indicators they are assigned.
• This way when it is time for reporting there
are no surprises
• An easy way to put this into the M&E plan
is to expand the indicators table with
additional columns for who is responsible
for each indicator, as shown below.
Indicator Data source Timing Data manager
Number of
trainings held with
health providers
raining
attendance sheets
Every 6 months Activity manager
Number of
outreach activities
conducted at
outreach locations
Activity sheet Every 6 months Activity manager
Number of
Vitamin A vitamin
A Capsules
distributed at
outreach locations
to under-five
children
Vitamin A
distribution
sheets/tally
Every 6 months Activity manager
Step 5: Create an Analysis Plan and Reporting
Templates
• Once all of the data have been collected,
someone will need to compile and analyze
it to fill in a results table for internal review
and external reporting.
• This is likely to be an in-house M&E
manager or research assistant for the
program.
• The M&E plan should include a section
with details about what data will be
analyzed and how the results will be
presented.
• Do research staff need to perform any
statistical tests to get the needed
answers? If so, what tests are they and
what data will be used in them?
• What software program will be used to
analyze data and make reporting tables?
Excel? SPSS? These are important
considerations.
• Another good thing to include in the plan is
a blank table for indicator reporting.
• These tables should outline the indicators,
data, and time period of reporting
• They can also include things like the
indicator target, and how far the program
has progressed towards that target.
• An example of a reporting table is below.
Indicator Baseline Year 1 Target % of target
achieved
Number of trainings
held with health
providers
0 5 10 50%
Number of outreach
activities conducted at
outreach locations
0 2 6 33%
Number of Vitamin A
vitamin A Capsules
distributed at outreach
locations
0 240000 1000000 50%
Step 6: Plan for Dissemination and Donor
Reporting
• The last element of the M&E plan
describes how and to whom data will be
disseminated.
• Data for data’s sake should not be the
ultimate goal of M&E efforts.
• Data should always be collected for
particular purposes.
Consider the following:
• How will M&E data be used to inform staff and
stakeholders about the success and progress of
the program?
• How will it be used to help staff make
modifications and course corrections, as
necessary?
• How will the data be used to move the field
forward and make program practices more
effective?
• The M&E plan should include plans for
internal dissemination among the program
team, as well as wider dissemination
among stakeholders and donors.

Unit IV_Monitoring_and_Evaluation.pptx

  • 1.
  • 2.
    The purpose ofcovering the topic To equip students with the knowledge that can can enable them to: ď‚§ Design an M&E system for a nutrition Programme ď‚§ Conduct M&E ď‚§ Effectively use a M&E data base to monitor progress in Programme implementation
  • 3.
    M&E cont. • TheM&E session covers the following areas: – Purpose of M&E in programming. – Components of an M&E plan /framework for a nutrition Programme – Conducting M&E activities – M&E indicators – Evaluation of nutrition programmes
  • 4.
    Monitoring and Evaluation •Monitoring: what is it? It is the ongoing collection and review of information on project implementation, coverage and utilization that is reported and acted on, on on-going basis.
  • 5.
    M&E cont. • Whydo monitoring? – To improve intervention programmes by identifying aspects that are working as planned and those that need correction. – To modify Programme as per identified need. – To ensure that all needy cases are reached: General food ration reaches all targeted persons By maximising on inclusion of deserving cases and exclusion of undeserving cases in special targeting I.e severely malnourished – To track ( and demonstrate) results at the program or population level.
  • 6.
    Monitoring cont. • Thusmonitoring is used to determine how well a Programme is being implemented – At different levels – At what cost – Also tracks the changes occurring due to interventions being implemented (positive or negative
  • 7.
    M&E cont Evaluation: Whatis it? • A process of data collection designed to assess the effectiveness of the project in attaining its originally stated objectives and the extent to which observed changes are attributable to the project. • Done at the end of the project but could be planned at strategic periods during during the life of the projects;inform of reviews e.g. mid- term reviews, or biennial reviews. Evaluation can use rigorous study designs e.g., experimental design or quasi-experimental-involving control groups.
  • 8.
    M&E • M&E happensonly after the decision to implement a certain intervention has been made • Both monitoring and evaluation need – clearly stated goals – clearly stated objectives.
  • 9.
    Components of M&E •Usually four components are considered – Inputs – Processes – Outputs – Outcomes
  • 10.
    Components • Inputs arethe set of resources dedicated to a Programme: • They include human and financial resources, physical facilities, equipment and operational policies that enable services to be delivered,I.e., – Personnel – Facilities – Space: storage, room for admitting children onTFP, for conducting SFP and space for guardians. – Equipment – Supplies
  • 11.
    Components • Process refersto the set of activities in which Programme inputs are utilized in pursuit of the results from the Programme • process refer to multiple activities that are carried out to achieve the objectives of the projects – Service delivery operations (food distribution) – Management oriented activities – Training, IES, research, etc.
  • 12.
    Components • Outputs arethe results obtained at the Programme level through execution of activities using its resources (inputs):
  • 13.
    Outputs cont. • Outputs –Could be staff performing better as a result of having been trained – As a result of clear policy environment – Staff better informed about policies- better in taking decisions.
  • 14.
    Components • Outputs classifiedinto three levels: • Functional outputs: number of nutrition IEC talks, food preparation demonstration, people trained • Service outputs:e.g, number benefiting/accessing TFP,SFP, quality of service, acceptability/ image • Service utilization: number using the service(collecting rations, visiting ANC clinics for supplements)
  • 15.
    Outcomes • Outcome: theset of results expected to occur at the population level due to Programme activities and generation of Programme outputs. The intermediate effects are often behavioral and result directly from project outputs. They may be necessary to achieve a desired impact.
  • 16.
    Outcomes cont. • Outcomesmay be divided into two components: Intermediate outcomes and long-term outcomes. • Immediate outcomes:are set of results at the population level that are closely and clearly linked to the Programme activities. • Long-term outcomes: refer to set of results at the population level that are long- term in nature and are produced through action of immediate outcomes.
  • 17.
    Outcomes cont. • Thereis generally a considerable time lag 5-10 years between inception and change in long range outcomes.Good health, reduced mortality rates.
  • 18.
    Components • Outcomes- twolevels • Intermediate outcomes Occur at population level and are closely associated with Programme inputs(drop in prevalence of severe cases of malnutrition, anemia). • Long range outcomes Refer to results at population level that are long term in nature and come through action of intermediate outcomes (drop in overall prevalence of malnutrition, deaths, better health)
  • 19.
    Outcomes cont. • Sixkey outcomes for nutrition – Exclusive breastfeeding – Appropriate complementary feeding, – Adequate iron intake – Adequate vitamin A intake – Adequate iodine intake – Adequate nutritional care during illness and severe malnutrition
  • 20.
    Elements of M&E Indicators •Indicators are variables that measure the different aspects of a given Programme. – The inputs, processes, outputs and outcomes. • An indicator can be assigned a numeric value, a percentage, a mean value, a ranking, an absolute number of yes/ no score e.g., presence verses absence.
  • 21.
    Indicators cont • Selectionof indicators Indicators must be selected to provide evidence that defines the extent to which project interventions are successful in achieving the set objectives.
  • 22.
    Indicators cont. Criteria forselection of indicators • Validity: It measures what it is intended to • Sensitivity: its changes reflect desired changes • Reliable: It produces the same results when repeated • Uni-dimensional: it measures only one phenomenon • Operational: it is measurable • Objective: it is not subject to Measurer’s biases • Practical: its data collection is reasonably feasible • Comparability (from time to time/ place to place)
  • 23.
  • 24.
    What is aMonitoring and Evaluation Plan?  A monitoring and evaluation (M&E) plan is a document that helps to track and assess the results of the interventions throughout the life of a program.  It is a living document that should be referred to and updated on a regular basis.  While the specifics of each program’s M&E plan will look different, they should all follow the same basic structure and include the same key elements.
  • 25.
    • An M&Eplan will include some documents that may have been created during the program planning process, and some that will need to be created new. • For example, elements such as the logical framework and monitoring indicators may have already been developed. • The M&E plan takes those documents and develops a further plan for their implementation.
  • 26.
    Why develop aMonitoring and Evaluation Plan? • It is important to develop an M&E plan before beginning any monitoring activities so that there is a clear plan for what questions about the program need to be answered. • It will help program staff decide how they are going to collect data to track indicators, how monitoring data will be analyzed, and how the results of data collection will be disseminated both to the donor and internally among staff members for program improvement.
  • 27.
    • M&E dataalone is not useful until someone puts it to use! • An M&E plan will help make sure data is being used efficiently to make programs as effective as possible and to be able to report on results at the end of the program.
  • 28.
  • 29.
    Step 1: IdentifyProgram Goals and Objectives • The first step to creating an M&E plan is to identify the program goals and objectives. • If the program already has a logical framework then the program goals are most likely already defined. • However, if not, the M&E plan is a great place to start.
  • 30.
    Step 2: DefineIndicators • Once the program’s goals and objectives are defined, it is time to define indicators for tracking progress towards achieving those goals. • Program indicators should be a mix of those that measure process, or what is being done in the program, and those that measure outcomes.
  • 31.
    • Process indicatorstrack the progress of the program. • They help to answer the question, “Are activities being implemented as planned?” Some examples of process indicators are: • Number of trainings held with health providers • Number of outreach activities conducted at youth-friendly locations
  • 32.
    • Outcome indicatorstrack how successful program activities have been at achieving program objectives. • They help to answer the question, “Have program activities made a difference?” • Some examples of outcome indicators are: • Percent of women consuming orange fleshed sweet potatos
  • 33.
    Step 3: DefineData Collection Methods and TImeline • After creating monitoring indicators, it is time to decide on methods for gathering data and how often various data will be recorded to track indicators. • This should be a conversation between program staff, stakeholders, and donors. • These methods will have important implications for what data collection methods will be used and how the results will be reported.
  • 34.
    • The sourceof monitoring data depends largely on what each indicator is trying to measure. • The program will likely need multiple data sources to answer all of the programming questions.
  • 35.
    • once itis determined how data will be collected, it is also necessary to decide how often it will be collected. • This will be affected by donor requirements, available resources, and the timeline of the intervention. • Some data will be continuously gathered by the program (such as the number of trainings), but these will be recorded every six months or once a year, depending on the M&E plan. • Other types of data depend on outside sources, such as clinic and DHS data.
  • 36.
    • After allof these questions have been answered, a table like the one below can be made to include in the M&E plan. • This table can be printed out and all staff working on the program can refer to it so that everyone knows what data is needed and when.
  • 37.
    Indicator Data sourceTiming Number of trainings held with health providers raining attendance sheets Every 6 months Number of outreach activities conducted at outreach locations Activity sheet Every 6 months Number of Vitamin A vitamin A Capsules distributed at outreach locations Vitamin A distribution sheets Every 6 months
  • 38.
    Step 4: IdentifyM&E Roles and Responsibilities • The next element of the M&E plan is a section on roles and responsibilities. • It is important to decide from the early planning stages who is responsible for collecting the data for each indicator. • This will probably be a mix of M&E staff, research staff, and program staff. • Everyone will need to work together to get data collected accurately and in a timely fashion. • .
  • 39.
    • Data managementroles should be decided with input from all team members so everyone is on the same page and knows which indicators they are assigned. • This way when it is time for reporting there are no surprises
  • 40.
    • An easyway to put this into the M&E plan is to expand the indicators table with additional columns for who is responsible for each indicator, as shown below.
  • 41.
    Indicator Data sourceTiming Data manager Number of trainings held with health providers raining attendance sheets Every 6 months Activity manager Number of outreach activities conducted at outreach locations Activity sheet Every 6 months Activity manager Number of Vitamin A vitamin A Capsules distributed at outreach locations to under-five children Vitamin A distribution sheets/tally Every 6 months Activity manager
  • 42.
    Step 5: Createan Analysis Plan and Reporting Templates • Once all of the data have been collected, someone will need to compile and analyze it to fill in a results table for internal review and external reporting. • This is likely to be an in-house M&E manager or research assistant for the program.
  • 43.
    • The M&Eplan should include a section with details about what data will be analyzed and how the results will be presented. • Do research staff need to perform any statistical tests to get the needed answers? If so, what tests are they and what data will be used in them?
  • 44.
    • What softwareprogram will be used to analyze data and make reporting tables? Excel? SPSS? These are important considerations. • Another good thing to include in the plan is a blank table for indicator reporting. • These tables should outline the indicators, data, and time period of reporting
  • 45.
    • They canalso include things like the indicator target, and how far the program has progressed towards that target. • An example of a reporting table is below.
  • 46.
    Indicator Baseline Year1 Target % of target achieved Number of trainings held with health providers 0 5 10 50% Number of outreach activities conducted at outreach locations 0 2 6 33% Number of Vitamin A vitamin A Capsules distributed at outreach locations 0 240000 1000000 50%
  • 47.
    Step 6: Planfor Dissemination and Donor Reporting • The last element of the M&E plan describes how and to whom data will be disseminated. • Data for data’s sake should not be the ultimate goal of M&E efforts. • Data should always be collected for particular purposes.
  • 48.
    Consider the following: •How will M&E data be used to inform staff and stakeholders about the success and progress of the program? • How will it be used to help staff make modifications and course corrections, as necessary? • How will the data be used to move the field forward and make program practices more effective?
  • 49.
    • The M&Eplan should include plans for internal dissemination among the program team, as well as wider dissemination among stakeholders and donors.