MEAL ETH11171-draft.pptx

Photo:
Welthungerhilfe
MEAL WORKSHOP ON ETH1171
Discussion on MEAL practice on ETH1117
April 26-29,2023
Abraham Lebeza

2
# Topic Timing
MEAL Training Outline- Training objective is being familiar/acquainted with the following topics
 General MEAL Concepts
 Theory of change /Logframe
 Indicator (Outcome/output)
 Measuring success at WHH)
 ETH1171 MEAL plan development
 Data management , Data quality and
beneficiary counting
 Accountability at different level(project,
organizational and social)
 CFRM and WHH code of conduct
 Setting and promotion of learnings drawn form
the project as consortium of IPs

3
# Topic Timing
General MEAL Concepts

4
# Topic Timing
What are these buzz words ?
Monitoirng ?
Evaluation?

5
# Topic Timing
Are monitoring and evaluation similar or identical to each other?
How is monitoring and evaluation are reinforcing to each other?
1. Group exercise
 to differentiate monitoring and evaluation?
 show how Monitoirng and evaluation are reinforcing to each other?

6
# Topic Timing
Characteristics Monitoring Evaluation
Timing Continuous /regular Periodic
Purpose Plan versus achievement,
track records
Judgmental of effectiveness
, what works well and what
doesn’t
Big question Are we doing what we are
expected to do
Are we doing the right things
Rigor /method Enough to drive continuous
improvement
Systematic design that
employs qualitative and
quantitative study
Actors/Staffing Internal Internal and external
Audience Internal Internal and external

7
# Topic Timing
MONITORING VERSUS EVALAUTION
Characteristics of monitoring and evaluation compared (adapted from Jaszczolt, Potkański, and Alwasiak 2003)
CHARACTERISTIC EVALUATION MONITORING
Subject Usually focused on strategic aspects. Operational management issues
addressed.
Character Subject and methods flexible. Systematic.
Frequency Periodic. Continuous.
Primary client Stakeholders and external audience. Program management.

8
# Topic Timing
MONITORING VERSUS EVALUATION
Characteristics of monitoring and evaluation compared (adapted from Jaszczolt, Potkański, and Alwasiak 2003)
CHARACTERISTIC EVALUATION MONITORING
Party conducting Can be external or internal. Internal.
Approach Objectivity, transparency. Utility.
Methodology Rigorous research methodologies,
sophisticated tools.
Rapid appraisal methods.
Primary focus Focus on relevancy, outcomes, impact,
and sustainability.
Focus on operational efficiency.
Objectives To check outcomes and impact, to verify
developmental hypothesis, and to
document successes and lessons
learned.
To identify and resolve implementation
problems; to assess progress toward
objectives

9
# Topic Timing
Monitoring and evaluation differ in terms of audience, timing, purposes and actors.
Good monitoring makes the evaluation more accurate: If an issue has been
spotted thanks to monitoring, the evaluation can investigate it.
Good monitoring makes the evaluation more focused: When data about
activities, outputs and first outcomes are already collected and analyzed, the
evaluation can build upon these data and focus on more interesting questions
such as the impacts of the programme.
Good monitoring makes the evaluation more relevant: If you already know about
what has changed, the evaluation can focus on the how and why, and learning
lessons from experience.

10
# Topic Timing
The six main components of a project M&E system
1. Clear statements of measurable objectives for the project and its components. Why
an M&E system is needed?
2. A structured set of indicators covering: inputs, process, outputs, outcomes, impact,
and exogenous factors. What do we need to know to manage the project?
3. Data collection mechanisms capable of monitoring progress over time, including
baselines and a means to compare progress and achievements against targets.
Monitoring plan

11
# Topic Timing
Contd..
4. Where applicable building on baselines and data collection with an evaluation
framework and methodology capable of establishing causation (i.e. capable of
attributing observed change to given interventions or other factors).
5. Clear mechanisms for reporting and use of M&E results in decision-making. How
and to whom do we need to communicate the information
6. Sustainable organizational arrangements for data collection, management, analysis,
and reporting. Capacity building and reinstalling M&E culture

12
Topic Slide
#
MEAL PURPOSES
2GROUP EXERCISE
What is existing practice at partners in implementing the M&E system ?
What do the strategies partners employed to translate accountability to project
beneficiaries?
Bottlenecks –what should we overcome in the future

13
Topic Slide
#
In their book on Reinventing Government (1992),
Osborne and Gaebler note that:
- What gets measured gets done.
- If you don’t measure results, you can’t tell success from failure.
- If you can’t see success, you can’t reward it.
- If you don’t recognize failure, you can’t correct it.

14
Topic Slide
#
MEAL WORKSHOP ON MEAL WORKSHOP ON
ETH1117
Failing to plan is Planning to fail

15
Topic Slide
#
MEAL PURPOSES
1. Harnessing project management decisions through tracking project
progress/performance against budget and physical plan
1.1. Measurement –Result Based Management
2. Forward and backward accountability
3. Lessons learning so that we can build up to improve our programming

16
Topic Slide
#
Discussions
 What good practices Partners would replicate to the coming project
Partners MEAL performance and constraining challenges
 What partners expect from WHH for further strengthening MEAL
 What was partners experience in M&E in general and in implementing previous
projects
 MEAL practices Challenges and good practices

17
MONITOIRNG ACTIVITIES
Monitoring Exercice and review Meetings
 Annual joint planning
 joint monitoring
 Revising plan
 Reflection workshop to share good practices /Review meetings
 Reporting (Quarter reporting of outcomes and outputs from partners and Interim report to the
donor/government sector office/ ACSI)
1. HARNESSING PROJECT MANAGEMENT
DECISIONS

18
Topic Slide
#
MEAL PURPOSES
]
What is existing practice at partners? With respect to evidence-
based project management ,accountability and continual
improvement
Strengths Weaknesses
Activity monitoring
Budget monitoring
Measuring results /outcomes
Redesign-mid-course adjustment
Compliance on project agreement /timely reporting to donors and
partners
Adaptability of project activities /external environment is changing
Quantitative survey and triangulation to substantiate results
Accountability and engagement of beneficiaries and government
Empower community ownership

19
 Annual joint planning
 joint monitoring
 Revising plan
 Reflection workshop to share good practices
 Reporting (Quarter reporting of outcomes and outputs from partners and Interim report to the donor
1. FACILITATE MANAGEMENT DECESIONS

20
Topic Slide
#
Why we use qualitative tools such as observation , KII , II and FGDs?
Qualitative tools during measuring/ field Monitoirng
Participants Reflection
Exercise to entail the difference results of qualitative and quantitative undertakings

21
 Data collection using quantitative and qualitative indicators
 Use triangulation method for complementarity
 Exercise
What do you think , qualitative methods would bring to a data collection to understand how food security and
nutrition project attains its objective ?
What are the tools for data collection ? Quantitative versus qualitative

22
REPORTING
 Reporting on results (outcomes and immediate results)
 Reporting to inform the project management
 Compliance reporting (complaint response mechanisms ensured/accountability to the beneficiaries)
 Reporting of good practices for cross fertilization of knowledge
 Physical versus financial performance to learn about our efficiency

23
REPORTING
 Quarterly reporting by partner
 Reporting challenges that needs management decisions

24
REPORTING CHALLENGES
 What are the challenges in reporting ?
Discussion
Why not meeting deadlines ?
Why not reporting completed information?
Why not address all the results in the project logical framework
Aligning reportable data elements with government sector reporting data elements to avoid duplication of
effort (one plan one report)
Infact some data elements required to be reported might be the partner's interest

25
PILLARS OF CRM SYSTEM
Accountability and Quality of Programming
1. Information sharing
2. Sensitizing the beneficiaries and staffs about the CRM system
3. Confidentiality of receiving complaints and feedback
4. Responsive action –program improvement
2.FORWARD AND BACK WARD ACCOUNTABILITY

26
PROJECT DESIGN , IMPLEMENTAITON , MONITORING AND EVALAUTATION
 Participatory planning –integrating plan
 Integrating implementaiton
 Participatory targeting of beneficiaries
 Clear targeting criteria
 Intersectionality/Inclusiveness the most affected (PLW, aged people, vulnerable and women HH heads)
 Joint Monitoirng and evaluation

27
THE PARIS DECLARATION- five principles of smart aid
1. OWNERSHIP Developing countries set their own development strategies,
improve their institutions and tackle corruption.
2. ALIGNMENT Donor countries and organisations bring their support in line
with these strategies and use local systems.
3. HARMONISATION Donor countries and organisations co-ordinate their actions,
simplify procedures and share information to avoid duplication.
4. MANAGING FOR
RESULTS
Developing countries and donors focus on producing – and
measuring – results.
5. MUTUAL
ACCOUNTABILITY
Donors and developing countries are accountable for
development results.

28
2005 THE PARIS DECLARATION- five principles of smart aid
1. OWNERSHIP Developing countries set their own development strategies,
improve their institutions and tackle corruption.
2. ALIGNMENT Donor countries and organizations bring their support in line
with these strategies and use local systems.
3. HARMONISATION Donor countries and organizations co-ordinate their actions,
simplify procedures and share information to avoid duplication.
4. MANAGING FOR
RESULTS
Developing countries and donors focus on producing – and
measuring – results.
5. MUTUAL
ACCOUNTABILITY
Donors and developing countries are accountable for
development results.

29
 Exclusion and inclusion criteria
GROUP DISCUSSION ON THE EXISTING INCLUSION AND EXCLUSION CRITERIA FOR THE MULTIPLE BENEFITS AND SERVICES AT
OUR DISPOSAL FOR BENEFICIARIES
 What are inclusion criteria to target beneficiaries of a food security and nutrition program.
 What are the rationality to select beneficiaries?
 List criteria for selection criteria

30
Leesons learning
 Benchmark meeting
 Quarterly review
 Joint evaluation
 Mid-term/Terminal Evaluation
3. GROUP EXERCISE
What lessons we have learned from previous projects if any?
What do you incorporate in the new programming for ETH1171?
3. LESSONS LEARNING SO THAT WE CAN BUILD
UP TO IMPROVE OUR PROGRAMMING

31
Leesons learning
Key components of a TOR include the following:
• Background of the evaluation
• Brief description of the program
• Purpose of the evaluation
• Evaluation questions
• Evaluation methodology

32
Leesons learning
Contd…
• Evaluation team
• Schedule and logistics
• Reporting and dissemination plan
• Budget
• Timeline
• Ethical consideration

33
TYPES OF EVALUATION –REFERENCE SOURCE-
JICA
Ex ante
evaluation
Monitoring Mid-term
evaluation
Terminal Ex-post
evaluation
Confirmation of
performance
Note 1 ● ● ● ◌
Grasping of
implementaiton
on process
◊ ● ● ● ◌
Five Evaluation criteria
Relevance ● - ● ● ◊
Effectiveness ◌ - ◊ ● -
Efficiency ◌ - ● ● -
Impact ◌ - ◊ ◊ ●
Sustainability ◌ - ◌ ◌ ●

34
● Examination based on the actual situation and performance
◌ Examination based on forecast and prospects
◊ Examination what is judged necessary or possible for the evaluation
-Full examination is not yet possible or completed in the previous phase
Note 1-In case of ex-ante evaluation , this means conducting baseline study/setting baseline values or pre-
feasibility study
Note 2:- An evaluation based on the five evaluation criteria is beyond the scope of evaluation , but for its
operation and management it is important to keep in mind the five evaluation criteria viewpoints always in
mind.
TYPES OF EVALUATION

35
LEESONS LEARNING
Leesons learning
3. Group exercise
What lessons we have learned from previous projects if any?
What do you incorporate in the new programming for ETH1171?

36
Topic Slide
#
Theory of change /Logframe

37
Topic Slide
#
What is theory of change ?
What is logical framework?

38
Topic Slide
#
LOGIC OF PROJECT-THEORY OF CHANGE
Logic of project (theory of change) – pathways for achieving project objectives that considers risks
and assumptions
Logical framework- planning and designing tool to portray the project/program progress from
investment to achieving its intended objective-A four by four matrix
What risks are inevitable and what assumptions are holding true
Understanding output indicators /quantitative and qualitative aspect
If then logic –logical framework
Cause and effect relationship –result framework
Partners dependent( USAID and World Bank-Result framework versus European Union-Logical
framework

39
Topic Slide
#
Theory of change versus logframe
Reference source:- USAID OFFICE OF FOOD FOR PEACE
Theory of change Logfrmae
1. Broad :-Shows all domains and pathways needed
to reach a goal, including those a project will not
directly address
2. Non-linear and adaptive
3. Describes and supports with evidence
assumptions and rationales for linkages along
causal pathways
4. Helpful tool for understanding the “big picture”,
reflecting, learning, adapting
1. Specific: -Shows only specific outcomes a project
will directly address
2. Linear and structured
3. Lists assumptions , indicators and data sources
4. Primarily a tool to provide a framework for an
M&E systems that assures accountability

40
Topic Slide
#
RESULT FRAMEWORK VERSUS THEORY OF
CHANGE

41
Topic Slide
#
The overall objective of the action is:-
Strengthening civil society actors and local capacities in the field of food and
nutrition security in Afar, Ethiopia

42
Topic Slide
#
Log frame and MEAL plan are interrelated and the later is build up on the preceding
Objectively verifiable indicators
Means of verification/source of data
data collection tools ,
assumption and risks
frequency of data collection/reporting and
for whom to be reported/audience of report

43
Topic Slide
#
LOGFRAME- A FOUR BY FOUR MATRIX
Overall objective Objectively verifiable
indicators (OVIs)
Sources of verification
(MOVs)
Assumptions
Project purpose Objectively verifiable
indicators
Sources of verification Assumptions
Results (intermediate results
)
Objectively verifiable
indicators
Sources of verification Assumptions
Activities Means Costs Assumptions

44
Topic Slide
#
Indicator (Outcome/output)

45
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
What are indicators ?
Baseline Targets
Benchmarks/Milestones
Endline Targets
Indicator development

46
Topic Slide
#
Indicators –signals whether or not desired changes are made (or lack thereof )
Targets -are values of indicators

47
Topic Slide
#
How to set targets ?
-Current situation/baseline
-What trends occur before the project
-What do research findings or similar programs suggests
-Expert judgment
-Motivational and required effort /value added by project

48
Topic Slide
#
Indicators and Targets:
Indicators are often confused with targets.
Indicators tell us what we want to measure. They are units of measure only.
Targets have specific values attached usually a number and or a date and help us to track
progress.
For example:
Percentage of children under one year fully immunized by 2015 is an indicator.
To make this indicator measurable a target will be added for example 60 percent of
children under one year fully immunized by 2015.

49
Topic Slide
#
Defining indicators
Indicators are developed according to the Quantity, Quality, Time and Place (QQTP) protocol.
This meant that an indicator would be defined to be specific about:
• Quantity: 300 HHs
• Quality: 300 HHs trained in homestead agriculture and provision of homestead seeds
• Time: 300 HHs trained in homestead agriculture and provision of homestead seeds by the end of
2023
• Place: 300 HHs in selected kebeles of targeted kebeles of the two Woredas of Afar-Project woredas
will be trained in homestead agriculture and provision of homestead seeds by the end of 2023

50
Topic Slide
#
Defining indicators
Contd…
• Place: 300 HHs trained in homestead agriculture and provision of homestead seeds in
targeted kebeles of the two Woredas of Afar-Project woredas by the end of 2023

51
Topic Slide
#
Indicator development/formulation :-Criteria for selection of indicators
 Pertinent/relevant/appropriate
 Sensitive/ measurable
 Effective
 Technically valid
 Feasible to collect data /cost effective
 Verifiable

52
Topic Slide
#
What makes a good indicator?
AIMS (developed by NEW ECONOMICS FOUNDATION) criteria for indicators, which
continues to be a useful guide in deciding if an indicator is a good indicator:
A-Action focused. Indicators should lead to action. If stakeholders cannot imagine what to do with the
data from an indicator, then it probably isn’t a good indicator.
I-Important. Stakeholders should agree that the indicator and the data it will generate make a relevant
and significant contribution to determining how to effectively respond to the epidemic.
M-Measurable. Not only must the data collection methodology be defined, it must also be feasible to
collect the data.
S-Simple. it is much better to identify good, simple indicators that provide data that can be put to use

53
Topic Slide
#
Types of indicators
Input indicators – immunization policies, resource inputs (human, material, financial). These are
pre-requisite indicators for implementation.
Process indicators – this area examine functionality and quality of immunization system and
include all activities: planning, financing, quality of service delivery, immunization safety,
assessment of the programme and its efficiency, training, etc.
Output indicators – program’s immediate results, e.g. vaccination coverage and other results or
products contributing to the achievements of the programme objectives.

54
Topic Slide
#
Contd…
Outcome indicators– relates to the objectives of the programme, i.e., achievement of final goal
of polio eradication, neonatal tetanus elimination, etc.
Impact indicators – relates to the goal of the programme, i.e. reduction of morbidity and
mortality of targeted diseases

55
Topic Slide
#
Also classified as
 Aggregate versus Operational Indicators
 Direct versus indirect (proxy)indicators
 Qualitative versus quantitative indicators

56
Topic Slide
#
We substantiate quantitative indicators using qualitative ones
(Triangulation of data to improve the informativeness of the data )

57
Topic Slide
#
Question:- Exercise to entail the difference results of qualitative and quantitative
undertakings / substantiating quantitative indicators using qualitative ones
3. Group exercise
What are the pros of qualitative tools against quantitative tools?

58
Topic Slide
#
TARGETING-ACCOUNTABILITY TO
BENEFICIARIES
Targeting-accountability to
beneficiaries

59
Topic Slide
#
TARGETING ? BLANKET VERSUS
PRIORITIZATION
Targeting and accountability to beneficiaries
There is no blanket targeting always (resource is scarce)
Blanket targeting –livestock vaccination
 Criteria for inclusion and exclusion
5. Group exercise
What are the beneficiary targeting criteria for this particular project
(ETH1171)?
Target registration (Asset/ family size/family age band/PLW/benefit packages
eligible )?

60
Topic Slide
#
TARGETING
SOPs and/or Steps in targeting process?
Step 1: Deciding whether targeting appropriate /community engagement (Does the
resource allows blanket targeting? )
Step 2: Selecting the targeting methodology (community based targeting/community
taskforce)
Step 3: Defining eligibility criteria

61
Topic Slide
#
TARGETING
Contd…
Step 4: Validating the targeting method and criteria before start of service provision
Step 5: Communication, appeals and feedback systems ( accountability and
transparency)
Step 6: Identification of eligible individuals or households-maintain beneficiary
master list- digital
Step 7: Monitoring program implementation (to learn changing situations under the
possible dynamic environment )

62
Topic Slide
#
TARGETING ? BLANKET VERSUS
PRIORITIZATION
What targeting strategies are better ?
 Self targeting- When there is an opportunity cost (conditional
support/labour-based works)
 Community based
 Proxy means test, categorical?

63
Topic Slide
#
ETH1171 MEAL PLAN DEVELOPMENT EXERCISE?
ETH1171 MEAL Plan development exercise?

64
Topic Slide
#
ETH1171 MEAL PLAN DEVELOPMENT EXERCISE?
5 Group exercise MEAL plan exercise and presentation

65
Topic Slide
#
 DATA MANAGEMENT , DATA QUALITY AND
BENEFICIARY COUNTING
Data management , Data quality and beneficiary
counting

66
Topic Slide
#
REPORTING FRAMEWORK
REPORTING FRAMEWORK
Frequency of reporting , reporting format and quality , audience of reports
Weekly/Monthly/Quarterly/Annually
Is it whole programme reporting versus project-based approach

67
Topic Slide
#
MULTI-STAKEHOLDER PARTNERSHIP
Audience of reports
Multi-stakeholder partnership (WHH and BMZ roles and responsibilities
Local partners (APDA,AISD & AFSA) roles and responsibilities
Compliance to partnership agreement about reporting requirements
Forward and backward accountability through communicating to community
,government and partners

68
Topic Slide
#
Reporting quality reports
Reporting in line with the result framework/logframework
Reporting disaggregated as well as aggregated data tailored to needs
Reporting on performance (complete information on physical activities and
budget)
Annual and long-term reports focus on outcomes (project objectives ) while
weekly and/or monthly updates can focus on activity level
If the reporting is annual, it should address outcome level indicators
Visualizing data to increase buy-in in your reports by the audience

69
Topic Slide
#
PARTICIPANTS REFLECTION
What are the challenges to have a complete report, incorporate budget section
to a report?
Data quality dimensions
Timeliness- currency of information to be used for decision making
Validity /Accuracy- does the indicator measure what it intended to report
Reliability – consistence of tool across time and geography so that it can serve
for comparison
Complete/Detail-Level of disaggregation
Integrity-Free from manipulation due to self interest

70
Topic Slide
#
Contd…
Validity/Accuracy
Beneficiary counting
What is beneficiary counting ?
What are target beneficiaries ?
What are direct beneficiaries
What are indirect beneficiaries
For whom the project is accountable ?

71
Topic Slide
#
• Direct beneficiaries, receiving direct humanitarian assistance, individually
(medical consultation), or at household level (food, water, shelter …)
• Indirect beneficiaries, benefitting indirectly from the activity (radio broadcast,
rehabilitation of health center). Note: some activities only have indirect
beneficiaries: repairing a road, or training civil officers benefit the whole
community indirectly, but no particular group directly.

72
Topic Slide
#
People reached: The number of people targeted who have benefitted from one
or several humanitarian activities at least once during the reporting period.
Activities may include goods, services, protection, cash-based assistance, etc.
People covered: The number of people targeted who have received the
planned assistance and whose targeted needs have been fully met, in terms of
type, quantity, quality and/or periodicity.
‘People reached’ is an inclusive measure, as it includes all people reached by
any activity; whereas ‘people covered’ is an exclusive measure, as it excludes
all people unless they are fully covered by all activities (which targeted them).

73
Topic Slide
#
COMPLAINT RESPONSE MECHANISM-CRM
Complaint response mechanism and accountability (Code of conduct)
 Community participation
 Information sharing
 Responsiveness to complaints
CFRM

74
Topic Slide
#
MEASURING SUCCESS AT WHH
Measuring success at WHH

75
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS _#MS
WHH zero hunger across the globe – how can we measure
this gaol?
WHH ADHERE TO EIGHT INDICATORS
# Topic/Sector Indicator
Please note: The square brackets […] indicate alternatives
#ms1 Food Access (MAHFP) Average months of food sufficiency (of targeted households) in the previous twelve months
#ms2 Nutrition Diversity (MDD-W) Number [or percentage] of targeted women aged 15–49 consuming a sufficiently diverse diet
#ms3 Drinking Water Number [or percentage] of households using safely managed drinking water services
#ms4 Sanitation Number [or percentage] of households using safely managed sanitation services

76
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS 02_#MS.HANDBOOK.2019_EN.PDF
# Topic/Sector Indicator
Please note: The square brackets […] indicate alternatives
#ms4 Sanitation Number [or percentage] of households using safely managed sanitation services
#ms5 Income Number of households increasing their cash income
#ms6 Women in Decision-making Number [or percentage] of adult women being able to influence decision-making
#ms7 Skills Number of trainees successfully completing skills-development training
#ms7 Project participants
satisfaction
The percentage of participants satisfaction with the services aNd benefits they have received
from WHH

77
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS
WHH zero hunger across the globe – how can we measure
this gaol?

78
MINIMUM DIET DIVERSITY- WOMEN

79
MINIMUM ACCEPTABLE DIET-CHILDREN

80
NUMBER [OR PERCENTAGE] OF ADULT WOMEN BEING
ABLE TO INFLUENCE DECISION-MAKING

Photo:
Haddad
Toni
/
Welthungerhilfe
Abraham Lebeza
MEAL
Abraham.Lebeza@welhungerhilfe.de
+251910818934

MEAL ETH11171-draft.pptx

Recommended

Recommended

More Related Content

Similar to MEAL ETH11171-draft.pptx

Similar to MEAL ETH11171-draft.pptx (20)

More from Abraham Lebeza

More from Abraham Lebeza (20)

Recently uploaded

Recently uploaded (20)

MEAL ETH11171-draft.pptx