Monitoring, Evaluation, Accountability and Learning

1. Photo:
Welthungerhilfe
MEAL WORKSHOP ON ETH1224
Discussion on MEAL practice on ETH1224
Dcember 6,2022
Abraham Lebeza

2. 2
# Topic Timing
MEAL WORKSHOP ON ETH1161
MEAL Training Outline
 General MEAL Concepts
 Theory of change /Logframe
 Indicator (Outcome/output)
 Measuring success at WHH
 ETH1224 MEAL plan development -
Exercise
 Data management , Data quality and
beneficiary counting
 Targeting (accountability to beneficiaries)
 CFRM and WHH code of conduct

3. 3
# Topic Timing
MEAL WORKSHOP ON ETH1161
General MEAL Concepts

4. 4
# Topic Timing
MEAL WORKSHOP ON ETH1161
What are these buzz words ?
Monitoirng ?
Evaluation?

5. 5
# Topic Timing
MEAL WORKSHOP ON ETH1161
Are monitoring and evaluation similar or identical to each other?
How is monitoring and evaluation are reinforcing to each other?
1. Group exercise
 to differentiate monitoring and evaluation?
 To show how Monitoirng and evaluation are reinforcing to each other?

6. 6
# Topic Timing
MEAL WORKSHOP ON ETH1224
Characteristics Monitoring Evaluation
Time frame Continuous /regular Periodic
Purpose Plan versus achievement,
track records
Judgmental of effectiveness
, what works well and what
doesn’t
Big question Are we doing what we are
expected to do
Are we doing the right things
Staffing Internal Internal and external
Rigor Enough to drive continuous
improvement
Systematic design that
employs qualitative and
quantitative study

7. 7
# Topic Timing
MEAL WORKSHOP ON ETH1224
The six main components of a project M&E system
1. Clear statements of measurable objectives for the project and its components.
2. A structured set of indicators covering: inputs, process, outputs, outcomes, impact,
and exogenous factors.
3. Data collection mechanisms capable of monitoring progress over time, including
baselines and a means to compare progress and achievements against targets.

8. 8
# Topic Timing
MEAL WORKSHOP ON ETH1224
Contd..
4. Where applicable building on baselines and data collection with an evaluation
framework and methodology capable of establishing causation (ie capable of attributing
observed change to given interventions or other factors).
5. Clear mechanisms for reporting and use of M&E results in decision-making.
6. Sustainable organisational arrangements for data collection, management, analysis,
and reporting.

9. 9
Topic Slide
#
MEAL PURPOSES
2Group exercise
What is existing practice at HUNDEE in implementing the M&E system ?
What do the strategies HUNDEE employed to translate accountability to project
beneficiaries?
Bottlenecks –what should we overcome in the future

10. 10
Topic Slide
#
In their book on Reinventing Government (1992),
Osborne and Gaebler note that:
- What gets measured gets done.
- If you don’t measure results, you can’t tell success from failure.
- If you can’t see success, you can’t reward it.
- If you don’t recognize failure, you can’t correct it.

11. 11
Topic Slide
#
Failing to plan is Planning to fail

12. 12
Topic Slide
#
MEAL PURPOSES
1. Harnessing project management decisions through tracking project
progress/performance against budget and physical plan
1.1. Measurement –Result Based Management
2. Forward and backward accountability
3. Lessons learning so that we can build up to improve our programming

13. 13
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Discussions
 What good practices Hundee would replicate to the coming project
Hundee MEAL performance and constraining challenges
 What Hundee expect from WHH for further strengthening MEAL
 What was Hundee experience in M&E in general and in implementing ETH1205 in
particular
 MEAL practices Challenges and good practices

14. 14
MONITOIRNG ACTIVITIES
Monitoring Exercice and review Meetings
 Annual joint planning
 joint monitoring
 Revising plan
 Reflection workshop to share good practices /Review meetings
 Reporting (Quarter reporting of outcomes and outputs from partners and Interim report to the
donor/government sector office/ ACSI)
1. HARNESSING PROJECT MANAGEMENT
DECISIONS

15. 15
Topic Slide
#
MEAL PURPOSES
]
What is existing practice at Hundee? With respect to evidence-based
project management ,accountability and continual improvement
Strengths Weaknesses
Activity monitoring
Budget monitoring
Measuring results /outcomes
Redesign-mid-course adjustment
Compliance on project agreement /timely reporting to donors and
partners
Adaptability of project activities /external environment is changing
Quantitative survey and triangulation to substantiate results
Accountability and engagement of beneficiaries and government
Empower community ownership
Scale up good practices /Continual improvement

16. 16
MONITOIRNG ACTIVITIES
Monitoring Exercice and review Meetings
 Annual joint planning
 joint monitoring
 Revising plan
 Reflection workshop to share good practices
 Reporting (Quarter reporting of outcomes and outputs from partners and Interim report to the donor
2. FACILITATE MANAGEMENT DECESIONS

17. 17
MONITOIRNG ACTIVITIES
Monitoring Exercice and review Meetings
 Data collection using quantitative and qualitative indicators
 Use triangulation method for complementarity
 Exercise
What do you think , qualitative methods would bring to a data collection to understand how food security and
nutrition project attains its objective ?
What are the tools for data collection ? Quantitative versus qualitative
2. FACILITATE MANAGEMENT DECESIONS

18. 18
REPORTING
Monitoring Exercice and review Meetings
 Reporting on results (outcomes and immediate results)
 Reporting to inform the project management
 Compliance reporting (complaint response mechanisms ensured/accountability to the beneficiaries)
 Reporting of good practices for cross fertilization of knowledge
 Physical versus financial performance to learn about our efficiency
2. FACILITATE MANAGEMENT DECESIONS

19. 19
REPORTING
Monitoring Exercice and review Meetings
 Quarterly reporting by partner
 Reporting challenges that needs management decisions
2. FACILITATE MANAGEMENT DECESIONS

20. 20
REPORTING CHALLENGES
Monitoring Exercice and review Meetings
 What are the challenges in reporting ?
Discussion
Why not meeting deadlines ?
Why not reporting completed information?
Why not address all the results in the project logical framework
Aligning reportable data elements with government sector reporting data elements to avoid duplication of
effort (one plan one report)
Infact some data elements required to be reported might be the partner's interest
2. FACILITATE MANAGEMENT DECESIONS

21. 21
PILLARS OF CRM SYSTEM
Accountability and Quality of Programming
1. Information sharing
2. Sensitizing the beneficiaries and staffs about the CRM system
3. Confidentiality of receiving complaints and feedback
4. Responsive action –program improvement
2.FORWARD AND BACK WARD ACCOUNTABILITY

22. 22
PROJECT DESIGN , IMPLEMENTAITON , MONITORING AND EVALAUTATION
Accountability and Quality of Programming
 Participatory planning –integrating plan
 Integrating implementaiton
 Participatory targeting of beneficiaries
 Clear targeting criteria
 Intersectionality/Inclusiveness the most affected (PLW, aged people, vulnerable and women HH heads)
 Joint Monitoirng and evaluation
2.FORWARD AND BACK WARD ACCOUNTABILITY

23. 23
PROJECT DESIGN , IMPLEMENTAITON , MONITORING AND EVALAUTATION
Accountability and Quality of Programming
 Exclusion and inclusion criteria
 Exercise
 What are inclusion criteria to target beneficiaries of a food security and nutrition program. What are the
rationality to select beneficiaries? List criteria for selection criteria
2.FORWARD AND BACK WARD ACCOUNTABILITY

24. 24
LEESONS LEARNING
Leesons learning
 Benchmark meeting
 Quarterly review
 Joint evaluation
3. Group exercise
What lessons we have learned from ETH1205?
3. LESSONS LEARNING SO THAT WE CAN BUILD
UP TO IMPROVE OUR PROGRAMMING

25. 25
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Theory of change /Logframe

26. 26
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
What is theory of change ?
What is logical framework?

27. 27
Topic Slide
#
LOGIC OF PROJECT-THEORY OF CHANGE
Logic of project (theory of change) – pathways for achieving project objectives that
considers risks and assumptions
Logical framework- planning and designing tool to portray the project/program
progress from investment to achieving its intended objective-A four by four matrix
What risks are inevitable and what assumptions are holding
Understanding output indicators /quantitative and qualitative aspect

28. 28
Topic Slide
#
LOGIC OF PROJECT-THEORY OF CHANGE
The overall objective of the action is
to improve the food and nutrition security of coffee
farming communities in Manna, Seka Chekorsa,
Gomma, Shabe Sombo and Gera districts, in Jimma
Zone, Oromia region.

29. 29
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Log frame and MEAL plan are interrelated and the later is build up on the preceding
Objectively verifiable indicators, Means of verification/source of data ,
assumption and risks
data collection tools , frequency of data collection/reporting and for whom to be
reported)
Qualitative tools during measuring/ field Monitoirng
Exercise to entail the difference results of qualitative and quantitative undertakings
Participants can exercise
Why we use qualitative tools such as observation , KII , II and FGDs?

30. 30
Topic Slide
#
LOGFRAM- A FOUR BY FOUR MATRIX
Overall objective Objectively verifiable
indicators (OVIs)
Sources of verification
(MOVs)
Assumptions
Project purpose Objectively verifiable
indicators
Sources of verification Assumptions
Results (intermediate results
)
Objectively verifiable
indicators
Sources of verification Assumptions
Activities Means Costs Assumptions

31. 31
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Indicator (Outcome/output)

32. 32
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
What are indicators ?
Baseline Targets
Milestones
Endline Targets
Indicator development
Direct versus indirect (proxy)targets
Qualitative versus quantitative INDICATORS

33. 33
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Indicator development/formulation :-Criteria for selection of indicators
 Pertinent (relevant)
 Sensitive
 Effective
 Technically valid
 Feasible to collect data
 Verifiable

34. 34
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Types of indicators
Input indicators – immunisation policies, resource inputs (human, material, financial). These are
pre-requisite indicators for implementation.
Process indicators – this area examine functionality and quality of immunization system and
include all activities: planning, financing, quality of service delivery, immunisation safety,
assessment of the programme and its efficiency, training, etc.
Output indicators – program’s immediate results, e.g. vaccination coverage and other results or
products contributing to the achievements of the programme objectives.

35. 35
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Types of indicators
Outcome indicators– relates to the objectives of the programme, i.e., achievement of final goal
of polio eradication, neonatal tetanus elimination, etc.
Impact indicators – relates to the goal of the programme, i.e. reduction of morbidity and
mortality of targeted diseases

36. 36
Topic Slide
#
MEAL COMPONENTS OBJECTIVE
Question:- Exercise to entail the difference results of qualitative and quantitative
undertakings
4. Group exercise
Why we use qualitative tools such as observation , KII , II and FGDs?
What are the pros of qualitative tools against quantitative tools?

37. 37
Topic Slide
#
TARGETING-ACCOUNTABILITY TO
BENEFICIARIES
Targeting-accountability to
beneficiaries

38. 38
Topic Slide
#
TARGETING ? BLANKET VERSUS
PRIORITIZATION
Targeting and accountability to beneficiaries
There is no blanket targeting always (resource is scarce)
Blanket targeting –livestock vaccination
 Criteria for inclusion and exclusion
5. Group exercise
What are the beneficiary targeting criteria for this particular project
(ETH1224)?
Target registration (Asset/ family size/family age band/PLW/benefit packages
eligible )?

39. 39
Topic Slide
#
TARGETING ? BLANKET VERSUS
PRIORITIZATION
What targeting strategies are better ?
 Self/
 Community based
 Proxy means test, categorical?
 Beneficiary master list- digital

40. 40
Topic Slide
#
TARGETING
SOPs and/or Steps in targeting process?
Step 1: Deciding whether targeting appropriate /blanket programming
Step 2: Selecting the targeting methodology
Step 3: Defining eligibility criteria
Step 4: Validating the targeting method and criteria
Step 5: Communication, appeals and feedback systems
Step 6: Identification of eligible individuals or households
Step 7: Monitoring

41. 41
Topic Slide
#
ETH1224 MEAL PLAN DEVELOPMENT EXERCISE?
ETH1224 MEAL Plan
development exercise?

42. 42
Topic Slide
#
ETH1224 MEAL PLAN DEVELOPMENT EXERCISE?
6 Group exercise MEAL plan exercise
and presentation

43. 43
Topic Slide
#
 DATA MANAGEMENT , DATA QUALITY AND
BENEFICIARY COUNTING
Data management ,
Data quality and
beneficiary counting

44. 44
Topic Slide
#
REPORTING FRAMEWORK
REPORTING FRAMEWORK
Frequency of reporting
Monthly update /Quarterly and annual reporting
Reporting on performance (physical activities and budget)
Reporting on outcomes (project objectives )
Reporting should focus on outcome level not only activity level
If the reporting is annual, it should address outcome level indicators

45. 45
Topic Slide
#
MULTI-STAKEHOLDER PARTNERSHIP
Multi-stakeholder partnership
WHH roles and responsibilities as partner
GIZ roles and responsibilities
Hundee roles and responsibilities
Compliance to partnership agreement
Forward and backward accountability
Reporting to partners
Monthly update, quarterly and annual reporting (Project based approach )

46. 46
Topic Slide
#
MULTI-STAKEHOLDER PARTNERSHIP
Reporting both physical and budget performance.
What are the challenges to incorporate budget section to a report?

47. 47
Topic Slide
#
COMPLAINT RESPONSE MECHANISM-CRM
Complaint response mechanism and accountability (Code of conduct)
 Community participation
 Information sharing
 Responsiveness to complaints
CFRM

48. 48
Topic Slide
#
MEASURING SUCCESS AT WHH
Measuring success at WHH

49. 49
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS _#MS
WHH zero hunger across the globe – how can we measure
this gaol?
WHH ADHERE TO EIGHT INDICATORS
# Topic/Sector Indicator
Please note: The square brackets […] indicate alternatives
#ms1 Food Access (MAHFP) Average months of food sufficiency (of targeted households) in the previous twelve months
#ms2 Nutrition Diversity (MDD-W) Number [or percentage] of targeted women aged 15–49 consuming a sufficiently diverse diet
#ms3 Drinking Water Number [or percentage] of households using safely managed drinking water services
#ms4 Sanitation Number [or percentage] of households using safely managed sanitation services

50. 50
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS 02_#MS.HANDBOOK.2019_EN.PDF
WHH ADHERE TO EIGHT INDICATORS
# Topic/Sector Indicator
Please note: The square brackets […] indicate alternatives
#ms4 Sanitation Number [or percentage] of households using safely managed sanitation services
#ms5 Income Number of households increasing their cash income
#ms6 Women in Decision-making Number [or percentage] of adult women being able to influence decision-making
#ms7 Skills Number of trainees successfully completing skills-development training

51. 51
THERE ARE EIGHT SUCCESS MEASUREMENT INDICATORS
WHH zero hunger across the globe – how can we measure
this gaol?
WHH ADHERE TO EIGHT INDICATORS
WHH ADHERE TO EIGHT INDICATORS

52. 52
MINIMUM DIET DIVERSITY- WOMEN

53. 53
MINIMUM ACCEPTABLE DIET-CHILDREN

54. 54
NUMBER [OR PERCENTAGE] OF ADULT WOMEN BEING
ABLE TO INFLUENCE DECISION-MAKING

55. Photo:
Haddad
Toni
/
Welthungerhilfe
Abraham Lebeza
MEAL
Abraham.Lebeza@welhungerhilfe.de
+251910818934