Presented by Birhan Abdulkadir, ILRI, at the Training of Trainers on Multi-Stakeholder Platform Facilitation, Gender and Data Management, ILRI, Addis Ababa, 20-21 November 2019
Africa RISING Monitoring and Evaluation Data Requirements
1. Africa RISING Monitoring and Evaluation Data
Requirements
Birhan Abdulkadir, ILRI
Training of Trainers on Multi-Stakeholder
Platform Facilitation, Gender and Data
Management, ILRI, Addis Ababa, 20-21
November 2019
2. What Changes Do We Expect to Monitor?
If you do not know where you are going, any destination is acceptable.
3. Monitoring and Evaluation
Monitoring:
A continuous function that uses the
systematic collection of data on
specified indicators to provide
management and the main stakeholders
of an ongoing development intervention
Routinely collects data on indicators,
compares actual results with targets
Document learning's generated through
monitoring
Evaluation:
The systematic and objective assessment
of an ongoing or completed project,
including its design, implementation,
and results.
It determine the relevance and
fulfillment of objectives, development
efficiency, effectiveness, impact, and
sustainability.
Assesses specific causal contributions of
activities to results
What Why How When Who
4. Where Monitoring happens
Inputs
Activities
Outputs
Outcomes
Impacts
Are finance, personnel and materials available on time
and in the right quantities and quality?
Are activities being implemented on schedule and
within budget?
Are activities leading to the expected outputs?
Are outputs leading to achievement
of the outcomes?
Measuring changes at higher level outcomes and impact level requires a longer time
frame and is dealt with by evaluation and not monitoring.
Monitoring
5. Where Evaluation happens
Inputs
Activities
Outputs
Outcomes
Impacts
Efficiency
• Were activities implemented on schedule and within budget?
• Were outputs delivered economically?
Effectiveness
• Were the projects objectives achieved?
• Did the outputs lead to the intended outcomes?
Relevance:
• Were the projects objectives consistent with beneficiaries’ needs?
Impact: • What changes did the project, bring about?
• Were there any unplanned, or unintended changes
Sustainability: • Are the benefits likely to be maintained for an extended period after project ends?
6. Two M&E Systems
Implementation-focused M&E
Designed to address compliance—
following questions
“Did they do it”
Did they mobilize the needed inputs?
Did they undertake and complete the
agreed activities?
Did they deliver the intended outputs
Results-based M&E
Designed to provide feedback on the
actual outcomes and goals of project
actions. And addresses the following
questions:
“So what” question
So what about the fact that outputs have
been generated?
So what that activities have taken place?
So what that the outputs from these
activities have been counted?
What Why How When Who
8. Vision of Success
Region Directly engaged
households
Potential
beneficiaries in ZoI
Amhara 215K 2.37 mln
Oromia 189K 2,.38 mln
SNNPR 169K 1.5 mln
Tigray 154K 1.15 mln
Total 700K 7.34 mln
Note: these figures assume the adoption of one innovation per
household. At implementation, there is likely to be duplication
with some households adopting more than one innovation.
9. Type of Organization
# of AR Local Partner
Organizations
BoA 37
Cooperative/ Union 8
Enterprise 1
NGO 5
Plc 1
Research Center 6
Research Institute 5
University 4
Grand Total 67
10. Our Beneficiary
Direct beneficiaries:
• Households directly testing AR SI innovations under the supervision of AR researchers through
mother-baby trials or other participatory approaches .
Indirect beneficiaries:
• Farmers exposed to AR innovations, without directly engaging in the testing of SI innovations
under the supervision of AR researchers.
Scaling beneficiaries
• AR is expected to reach hundreds of thousands of households working with development
partners. During this phase, basic information about the SI innovations(s) that are being
scaled up, the development partners assisting with the scaling up efforts, as well as the total
number of households benefiting from scaling activities shall be collected.
11. Minimum monitoring data requirement
Data for monitoring research activities:
• detailed geographic and household identifiers,
• household-specific innovations being tested,
• dates when households joined the program,
• Partner organizations involved.
Data for monitoring scaling activities:
• basic information about innovations being scaled up
• development partners in charge of the scaling,
• the total number of households that benefited from scaling-up activities (disaggregated by the type of
AR innovation, gender of beneficiaries (e.g., gender of the household head or individual beneficiary as
applicable),
• reporting period.
13. AR M&E Data Requirements
Data type Tools Timing of data collection Responsibility
Direct beneficiaries &
technologies
Technology cataloging tool Before and after growing
season
AR researchers
Direct beneficiaries &
technologies
Beneficiary and technology
tracking tool (BTTT)
After each growing season AR researchers
Indirect beneficiaries and
technologies
Exposure tool (trainings, field
days, visits, etc.)
After every incidence of the
exposure
AR researchers
Beneficiaries of scaling up/out Scaling tool Quarterly AR researchers,
development partners
Agronomic Various tools As necessary
Socio-economic Various tools As necessary
FtF Indicators Project mapping and
monitoring tool (PMMT)
Once a year AR researchers
Scaling-up process evaluation Yearly
15. “So what?”
Managing for results
Implement,
(Innovation
platforms), and
strategies to do so
Measure
performance
Learn,
document
and report
We Plan
LEARNINGKey question: To What extent are we going to use the feedback from
2017-2019 data for 2020 planning and implementations?
16. Contents of Africa RISING
Data Collection Tools
Africa RISING_Data Collection Tools_2019_Final_2
Name of the Sheet Name of the tool
Site
Coordinators
Scaling
Partners/ Focal
Persons Frequency of Data Collection
T1_AR Techs-Practices_Approach Lists of Africa RISING Technologies, Practices & Approaches in 2019 Cropping Season X After each growing season or as necessary
T2_DegreeTrainings Africa RISING Supported Degree Seeking Trainees (new & continuing) in 2019 Cropping Season Twice in a year
T3_non-degree_Trainings Africa RISING Supported TRAININGS in 2019 Cropping Season X X After incidence of the training
T4_Events Africa RISING/ Partners Supported EVENTS in 2019 Cropping Season X X After incidence of the event
T5_AfRISING_#HHs Lists of Household Beneficiaries Directly Reached by Africa RISING SCALING and R4D in 2019 Cropping SeasonX Twice (Before & after each growing season or as necessary)
T6_Partners_Scaling_#HHs Number of Household Beneficiaries Reached by SCALING Partners in 2019 Cropping Season X X Quarterly/ bi-annually
AR_FtF_Indicators_ReportingT FtF Indicators for Africa RISING Reporting Template Once a year
AR-2019-FtF Indicators Definitions Definitions of FtF Indicators for Africa RISING in 2019 Cropping Season
Form Filled by
17. Group Work 1: Hands on Data Collection Tools
Discuss on the contents of the templates
Tool To capture
T6_Partners_Scaling_#HHs
Number of Household Beneficiaries Reached by SCALING Partners in
2019 Cropping Season
T3_non-degree_Trainings Africa RISING Supported TRAININGS in 2019 Cropping Season
T4_Events Africa RISING/ Partners Supported EVENTS in 2019 Cropping Season
(and if possible, fill the information at hand)
18. Group Work 2: Partners’ M&E Data Collection
Strategy
How will your Organization integrate (collect, compile & share data, etc)
the Africa RISING M&E needs?
a) Refer to your organization data collection processes
b) Data collection tools: which data tools do you have? Are there
common data in both tools?
c) Are there additional data required by Africa RISING? How will you
integrate additional data for Africa RISING?
19. Wachemo University Mekelle University Madda Walabu University Debre Birhan University Hawassa University
Amhara Region Agricultural Research
Institute (ARARI)
South Agricultural Research
Institute (SARI)
Tigray Agricultural Research
Institute (TARI)
Oromia Agricultural Research
Institute (OARI)
Ethiopian Institute of Agricultural
Research (EIAR)
Fuji integrated Farm
Hundie
REST-GRAD Sunarma SOS Sahel Ethiopia
Ethiopian Agricultural Transformation
Agency (ATA)
Offices of Agriculture: Endamekoni (Tigray) Basona Worena (Amhara) Lemo (SNNRP) Sinana (Oromia)
Innovation laboratories: SIIL ILSSI PHIL LSIL
Africa RISING
Local Partners
(Phase I)
21. Africa Research in Sustainable Intensification for the Next Generation
africa-rising.net
This presentation is licensed for use under the Creative Commons Attribution 4.0 International Licence.
Editor's Notes
If you do not know where you are going, any destination is acceptable
Clear Theory of Change and Impact Pathway
Results Framework (what specifically will you look out for as results?)
Objectives
Indicators (at various levels of outputs, outcomes, impacts)
Evidences (quantitative and qualitative data against set targets/planned results)
Activities
Inputs
1 Farmers ‘adopt’ if they gain access to, test, modify and use an innovation.
2 Benefits are linked to the five domains of the SI framework.
3 Households that are aware of a certain innovation and able to act on such awareness.
BTTT
The unit of data collection for indirect beneficiaries can be either an individual or a household, as appropriate. In the case of a field day, for example, it will be logistically difficult to try to map field day attendees to a specific household (since different household members could attend the field day) and so data on field day attendees shall be collected at the farmer level.
These data are especially important for Africa RISING, given the diversity of innovations and farming systems as well as the number of partners involved.
gender of beneficiaries (e.g., gender of the household head or individual beneficiary as applicable)
Distance between farm and homestead
NO BIG DATA without SMALL DATA.
73.7%
So what? What the 3 can tell us? Are HHs benefitted from the technology, simply counted, their lives changed?