value chain development

1. April 19, 2018
Introduction to the revised Feed the
Future Monitoring, Evaluation, and
Learning (MEL) system:
More than just standard indicators!
Feed the Future MEL webinar series
1

2. Presenters
2
Jessica Cagley
Zachary Baquet
Tatiana Pulido
Anne Swindale
Julie MacCartee

3. Feed the Future MEL Webinar Series
• Intro to the MEL System – April 19, 2018
• Standard Indicator Overview – May 15, 2018
• New Indicators 1: Application of improved practices and
technologies
• New Indicators 2: Yield and geospatial
• New Indicators 3: Sales and investment indicators
• Learning Agenda
• Market Systems Measurement
• Annual FTFMS users webinar
3

4. Survey responses
4

5. Objectives (and outline)
• For participants to understand:
– The purpose of the Feed the Future Monitoring,
Evaluation, and Learning (MEL) system
– Major revisions made to reflect lessons learned
– The components of the MEL system
– The importance of logic models in addressing multiple
MEL challenges
5

6. The Purpose of the MEL System
6

7. The Global Food Security Act’s Call for
Monitoring, Evaluation, and Learning (MEL)
• The President shall seek to ensure that assistance to
implement the Global Food Security Strategy is provided
under established parameters for a rigorous
accountability system to monitor and evaluate
progress and impact of the strategy, including by
reporting to the appropriate congressional committees
and the public on an annual basis. (Sec. 6.c)
• The President shall submit … reports that describe the
status of the implementation of the Global Food Security
Strategy …, which shall… indicate how findings from
monitoring and evaluation were incorporated into
program design and budget decisions; (Sec. 8.a.6)
7

8. Purposes of the MEL System
• Accountability
• Learning
• Strengthening national data systems
8

9. Vision of the MEL System
Our vision is that…
Quality evidence is used appropriately to
1. Design, manage and adapt programs to achieve
Feed the Future goals and objectives,
2. Inform initiative strategic and budget decisions, and
3. Communicate that what we do matters.
9

10. MEL System Stakeholders
Advocacy Orgs
Partner
governments
Academia
Private
sector
Multilateral
organizations 10

11. Lessons Learned from Feed the
Future, Phase 1
11

12. Revisions to the MEL System Based on
Lessons Learned
1. Better capture of systems-
level work
2. Linking interventions to
higher level impacts
3. Improving the use of
evidence generated through
MEL system
4. Leveraging national data
systems to measure
population-level results
12

13. Components of the MEL System
13

14. MEL System Components
Analysis &
Learning
Results
Framework
Monitoring
Evaluation
Learning
Agenda
National data
systems
14

15. Component 1. Results framework
15

16. Component 1. Results framework (p.2)
16

17. Component 2. Monitoring
• Standard indicators: New and
improved Indicator Handbook!
• Types of standard indicators:
– Performance
– Context New!
• Sources of indicator data:
– Zone of Influence (ZOI) surveys:
Target countries only, population-
based, every 3 years, one
survey per country to cover all
USG contributions
– Implementing partner reporting:
Annual, all countries, all
activities, all agencies New!
– Other: Remotely sensed data,
national surveys, global sources
etc.
54 Performance
Indicators
•27 Implementing
mechanism
•20 ZOI
•6 National
25 Context
Indicators
•5 ZOI
•17 National
•2 Recurrent crisis
area/national
•1 Global
17

18. Component 2. Monitoring (other
changes)
• Importance of custom indicators and
disaggregates… standard indicators are not
enough!
• Monitoring market system changes through:
– Clear and detailed logic model
– Standard indicators
– Custom indicators
– Qualitative measures
18

19. Component 3. Evaluations
• Performance evaluations
• Impact evaluations
– Rigorous counterfactual
– Examines attribution
• Lessons learned
– Increase use of multi-
activity, high-level
evaluations
– Use balance of evaluation
methods and types
19

20. Component 4. Learning Agenda
20
Source: USAID/PPL

21. Component 4. Learning Agenda
21
Source: USAID/PPL
• Revised learning agenda will:
– Use a variety of learning activities (e.g., research,
evaluations, monitoring, assessments) to address
questions
– Be co-owned by technical experts
– Have more frequent learning products to synthesize
and disseminate evidence

22. Component 5. National data systems
22
• Piloting use of the World Bank
LSMS-ISA for Feed the Future
surveys
• Support to the Global Rural
and Agricultural Integrated
Surveys (GRAInS) Partnership
to harmonize national data
system initiatives
• Supporting pilots of the farm-
based Agricultural Integrated
Survey (AGRISurvey)
Strengthening
national data
systems

23. Component 6. Analysis and Learning
23
• Dynamic, practical
revised Learning
Agenda
• Increased analysis of
existing data
• Innovative evaluations
and analysis methods
• Testing technologies
Improving
use of
M&E data

24. Logic Models: A Crucial Tool
24

25. What is a logic model?
• Visual representation of a theory of change,
detailing the expected causal pathways linking
our activities to our ultimate objectives and goals
• Used to:
– Design programs, projects, and activities
– Track whether results and assumptions are
happening as expected
– Adaptively manage based on data
– Tell your story!
25

26. Why are high-quality, detailed, and
dynamic logic models needed?
• To clearly articulate the hypotheses and
assumptions around what is expected to happen
step-by-step
• To identify the appropriate data sources (standard
and custom indicators + qualitative data) needed to
manage programs and tell your story
• To set ourselves up to have the data needed to tell a
plausible story (if it exists!) connecting USG
activities to changes observed at the population or
systems level
• We are not building widgets!
26

27. Logic model example
Tracked through
custom indicators or
qualitative methods
Tracked through
standard indicator
Key:
Increased
government
revenue from
targeted
crops
Increased
employment
in the ag
system
Targeted
firms
increase
employment
Increased
organizational
performance of
targeted firms
Business
development
training offered to
export crop
processing firms
Increased
sales for
targeted
firms
Increased
purchases/
investment of
crop from
smallholders
Procuers adopt
key practices and
technologies
Growth in
targeted
crop sub-
sector
Increased
export of
targeted
commodities
Gov’t
budgets and
spends more
on social
safety nets
Increased
sales for
producers
Producers
increase ag
productivity
Producers
increase
quality of
produce
Targeted
firms train
farmer orgs
Increased
financing
accessed
Partial loan
guarantee
Decreased
national
poverty
Decreased
ZOI
poverty
Decreased
national
hunger
Decreased
ZOI hunger
PPP e-
verification
program
for seeds,
fertilizer
Reduced
counterfeit
inputs
Stronger
relationships
between
input
wholesalers
& retailers
Greater
access to
inputs
Greater
trust in
quality of
inputs
Interventions
/ Outputs Outcomes for participants or system
Impacts across
population
27
Increased
incomefor
producers
Increased
income

28. Logic model reminders
• Not every result necessarily needs an indicator
• Think about scale and space in addition to the
direction of results
• Time element needs to be considered.
28

29. Take-home messages
• Standard indicators are not enough! Use custom
indicators, disaggregates, and other measures for
adaptive management and telling your story.
• Good MEL starts with a theory of change, ideally
illustrated through a detailed, dynamic logic model
tied to indicators and other measures.
• Generating data without analyzing and using it is
a waste of resources. Build in time and resources
to analyze, learn, and apply the evidence.
29

30. Questions and discussion
30

31. Feed the Future MEL Webinar Series
• Intro to the MEL System - April 19, 2018
• Standard Indicator Overview – May 15, 2018
• New Indicators 1: Application of improved practices and
technologies
• New Indicators 2: Yield and geospatial
• New Indicators 3: Sales and investment indicators
• Learning Agenda
• Market Systems Measurement
• Annual FTFMS users webinar
31