Uruguay Ministry of Livestock, Agriculture and Fisheries (MGAP) strategy for the evaluation of agricultural policies http://www.fao.org/in-action/naps/resources/webinars/en/

1. 1
Agricultural Policies Evaluation Area
MGAP strategy for
Evaluation of Agricultural Policies
Bureau of Programming and Agricultural Policy (OPYPA)
Ministry of Livestock, Agriculture and Fisheries (MGAP)
URUGUAY

2. 2
Schedule
I. Institutionalizing agriculture policies evaluation at
MGAP
a) Why evaluate at MGAP?
b) Building internal capabilities
c) Evaluation Scope and methodologies.
II. Evaluation agenda prioritized by authorities
a) Progress in evaluation strategy
b) Information System for evaluation.
III. Lessons learned

3. 3
The MGAP is making progress in institutionalizing evaluation of
policies and programs, with emphasis on impact assessment and
evaluation of results.
Why?
a) Why evaluate at MGAP?
Need evidence on what works
► Justification for program ($$)
► Scale up or not – did it work?
► Compare different policy options
11
Improve program/policy
► Design (eligibility, benefits)
► Implementation (efficiency & targeting)
22
Be accountable and transparent to the people
(stakeholders) for whom we are working33
Impact
evaluation
for evidence
based
policy
making

4. 4
b) Building internal capabilities
► Agricultural Policies Evaluation Area has a specialized technical
team.
▪ Evaluators are responsible for the technical aspects: the evaluation
methodology, sampling design, data collection, analysis, and reports.
► Institutionally, the Agricultural Policies Evaluation Area is
located within OPYPA but is external to the implementing
programs units

5. 5
b) Building internal capabilities
Why internal Evaluations?
Advantages Disadvantages
● To build a "culture of evaluation"
● Evaluation needs a deep
understanding of the theory of
change and the context.
● Evaluation reports and policy
recommendations would be more
accepted if they came from inside.
Challenges to maintain objectivity and
credibility.
Agreement OPYPA-AGEV:
Key partner who gives technical guarantees on the
independence of evaluation (an external view).

6. 6
c) Evaluation Scope
What was the effect of the program on
outcomes? (before-after, no causality)
How much better off are the beneficiaries
because of the program/policy (attribution)?
Is the program cost-effective? Traditional
Monitoring
cannot answer
these.

7. 7
c) Methodologies
► Quantitative methods: Matching, Difference-
in-Difference, Instrumental variables,
Regression discontinuity design, others.
►Qualitative methods: including semi-structured
interviews and focus group discussions
► A mix of methods
Do what works best in
the context.
Be flexible, be
creative.

8. 8
Schedule
I. Institutionalizing agriculture policies evaluation at
MGAP
a) Why evaluate at MGAP?
b) Building internal capabilities
c) Evaluation Scope and methodologies.
II. Evaluation agenda prioritized by authorities
a) Progress in evaluation strategy
b) Information System for evaluation.
III. Lessons learned

9. 9
1. Adaptation of production systems to climate variability
▪Building resilience to climate change and variability in
vulnerable smallholders (GFCC)
▪Water for Animal Production (APA)
▪Agricultural insurance
2. Rural development: competitive insertion of family farming
in value chains
▪Institutional Strengthening Plans (PFI)
▪Family, Integral and Sustainable Production (PFIS)
a) Progress in evaluation strategy at MGAP
Baseline
Final Report
Baseline
Design

10. 10
•1
•Idea
•2
Impact
Assessment
APA
(Efects medium and
long term)
Inputs for Design and Redesign
Results
Evaluation PFI
(End of
implementation)
Baseline
GFCC-PFIS
(start of
implementation)
Process
Evaluation
DID-AGEV
(during
implementation)
Contexto
a) Progress in evaluation strategy

11. 11
National
Level
National
Level
BeneficiaryBeneficiary
Producers /
Rural
organizations
Producers /
Rural
organizations
Project
Level
Project
Level
Farms
Level
Farms
Level
Programs application forms
DB
National
Agricultural
Census 2011
b) Information system
ApplicantsApplicants
Data panel
SNIG-Dicose
GFCC Survey
PFIS Survey
Family Farmers
Registration DB
PFI Survey and
Qualitative data

12. 12
Schedule
I. Institutionalizing agriculture policies evaluation at
MGAP
a) Why evaluate in MGAP?
b) Building internal capabilities
c) Evaluation Scope and methodologies.
II. Evaluation agenda prioritized by authorities
a) Progress in evaluation strategy
b) Information System for evaluation.
III. Lessons learned

13. 13
1. It is important to plan for impact (the ‘start point’). Some
changes in MGAP interventions would result in better
evaluation and strengthen impact.
►Encourage the development of an explicit Theory of Change, measure
target population and indicators.
►Embed impact assessments in organizational learning and planning.
►Ensure input from policy makers and implementers at all stages of design
and development of impact assessment processes.
Lessons learned

14. 14
2. MGAP information systems need further refinements to
maximize the use of administrative data and statistics, which have
great potential in Uruguay and are less expensive than surveys.
3. It is convenient to deploy both quantitative and qualitative
assessments to better understand causes and effects, which leads
to better policy recommendations. Main challenge: develop an
adequate “methodology toolbox”.
4. Focus on producing inputs for program design and redesign.
Use findings widely and creatively for improved impact of impact
assessments!
Lessons learned

15. 15