Improving evidence on the impact of
agricultural research & extension
Reflections on CSISA’s experience
David J. Spielman, Patrick Ward, and Simrin Makhija
International Food Policy Research Institute
Africa RISING Monitoring and Evaluation
Meeting
Arusha, Tanzania, 13-14 November 2014

A quick outline
• Constraints to adoption
• Uses and utility of evaluation
• CSISA’s experience

Why do farmers adopt new
tEexcpehctnedo ultoilitgy imeasxi?mization
 Individuals maximize the expected “utility” they get from
consuming food, non-food goods/services, and leisure with
products and income derived from agricultural production
 Individuals compare utility that would be achieved under
different states of the world and the probability of that state of
the world actually occurring
 If individuals expect that adopting a new technology will
maximize their total utility, then they tend to adopt…
 Subject to several important constraints

What constrains adoption?
Known, readily measured constraints
• Biophysical characteristics
• Land, soil, water, biology
• Farm characteristics
• Farm size, scale, system
• Individual characteristics
• Experience (age), education, hh labor supply
• Market factors
• Access to credit, input, and output markets
• Institutional factors
• land tenure arrangement, group membership
• Returns to adoption
• Net returns, variability in returns
∗ =
푍푖
푟
1 푖푓 푈푖 > 푈푛
푟
0 푖푓 푈푖 ≤ 푈푛

What else constrains adoption?
Lesser-known, harder-to-measure, and unobserved
constraints
Informational constraints
 Experimentation, trialing (Linder et al. 1982; Marra et al. 2003)
 Exposure, awareness (Diagne & Demont 2007)
 Learning by doing (Foster & Rosenzweig 1995)
 learning from others and peer effects (Besley & Case 1994; Conley & Udry
2010)
 Learning by noticing (Hanna & Mullainathan 2012)
Individual preferences
 Risk preferences (Just & Zilberman 1983; Smale et al. 1994)
 Loss aversion (Tanaka et al. 2010; Liu 2013)
 Time-inconsistent preferences and present bias (Duflo et al. 2008)
 Aspirations (Bernard et al. 2014)

So… what questions should we be
asking?
1. How do different approaches to adult
education affect learning outcomes?
2. How do learning outcomes lead to
adoption?
3. How does adoption result in desirable
social, economic, and environmental
outcomes?

How do evaluations answer these
questions?
• Measurement: to gauge impact of alternative extension
approaches on
• Adoption
• Productivity
• Sustainability
• Incomes and welfare
• Learning: to assemble evidence about what works and why
• Feedback: to improve project management and implementation
• Accountability
• Transparency
• Policy design: to credibly inform large-scale replications by
government

Why do we need better designs, more
rigor?
• Sample selection bias
• Those who learn/adopt may be fundamentally different from those
who don’t
• Bias limits our ability to make wider inferences, to scale up, and to
design policies
• Endogeneity
• Reverse Causation: A → B or B → A ?
• Simultaneity: the “Reflection Problem”
• Heterogeneity
• Beyond average effects: Measuring outcomes for specific groups
within a population

With a better toolkit, we can do a lot
more…
• Qualitative
• Quantitative
• Mixed
Methods
• Understanding context
• Understanding impact pathways and theories
of change
• Internal validity: good identification
strategies
1. Experimental Methods: RCTs
2. Non-experimental methods: PSM, RDD, Ivs,
D-in-D
• External validity: generalizability

…to explore complex processes
• Combine economics, education, and social psychology
 behavioral dimensions of learning and adoption
• Evaluate type and intensity of training
• Study the step-by-step process of learning
• Evaluate changes in aspirations and locus of control
• Evaluate learning failures
• Evaluate peer effects

But simplicity is key to understanding
complex innovation processes
Take a single technology, practice, or system…
1. Evaluate which extension approach better facilitates
learning/adoption
2. Compare different extension approaches
• Training & Visit vs. Farmer Field Schools vs. Mother-Baby Trials vs.
Chalk-and-Talk
3. Measure the cost-effectiveness of each extension approach
4. Open the door to evaluation of learning approaches

Monitoring & evaluation in CSISA
Project
Monitoring
Project outputs,
deliverables
FTF indicators
External
project reviews
Feedback to
management
Project
evaluation
Station, on-farm
agronomic
trials
Costs and
returns to
farmers
Costs and
returns to
service
providers
Determinants
of adoption
Impact
evaluation
Productivity,
sustainability
impacts
Income,
welfare impacts
(Nutrition,
health impacts)
Policy
analysis
Ag subsidy
policy
Industrial
policy
National R&D
priorities
(Ex ante
scenario
analysis)

What role for baseline surveys in CSISA?
• As a diagnostic tool
• Characterizing farming systems, households, development domains
• Measuring farm-level productivity and poverty levels
• Characterizing variation in these indicators
• As an analytical tool
• As data for modeling ex ante impacts at scale
• As an evaluation tool?
• Too many simultaneous interventions to establish causality
• “Saturation approach” to project rollout leads to contamination of
treatments
• Extremely costly and time-consuming (3 yrs just to get a report out!)
The solution? We abandoned baseline surveys, focused on individual
interventions

Our emerging approach
Diagnostics
(Consultatio
ns, FGDs, lit
reviews)
Valuations
(DCEs,
BDMs)
Evaluations
(RCTs and
quasi-experimental
designs)
Scale-up
Scenarios
(Simulations
models)
Tangents
(Deeper
analysis of
second-order
topics)

Our approach in action: LLLs in India
Diagnostic Valuation Evaluation Scenario
Is there
demand for
laser land
leveling?
What will
farmers pay
for leveling?
What are the
net gains to
leveling?
How do alternative
targeting strategies
impact welfare?
How to
gendered peer
effects
influence
valuation and
adoption?
analysis
Tangents

Demand for LLL services
200 400 600 800
Rs./hour
0 10 20 30 40 50 60 70 80
Percent of area
2012 2011
Source: Lybbert et al.
2014

Demand for LLL services
200 400 600 800
Rs./hour
0 10 20 30 40 50 60 70 80
Percent of area
Deoria
Gorakhpur
Maharajganj
District
200 400 600 800
Rs./hour
BPL
Non-BPL
BPL
0 10 20 30 40 50 60 70 80
Percent of area
200 400 600 800
Rs./hour
Landholdings
<1 acres
1-2.5 acres
>2.5 acres
0 10 20 30 40 50 60 70 80
Percent of area
200 400 600 800
Rs./hour
Plot size
First (<= .3 acres)
Second (.3-1 acre)
Third (1 or more acres)
0 10 20 30 40 50 60 70 80
Percent of area
Source: Lybbert et al.
2014

Our approach in action: DT/WII in
Bangladesh
Diagnostic Valuation Evaluation Scenario
Is there
demand for
weather
index
insurance
vs. a DT
cultivar?
What will
farmers
pay for
insurance?
Does
behavior
change
with
insurance
or a DT
rice
cultivar?
How cost-effective
is
a large-scale
rollout of
DT vs.
WII?
How do farmers
learn about risk
management from
infrequent
stochastic events?
analysis
Tangents

Thank you

References
Bernard, T., S. Dercon, K. Orkin, & A.S. Taffesse. 2014. “The Future in Mind: Aspirations and Forward-Looking
Behaviour in Rural Ethiopia.” Centre for the Study of African Economies Working Paper Series no. 2014-16. Oxford,
UK: University of Oxford.
Davis, K., Nkonya, E., Kato, E., Mekonnen, D. A., Odendo, M., Miiro, R., & Nkuba, J. 2012. “Impact of Farmer Field
Schools on Agricultural Productivity and Poverty in East Africa.” World Development 40(2): 402-413.
Feder, G., R.E. Just, & D. Zilberman. 1985. “Adoption of Agricultural Innovations in Developing Countries: A Survey.”
Economic Development and Cultural Change 33(2): 255-298.
Hanna, R., S. Mullainathan, & J. Schwartzstein. 2012. “Learning through Noticing: Theory and Experimental Evidence in
Farming.” Working Paper no. 18401. Cambridge, MA: National Bureau of Economic Research.
Haug, R. 1999. “Some Leading Issues in International Agricultural Extension, a Literature Review.” Journal of Agricultural
Education and Extension 5(4): 263-274.
IPPRI (International Food Policy Research Institute). 2011. Policies, Institutions, and Markets to Strengthen Food
Security and Incomes for the Rural Poor. A revised proposal submitted to the CGIAR Consortium Board. Washington,
DC: IFPRI.
Jack, B.K. 2011. “Market Inefficiencies and the Adoption of Agricultural Technologies in Developing Countries.” White
paper prepared for the Agricultural Technology Adoption Initiative: Cambridge, MA/Berkeley, CA: Abdul Latif Jameel
Poverty Action Lab (MIT)/Center for Effective Global Action.
Kondylis, F., & Mueller, V. 2010. “Seeing is Believing? Evidence from a Demonstration Plot Experiment in
Mozambique.” Mozambique Strategy Support Program working paper no. 1. Washington, DC: IFPRI.
Lambrecht, I., Vanlauwe, B., Merckx, R., & Maertens, M. 2014. “Understanding the Process of Agricultural Technology
Adoption: Mineral Fertilizer in Eastern DR Congo.” World Development 59: 132-146.
Lybbert, T.J., N. Magann, D.J. Spielman, A. Bhargava, and K. Gulati. 2014. Targeting technology to increase smallholder
profits and conserve resources: Experimental provision of laser land leveling services to Indian farmers . mimeo.
Manski, C.F. 1993. “Identification of Endogenous Social Effects: The Reflection Problem.” Review of Economic Studies