This document summarizes the key points from a presentation on assessing the impact of natural resource management (NRM) research. It discusses CGIAR's NRM research agenda, past impact assessments of NRM innovations, gaps in the evidence base, and insights on ways to strengthen impact assessment going forward. The presentation calls for rethinking the focus on technology adoption and instead documenting how NRM research changes discourses and understanding. It emphasizes the need for new impact assessment methods that can evaluate outcomes at farm and landscape scales and account for non-linear impacts over time.

1. Impact of NRM research – the way forward
Karen Macours
Paris School of Economics / ISPC’s Standing Panel on Impact
Assessment

2. Outline
• CGIAR’s Natural Resource Management
Research (NRMR) agenda
• Impact assessments of NRM research
• SPIA’s work to-date
• Findings and gaps
• Insights from the workshop
• Ways forward: challenges and priorities

3. CGIAR’s NRM research agenda
• Environmental and social benefits (well-being)
• Micro- & macro- interventions
Spans
• Productivity-related: NRM for agriculture
• Incld. production & natural resource systems for household /
community use (fisheries, forestry, aquaculture)
• Resource conservation or restoration related
• Policies for governance/management of natural resources
• Human & environmental health via reduced agricultural pollution

4. Evolution of investments in NRM research
1988: TAC paper on sustainability, conservation, management of NR
US$ 380m
(Protecting
environment)
US$ 780m
(Production
systems)

5. Investment in (CGIAR) development outcomes 2017-
2022
- USD 5.3 billion funding requested
- USD 1.2 billion mapped to
• natural resource / ecosystem services (15%)
• climate change- related outcomes (7%)
 Both historically and currently an important part
of CGIAR agenda
What do we know about impacts?
Last updated April 2016. Source: ISPC

6. NRMR Impacts: Evidence from CGIAR, 2006
Response to ‘research findings on the impact of NRMR remain sparse.’
1. Zero-till in rice-wheat systems, India, CIMMYT
2. Participatory NRM in cassava-based systems, Vietnam, CIAT
3. Integrated aquaculture-agriculture technologies, Malawi,
WorldFish
4. Improved tree fallow, Zambia, ICRAF
5. NRM in crop-livestock systems, Morocco/Tunisia, ICARDA
6. Irrigation management transfer, Global, IWMI
7. Forest management (criteria and indicators) - SFM, Global, CIFOR

7. Impact of Policy-oriented Research, 2006-10
• Pulp and paper industry, Indonesia, CIFOR
National
• Policy outcome: mills to source wood from
plantations
• Pesticide regulatory policy, Philippines, IRRI
National
• Policy outcome: highly toxic insecticides in rice
regulated
• Six studies, of which two NRM POR

8. Assessing Environmental Impacts, 2008-11
Six studies, of which four NRMR; non-
market valuation techniques
1. Zero-till in rice-wheat systems, IGP SA, ICAR
Regional
2. Improved supplemental irrigation, Syria,
ICARDA National
3. Alternative sluice management, Vietnam,
IWMI Watershed
4. Rubber agroforestry, Indonesia, ICRAF
Provincial

9. SIAC program, 2013-17
• Adoption studies
• 9 work packages, 6 NRMR outputs
• Nine impact assessment studies
• Agroforestry (ICRAF, Univ. of Illinois, ViAgroforestry)
• Alternative Wetting and Drying (North Carolina State Univ., IRRI)
• Conservation Agriculture (Univ. of Illinois, ICRISAT)
• Drip irrigated horticulture (Johns Hopkins, George Washington
Univ., MDG Center)*
• Forest co-management (Virginia Tech, CIFOR)
• Intercropping (Yale, ICIMOD)
• Integrated Soil Fertility Management (Paris School of
Economics, IITA)
• Rainwater harvesting (Tufts Univ.)
• Soil fertility analysis based extension (UC Berkeley, World Bank,
QFD)

10. NRMR innovations examined in IAs

11. NRMR innovations & outcomes in IAs

12. Impact evidence base
• Mixed evidence
• Individual studies – some positive impacts
• Yield
• Household income or food consumption
• Water savings and other input costs
• Tree cover
• Starting to see application of rigorous,
innovative methodologies
• Use of secondary data. In particular, land use,
tree / forest cover changes using remote sensing

13. Impact evidence base
• Limited documentation of environmental
outcomes
• Limited evidence of scale – ‘project mode’
• Dearth of synthesized evidence on impact, in
contrast to crop genetic improvement
• Evenson and Gollin (2003)
• Attribution of CGIAR role – challenging

14. Insights from workshop Feb 26-27, 2018
• Starting point: Evidence of low levels of adoption of
series of sustainable production systems practices
• Why? => research agenda moving forward
• Innovations not beneficial for (many) farmers?
~ questions on development & testing innovations
• Farmers don’t know innovations and their returns
• Complexity practices & their costs and benefits
~ questions on extension & diffusion at scale
• Innovations not (perceived) beneficial to individual farmers?
• Social returns > Private returns
• Long-term returns discounted/unknown
~ questions on (new methods) of measurement (at scale)
CGIAR researchers and outside experts
Bio-physical and social scientists

15. Some general thoughts on the way forward
• Re-think ‘adoption of technologies/practices’ paradigm
• CGIAR NRMR provides principles, changes
discourses
• document what and how
• methodological development for impact at farm and
landscape level, and interlinkages
• ~ attribution challenge
• Non-linearity of NRM research
• Not: development => testing => diffusion
• Impact assessments also need to study different
parts of causal chain, including development and
testing stages

16. Research agenda: heterogeneity and targeting
~ Innovations not beneficial for (many) farmers?
• Goal is not all farmers adopting all innovations
• Heterogeneity and variability of benefits challenge for scaling
- Targeting on observables shown to be difficult
- Understand farmers’ objective => Revelation mechanisms
- Revealed preferences may help predictions for scale-up
• Participatory experimentation with large N
- Mix of innovations suggested by researchers and those chosen by farmers
- Self-targeting by farmers based on demand
- But also allow for farmers information and learning to be imperfect
=> dynamic process & two-way feedback
- Bring measurement for research and measurement for farmers’ learning
closer together
- Farmers’ perceptions versus detailed “objective” measures
- Combine biophysical and social science perspectives

17. Research agenda: measurement
• Measurement of which outcomes?
- Understand trade-offs, private & public costs and benefits
• Don’t forget but go beyond resource-related benefit
• Go beyond productivity (and certainly beyond yield)
• Document spillovers, externalities
• Farmers & policymakers need information to assess cost effectiveness
- Given that objective is to change principles (rather than particular
practices/technologies): outcomes to be measured not binary
• With what frequency and which timelines?
- Dynamic process and trajectories: Adoption and dis-adoption
• Panels and long-term follow-ups
• Impact ex-post has to be sufficiently long-term
• Which methods? leverage combination of:
- Households/farm (plot) level survey
- Phone surveys (high frequency outcomes)
- Remote sensing and drones
- Secondary data (ag & population census, …)
- …

18. Research agenda: understanding diffusion at scale
- Start from expected private and social benefits and costs
- If important externalities => will need different mechanisms for
diffusion
- Interventions addressing information and other binding
constraints
- Which approach for which objective?
- How we transmit information matters (adult learning, learning-
by-doing versus learning-from-others, …)
- Complexity of innovation matters
- Bring in behavioral insights
- Link with existing programs /partners outside of CGIAR
- Lots of recent advances on understanding constraints to
adoption and learning on scaling (extension)

19. Thank you
http://ispc.cgiar.org