Presentation by Brian Belcher

1. White paper and CRP6: Co-learning on
Impact Evaluation Design in NRM
Research Programmes
Presented to NRM Impact Community of Practice
The Worldfish Center, Penang, Malaysia, Sept. 4-5, 2012
Brian Belcher, CIFOR/Royal Roads University

2. CRP6 Aims
to enhance livelihoods through forestry,
agroforestry and other uses of forest
resources while sustaining environmental
services and resource resilience.
THINKING beyond the canopy

3. CRP6 Components
THINKING beyond the canopy

4. What’s new in
CGIAR/CRP6?
• Stronger collaboration, focus and
coherence within CG
• More, stronger and more diverse
partnerships
• Results focus, shared responsibility
for outcomes
• Sentinel landscapes
• Emphasis on learning by doing and
on verification of progress
THINKING beyond the canopy

5. CRP6 Proposal & MEIA Strategy
 Discuss complexity in NRMR
 Recognize experimental design IE approaches not
necessarily appropriate or best
 Appreciate evaluation for learning & for accountability
 Emphasize clarifying causal assumptions through
participatory impact pathway development
 Recognize multiple impact pathways and multiple scales
 Propose using tools of Outcome Mapping, PIPA,
integrating monitoring, evaluation and impact
assessment, and collaboration with other CRPs and
experts
THINKING beyond the canopy

6. White Paper and process
 White Paper and COP helps explain the need for
alternative approaches and provides legitimacy for mixed
methods approaches
 Develop/promote consistent definitions and approaches
 Forum for sharing ideas, perspectives, expertise
 Overview and access to a range of relevant methods and
related literature
 Simultaneously developing “IDOs”
 Presentation will reflect on experience to date within
CRP6 evaluation planning and implications for WP
THINKING beyond the canopy

7. Concept of NRMR
 CRP6 primarily contributes to SLO 1(reducing rural
poverty) and 4 (sustainable NRM)
 Supplementary contributions to 2 (improving food
security) and 3 (improving nutrition and health)
 Bottom-up approaches (“trajectories”): develop and
support technology and institutions to benefit small-
holders, communities)
 Intermediate approaches: influence research agenda;
support capacity strengthening; mainstream gender
analysis)
 Top-down approaches: influence policy at level of
conservation orgs, development orgs, national and
international policy) THINKING beyond the canopy

8. Research for policy influence
 predict (ex ante) or measure (ex post) effects of policy
options and policy tools
 provide knowledge for forming, implementing or
contesting policy
 identify and explain trends
 raise awareness of a problem
 improve understanding of underlying causes of economic
behaviour and environmental outcomes
 challenge conventional wisdom
 develop/influence research methods
 develop useful theory or conceptual framework
THINKING beyond the canopy

9. Implications for WP
Begin with definition of NRMR that encompasses and
explains the range of ways that NRMR contributes to
improvements in social and natural systems
Explicitly recognize that research processes and products
contribute to change
Make stronger argument that we need alternative ways to
establish “counterfactuals”
THINKING beyond the canopy

10. Terminology
 Major source of confusion – need good, clear definitions
and consistent use of key terms
 e.g. SLOs are really “impacts” by contemporary
definitions
 still unclear whether IDOs should be defined as changes
in behaviour or changes in state
THINKING beyond the canopy

11. Implications for WP
 Distinguish “intervention” from “programme” (programme
can include/support many interventions/kind of intervention)
 Use “boundary partner” instead of “working partner”
(consistent with K to A literature)
 Distinguish & clarify “target groups” & “beneficiary partners”
 Distinguish between results from the use of new knowledge
and results from the process of doing the research
 Define outcomes as behavioural change
 Intended impacts include changes/conservation of
biophysical resources
THINKING beyond the canopy

12. Scale issues
 Nested impact pathways at several scales
(project, theme, component, “sentinel sites”,
CRP6, SLOs)
 Project-scale boundary partners & outcomes
typically more direct, more tangible
 Component-scale boundary partners and
outcomes more difficult to define, identify and
measure
THINKING beyond the canopy

13. Implications for WP
 Need more attention to program-level issues and
approaches
 How to define, identify, and measure outcomes that are
manifold and diffuse
THINKING beyond the canopy

14. How will adopting the framework
change M&E and IE practice in
CRP6?
 supports and strengthens MEIA approaches in
development in CRP6
 Provides structure for building, testing methods

15. What benefits will this bring?
 Encourages “impact culture”, clearer and more
comprehensive project/program
conceptualization, design and implementation
 Support and encourage broader range of
partnerships and interventions
 Learning, feedback (monitoring for adaptive
management)

16. What enabling changes are
needed?
 Recognition and legitimacy of TOC approaches
within CGIAR and donors
 Capacity building internally

17. Next steps for implementation?
 TOC approaches to be used in forthcoming
evaluations of past work
 Recognition and legitimacy of TOC approaches
within CGIAR and donors
 Capacity building internally

18. CRP6 IDO Template
SLO
Comments [on ability
Geographic
Quantified targets to quantify
IDO Baseline focus
# How? (10 Years) targets/baseline/geo...
/scope
]
Enhanced contribution of FAT to income, food security, and nutrition
Intermediate Development
Impacts
Increased revenue from sale of 1 Higher productivity of • Incomes from ICRAF: new data ICRAF: new We have households who
trees and forests tree and forest from Sahel; data on projects in already sell and may enjoy
tree products
coupled with improved products fruits in Kenya and Mali, BF, better incomes; we will have
markets and policy doubled for Malawi; range of Niger, Sierra households new to receiving
environment enables target tree products in Leone, CDI, income from FAT. We can
more rural households households Cameroon, Cote Malawi, estimate this once we
to participate in tree d’Ivoire and Ethiopia, decide on the locations. As
product markets and to Sumatra; timber in Rwanda, for the projects mentioned,
earn more money from NW India and fruits Burundi, most have baselines but
them. Higher elsewhere; Uganda, there is need to collate
productivity comes Kenya; results in 2012/13.
from improved Allanblackia
germplasm and project in
management. Tanzania and
Ghana, timber
in Sulawezi,
fruits in
Vietnam
THINKING beyond the canopy

19. www.cifor.cgiar.org
THINKING beyond the canopy