Presentation for ICCB 2017 by Edwin Pynegar based on work by Edwin Pynegar, James Gibbons, Nigel Asquith and Julia P G Jones

1. An evaluation of an incentive-based
watershed conservation program using a
Randomized Control Trial:
Lessons for implementers and evaluators
Edwin Pynegar, James Gibbons, Nigel Asquith and Julia Jones
ICCB Cartagena, Wednesday 26th July 2017
@EdwinPynegar

2. Impact Evaluation in Conservation:
think counterfactual!

3.  Randomized Control Trial (RCT)
of Watershared: incentive-
based forest conservation in
Latin America, pioneered by
Fundación Natura Bolivia
 129 communities (64
treatment, 65 control)
 7340 km2 protected area in the
Bolivian Andean forests

4. Watershared in the Santa Cruz Valleys (Bolivia)
Landowner NGO/local authority
zero deforestation
zero cattle presence
(for higher-level
agreement)
Delivery of
incentives (goods)
at regular intervals
on compliance with
the agreement

5. E. coli: A quantitative metric of the ecosystem
service of clean water
Measurement of faecal indicator organisms and other metrics of
water quality at community water intakes (left) and taps (right)

6. E. coli: A quantitative metric of the ecosystem
service of clean water

7. Watershared in the Santa Cruz Valleys (Bolivia):
A Randomized Control Trial (RCT)
No difference in rates of change between treatment and control groups…
(site n=83, community n=47, treatment-time interaction coefficient 95% CI = -0.39, 2.28)

8. Watershared in the Santa Cruz Valleys (Bolivia):
A Randomized Control Trial (RCT)
…but actions associated with Watershared may have major impacts on
ecosystem service indicators.
(site n=219, community n=124, ‘faeces in water/on riverbanks’ coefficient 95% CI = 0.26, 3.97)

9. Paradox: the action incentivized can
increase supply of ESs, but the
intervention as designed does not.
 How to explain it?
 And what does it teach us about designing PES, and
designing conservation RCTs?

10. 1. Conserved areas may have different effects
on outcomes depending on their locations
Targeting prevents
waste, but implies
higher transaction costs
(Jack et al. 2008)
Potential for spillover
due to misalignment of
RCT unit with ecological
unit of interest
Designing payments for ecosystem services: Lessons from previous experience with incentive-based mechanisms.
Jack, B. K., Kousky, C., Sims, S. P.N.A.S. USA 105(28), 9465-9470 (2008).

11. 2. Relationships between implemented actions
and ecological outcomes may vary greatly
Implies requirement for an
‘agglomeration bonus’ (e.g.
Engel 2016) or different model
of contracting agreements
The devil in the detail: A practical guide on designing payments for environmental services. Engel, S.
International Review of Environmental and Resource Economics 9, 131-177 (2016).

12. 3. Communities conserve land separately from
the Watershared program – an RCT confounder
Water intake, Minas (control community) Reserva de Patrimonio Natural, La Aguada

13. Take-home messages
 Incentive-based conservation did not lead to a significant
improvement in water quality at the community scale (RCT
analysis), but it can do at the local scale where actions
actually implemented
 Effective incentive-based conservation requires careful
consideration of targeting and action-outcome relationships
 Spatial and behavioural spillover and confounders make
designing high-quality RCTs to evaluate such interventions
challenging.

14. Thank you for listening!
Further reading:
 What role should Randomized Control Trials play in providing the
evidence base underpinning conservation? Pynegar, E. L., Gibbons,
J. M., Asquith, N. M., Jones, J. P. G. In review.
 An Evaluation of the Impact of Payments for Ecosystem Services
using a Randomized Control Trial. Pynegar, E. L., Jones, J. P. G,
Gibbons, J. M., Asquith, N. M. In review.
 ‘Watershared’ – Adaptation, mitigation, watershed protection and
economic development in Latin America. Asquith, N. M. Climate &
Development Knowledge Network (2016).
@EdwinPynegar