This document summarizes the results of an assessment of 23 REDD+ initiatives using a before-after/control-intervention (BACI) approach. The BACI ratio compares the average annual deforestation rates in the intervention area before and after to rates in a control area. Results found good performance in 11 initiatives, neutral in 8, and poor in 4. Poor performance does not necessarily mean failure, as further analysis found various contextual factors that influenced results, including bias in baseline periods, low overall deforestation, peak deforestation years, limited additionality compared to controls, and initiatives ceasing projects. The conclusions are that REDD+ sites generally perform well compared to controls when analyzing relative change, but
3. Performance assessment
Reference levels vs. Before-After/Control-Intervention
3
𝐵𝐴𝐶𝐼 𝑟𝑎𝑡𝑖𝑜 𝛽 = 𝑥 𝐴𝐼 − 𝑥 𝐵𝐼 − 𝑥 𝐴𝐶 − 𝑥 𝐵𝐶
𝑤𝑖𝑡ℎ 𝑥 𝐴𝐼 =
1
𝑛 𝑎
𝑖=1
𝑛 𝑎
𝑥𝑖
𝑤ℎ𝑒𝑟𝑒 𝑥 𝐴𝐼 𝑟𝑒𝑝𝑟𝑒𝑠𝑒𝑛𝑡𝑠 𝑡ℎ𝑒 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑎𝑛𝑛𝑢𝑎𝑙 𝑑𝑒𝑓𝑜𝑟𝑒𝑠𝑡𝑎𝑡𝑖𝑜𝑛 𝑟𝑎𝑡𝑒 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑒𝑟𝑖𝑜𝑑 𝑎𝑓𝑡𝑒𝑟 𝑡ℎ𝑒 𝑖𝑛𝑡𝑒𝑟𝑣𝑒𝑛𝑡𝑖𝑜𝑛 𝑠𝑡𝑎𝑟𝑡𝑒𝑑
𝑎𝑛𝑑 𝑛 𝑎 𝑖𝑠 𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑦𝑒𝑎𝑟𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑒𝑟𝑖𝑜𝑑 𝑎𝑓𝑡𝑒𝑟 𝑡ℎ𝑒 𝑖𝑛𝑡𝑒𝑟𝑣𝑒𝑛𝑡𝑖𝑜𝑛 𝑠𝑡𝑎𝑟𝑡𝑒𝑑
B ACI
CI
B A
B A
B A
4. Data: Tree cover and tree-cover change
Global Forest Change 2000–2014 (Hansen et al., Science 2013)
Forest definition: 10% tree cover (FAO)
Regional uncertainty no effect on local trend analysis
4
5. Results
difference Before-After & Before-After/Control-Intervention ratio
5
good 7 30.4%
neutral 7 30.4%
poor 9 39.1%
good 8 34.8%
neutral 9 39.1%
poor 6 26.1%
good 9 40.9%
neutral 4 18.2%
poor 9 40.9%
good 11 50.0%
neutral 8 36.4%
poor 3 13.6%
6. ● Deforestation intervention > control REDD+ in frontier (e.g. Brazil_3)
● Deforestation intervention < control Conservation area (e.g. Indonesia_4)
Results explained
(1) Bias in before period
6
Average annual deforestation rate
in intervention area
(initiative boundaries)
Average annual deforestation rate
in control area
(district)
Average annual deforestation rate
in intervention area
(initiative boundaries)
Average annual deforestation rate
in control area
(district)
7. Results explained
(2) Low absolute deforestation
7
small differences high uncertainty big influence on score (e.g. Tanzania_1)
8. Results explained
(3) Peak years
8
Tanzania_1 control area (district)
Tanzania_5 intervention area (initiative)
In before period (in control area)
“better” Before-After score for control
“poorer” BACI
(e.g. Brazil_1/Tanzania_1/Tanzania_6)
In after period (both control & intervention)
Poor performance?
REDD+ not addressing big event drivers
(e.g. Tanzania_5)
9. Results explained
(4) Limited additionality
9
Decrease in deforestation, but limited additionality
(control area performs even better than intervention villages) (e.g. Brazil_2)
Brazil_2 intervention (villages) Brazil_2 control (villages)
10. Results explained
(5) Poor performance?
10
Poor performance?
3 consecutive years in after period in intervention area with high deforestation
Vietnam_1 ceased project in 2012
11. Conclusions (preliminary)
Performance measure itself has implications on results
Overall, REDD+ sites perform relatively well when
compared to control units (here: only relative change is
analysed)
Causes of “poor” BACI score vary widely
● Random/contextual factors
o Bias
o Low absolute deforestation
o Peaks
● Limited additionality
● Poor performance (incl. cease initiative)
For result-based finance, it is important to understand
causes of change
11
Explain GCS + module 2
Diversity between initiatives: state-led programs versus small-scale NGO projects that focus on a couple of villages
Private companies as well.
Recent comparison: show reasonable representativeness of global redd+ initiatives (170 gabriella simonet study)
Performance defined as: in areas where there is an REDD+ intervention, is there less deforestation compared to the situation without the intervention?
Traditional: compare a trendline with a reference level based on period before the period started
Rel. new: BACI. Explain control vs intervention
Comparing average annual deforestation rates score: the lower the score the better the performance
BACI link with socio-economic research.
3rd phase long term data collection
Landsat based
Advantages: worldwide, preprocessed: easy to implement own forest definition (tree cover percentage)
Disadvantages: criticised for use at local level local calibrated products are probably more accurate. No distinction between plantation and natural forest, although state that with the latest version, that has improved:
Improved detection of boreal forest loss due to fire.
Improved detection of smallholder rotation agricultural clearing in dry and humid tropical forests.
Improved detection of selective logging.
Improved detection of the clearing of short cycle plantations in sub-tropical and tropical ecozones.
Some comments on regional uncertainty. Justification: we only focus on local trends, and relative changes. Uncertainty: error is correlated at the local level not influencing our results.
Explain control vs intervention
0.1 threshold for good/neutral/poor categories
At both site and village level: biggest difference in poor neutral when adding control regions *click*
Village level is showing bigger change, and better BACI performance, than site level *click*
Brazil_3: bias in before period(Deforestation intervention > deforestation control (district)) REDD+ location in deforestation frontier?
Indonesia_4: Bias in before period (traditional conservation area?)(Deforestation intervention < deforestation control (district))
Tanzania_1: variable trend with peaks. But low absolute deforestation because of low deforestation, BACI score affected by small differences
Low deforestation high variability high uncertainty
Tanzania_5: Peak in control & intervention in after period. poor performance? REDD+ not addressing big event drivers
Tanzania_6: Peak in control in before period
Brazil_2: poor score, but decrease in deforestation. Decrease is larger in control area
No other explanation found yet may be “true” poor performance.
Control areas are necessary to put changes in forest into context