SlideShare a Scribd company logo

Introduction of impact evaluation: What is it and how is it done?

Presented by Colas Chervier (CIRAD) at "Workshop on impact evaluation methods and research collaboration kick-off", Samarinda, Indonesia, on 10 October 2022

1 of 15
Download to read offline
Counterfactual impact
evaluation
Introduction
Colas Chervier
CIRAD
colas.chervier@cirad.fr
1
Content
• Session 1.
• What questions can these methods help address?
• What are the main features of these methods and why use them?
• Session 2.
• What is a counterfactual?
• What are the specific features of these methods?
• How to implement them in practice (steps)?
• Presentation of an example of an impact assessment study
2
The policy-relevant questions that these
methods can help address.
3
Examples of previous research work (1/2)
[Peru] In the 3 years between surveys, we observed a severe decline in forest revenue.
However, by using a BACI study design and matching, we show that this decrease was
not caused by the REDD+ interventions. Thus, REDD+ “did no harm” to local people, at
least in terms of forest revenues (Solis et al, 2021)
[Brazil] We find significant but small additional conservation effects from the
implementation of the PES program. Notwithstanding, treatment effects are relatively
larger in areas with higher deforestation pressure and higher potential agricultural
income (Cisneros et al, 2022)
[Brazil] We find that it is more effective to locate parks and payments away from each
other, rather than in the same location or near each other. (Robalino et al. 2015)
4
Examples of previous research work (2/2)
[Brazil] Results suggest that the accepted methodologies for quantifying carbon
credits overstate impacts on avoided deforestation and climate change
mitigation.(West et al. 2020)
[Indonesia] Contrary to the objective of the program, community titles aimed at
conservation did not decrease deforestation; if anything, they tended to increase forest
loss. In contrast, community titles in zones aimed at timber production decreased
deforestation, albeit from higher baseline forest loss rates. (Kraus et al. 2021)
[Nepal] Our results indicate that CFM has, on average, contributed to significant net
reductions in both poverty and deforestation across Nepal, and that CFM increases the
likelihood of win–win outcomes. We also find that the estimated reduced deforestation
impacts of community forests are lower where baseline poverty levels are high, and
greater where community forests are larger and have existed longer (Oldekop 2021).
5
Example of questions applied to community forests (1/2)
Directly related to the measure of impact…
• Do community forests have an additional effect on deforestation and
community incomes?
• Does CF effectiveness depends on the context (e.g. in a REDD+ area vs.
outside it), characteristics of the beneficiaries or the type of CF scheme?
• Are community forestry schemes more effective when combined with other
interventions?
• Is it possible to achieve win-win outcomes through CF?
• Etc.
6
Example of questions applied to community forests (2/2)
… and beyond
• Is CF worth replicating and if so, where?
• By putting the measured impact into perspective with the
costs incurred: is the CF model an efficient approach (for
example compared to other management methods)?
7
Definition and justification of
counterfactual impact evaluation
8
Overall definition
• Impact evaluation is a quantitative assessment of
the extent to which an intervention affects
outcomes
• The proper analysis of impact requires a
counterfactual, i.e. a control group representing
what those outcomes would have been in the
absence of the intervention
→ By comparing counterfactual and intervention
group, we can say how many hectares of forest have
been saved or by how many times the income of a
group of individuals has been increased by a program,
and if this result if significant
9
Differences with more widespread methods
• Monitoring tools
• Following a number of indicators over time in
program area
• E.g. LTKL monitoring tool →
• Different goal :does not aim to provide
information about if an intervention/policy
affects indicators measured
• Carbon sequestration scenarios
• Based on projections of historical trends in
target areas
• E.g. EK or any other forest carbon projects →
• Biased: changes cannot be attributed to
REDD+ interventions as lots of changes in the
context can influence deforestation
ERPD - FCPF
10
Biases associated with widespread methods
Before/after comparison
• Ex. carbon credits based on historical trends.
• Time-varying conditions that influence the target
outcome (e.g. change in the price of agricultural
products)
« Simple » with/without comparison
• Selection bias, there are initial differences
between control group and intervention (non-
random placement) that influence the results
Before/after PES
Control
Intervention (PES)
11
Introduction to quasi-experimental
methods and data needs
12
Definition of quasi-experimental methods
• Statistical methods that help identifies a control
group (e.g. households or forest plots that did not
receive the program) that represent a good
counterfactual
• Allows controlling for the main biases
• Applied in cases where the target area of the
intervention has already been selected (this
decision cannot be influenced)
• Specific data needs : baseline data (before the
program starts) & in "non-project" areas
13
Statistical methods to get rid of these biases
• Matching
• Selects two "similar" groups based on observable characteristics
(i.e. for which we have data)
• Double difference
• compares differences in outcomes over time between a
population participating in a program and one not participating.
• Often associated with matching
• Synthetic control method
• creates a control obtained as a weighted combination of
control units.
• Weights are assigned to control units so that their
combination is as close as possible to the treated unit's and
minimize differences in pre-treatment outcomes.
14
Conclusion day 1
• Provide answers to a wide range of questions relating to the
effectiveness of policies with a view to their
improvement/replication.
• Because these methods identify robust control groups, allow
to have a precise quantified measure free from bias.
• Specific data needs, especially pre-intervention data.
15

Recommended

Quasi-experimental impact evaluation methods: an introduction
Quasi-experimental impact evaluation methods: an introductionQuasi-experimental impact evaluation methods: an introduction
Quasi-experimental impact evaluation methods: an introductionCIFOR-ICRAF
 
Effective Useful Life and Persistence for Behavioral Programs and Framing the...
Effective Useful Life and Persistence for Behavioral Programs and Framing the...Effective Useful Life and Persistence for Behavioral Programs and Framing the...
Effective Useful Life and Persistence for Behavioral Programs and Framing the...Opinion Dynamics
 
Evaluating the impacts of REDD+ interventions on forests and people
Evaluating the impacts of REDD+ interventions on forests and peopleEvaluating the impacts of REDD+ interventions on forests and people
Evaluating the impacts of REDD+ interventions on forests and peopleCIFOR-ICRAF
 
Towards a global typology of forest policies and measures
Towards a global typology of forest policies and measuresTowards a global typology of forest policies and measures
Towards a global typology of forest policies and measuresCIFOR-ICRAF
 
Moving from readiness to performance-based payments
Moving from readiness to performance-based paymentsMoving from readiness to performance-based payments
Moving from readiness to performance-based paymentsCIFOR-ICRAF
 
Can REDD+ achieve its goals? Insights from research on forests and climate in...
Can REDD+ achieve its goals? Insights from research on forests and climate in...Can REDD+ achieve its goals? Insights from research on forests and climate in...
Can REDD+ achieve its goals? Insights from research on forests and climate in...CIFOR-ICRAF
 

More Related Content

Similar to Introduction of impact evaluation: What is it and how is it done?

Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...
Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...
Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...CIFOR-ICRAF
 
CIFOR and Global Comparative Study on REDD+
CIFOR and Global Comparative Study on REDD+CIFOR and Global Comparative Study on REDD+
CIFOR and Global Comparative Study on REDD+CIFOR-ICRAF
 
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...Prabhakar SVRK
 
Effectiveness of REDD+: Meta-study results
Effectiveness of REDD+:  Meta-study resultsEffectiveness of REDD+:  Meta-study results
Effectiveness of REDD+: Meta-study resultsCIFOR-ICRAF
 
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...CIFOR-ICRAF
 
Social Forestry & the Paris agreement: Lessons for benefit sharing
Social Forestry & the Paris agreement: Lessons for benefit sharingSocial Forestry & the Paris agreement: Lessons for benefit sharing
Social Forestry & the Paris agreement: Lessons for benefit sharingCIFOR-ICRAF
 
Performance assessment of 23 REDD+ initiatives: A tree cover change perspective
Performance assessment of 23 REDD+ initiatives: A tree cover change perspectivePerformance assessment of 23 REDD+ initiatives: A tree cover change perspective
Performance assessment of 23 REDD+ initiatives: A tree cover change perspectiveCIFOR-ICRAF
 
Transforming REDD+ lessons learned and way forward
Transforming REDD+ lessons learned and way forwardTransforming REDD+ lessons learned and way forward
Transforming REDD+ lessons learned and way forwardCIFOR-ICRAF
 
Towards green budgeting: a roadmap -- Juliane Jansen, OECD
Towards green budgeting: a roadmap -- Juliane Jansen, OECDTowards green budgeting: a roadmap -- Juliane Jansen, OECD
Towards green budgeting: a roadmap -- Juliane Jansen, OECDOECD Governance
 
Successful Adaptation to Climate Change
Successful Adaptation to Climate ChangeSuccessful Adaptation to Climate Change
Successful Adaptation to Climate Changevishvjeet tholia
 
This is the subject of Business Ethics and Corporate Social Responsi.pdf
This is the subject of Business Ethics and Corporate Social Responsi.pdfThis is the subject of Business Ethics and Corporate Social Responsi.pdf
This is the subject of Business Ethics and Corporate Social Responsi.pdfvicky309441
 
Impact prediction, evaluation and mitigation in eia
Impact prediction, evaluation and mitigation in eiaImpact prediction, evaluation and mitigation in eia
Impact prediction, evaluation and mitigation in eiaMizanur R. Shohel
 
Cost Effectiveness Analysis
Cost Effectiveness AnalysisCost Effectiveness Analysis
Cost Effectiveness Analysisclearsateam
 
Global typology of policies to reduce deforestation
Global typology of policies to reduce deforestationGlobal typology of policies to reduce deforestation
Global typology of policies to reduce deforestationCIFOR-ICRAF
 
Day 2: Prioritisation of Adaptation Options
Day 2: Prioritisation of Adaptation OptionsDay 2: Prioritisation of Adaptation Options
Day 2: Prioritisation of Adaptation OptionsNAP Global Network
 
The REDD+ Policy Arena: where are the bridges and brokers ?
 The REDD+ Policy Arena: where are the bridges and brokers ? The REDD+ Policy Arena: where are the bridges and brokers ?
The REDD+ Policy Arena: where are the bridges and brokers ?CIFOR-ICRAF
 
Global Comparative Study on REDD+: Inputs for ASEAN region
Global Comparative Study on REDD+: Inputs for ASEAN regionGlobal Comparative Study on REDD+: Inputs for ASEAN region
Global Comparative Study on REDD+: Inputs for ASEAN regionCIFOR-ICRAF
 
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific Region
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific RegionEvidence-Based Forestry: Approaches and Results in the Asia-Pacific Region
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific RegionCIFOR-ICRAF
 

Similar to Introduction of impact evaluation: What is it and how is it done? (20)

Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...
Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...
Benefit and burden sharing in forest conservation and REDD+: A conceptual fra...
 
CIFOR and Global Comparative Study on REDD+
CIFOR and Global Comparative Study on REDD+CIFOR and Global Comparative Study on REDD+
CIFOR and Global Comparative Study on REDD+
 
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...
Adaptation Metrics: Community Based Measuring And Prioritizing Adaptation Act...
 
Effectiveness of REDD+: Meta-study results
Effectiveness of REDD+:  Meta-study resultsEffectiveness of REDD+:  Meta-study results
Effectiveness of REDD+: Meta-study results
 
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...
The State of Jurisdictional Sustainability: Synthesis for Practitioners and P...
 
Social Forestry & the Paris agreement: Lessons for benefit sharing
Social Forestry & the Paris agreement: Lessons for benefit sharingSocial Forestry & the Paris agreement: Lessons for benefit sharing
Social Forestry & the Paris agreement: Lessons for benefit sharing
 
Oecd KE4CAP session 1_may 11
Oecd KE4CAP session 1_may 11Oecd KE4CAP session 1_may 11
Oecd KE4CAP session 1_may 11
 
Performance assessment of 23 REDD+ initiatives: A tree cover change perspective
Performance assessment of 23 REDD+ initiatives: A tree cover change perspectivePerformance assessment of 23 REDD+ initiatives: A tree cover change perspective
Performance assessment of 23 REDD+ initiatives: A tree cover change perspective
 
Transforming REDD+ lessons learned and way forward
Transforming REDD+ lessons learned and way forwardTransforming REDD+ lessons learned and way forward
Transforming REDD+ lessons learned and way forward
 
Towards green budgeting: a roadmap -- Juliane Jansen, OECD
Towards green budgeting: a roadmap -- Juliane Jansen, OECDTowards green budgeting: a roadmap -- Juliane Jansen, OECD
Towards green budgeting: a roadmap -- Juliane Jansen, OECD
 
Successful Adaptation to Climate Change
Successful Adaptation to Climate ChangeSuccessful Adaptation to Climate Change
Successful Adaptation to Climate Change
 
This is the subject of Business Ethics and Corporate Social Responsi.pdf
This is the subject of Business Ethics and Corporate Social Responsi.pdfThis is the subject of Business Ethics and Corporate Social Responsi.pdf
This is the subject of Business Ethics and Corporate Social Responsi.pdf
 
Impact prediction, evaluation and mitigation in eia
Impact prediction, evaluation and mitigation in eiaImpact prediction, evaluation and mitigation in eia
Impact prediction, evaluation and mitigation in eia
 
Cost Effectiveness Analysis
Cost Effectiveness AnalysisCost Effectiveness Analysis
Cost Effectiveness Analysis
 
Global typology of policies to reduce deforestation
Global typology of policies to reduce deforestationGlobal typology of policies to reduce deforestation
Global typology of policies to reduce deforestation
 
Day 2: Prioritisation of Adaptation Options
Day 2: Prioritisation of Adaptation OptionsDay 2: Prioritisation of Adaptation Options
Day 2: Prioritisation of Adaptation Options
 
The REDD+ Policy Arena: where are the bridges and brokers ?
 The REDD+ Policy Arena: where are the bridges and brokers ? The REDD+ Policy Arena: where are the bridges and brokers ?
The REDD+ Policy Arena: where are the bridges and brokers ?
 
Balancing Acceleration and Systemic Impact
Balancing Acceleration and Systemic ImpactBalancing Acceleration and Systemic Impact
Balancing Acceleration and Systemic Impact
 
Global Comparative Study on REDD+: Inputs for ASEAN region
Global Comparative Study on REDD+: Inputs for ASEAN regionGlobal Comparative Study on REDD+: Inputs for ASEAN region
Global Comparative Study on REDD+: Inputs for ASEAN region
 
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific Region
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific RegionEvidence-Based Forestry: Approaches and Results in the Asia-Pacific Region
Evidence-Based Forestry: Approaches and Results in the Asia-Pacific Region
 

More from CIFOR-ICRAF

Cenários de desmatamento para o Brasil Plataforma REDD BRASIL
Cenários de desmatamento para o Brasil Plataforma REDD BRASILCenários de desmatamento para o Brasil Plataforma REDD BRASIL
Cenários de desmatamento para o Brasil Plataforma REDD BRASILCIFOR-ICRAF
 
Research priorities in seagrass ecosystems to enhance blue carbon as Natural...
Research priorities in seagrass ecosystems to enhance blue carbon  as Natural...Research priorities in seagrass ecosystems to enhance blue carbon  as Natural...
Research priorities in seagrass ecosystems to enhance blue carbon as Natural...CIFOR-ICRAF
 
The role of mangrove blue carbon research to support national climate mitigat...
The role of mangrove blue carbon research to support national climate mitigat...The role of mangrove blue carbon research to support national climate mitigat...
The role of mangrove blue carbon research to support national climate mitigat...CIFOR-ICRAF
 
Mangrove restoration – where to restore & can we monitor? case for Indonesia
Mangrove restoration – where to restore & can we monitor? case for IndonesiaMangrove restoration – where to restore & can we monitor? case for Indonesia
Mangrove restoration – where to restore & can we monitor? case for IndonesiaCIFOR-ICRAF
 
Trees, forests and landscapes for people and the planet
Trees, forests and landscapes for people and the planetTrees, forests and landscapes for people and the planet
Trees, forests and landscapes for people and the planetCIFOR-ICRAF
 
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...CIFOR-ICRAF
 
Academic Framework of Blue Carbon Ecosystems (BCE)
Academic Framework of Blue Carbon Ecosystems (BCE)Academic Framework of Blue Carbon Ecosystems (BCE)
Academic Framework of Blue Carbon Ecosystems (BCE)CIFOR-ICRAF
 
Kebijakan Blue Carbon dalam operasionalisasi global
Kebijakan Blue Carbon dalam operasionalisasi globalKebijakan Blue Carbon dalam operasionalisasi global
Kebijakan Blue Carbon dalam operasionalisasi globalCIFOR-ICRAF
 
Blue Carbon Dialogue: From science-based policy to implementation
Blue Carbon Dialogue: From science-based policy to implementationBlue Carbon Dialogue: From science-based policy to implementation
Blue Carbon Dialogue: From science-based policy to implementationCIFOR-ICRAF
 
The evolution of REDD+ finance in DRC across the years: The rise of private R...
The evolution of REDD+ finance in DRC across the years: The rise of private R...The evolution of REDD+ finance in DRC across the years: The rise of private R...
The evolution of REDD+ finance in DRC across the years: The rise of private R...CIFOR-ICRAF
 
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...CIFOR-ICRAF
 
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​Simulation of Indonesian Palm Oil Sustainability (SIPOS)​
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​CIFOR-ICRAF
 
Tantangan penerapan pendekatan yuridiksi dari tinjauan rantai nilai industri...
Tantangan penerapan pendekatan yuridiksi  dari tinjauan rantai nilai industri...Tantangan penerapan pendekatan yuridiksi  dari tinjauan rantai nilai industri...
Tantangan penerapan pendekatan yuridiksi dari tinjauan rantai nilai industri...CIFOR-ICRAF
 
Peluang dan tantangan untuk rantai suplai sawit berkelanjutan
Peluang dan tantangan untuk rantai suplai sawit berkelanjutanPeluang dan tantangan untuk rantai suplai sawit berkelanjutan
Peluang dan tantangan untuk rantai suplai sawit berkelanjutanCIFOR-ICRAF
 
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...CIFOR-ICRAF
 
Balance de los tres años de las plataformas de ciencia y políticas públicas d...
Balance de los tres años de las plataformas de ciencia y políticas públicas d...Balance de los tres años de las plataformas de ciencia y políticas públicas d...
Balance de los tres años de las plataformas de ciencia y políticas públicas d...CIFOR-ICRAF
 
The evolution of REDD+ finance in Peru across the years
The evolution of REDD+ finance in Peru across the yearsThe evolution of REDD+ finance in Peru across the years
The evolution of REDD+ finance in Peru across the yearsCIFOR-ICRAF
 
WP4.2 – Escenarios de deforestación para construir políticas públicas
WP4.2 – Escenarios de deforestación para construir políticas públicasWP4.2 – Escenarios de deforestación para construir políticas públicas
WP4.2 – Escenarios de deforestación para construir políticas públicasCIFOR-ICRAF
 
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...CIFOR-ICRAF
 
Comunicar la ciencia para mejores políticas públicas
Comunicar la ciencia para mejores políticas públicasComunicar la ciencia para mejores políticas públicas
Comunicar la ciencia para mejores políticas públicasCIFOR-ICRAF
 

More from CIFOR-ICRAF (20)

Cenários de desmatamento para o Brasil Plataforma REDD BRASIL
Cenários de desmatamento para o Brasil Plataforma REDD BRASILCenários de desmatamento para o Brasil Plataforma REDD BRASIL
Cenários de desmatamento para o Brasil Plataforma REDD BRASIL
 
Research priorities in seagrass ecosystems to enhance blue carbon as Natural...
Research priorities in seagrass ecosystems to enhance blue carbon  as Natural...Research priorities in seagrass ecosystems to enhance blue carbon  as Natural...
Research priorities in seagrass ecosystems to enhance blue carbon as Natural...
 
The role of mangrove blue carbon research to support national climate mitigat...
The role of mangrove blue carbon research to support national climate mitigat...The role of mangrove blue carbon research to support national climate mitigat...
The role of mangrove blue carbon research to support national climate mitigat...
 
Mangrove restoration – where to restore & can we monitor? case for Indonesia
Mangrove restoration – where to restore & can we monitor? case for IndonesiaMangrove restoration – where to restore & can we monitor? case for Indonesia
Mangrove restoration – where to restore & can we monitor? case for Indonesia
 
Trees, forests and landscapes for people and the planet
Trees, forests and landscapes for people and the planetTrees, forests and landscapes for people and the planet
Trees, forests and landscapes for people and the planet
 
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...
Blue Carbon Dialogue: Bridging Science and Policy for Effective Implementatio...
 
Academic Framework of Blue Carbon Ecosystems (BCE)
Academic Framework of Blue Carbon Ecosystems (BCE)Academic Framework of Blue Carbon Ecosystems (BCE)
Academic Framework of Blue Carbon Ecosystems (BCE)
 
Kebijakan Blue Carbon dalam operasionalisasi global
Kebijakan Blue Carbon dalam operasionalisasi globalKebijakan Blue Carbon dalam operasionalisasi global
Kebijakan Blue Carbon dalam operasionalisasi global
 
Blue Carbon Dialogue: From science-based policy to implementation
Blue Carbon Dialogue: From science-based policy to implementationBlue Carbon Dialogue: From science-based policy to implementation
Blue Carbon Dialogue: From science-based policy to implementation
 
The evolution of REDD+ finance in DRC across the years: The rise of private R...
The evolution of REDD+ finance in DRC across the years: The rise of private R...The evolution of REDD+ finance in DRC across the years: The rise of private R...
The evolution of REDD+ finance in DRC across the years: The rise of private R...
 
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...
Diskusi penguatan Rencana Aksi Nasional kelapa sawit berkelanjutan melalui Te...
 
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​Simulation of Indonesian Palm Oil Sustainability (SIPOS)​
Simulation of Indonesian Palm Oil Sustainability (SIPOS)​
 
Tantangan penerapan pendekatan yuridiksi dari tinjauan rantai nilai industri...
Tantangan penerapan pendekatan yuridiksi  dari tinjauan rantai nilai industri...Tantangan penerapan pendekatan yuridiksi  dari tinjauan rantai nilai industri...
Tantangan penerapan pendekatan yuridiksi dari tinjauan rantai nilai industri...
 
Peluang dan tantangan untuk rantai suplai sawit berkelanjutan
Peluang dan tantangan untuk rantai suplai sawit berkelanjutanPeluang dan tantangan untuk rantai suplai sawit berkelanjutan
Peluang dan tantangan untuk rantai suplai sawit berkelanjutan
 
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...
Pengenalan Riset: Perluasan Pendekatan Yurisdiksi pada Sektor Kelapa Sawit di...
 
Balance de los tres años de las plataformas de ciencia y políticas públicas d...
Balance de los tres años de las plataformas de ciencia y políticas públicas d...Balance de los tres años de las plataformas de ciencia y políticas públicas d...
Balance de los tres años de las plataformas de ciencia y políticas públicas d...
 
The evolution of REDD+ finance in Peru across the years
The evolution of REDD+ finance in Peru across the yearsThe evolution of REDD+ finance in Peru across the years
The evolution of REDD+ finance in Peru across the years
 
WP4.2 – Escenarios de deforestación para construir políticas públicas
WP4.2 – Escenarios de deforestación para construir políticas públicasWP4.2 – Escenarios de deforestación para construir políticas públicas
WP4.2 – Escenarios de deforestación para construir políticas públicas
 
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...
¿Cómo la ciencia forestal ayuda a construir la política pública climática? La...
 
Comunicar la ciencia para mejores políticas públicas
Comunicar la ciencia para mejores políticas públicasComunicar la ciencia para mejores políticas públicas
Comunicar la ciencia para mejores políticas públicas
 

Recently uploaded

Role of soil aeration for crop growth and its development.ppt
Role of soil aeration for crop growth and its development.pptRole of soil aeration for crop growth and its development.ppt
Role of soil aeration for crop growth and its development.pptrangaswamyranga8341
 
That tree related to god(wild's gravity).pdf
That tree related to god(wild's gravity).pdfThat tree related to god(wild's gravity).pdf
That tree related to god(wild's gravity).pdfdrsk203
 
Coffee Capsules - Recycling Campaign for Emploees
Coffee Capsules - Recycling Campaign for EmploeesCoffee Capsules - Recycling Campaign for Emploees
Coffee Capsules - Recycling Campaign for EmploeesDébora Salviato
 
CSR01P2 The foundations of CSR Part 2 Planet
CSR01P2 The foundations of CSR Part 2 PlanetCSR01P2 The foundations of CSR Part 2 Planet
CSR01P2 The foundations of CSR Part 2 PlanetMatthieu Bruckert
 
fish starting coming out of the water.pdf
fish starting coming out of the water.pdffish starting coming out of the water.pdf
fish starting coming out of the water.pdfbraydenstoch777
 
Invasives Management-Big Picture-MCD.pdf
Invasives Management-Big Picture-MCD.pdfInvasives Management-Big Picture-MCD.pdf
Invasives Management-Big Picture-MCD.pdfadmin944664
 
National Safety Week 2024 ppt by Neeraj Rai.pptx
National Safety Week 2024 ppt by Neeraj Rai.pptxNational Safety Week 2024 ppt by Neeraj Rai.pptx
National Safety Week 2024 ppt by Neeraj Rai.pptxNeeraj Kumar Rai
 
Scope 3 Bootcamp S3 history - Marty Spitzer.pdf
Scope 3 Bootcamp S3 history - Marty Spitzer.pdfScope 3 Bootcamp S3 history - Marty Spitzer.pdf
Scope 3 Bootcamp S3 history - Marty Spitzer.pdfGreenBiz Group
 
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...GreenBiz Group
 
Incorporating Water and Energy Data into Your Sustainability and ESG Goals
Incorporating Water and Energy Data into Your Sustainability and ESG GoalsIncorporating Water and Energy Data into Your Sustainability and ESG Goals
Incorporating Water and Energy Data into Your Sustainability and ESG GoalsGreenBiz Group
 
Why collaborating with your communications team is key to driving progress
Why collaborating with your communications team is key to driving progressWhy collaborating with your communications team is key to driving progress
Why collaborating with your communications team is key to driving progressGreenBiz Group
 
Types of Natural Vegetation in India PPT
Types of Natural Vegetation in India PPTTypes of Natural Vegetation in India PPT
Types of Natural Vegetation in India PPTrakesh64416
 
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...GreenBiz Group
 
Partnering for Resilient Forests - Climate, Fires, and Communities
Partnering for Resilient Forests - Climate, Fires, and CommunitiesPartnering for Resilient Forests - Climate, Fires, and Communities
Partnering for Resilient Forests - Climate, Fires, and CommunitiesGreenBiz Group
 

Recently uploaded (20)

Role of soil aeration for crop growth and its development.ppt
Role of soil aeration for crop growth and its development.pptRole of soil aeration for crop growth and its development.ppt
Role of soil aeration for crop growth and its development.ppt
 
How to do note: Formulating a CR-CVP - Lessons from Brazil
How to do note: Formulating a CR-CVP - Lessons from BrazilHow to do note: Formulating a CR-CVP - Lessons from Brazil
How to do note: Formulating a CR-CVP - Lessons from Brazil
 
That tree related to god(wild's gravity).pdf
That tree related to god(wild's gravity).pdfThat tree related to god(wild's gravity).pdf
That tree related to god(wild's gravity).pdf
 
Coffee Capsules - Recycling Campaign for Emploees
Coffee Capsules - Recycling Campaign for EmploeesCoffee Capsules - Recycling Campaign for Emploees
Coffee Capsules - Recycling Campaign for Emploees
 
CSR01P2 The foundations of CSR Part 2 Planet
CSR01P2 The foundations of CSR Part 2 PlanetCSR01P2 The foundations of CSR Part 2 Planet
CSR01P2 The foundations of CSR Part 2 Planet
 
fish starting coming out of the water.pdf
fish starting coming out of the water.pdffish starting coming out of the water.pdf
fish starting coming out of the water.pdf
 
Smiling Faces Photo Reel 2024 Annual Meeting
Smiling Faces Photo Reel 2024 Annual MeetingSmiling Faces Photo Reel 2024 Annual Meeting
Smiling Faces Photo Reel 2024 Annual Meeting
 
Invasives Management-Big Picture-MCD.pdf
Invasives Management-Big Picture-MCD.pdfInvasives Management-Big Picture-MCD.pdf
Invasives Management-Big Picture-MCD.pdf
 
National Safety Week 2024 ppt by Neeraj Rai.pptx
National Safety Week 2024 ppt by Neeraj Rai.pptxNational Safety Week 2024 ppt by Neeraj Rai.pptx
National Safety Week 2024 ppt by Neeraj Rai.pptx
 
Scope 3 Bootcamp S3 history - Marty Spitzer.pdf
Scope 3 Bootcamp S3 history - Marty Spitzer.pdfScope 3 Bootcamp S3 history - Marty Spitzer.pdf
Scope 3 Bootcamp S3 history - Marty Spitzer.pdf
 
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...
Meeting Your Energy Sustainability Goals - A Case Study with Top Industry Exp...
 
Incorporating Water and Energy Data into Your Sustainability and ESG Goals
Incorporating Water and Energy Data into Your Sustainability and ESG GoalsIncorporating Water and Energy Data into Your Sustainability and ESG Goals
Incorporating Water and Energy Data into Your Sustainability and ESG Goals
 
Why collaborating with your communications team is key to driving progress
Why collaborating with your communications team is key to driving progressWhy collaborating with your communications team is key to driving progress
Why collaborating with your communications team is key to driving progress
 
Types of Natural Vegetation in India PPT
Types of Natural Vegetation in India PPTTypes of Natural Vegetation in India PPT
Types of Natural Vegetation in India PPT
 
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...
The Power of Public-Private Partnerships: Cross Sector Collaboration for Futu...
 
Partnering for Resilient Forests - Climate, Fires, and Communities
Partnering for Resilient Forests - Climate, Fires, and CommunitiesPartnering for Resilient Forests - Climate, Fires, and Communities
Partnering for Resilient Forests - Climate, Fires, and Communities
 
THRIVE Project - Towards Holistic Resilience in Vulnerable Environments
THRIVE Project - Towards Holistic Resilience in Vulnerable EnvironmentsTHRIVE Project - Towards Holistic Resilience in Vulnerable Environments
THRIVE Project - Towards Holistic Resilience in Vulnerable Environments
 
Indigenous Peoples, and local communities: Why the distinction is important
Indigenous Peoples, and local communities: Why the distinction is importantIndigenous Peoples, and local communities: Why the distinction is important
Indigenous Peoples, and local communities: Why the distinction is important
 
COSME Project - Community Engagement Experiences & Best Practice
COSME Project - Community Engagement Experiences & Best PracticeCOSME Project - Community Engagement Experiences & Best Practice
COSME Project - Community Engagement Experiences & Best Practice
 
Climate Resilience Community Village Plans - Lessons from Sudan
Climate Resilience Community Village Plans - Lessons from SudanClimate Resilience Community Village Plans - Lessons from Sudan
Climate Resilience Community Village Plans - Lessons from Sudan
 

Introduction of impact evaluation: What is it and how is it done?

  • 2. Content • Session 1. • What questions can these methods help address? • What are the main features of these methods and why use them? • Session 2. • What is a counterfactual? • What are the specific features of these methods? • How to implement them in practice (steps)? • Presentation of an example of an impact assessment study 2
  • 3. The policy-relevant questions that these methods can help address. 3
  • 4. Examples of previous research work (1/2) [Peru] In the 3 years between surveys, we observed a severe decline in forest revenue. However, by using a BACI study design and matching, we show that this decrease was not caused by the REDD+ interventions. Thus, REDD+ “did no harm” to local people, at least in terms of forest revenues (Solis et al, 2021) [Brazil] We find significant but small additional conservation effects from the implementation of the PES program. Notwithstanding, treatment effects are relatively larger in areas with higher deforestation pressure and higher potential agricultural income (Cisneros et al, 2022) [Brazil] We find that it is more effective to locate parks and payments away from each other, rather than in the same location or near each other. (Robalino et al. 2015) 4
  • 5. Examples of previous research work (2/2) [Brazil] Results suggest that the accepted methodologies for quantifying carbon credits overstate impacts on avoided deforestation and climate change mitigation.(West et al. 2020) [Indonesia] Contrary to the objective of the program, community titles aimed at conservation did not decrease deforestation; if anything, they tended to increase forest loss. In contrast, community titles in zones aimed at timber production decreased deforestation, albeit from higher baseline forest loss rates. (Kraus et al. 2021) [Nepal] Our results indicate that CFM has, on average, contributed to significant net reductions in both poverty and deforestation across Nepal, and that CFM increases the likelihood of win–win outcomes. We also find that the estimated reduced deforestation impacts of community forests are lower where baseline poverty levels are high, and greater where community forests are larger and have existed longer (Oldekop 2021). 5
  • 6. Example of questions applied to community forests (1/2) Directly related to the measure of impact… • Do community forests have an additional effect on deforestation and community incomes? • Does CF effectiveness depends on the context (e.g. in a REDD+ area vs. outside it), characteristics of the beneficiaries or the type of CF scheme? • Are community forestry schemes more effective when combined with other interventions? • Is it possible to achieve win-win outcomes through CF? • Etc. 6
  • 7. Example of questions applied to community forests (2/2) … and beyond • Is CF worth replicating and if so, where? • By putting the measured impact into perspective with the costs incurred: is the CF model an efficient approach (for example compared to other management methods)? 7
  • 8. Definition and justification of counterfactual impact evaluation 8
  • 9. Overall definition • Impact evaluation is a quantitative assessment of the extent to which an intervention affects outcomes • The proper analysis of impact requires a counterfactual, i.e. a control group representing what those outcomes would have been in the absence of the intervention → By comparing counterfactual and intervention group, we can say how many hectares of forest have been saved or by how many times the income of a group of individuals has been increased by a program, and if this result if significant 9
  • 10. Differences with more widespread methods • Monitoring tools • Following a number of indicators over time in program area • E.g. LTKL monitoring tool → • Different goal :does not aim to provide information about if an intervention/policy affects indicators measured • Carbon sequestration scenarios • Based on projections of historical trends in target areas • E.g. EK or any other forest carbon projects → • Biased: changes cannot be attributed to REDD+ interventions as lots of changes in the context can influence deforestation ERPD - FCPF 10
  • 11. Biases associated with widespread methods Before/after comparison • Ex. carbon credits based on historical trends. • Time-varying conditions that influence the target outcome (e.g. change in the price of agricultural products) « Simple » with/without comparison • Selection bias, there are initial differences between control group and intervention (non- random placement) that influence the results Before/after PES Control Intervention (PES) 11
  • 13. Definition of quasi-experimental methods • Statistical methods that help identifies a control group (e.g. households or forest plots that did not receive the program) that represent a good counterfactual • Allows controlling for the main biases • Applied in cases where the target area of the intervention has already been selected (this decision cannot be influenced) • Specific data needs : baseline data (before the program starts) & in "non-project" areas 13
  • 14. Statistical methods to get rid of these biases • Matching • Selects two "similar" groups based on observable characteristics (i.e. for which we have data) • Double difference • compares differences in outcomes over time between a population participating in a program and one not participating. • Often associated with matching • Synthetic control method • creates a control obtained as a weighted combination of control units. • Weights are assigned to control units so that their combination is as close as possible to the treated unit's and minimize differences in pre-treatment outcomes. 14
  • 15. Conclusion day 1 • Provide answers to a wide range of questions relating to the effectiveness of policies with a view to their improvement/replication. • Because these methods identify robust control groups, allow to have a precise quantified measure free from bias. • Specific data needs, especially pre-intervention data. 15