Metrics for CSA: increasing programming effectiveness and outcome tracking

Osana Bonilla-Findji, Andy Jarvis, Marie Quinney,Todd Rosenstock,
Christine Lamanna, Lini Wollemberg, Meryl Richard, Sonja Vermulen,
Dhanush Dinesh
Metrics for CSA
increasing programming
effectiveness and outcome tracking
GACSA’s joint workshop ‘Metrics for Climate-Smart Agriculture’
Rome, FAO HQ, June 15th 2017
ccafs.cgiar.org

CSA: Agriculture and Food systems that
deliver outcomes
Issues with metrics
reduced
emissions
resilience/
adaptation
food
security/
productivity
nutrition
outcomes
costly to
quantify
agreement
on core
indicators
 CSA is emerging as a mechanism for coherent and coordinated action
 Measuring processes or Outcomes?
Standard of measurements
That show progress

500 million farmers globally
25 million by 2025
Global momentum still building for CSA
6 million by 2021

CSA Complexity & Uncertainty
Complexity stemming from diversity of:
1. Interventions (micro to macro level)
2. Farming systems and households
3. Objectives/ potential outcomes of success (ranging from increased soil carbon to
maternal dietary diversity), with trade-offs possible.
Uncertainty and Context Specificity:
Lack of information and data
• Outcomes = CSA practices x social-environmental
context
• Need to identify where biggest wins occur across
landscapes/regions
• Gradient of CSA-ness
• CSA today, may not be CSA tomorrow

Review CSA metrics : What's there ?
High demand for useful/actionable tools to design, prioritize and
monitor useful interventions and investments: Paris Workshop 2015.
0
50
100
150
200
Indicators types (%)
Readiness Process Outcome
0
50
100
150
200
Indicators relative contribution to CSA
objectives (%)
Productivity Adaptation Mitigation
- Most agencies covered all the types of indicators but focused on Process and Outcomes
- Mostly covering Adaptation pillar (81%) and Productivity (40%).
Adaptation => Guidance needed
- Largely geared towards risk management
- Multidimensional nature of resilience (economic, financial social) and
capacities (absortive, adaptive, transformative) often not factored
- Measure potential adaptation rather than actual adaptation
- Lack ability to track changes over time (e.g lean season)
Mitigation
Significant lack of indicators related to mitigation outcomes
Global CSA indicators database:
 378 relevant indicators (USAID, DFID, World Bank, IFAD, GIZ, FAO, CCAFS)
 Revisited thought the “CSA lenses” and Classified
* Readiness/Enabling environment
* Process (outputs)
* Outcome indicators

What to measure and how?
Some CCAFS metrics tools
CSA
Evidence
building
CSA targeting/
Programming
CSA
Measures
and
Monitoring
- Compendium of CSA (Practice)
- CSA Calculator (Farm)
- Diverse Prioritization tools and approaches
- Risk-household-option modelling (RhOMIS)
- CSA programming and Indicator Tool
- SHAMBA (Small-Holder Ag. Monitoring and
Baseline Assessment
- CCAFS Mitigation Option Tool (MOT)
$

CSA Compendium
Rosenstock et al. 2016
Systematic review
• 1,200 peer reviewed
studies for Africa
• 100,000+ comparisons to
baselines,
• 70 practices
• 55 outcome indicators
Largest meta analysis to date
♦ What is the evidence base for field interventions’ potential impact on three pillars?
 Outcomes vary (crop, farming system, agroecology…)
 There is no best-bet practice across all outcomes and
systems
 Prioritization of outcomes and tailoring is key
Integration with other initiatives: WB, USAID, World Bank,
GIZ, Care, CRS, Oxfam, World Vision, Concern, NEPAD,
COMESA, national governments in multiple African Countries
Evidence building

♦ Farm model allowing a prospective assessment of synergies &
trade-offs related to implementation of different CSA options and
outcomes that can be expected (three pillars).
Main indicators of the CSA calculator
Pilar ASAC Indicador Calculation
Productivity Caloric ratio of the farm (%) Caloric supply/Caloric demand x 100
Fodder ratio of the farm (%) Fodder supply/Fodder demand x 100
Cost benefit ratio (%) Benefit/Cost x 100
Adaptation Biodiversity index (%) Assessment based on Gobbi, J.,
Casasola, F., 2003.
Water balance (%) Water supply/water demand x 100
Nutrient balance (%) Nutrient supply/nutrient demand x 100
Mitigation Emission/Sequestration of CO2 CoolFarmTool
Measurement & Monitoring
CSA Calculator of farm performance
(Synergies and Trade-offs)
Users: Piloting ongoing in
CCAFS CSV monitoring
Evidence building

Risk-household-option modelling (RhOMIS)
http://rhomis.net/blog/
CSA Programming
 Captures information on farm productivity and
practices, nutrition, food security, gender
equity, climate and poverty
 Up to 20 important performance and welfare
indicators + key farm level drivers, livelihood
data and management decisions
 Compare changes in farming practice and
livelihoods over time, stated plans for the
future, and farmers intrinsic values and attitudes
 Identifies ‘positive deviant’ farmers
Users: tested and adapted for diverse systems in more
than 7,000 households across the global tropics
Implementations by 10 projects, 15 diverse farming
systems on 4 continents. Lutheran World Relief (Kenya);
TreeAID (northern Ghana), CONABIO (in Mexico)
Citation: Hammond, J. et al. 2017
♦ Household-level Survey and Analytical engine for characterizing,
targeting and monitoring agricultural performance and the effectiveness
of ongoing out-scaling efforts

Mitigation measurement tools
Mitigation option tool (MOT) Small Holder Agriculture
Monitoring and Baseline
Assessment (SHAMBA)
 Provides fast, accessible, and
reliable information to make
informed decisions about
emissions reductions
 Assesses GHG emissions estimates
and Ranks the most effective
mitigation options for dozens of
different crops and livestock systems
 Low input data requirements
 Runs in Excel bringing together
several empirical models
• Estimates GHG emissions or
removals resulting from a change in
land management practices
(biomass burning, plant nitrogen inputs
to soils, and fertiliser use)
• Models a baseline scenario (where
land management activities continue
as business as usual) and a CSA
intervention scenario
• Set to calculate estimates on a yearly
basis, in tonnes of CO2e per hectare.

Case study : working with the public
sector on metrics (USAID)
CSA Programming and Indicator tool (CCAFS)
https://ccafs.cgiar.org/csa-programming-and-indicator-tool#.WT7e7usrJhE
CSA Programming Measurement & Monitoring
• Provide common framework to guide for
agriculturally focused programs/donors on the
design of CSA interventions
• Provide a robust and transparent process to
examine to which extent a specific program
addresses the three CSA pillars
• Support the selection of appropriate
indicators to measure progress and monitor
impact
Productivity
Adaptation
Mitigation
To increase the effectiveness of CSA interventions we need:
 How are agencies currently addressing CSA or how they can make their future
programing process more climate-smart

Case study : working with the public
sector on metrics (USAID)
CSA Programming and Indicator tool
https://ccafs.cgiar.org/csa-programming-and-indicator-tool#.WT7e7usrJhE
CSA Indicators data base = initial input

Step 1
Questions to be
addressed & intentionality
of desired outcomes
CSA
Programming
Measurement &
Monitoring

CSA
Programming
Measurement &
Monitoring

Results summary and visualisation3
Shared frameworks it enables to
programs/ donors :
 To Assess scope
 Compares intentionality
 Support indicators identification
CSA
Programming
Measurement &
Monitoring

Case study : working with the private
sector on metrics
PRODUCTIVITY
OUTCOME: 50%
more nutritious
food available
ACTIVITYe.g.raiseyields
RESILIENCE, INCOMES
& LIVELIHOODS
OUTCOME: climate
resilient agricultural
landscapes and
farming communities
ACTIVITYe.g.skillstransfer
MITIGATION
OUTCOME: food GHG
emissions 30% lower &
land use change
emissions eliminated
ACTIVITY:e.g.haltforestconversion
Source: Vermeulen & Frid-
Nielsen, 2017

Basic framework for measurement
INPUTS &
FARMING
PROCESSING &
LOGISTICS
STORAGE &
TRANSPORT
TRADING &
PURCHASE
GLOBAL INDICATORS
(some are OUTCOME indicators and some are ACTIVITY indicators)
COMPANY INDICATORS
(some are OUTCOME indicators and some are ACTIVITY indicators)
Enabling and regulatory environment

Companies included in the analysis
Company Sub-sector
Coca-Cola Food processing
CP Foods Food processing
Diageo Food processing
DuPont Agricultural inputs
Kellogg Food processing
Monsanto Agricultural inputs
Olam Food processing
PepsiCo Food processing
Starbucks Retail
Syngenta Agricultural inputs
Tyson Foods Food processing
Unilever Food processing
Yara Agricultural inputs
Source: Vermeulen & Frid-Nielsen, 2017

Results
Productivity
• On track to produce enough food to meet the demand for 50% more
food by 2030.
• No direct evidence that this food will be equally or more nutritious
Resilience
• No relevant global data that match the indicators that companies use
for resilience.
• Need to provide quantitative information on indicators that cover both
activities (e.g. training, on-farm agroecological practices) and
outcomes (e.g. incomes, women’s share of assets and decisions).
Mitigation
• The 2030 goal of 30% emissions reductions compared to 2010 will
not be met unless the trend 2010-30 is changed.
• Companies generally reduced the intensity of their own operations,
but reporting on Scope 3 emissions is not currently pervasive enough.
Source: Vermeulen & Frid-Nielsen, 2017

Next steps/opportunities
• Building CSA metrics into regular practice
• Considering how any activity or intervention will lead to desired
outcomes for productivity, resilience and mitigation
• Road-test countries and regions provide an innovative opportunity
to test and measure how scale effects and trade-offs can be
managed
• Lot more work needs to be done – on measurement and action

Indicator overload
• SDGs - 230 indicators
• USAID 500 indicators
• UNICEF 300+
Figures and graphic courtesy
of T. Rosenstock

CSA M&E challenges
• Multi-objective complexity
• Heterogeneity, complexity, and diversity of indicators
• Maintaining simplicity and cost effectiveness
• Logistical, statistical, technical and practical challenges.
• M&E at different scales of impact
• Multi-institutional coordination
Source: Rosenstock & Corner-Dolloff, 2015

A way forward for GACSA
A common approach to metrics to CSA which takes into
account:
• Rigour Vs Simplicity
• Cost effectiveness
• Harmonisation Vs flexibility
• Role of ICT across scales
• Continual feedback for adaptive programming and monitoring

Further reading
• CSA Programming and Indicator Tool (https://ccafs.cgiar.org/csa-programming-and-indicator-tool)
• Info note on livestock MRV (http://hdl.handle.net/10568/80890)
• Measuring Progress Towards the WBCSD Statement of Ambition on Climate-Smart Agriculture: Improving
Businesses’ Ability to Trace, Measure and Monitor CSA (http://hdl.handle.net/10568/80652)
• Monitoring, evaluation and learning for CSA (https://csa.guide/csa/monitoring-evaluation-and-learning)
• National level indicators for gender, poverty, food security, nutrition and health in Climate-Smart Agriculture (CSA)
activities (https://ccafs.cgiar.org/publications/national-level-indicators-gender-poverty-food-security-nutrition-and-
health-climate#.WSh81OsrJhE)
• A Monitoring Instrument for Resilience (https://cgspace.cgiar.org/handle/10568/56757)
• Info Note: A rapid, cost-effective and flexible tool for farm household characterisation, targeting interventions and
monitoring progress towards climate-smart agriculture http://rhomis.net/blog/wp-content/uploads/2016/09/CCAFS-
info-note-VanWijkHammond-etal-final.pdf
• Info Note: Surveillance of Climate-smart Agriculture for Nutrition (SCAN): Innovations for monitoring at scale :
https://cgspace.cgiar.org/handle/10568/79903
• Journal paper: When less is more: Innovations for tracking progress toward global targets :
• Info Note: What is the scientific basis for climate-smart agriculture? https://ccafs.cgiar.org/publications/what-scientific-
basis-climate-smart-agriculture
• Evidence-based opportunities for out-scaling climate-smart agriculture in East Africa:
• CSA X-Rays: Conservation agriculture in East and Southern Africa main messages
(https://cgspace.cgiar.org/handle/10568/79900)
• CCAFS Mitigation Option Tool: https://ccafs.cgiar.org/es/mitigation-option-tool-agriculture#.WUGSU-t97IU
• SHAMBA Tool https://ccafs.cgiar.org/es/small-holder-agriculture-monitoring-and-baseline-assessment-
tool#.WUGO2ut97IU

Risks-Households-Options (RHO) integrated modeling
system
CSA Programming
Applications/users to date
Niger case study using the HH-level data from
the World Bank’s Living Standards
Measurement Survey
 Integrates available datasets
combined field level and
modeling information + CSA
evidence
♦ Three-step (cost effective) approach that
integrates information on
- HH characteristics (Survey datasets),
- risk exposure levels (Crop modeling)
- for determining best bet CSA practices
to support efficient and relevant CSA
programming
♦ Ultimately addresses Food Security under CC
♦ Supports building robust portfolio of best bet
CSA options that can be implemented, using
currently available data!
Citation: Lamanna et al 2015

Metrics for CSA: increasing programming effectiveness and outcome tracking

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Metrics for CSA: increasing programming effectiveness and outcome tracking

Similar to Metrics for CSA: increasing programming effectiveness and outcome tracking (20)

More from CCAFS | CGIAR Research Program on Climate Change, Agriculture and Food Security

More from CCAFS | CGIAR Research Program on Climate Change, Agriculture and Food Security (20)

Recently uploaded

Recently uploaded (20)

Metrics for CSA: increasing programming effectiveness and outcome tracking