Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metrics for CSA: increasing programming effectiveness and outcome tracking


Published on

Presentation by Osana Bonilla-Findji and Dhanush Dinesh at GACSA’s joint workshop on ‘Metrics for Climate-Smart Agriculture’ in Rome, FAO HQ, 15 June 2017.

Published in: Environment
  • Be the first to comment

  • Be the first to like this

Metrics for CSA: increasing programming effectiveness and outcome tracking

  1. 1. Osana Bonilla-Findji, Andy Jarvis, Marie Quinney,Todd Rosenstock, Christine Lamanna, Lini Wollemberg, Meryl Richard, Sonja Vermulen, Dhanush Dinesh Metrics for CSA increasing programming effectiveness and outcome tracking GACSA’s joint workshop ‘Metrics for Climate-Smart Agriculture’ Rome, FAO HQ, June 15th 2017
  2. 2. CSA: Agriculture and Food systems that deliver outcomes Issues with metrics reduced emissions resilience/ adaptation food security/ productivity nutrition outcomes costly to quantify agreement on core indicators  CSA is emerging as a mechanism for coherent and coordinated action  Measuring processes or Outcomes? Standard of measurements That show progress
  3. 3. 500 million farmers globally 25 million by 2025 Global momentum still building for CSA 6 million by 2021
  4. 4. CSA Complexity & Uncertainty Complexity stemming from diversity of: 1. Interventions (micro to macro level) 2. Farming systems and households 3. Objectives/ potential outcomes of success (ranging from increased soil carbon to maternal dietary diversity), with trade-offs possible. Uncertainty and Context Specificity: Lack of information and data • Outcomes = CSA practices x social-environmental context • Need to identify where biggest wins occur across landscapes/regions • Gradient of CSA-ness • CSA today, may not be CSA tomorrow
  5. 5. Review CSA metrics : What's there ? High demand for useful/actionable tools to design, prioritize and monitor useful interventions and investments: Paris Workshop 2015. 0 50 100 150 200 Indicators types (%) Readiness Process Outcome 0 50 100 150 200 Indicators relative contribution to CSA objectives (%) Productivity Adaptation Mitigation - Most agencies covered all the types of indicators but focused on Process and Outcomes - Mostly covering Adaptation pillar (81%) and Productivity (40%). Adaptation => Guidance needed - Largely geared towards risk management - Multidimensional nature of resilience (economic, financial social) and capacities (absortive, adaptive, transformative) often not factored - Measure potential adaptation rather than actual adaptation - Lack ability to track changes over time (e.g lean season) Mitigation Significant lack of indicators related to mitigation outcomes Global CSA indicators database:  378 relevant indicators (USAID, DFID, World Bank, IFAD, GIZ, FAO, CCAFS)  Revisited thought the “CSA lenses” and Classified * Readiness/Enabling environment * Process (outputs) * Outcome indicators
  6. 6. What to measure and how? Some CCAFS metrics tools CSA Evidence building CSA targeting/ Programming CSA Measures and Monitoring - Compendium of CSA (Practice) - CSA Calculator (Farm) - Diverse Prioritization tools and approaches - Risk-household-option modelling (RhOMIS) - CSA programming and Indicator Tool - SHAMBA (Small-Holder Ag. Monitoring and Baseline Assessment - CCAFS Mitigation Option Tool (MOT) $
  7. 7. CSA Compendium Rosenstock et al. 2016 Systematic review • 1,200 peer reviewed studies for Africa • 100,000+ comparisons to baselines, • 70 practices • 55 outcome indicators Largest meta analysis to date ♦ What is the evidence base for field interventions’ potential impact on three pillars?  Outcomes vary (crop, farming system, agroecology…)  There is no best-bet practice across all outcomes and systems  Prioritization of outcomes and tailoring is key Integration with other initiatives: WB, USAID, World Bank, GIZ, Care, CRS, Oxfam, World Vision, Concern, NEPAD, COMESA, national governments in multiple African Countries Evidence building
  8. 8. ♦ Farm model allowing a prospective assessment of synergies & trade-offs related to implementation of different CSA options and outcomes that can be expected (three pillars). Main indicators of the CSA calculator Pilar ASAC Indicador Calculation Productivity Caloric ratio of the farm (%) Caloric supply/Caloric demand x 100 Fodder ratio of the farm (%) Fodder supply/Fodder demand x 100 Cost benefit ratio (%) Benefit/Cost x 100 Adaptation Biodiversity index (%) Assessment based on Gobbi, J., Casasola, F., 2003. Water balance (%) Water supply/water demand x 100 Nutrient balance (%) Nutrient supply/nutrient demand x 100 Mitigation Emission/Sequestration of CO2 CoolFarmTool Measurement & Monitoring CSA Calculator of farm performance (Synergies and Trade-offs) Users: Piloting ongoing in CCAFS CSV monitoring Evidence building
  9. 9. Risk-household-option modelling (RhOMIS) CSA Programming  Captures information on farm productivity and practices, nutrition, food security, gender equity, climate and poverty  Up to 20 important performance and welfare indicators + key farm level drivers, livelihood data and management decisions  Compare changes in farming practice and livelihoods over time, stated plans for the future, and farmers intrinsic values and attitudes  Identifies ‘positive deviant’ farmers Users: tested and adapted for diverse systems in more than 7,000 households across the global tropics Implementations by 10 projects, 15 diverse farming systems on 4 continents. Lutheran World Relief (Kenya); TreeAID (northern Ghana), CONABIO (in Mexico) Measurement & Monitoring Citation: Hammond, J. et al. 2017 ♦ Household-level Survey and Analytical engine for characterizing, targeting and monitoring agricultural performance and the effectiveness of ongoing out-scaling efforts
  10. 10. Mitigation measurement tools Mitigation option tool (MOT) Small Holder Agriculture Monitoring and Baseline Assessment (SHAMBA)  Provides fast, accessible, and reliable information to make informed decisions about emissions reductions  Assesses GHG emissions estimates and Ranks the most effective mitigation options for dozens of different crops and livestock systems  Low input data requirements  Runs in Excel bringing together several empirical models • Estimates GHG emissions or removals resulting from a change in land management practices (biomass burning, plant nitrogen inputs to soils, and fertiliser use) • Models a baseline scenario (where land management activities continue as business as usual) and a CSA intervention scenario • Set to calculate estimates on a yearly basis, in tonnes of CO2e per hectare. Measurement & Monitoring
  11. 11. Case study : working with the public sector on metrics (USAID) CSA Programming and Indicator tool (CCAFS) CSA Programming Measurement & Monitoring • Provide common framework to guide for agriculturally focused programs/donors on the design of CSA interventions • Provide a robust and transparent process to examine to which extent a specific program addresses the three CSA pillars • Support the selection of appropriate indicators to measure progress and monitor impact Productivity Adaptation Mitigation To increase the effectiveness of CSA interventions we need:  How are agencies currently addressing CSA or how they can make their future programing process more climate-smart
  12. 12. Case study : working with the public sector on metrics (USAID) CSA Programming and Indicator tool CSA Programming Measurement & Monitoring CSA Indicators data base = initial input
  13. 13. Step 1 Questions to be addressed & intentionality of desired outcomes CSA Programming Measurement & Monitoring
  14. 14. CSA Programming Measurement & Monitoring
  15. 15. Results summary and visualisation3 Shared frameworks it enables to programs/ donors :  To Assess scope  Compares intentionality  Support indicators identification CSA Programming Measurement & Monitoring
  16. 16. Case study : working with the private sector on metrics CSA Programming Measurement & Monitoring PRODUCTIVITY OUTCOME: 50% more nutritious food available ACTIVITYe.g.raiseyields RESILIENCE, INCOMES & LIVELIHOODS OUTCOME: climate resilient agricultural landscapes and farming communities ACTIVITYe.g.skillstransfer MITIGATION OUTCOME: food GHG emissions 30% lower & land use change emissions eliminated ACTIVITY:e.g.haltforestconversion Source: Vermeulen & Frid- Nielsen, 2017
  17. 17. Basic framework for measurement INPUTS & FARMING PROCESSING & LOGISTICS STORAGE & TRANSPORT TRADING & PURCHASE GLOBAL INDICATORS (some are OUTCOME indicators and some are ACTIVITY indicators) COMPANY INDICATORS (some are OUTCOME indicators and some are ACTIVITY indicators) Enabling and regulatory environment
  18. 18. Companies included in the analysis Company Sub-sector Coca-Cola Food processing CP Foods Food processing Diageo Food processing DuPont Agricultural inputs Kellogg Food processing Monsanto Agricultural inputs Olam Food processing PepsiCo Food processing Starbucks Retail Syngenta Agricultural inputs Tyson Foods Food processing Unilever Food processing Yara Agricultural inputs Source: Vermeulen & Frid-Nielsen, 2017
  19. 19. Results Productivity • On track to produce enough food to meet the demand for 50% more food by 2030. • No direct evidence that this food will be equally or more nutritious Resilience • No relevant global data that match the indicators that companies use for resilience. • Need to provide quantitative information on indicators that cover both activities (e.g. training, on-farm agroecological practices) and outcomes (e.g. incomes, women’s share of assets and decisions). Mitigation • The 2030 goal of 30% emissions reductions compared to 2010 will not be met unless the trend 2010-30 is changed. • Companies generally reduced the intensity of their own operations, but reporting on Scope 3 emissions is not currently pervasive enough. Source: Vermeulen & Frid-Nielsen, 2017
  20. 20. Next steps/opportunities • Building CSA metrics into regular practice • Considering how any activity or intervention will lead to desired outcomes for productivity, resilience and mitigation • Road-test countries and regions provide an innovative opportunity to test and measure how scale effects and trade-offs can be managed • Lot more work needs to be done – on measurement and action
  21. 21. CSA M&E challenges
  22. 22. Indicator overload • SDGs - 230 indicators • USAID 500 indicators • UNICEF 300+ Figures and graphic courtesy of T. Rosenstock
  23. 23. CSA M&E challenges • Multi-objective complexity • Heterogeneity, complexity, and diversity of indicators • Maintaining simplicity and cost effectiveness • Logistical, statistical, technical and practical challenges. • M&E at different scales of impact • Multi-institutional coordination Source: Rosenstock & Corner-Dolloff, 2015
  24. 24. A way forward for GACSA A common approach to metrics to CSA which takes into account: • Rigour Vs Simplicity • Cost effectiveness • Harmonisation Vs flexibility • Role of ICT across scales • Continual feedback for adaptive programming and monitoring
  25. 25. Further reading • CSA Programming and Indicator Tool ( • Info note on livestock MRV ( • Measuring Progress Towards the WBCSD Statement of Ambition on Climate-Smart Agriculture: Improving Businesses’ Ability to Trace, Measure and Monitor CSA ( • Monitoring, evaluation and learning for CSA ( • National level indicators for gender, poverty, food security, nutrition and health in Climate-Smart Agriculture (CSA) activities ( health-climate#.WSh81OsrJhE) • A Monitoring Instrument for Resilience ( • Info Note: A rapid, cost-effective and flexible tool for farm household characterisation, targeting interventions and monitoring progress towards climate-smart agriculture info-note-VanWijkHammond-etal-final.pdf • Info Note: Surveillance of Climate-smart Agriculture for Nutrition (SCAN): Innovations for monitoring at scale : • Journal paper: When less is more: Innovations for tracking progress toward global targets : • Info Note: What is the scientific basis for climate-smart agriculture? basis-climate-smart-agriculture • Evidence-based opportunities for out-scaling climate-smart agriculture in East Africa: • CSA X-Rays: Conservation agriculture in East and Southern Africa main messages ( • CCAFS Mitigation Option Tool: • SHAMBA Tool tool#.WUGO2ut97IU
  26. 26. Thank you
  27. 27. Questions Answers &
  28. 28. Risks-Households-Options (RHO) integrated modeling system CSA Programming Applications/users to date Niger case study using the HH-level data from the World Bank’s Living Standards Measurement Survey  Integrates available datasets combined field level and modeling information + CSA evidence ♦ Three-step (cost effective) approach that integrates information on - HH characteristics (Survey datasets), - risk exposure levels (Crop modeling) - for determining best bet CSA practices to support efficient and relevant CSA programming ♦ Ultimately addresses Food Security under CC ♦ Supports building robust portfolio of best bet CSA options that can be implemented, using currently available data! Citation: Lamanna et al 2015