Evaluating farmer centred innovation
•Download as PPTX, PDF•
0 likes•62 views
Julie Ingram's presentation at the CCRI webinar on October 8th 2020 entitled 'Evaluation Approaches – Capturing Complexity'
Recommended
Recommended
More Related Content
What's hot
What's hot (12)
Similar to Evaluating farmer centred innovation
Similar to Evaluating farmer centred innovation (20)
More from Countryside and Community Research Institute
More from Countryside and Community Research Institute (20)
Recently uploaded
Recently uploaded (20)
Evaluating farmer centred innovation
- 1. Evaluating farmer-centred innovation Julie Ingram CCRI Seminar series 8 October, 2020
- 2. Farmer-centred innovation • Farmer-centred innovation (FCI) approaches are increasingly popular in UK and Europe • Initiatives that facilitate and foster collaborative participatory activities, interactive innovation and learning • UK examples include: Farmer field labs, Monitor Farms, Countryside Stewardship Facilitation Fund, Farmer Clusters, operational groups
- 3. Evaluation challenges • FCI carry a high level of expectation • Requirement for evidence-based policy, accountability and impact from research and public funders • Policy makers and funders traditionally require output measures –practice/behaviour change, participation, or acquired knowledge as evidence of effectiveness
- 4. Evaluation challenges . But FCI approaches promote outcomes and processes like learning, empowerment, capacity building which are hard to capture Evaluation methods are multiple – no consensus on method to apply, or metrics and evidence to collect, to capture the multiple outcomes expected from FCI
- 5. Evaluation – definition The systematic collection of information about the: activities, characteristics, and outcomes of programs to make judgments about the program, improve the program, and/or inform decision about future programming Patton (1997)
- 6. Evaluation – definition • Monitoring is generally to assess whether the ‘project is doing things right’ • Evaluation is to assess whether the ‘project is doing the right things’
- 7. Evaluation -types Evaluations can have different objectives which are linked to the purpose of the evaluation: MEASURE, UNDERSTAND, LEARN
- 8. Evaluation -what counts as evidence? • Evidence of presence • Evidence of difference making/effectiveness • Evidence of mechanisms/cause and effect Qual data single cases Expert opinion Quant data, cohort studies, RCT
- 9. What are appropriate evaluation approaches for FCI? FLIN Farmer-Led Initiatives Network (FLIN): a Community of Practice of organisations working together to support farmer-centred innovation and research Lisa Williams van Dijk Beth Dooley
- 10. What are appropriate evaluation approaches for FCI? We have drawn on: • Large body of research & expertise about participatory approaches (e.g farmer field schools) • Practical experiences in UK- FLIN • Expertise & experiences- 24th European Seminar on Extension & Education 2019, workshop
- 11. Participatory approaches Attribution of impact is difficult: • Outcomes are intangible, knowledge gained and learning, sustainable livelihoods, attitudes to risk and innovation • Processes occur at many different levels • Diversity of farms/contexts and farmers – no such thing as an average farmer • Hard to compare between those participating/receiving support and those not • Need to account for other interventions/influences
- 12. Participatory approaches • Range of approaches • As well as ‘adoption’ • Take a broader view, of improved: analytical skills, critical thinking, ability to make better decisions, familiarity with practices, empowerment and enhanced capacity to learn • Participatory evaluation, joint reflection and iterative methods
- 13. FLIN evaluation review FLIN members -diverse monitoring and evaluating of initiatives BEST PRACTICE Participatory approaches - include farmers and foster an increased sense of ownership Facilitation: involve facilitators in the M&E process to support participants in reflecting on their experiences and evaluating their performance Formal evaluation: regularly review progress towards the initiative’s objectives and milestones
- 14. Expert workshop- adapting evaluation approaches OBJECTIVES- involve farmer in deciding what to evaluate, agree objectives and methods. Create dialogue with policy makers, make the evaluation adaptable to their needs OUTCOMES & PROCESSES–design evaluation to account for social dimensions including changes in attitudes, capacity building, network development, learning TOOLS utilise tools to convert QL into QT (e.g. SNA, Q methodology) to meet policy makers’ need for figures. Use, visualisations, infographics and case studies in a creative way
- 15. Expert workshop - adapting evaluation approaches Infographic – see CCRI website http://www.ccri.ac.uk/fciinfographic/
- 16. Question • What are the key question to ask when evaluating FCI approaches and what sort of evidence is needed to answer these questions?
Editor's Notes
- Farmer-centred innovation (FCI) approaches are increasingly popular in Europe and carry a high level of expectation Requirement for evidence-based policy, accountability and impact from research and public funders Evaluation methods are multiple– no consensus to apply, or metrics and evidence to collect, to capture the multiple outcomes expected from FCI Challenge in evaluating softer outcomes such as empowerment and capacity development in a language policy makers/funders can understand
- Evaluation is the systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and funders. Why the evaluation is being done How evaluation is done Who is doing the evaluating What is being evaluated For whom evaluation is being done. 1)Quantification of programme impacts- reference to a counterfactual can be used to identify outcomes which are specific to the program under evaluation’ 2) measuring efficiency- measuring the value of goods or services produced through public programs against the cost of their production
- to measure the impact; to understand the causal path that generates changes; to support learning processes for stakeholders Berriet-Solliec et al. (2014)
- the opinions of respected authorities; evidence obtained by single case observations; evidence obtained from historical or geographical comparisons; evidence obtained from cohort studies or controlled case studies; evidence obtained through randomized controlled trials (RCT). studies based on probabilistic models But where understanding of mechanism is needed- in depth qualitative studies including single case observations provide a higher level of evidence than results of cohort studies
- In FFS evaluations, learning tends to be seen as enhanced knowledge of farming technology such as Integrated Pest Management practices and pest identification. Often these studies provide evidence via adoption, or, taking a broader view, of improved: analytical skills, critical thinking, ability to make better decisions, familiarity with practices, which lead to better decisions regarding inputs, yields and costs (Waddington & White, 2014). At a deeper level, empowerment and enhanced capacity to learn are also indicative of improved and more transformative learning (Duveskog, Friis-Hansen, & Taylor, 2011).
- Programme theory, theory of change diagrams are used as a simplified representation of the proposed causal structure (e.g. intervention logic, theory of action, theory of program, theory-driven evaluation, contribution analysis, outcome mapping) tend to follow a theory of change, consultative- outcome mapping/joint visioning, use participatory evaluation and joint reflection)
- We asked What are the key question to ask when evaluating FCI approaches and what sort of evidence is needed to answer these questions?