Setting the scene – Trends in programming Research and Innovation for Impact Francois Stepman
6 April 2018. Rome. The SCAR Strategic Working Groups ARCH, AKIS and Food Systems organised jointly the Workshop: Programming Research and Innovation for Improved Impact
Presentation by Paul Winter
An overview of impact evaluation for organizations based on a program's Theory of Change, highlighting the need for a counterfactual and randomization (when possible) in order to convincingly demonstrate the effect of the program.
I gave this talk at a Nigeria Health Summit in March 2016. It was an introduction to impact evaluation: what it is, when it's a good idea, and some possible approaches.
Setting the scene – Trends in programming Research and Innovation for Impact Francois Stepman
6 April 2018. Rome. The SCAR Strategic Working Groups ARCH, AKIS and Food Systems organised jointly the Workshop: Programming Research and Innovation for Improved Impact
Presentation by Paul Winter
An overview of impact evaluation for organizations based on a program's Theory of Change, highlighting the need for a counterfactual and randomization (when possible) in order to convincingly demonstrate the effect of the program.
I gave this talk at a Nigeria Health Summit in March 2016. It was an introduction to impact evaluation: what it is, when it's a good idea, and some possible approaches.
Gender in monitoring, evaluation and impact assessmentILRI
Presentation by Jemimah Njuki at the FAO-ILRI Workshop on Integrating Gender in Livestock Projects and Programs, ILRI, Addis Ababa, 22-25 November 2011.
A presentation by Respichius Mitti as part of the Practicalities of Cohort and Longitudinal Research panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
A presentation by Nalini Takeshwar as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
Impact Evaluation Training with AERC: China Cash Transfer Programme Technical...The Transfer Project
A hypothetical technical proposal for China's conditional cash transfer programme from our impact evaluation training with AERC in Nairobi, Kenya in July 2019.
Monitoring and Evaluation system for PFES: Key findings and policy recommenda...CIFOR-ICRAF
This presentation by Pham Thu Thuy, Karen Bennet, Vu Tan Phuong and Le Ngoc Dung shows the key findings for M&E for Environmental Service, Social Impact, PFES contract and financial flows.
Webinar presentation by Susan Pietryzk. Access the webinar recording at http://www.measureevaluation.org/resources/webinars/measuring-impact-qualitatively
Planning the Evaluation
Impact models
Types of inference and choice of design
Defining the indicators and obtaining the data
Carrying out the evaluation
Disseminating evaluation findings
Working in large-scale evaluations
A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 7 of 11.
There are two handouts to go with this module, Population Indicators, and a Logframe with blanks. http://www.slideshare.net/Makewa/population-indicators-handout and http://www.slideshare.net/Makewa/exercise-watsan-logframe-with-blanks
Two Examples of Program Planning, Monitoring and EvaluationMEASURE Evaluation
Presented by Laili Irani, Senior Policy Analyst for the Population Reference Bureau, as part of the Measuring Success Toolkit webinar in September 2012.
An 8-step guide to help not-for-profit organizations develop a Project Impact Pathway or Logic Model for their programs - The first presentation in our Project Evaluation Series. By DevMetrica
Build Your NGO: Monitoring & Evaluation Allie Hoffman
The presentation attached is designed for grassroots NGOs wanting to learn more about monitoring and evaluation.
The presentation is a mini 'how to', in addition to providing an overview of strategic planning
To learn more or with any direct questions, please visit our website: www.thepariproject.com
A presentation by Rachel Hinton as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
Gender in monitoring, evaluation and impact assessmentILRI
Presentation by Jemimah Njuki at the FAO-ILRI Workshop on Integrating Gender in Livestock Projects and Programs, ILRI, Addis Ababa, 22-25 November 2011.
A presentation by Respichius Mitti as part of the Practicalities of Cohort and Longitudinal Research panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
A presentation by Nalini Takeshwar as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
Impact Evaluation Training with AERC: China Cash Transfer Programme Technical...The Transfer Project
A hypothetical technical proposal for China's conditional cash transfer programme from our impact evaluation training with AERC in Nairobi, Kenya in July 2019.
Monitoring and Evaluation system for PFES: Key findings and policy recommenda...CIFOR-ICRAF
This presentation by Pham Thu Thuy, Karen Bennet, Vu Tan Phuong and Le Ngoc Dung shows the key findings for M&E for Environmental Service, Social Impact, PFES contract and financial flows.
Webinar presentation by Susan Pietryzk. Access the webinar recording at http://www.measureevaluation.org/resources/webinars/measuring-impact-qualitatively
Planning the Evaluation
Impact models
Types of inference and choice of design
Defining the indicators and obtaining the data
Carrying out the evaluation
Disseminating evaluation findings
Working in large-scale evaluations
A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries.
Part 7 of 11.
There are two handouts to go with this module, Population Indicators, and a Logframe with blanks. http://www.slideshare.net/Makewa/population-indicators-handout and http://www.slideshare.net/Makewa/exercise-watsan-logframe-with-blanks
Two Examples of Program Planning, Monitoring and EvaluationMEASURE Evaluation
Presented by Laili Irani, Senior Policy Analyst for the Population Reference Bureau, as part of the Measuring Success Toolkit webinar in September 2012.
An 8-step guide to help not-for-profit organizations develop a Project Impact Pathway or Logic Model for their programs - The first presentation in our Project Evaluation Series. By DevMetrica
Build Your NGO: Monitoring & Evaluation Allie Hoffman
The presentation attached is designed for grassroots NGOs wanting to learn more about monitoring and evaluation.
The presentation is a mini 'how to', in addition to providing an overview of strategic planning
To learn more or with any direct questions, please visit our website: www.thepariproject.com
A presentation by Rachel Hinton as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
Examples of metagenomics use cases for the Phylotastic! web tools. Presented a the Phylotastic hackathon, June 4-8 2012: http://www.evoio.org/wiki/Phylotastic
Science publications used to have the joint function of keeping the 'Minutes of Science' as well as transferring knowledge. The sheer amounts of material published (2 new articles in PubMed every minute of every day) make comprehensive knowledge transfer via reading of articles virtually impossible. When the literature is open, though, much of the essential knowledge it contains can be distilled and the big picture obtained without having to read all the articles, so that reading can then be reserved for those key articles that give insight in the reasoning and argumentation that leads to consensus. The result is a much more efficient knowledge transfer that doesn't have to compromise on comprehensiveness.
Presentación de Germán A. Flores, PESA en Honduras en el marco del Primer Taller de Trabajo: Desafíos de la Cooperación Sur-Sur para fortalecimiento de la agricultura familiar algodonera, realizado los días 22, 23 y 24 de Octubre de 2013 en Lima, Perú.
http://www.fao.org/save-food/info-resources/en/
FAO's 'Global Initiative on Food Loss and Waste Reduction' has taken a coordinating role, to enhance information exchange, collaboration, synergy and harmonization of strategies and methodologies.
El remplazo de alimentos por productos ultra-procesados: ¿que debemos hacer?FAO
Presentación de Jean Claude Moubarac durante la Consulta técnica "Una mirada integral a las políticas públicas de agricultura familiar, seguridad alimentaria, nutrición y salud pública: Acercando las agendas de trabajo en el seno del Sistema de Naciones Unidas", realizada en Lima, Perú, 19 y 20 de Mayo de 2014.
http://www.fao.org/americas/eventos/ver/es/c/231982/
http://www.fao.org/agora
This module is part of the AGORA Basic Course. The AGORA Basic Course highlights the baseline skills necessary to use the AGORA program effectively and efficiently. The AGORA programme (Access to Global Online Research in Agriculture), set up by the Food and Agriculture Organization of the UN (FAO) together with major publishers, enables developing countries to gain access to an outstanding digital library collection in the fields of food, agriculture, environmental science and related social sciences. Module 4 covers CAB Abstracts
AGORA Basic Course: Module 7.3: E-journal, E-books and Internet Resources: Ot...FAO
http://www.fao.org/agora
This module is part of the AGORA Basic Course. The AGORA Basic Course highlights the baseline skills necessary to use the AGORA program effectively and efficiently. The AGORA programme (Access to Global Online Research in Agriculture), set up by the Food and Agriculture Organization of the UN (FAO) together with major publishers, enables developing countries to gain access to an outstanding digital library collection in the fields of food, agriculture, environmental science and related social sciences. Module 7.3 covers E-journal, E-books and Internet Resources (Agriculture Gateways/Portals)
AGORA Basic Course: Additional Resources. Tips for TrainersFAO
http://www.fao.org/agora
This module is part of the AGORA Basic Course. The AGORA Basic Course highlights the baseline skills necessary to use the AGORA program effectively and efficiently. The AGORA programme (Access to Global Online Research in Agriculture), set up by the Food and Agriculture Organization of the UN (FAO) together with major publishers, enables developing countries to gain access to an outstanding digital library collection in the fields of food, agriculture, environmental science and related social sciences. The module with Additional Resources includes tips for trainers presenting this course.
http://www.fao.org/agora
This module is part of the AGORA Basic Course. The AGORA Basic Course highlights the baseline skills necessary to use the AGORA program effectively and efficiently. The AGORA programme (Access to Global Online Research in Agriculture), set up by the Food and Agriculture Organization of the UN (FAO) together with major publishers, enables developing countries to gain access to an outstanding digital library collection in the fields of food, agriculture, environmental science and related social sciences. Module 1 covers Background, partners, eligibility, use, copyright.
Training Impact assessment or capacity development impact assessment pdfJayanta Dutta
The document contains details of concept, methods and approaches of training impact assessment which will be helpful for M.Sc(Agri) students under the course titled Capacity Development. The examples given in the document will help the students to understand the concept well.
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...OECD CFE
Presentation by Jonathan Potter, OECD LEED Senior Policy Analyst, and Stuart Thompson, OECD LEED Policy Analys, tat the seminar organised by the OECD LEED Trento Centre for the Officers of the Autonomous Province of Trento on 13 November 2015.
https://www.trento.oecd.org
Applying impact evaluation tools for integrating agricultural sectors in Nati...UNDP Climate
- Uganda and Zambia are carrying out activities to better assess adaptation options through cost-benefit analysis and impact evaluation exercises, as part of the Integrating Agriculture in National Adaptation Plans (NAP-Ag) Programme led by FAO and UNDP.
Both Uganda and Zambia are also paving way for gender mainstreaming into National Adaptation Plans, with recent cross-sectoral workshops held in May and June to discuss these topics and pave the way for integrated strategies.
Impact Management Principles. EVPA, European Venture Philanthropy AssociationDominique Gross
EVPA’s guidance for impact management and Social Value International’s
(SVI) Principles are in many ways interlinked. This document shows EVPA and SVI’s position on impact measurement and management.
The European Venture Philanthropy Association (EVPA) supports a fivestep
process to help organisations measure and manage their social
impact. These steps aim to help venture philanthropy organisations and
social investors (VPO/SIs) and social purpose organisations (SPOs) to implement a system to collect information in order to improve the products and services offered to the final beneficiaries.
Quality is like motherhood and apple pie: something that everyone can support. But ‐‐ as any mother will tell you ‐‐ that doesn't make it easy. And, as the world changes, the definition of quality is changing as well. Change and transformation are the emerging tools of quality. Leading organizations are listening to the voice of the customer and actively using this information to reinvent themselves to meet their customer’s increasing expectations. This increasing rate of change can have adverse consequences throughout the organization if not properly managed. This presentation outlines a model for change, explores best practices, and describes practical tips for getting started.
Improving Quality with Change ManagementKaren Maskell
Quality is like motherhood and apple pie: something that everyone can support. But ‐‐ as any mother will tell you ‐‐ that doesn't make it easy. And, as the world changes, the definition of quality is changing as well. Change and transformation are the emerging tools of quality. Leading organizations are listening to the voice of the customer and actively using this information to reinvent themselves to meet their customer’s increasing expectations. This increasing rate of change can have adverse consequences throughout the organization if not properly managed. This presentation identifies a model for change, explore best practices, and outline practical tips for getting started.
Evaluation serves two main purposes: accountability and learning. Development agencies have tended to prioritize the first, and given responsibility for that to centralized units. But evaluation for learning is the area where observers find the greatest need today and tomorrow. A learning approach to evaluation looks to designing evaluation with learning in mind.
2. Measuring impact:
Understanding pathways to rural transformation
The Importanceof Measuring the Impact of
The Importance of Measuring the Impact
of Development Interventions
Development Interventions
Philip Davies PhD
Head of London Office of 3ie
[International Initiative for Impact Evaluation]
3. The Importance of Measuring the Impact of
Development Interventions
Why Evaluate Impact ?
• Effectiveness - ensure that interventions have a positive impact
• Attribution – ensure that the outcomes are the results of (are caused by) the
intervention
• Efficiency – use scarce public resources to maximum effect (VfM)
• Implementation – ensure that interventions are/can be implemented
successfully and effectively.
• Service Orientation – ensure we meet citizens’ needs/expectations
• Adaptation – allow policy makers to know when a change of
policy/different intervention is required
• Accountability - transparency of what is done and why
• Democracy - enhance the democratic process
4. The Importance of Measuring the Impact of
Development Interventions
What Is To Be Evaluated?
• Intervention effectiveness - what works?
• Implementation effectiveness – how/why it works?
• Diversity of effectiveness across different groups –
what works for whom and when?
• Experiential effectiveness - users’ views
• Resource effectiveness - at what cost/benefit?
5. The Importance of Measuring the Impact of
Development Interventions
The Need to Measure Impact Properly
It is difficult to gauge whether
agricultural interventions improve
the nutritional status of children.
Many studies were conducted on
small samples. Several animal
husbandry and home gardening
studies found no change in the
nutritional status of the children Source: Massett, E. et al, 2011, A Systematic Review of
Agricultural Interventions That Aim to Improve The
involved. More importantly, the Nutritional Status of Children, London, EPPI-Centre.
studies reviewed were statistically
unable to identify impact if any
existed.
6. The Importance of Measuring the Impact of
Development Interventions
Key Features of Good Impact Evaluation
• Ask a clear and answerable question
• Have a clear and explicit theory of change
• Have an appropriate comparison/counterfactual
• Have adequate sample sizes, and sufficient statistical power
• Have meaningful outcomes, and data on those outcomes
• Choose the most appropriate evaluation design
• Ensure that the evaluation in undertaken properly
• Check for internal and external validity
• Check for adequacy of reporting and analysis
7. The Importance of Measuring the Impact of
Development Interventions
Types of Impact Evaluation Design
• Randomised Controlled Experiments
[Allocate the intervention to groups on a random basis]
• Matched Comparison Designs
[Carefully matched those who get the intervention with those who do not]
• Regression Discontinuity Designs
[Allocate the intervention on a clearly define basis – check for
differences post-intervention]
• Difference-in-Difference Designs
[Accept initial differences between intervention and control group –
check on the size of the difference post-intervention]
8. The Importance of Measuring the Impact of
Development Interventions
Types of Evaluation for Different Policy Questions
9. The Importance of Measuring the Impact of
Development Interventions
Thank you
Philip Davies
Email: pdavies@3ieimpact.org
+44 (0)207 958 8350
Visit www.3ieimpact.org
10. The Importance of Measuring the Impact of
Development Interventions
Thank you
Philip Davies
Email: pdavies@3ieimpact.org
+44 (0)207 958 8350
Visit www.3ieimpact.org