Measuring the impact of research: experience from ODI’s RAPID Programme<br />John Young<br />Overseas Development Institut...
Commission research<br />Analyse the results<br />Choose the best option<br />Establish the policy<br />Implement the poli...
Policy processes are...<br />Cabinet<br />Donors<br />Policy Formulation<br />Parliament<br />Agenda <br />Setting<br />De...
Policy is chaos<br />“The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rati...
Policy is complex<br />Interconnected<br />Feedback<br />Emergence<br /><ul><li>Nonlinearity
Sensitivity
Changing
Edge of chaos
Adaptive agents
Self-organising
Co-evolution</li></li></ul><li>Research plays a minor role<br />Kate Bird et al, Fracture Points in Social Policies for Ch...
Research can transform lives<br />“The results of household disease surveys informed processes of health service reform wh...
An iterative flexible approach<br />
Policy objectives<br /><ul><li>Discursive: Client-focused services
Attitudinal: Farmers have good ideas
Procedural: Participatory approaches to service development
Content: UU20, UU25. New guidelines
Behavioural: Approach being applied in practice</li></li></ul><li>The Cynefn Framework<br />
What should you measure?<br />“If you don't know where you are going, any road will get you there”<br />
Strategy and direction –are you doing the right thing?<br />Management –are you doing what you planned to do?<br />Outputs...
Upcoming SlideShare
Loading in...5
×

IFPRI Policy Seminar "Evidence-Based Policymaking" Presentation by John Young

2,011

Published on

IFPRI Policy

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,011
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
63
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • What are you trying to learn:Impact of research on policy and practice?Impact of changes in policy / practice on livelihoods?Why:To improve your own practice?To satisfy the donor?To demonstrate value for money?
  • IFPRI Policy Seminar "Evidence-Based Policymaking" Presentation by John Young

    1. 1. Measuring the impact of research: experience from ODI’s RAPID Programme<br />John Young<br />Overseas Development Institute J.young@odi.org.uk<br />
    2. 2. Commission research<br />Analyse the results<br />Choose the best option<br />Establish the policy<br />Implement the policy<br />Evaluation<br />Policy processes are not... <br />Identify the problem<br />
    3. 3. Policy processes are...<br />Cabinet<br />Donors<br />Policy Formulation<br />Parliament<br />Agenda <br />Setting<br />Decision<br />Making<br />Civil Society<br />Ministries<br />Monitoring and Evaluation<br />Policy Implementation<br />Private Sector<br />
    4. 4. Policy is chaos<br />“The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies”<br />Clay & Schaffer (1984), Room for Manoeuvre: An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London<br />
    5. 5. Policy is complex<br />Interconnected<br />Feedback<br />Emergence<br /><ul><li>Nonlinearity
    6. 6. Sensitivity
    7. 7. Changing
    8. 8. Edge of chaos
    9. 9. Adaptive agents
    10. 10. Self-organising
    11. 11. Co-evolution</li></li></ul><li>Research plays a minor role<br />Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf)<br />
    12. 12. Research can transform lives<br />“The results of household disease surveys informed processes of health service reform which contributed to a 43 and 46 per cent reduction in infant mortality between 2000 and 2003 in two districts in rural Tanzania.”<br />TEHIP Project, Tanzania: www.idrc.ca/tehip <br />
    13. 13. An iterative flexible approach<br />
    14. 14. Policy objectives<br /><ul><li>Discursive: Client-focused services
    15. 15. Attitudinal: Farmers have good ideas
    16. 16. Procedural: Participatory approaches to service development
    17. 17. Content: UU20, UU25. New guidelines
    18. 18. Behavioural: Approach being applied in practice</li></li></ul><li>The Cynefn Framework<br />
    19. 19. What should you measure?<br />“If you don't know where you are going, any road will get you there”<br />
    20. 20. Strategy and direction –are you doing the right thing?<br />Management –are you doing what you planned to do?<br />Outputs– are the outputs appropriate for the audience?<br />Uptake – are people aware of your work?<br />Outcomes and impacts –are you having any impact?<br />Strategy and direction<br />Management<br />Outputs<br />Uptake<br />Outcomes and impacts<br />What should you measure?<br />
    21. 21. Focus on behaviour change<br />Project <br />Team<br />Impact<br />Outcomes<br />Other Actors<br />Inputs<br />Activities<br />Outputs<br />Outcomes<br />Impact<br />Outcomes<br />Impact<br />BehaviourChange<br />
    22. 22. International Research<br />The Hubl Study<br />Episode Studies<br />1970s<br />1980s<br />1990s<br />2000s<br />Professionalisation of Public Services.<br />Structural Adjustment -> collapse of services.<br />Paravet projects emerge.<br />ITDG projects. <br />Privatisation.<br />ITDG Paravet network.<br />Rapid spread in North.<br />KVB letter (January 1998).<br />Multistakeholder WSs -> new policies.<br />Still not approved / passed!<br />Professionalisationof Public Services.<br />Structural Adjustment<br />Privatisation <br />ITDG Paravet networkand change of DVS.<br />KVB letter(January 1998).<br />Multistakeholder WSs ->new policies.<br />ITDG projects – collaborative research.<br />Dr Kajume<br />
    23. 23. Outcome Mapping<br />OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo<br />http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html<br />
    24. 24. Social Network Analysis<br />www.odi.org.uk/RAPID/Tools/Toolkits/KM/Social_network_analysis.html<br />
    25. 25. RAPID Outcome Assessment<br />www.odi.org.uk/RAPID/Publications/RAPID_WP_266.html<br />
    26. 26. Most Significant Change<br />Collect “stories of change” from different stakholders<br />Systematic analysis of “significance”.<br />http://www.mande.co.uk/docs/MSCGuide.pdf<br />
    27. 27. Others....<br />Classical case studies (IDRC, IFPRI)<br />Stories of Change (Denning)<br />Innovation Histories (CIAT)<br />HERG Payback Framework (Hanney)<br />Micro-Narratives (Snowden)<br />Impact matrices (Davies)<br />Peer evaluations (CHSRF)<br />Systematic reviews?<br />RCTs?<br />
    28. 28. Conclusions<br />Research to influence policy/practice:<br />Clear objectives<br />Understand the context<br />Theory of change<br />Iterative / learning approach<br />Measuring impact<br />Clear objectives<br />Theory of change<br />5-levels<br />Multiple methods<br />Triangulation<br />Expect the unexpected<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×