Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IFPRI Policy Seminar "Evidence-Based Policymaking" Presentation by John Young

2,527 views

Published on

IFPRI Policy

Published in: Education
  • Be the first to comment

  • Be the first to like this

IFPRI Policy Seminar "Evidence-Based Policymaking" Presentation by John Young

  1. 1. Measuring the impact of research: experience from ODI’s RAPID Programme<br />John Young<br />Overseas Development Institute J.young@odi.org.uk<br />
  2. 2. Commission research<br />Analyse the results<br />Choose the best option<br />Establish the policy<br />Implement the policy<br />Evaluation<br />Policy processes are not... <br />Identify the problem<br />
  3. 3. Policy processes are...<br />Cabinet<br />Donors<br />Policy Formulation<br />Parliament<br />Agenda <br />Setting<br />Decision<br />Making<br />Civil Society<br />Ministries<br />Monitoring and Evaluation<br />Policy Implementation<br />Private Sector<br />
  4. 4. Policy is chaos<br />“The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies”<br />Clay & Schaffer (1984), Room for Manoeuvre: An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London<br />
  5. 5. Policy is complex<br />Interconnected<br />Feedback<br />Emergence<br /><ul><li>Nonlinearity
  6. 6. Sensitivity
  7. 7. Changing
  8. 8. Edge of chaos
  9. 9. Adaptive agents
  10. 10. Self-organising
  11. 11. Co-evolution</li></li></ul><li>Research plays a minor role<br />Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf)<br />
  12. 12. Research can transform lives<br />“The results of household disease surveys informed processes of health service reform which contributed to a 43 and 46 per cent reduction in infant mortality between 2000 and 2003 in two districts in rural Tanzania.”<br />TEHIP Project, Tanzania: www.idrc.ca/tehip <br />
  13. 13. An iterative flexible approach<br />
  14. 14. Policy objectives<br /><ul><li>Discursive: Client-focused services
  15. 15. Attitudinal: Farmers have good ideas
  16. 16. Procedural: Participatory approaches to service development
  17. 17. Content: UU20, UU25. New guidelines
  18. 18. Behavioural: Approach being applied in practice</li></li></ul><li>The Cynefn Framework<br />
  19. 19. What should you measure?<br />“If you don't know where you are going, any road will get you there”<br />
  20. 20. Strategy and direction –are you doing the right thing?<br />Management –are you doing what you planned to do?<br />Outputs– are the outputs appropriate for the audience?<br />Uptake – are people aware of your work?<br />Outcomes and impacts –are you having any impact?<br />Strategy and direction<br />Management<br />Outputs<br />Uptake<br />Outcomes and impacts<br />What should you measure?<br />
  21. 21. Focus on behaviour change<br />Project <br />Team<br />Impact<br />Outcomes<br />Other Actors<br />Inputs<br />Activities<br />Outputs<br />Outcomes<br />Impact<br />Outcomes<br />Impact<br />BehaviourChange<br />
  22. 22. International Research<br />The Hubl Study<br />Episode Studies<br />1970s<br />1980s<br />1990s<br />2000s<br />Professionalisation of Public Services.<br />Structural Adjustment -> collapse of services.<br />Paravet projects emerge.<br />ITDG projects. <br />Privatisation.<br />ITDG Paravet network.<br />Rapid spread in North.<br />KVB letter (January 1998).<br />Multistakeholder WSs -> new policies.<br />Still not approved / passed!<br />Professionalisationof Public Services.<br />Structural Adjustment<br />Privatisation <br />ITDG Paravet networkand change of DVS.<br />KVB letter(January 1998).<br />Multistakeholder WSs ->new policies.<br />ITDG projects – collaborative research.<br />Dr Kajume<br />
  23. 23. Outcome Mapping<br />OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo<br />http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html<br />
  24. 24. Social Network Analysis<br />www.odi.org.uk/RAPID/Tools/Toolkits/KM/Social_network_analysis.html<br />
  25. 25. RAPID Outcome Assessment<br />www.odi.org.uk/RAPID/Publications/RAPID_WP_266.html<br />
  26. 26. Most Significant Change<br />Collect “stories of change” from different stakholders<br />Systematic analysis of “significance”.<br />http://www.mande.co.uk/docs/MSCGuide.pdf<br />
  27. 27. Others....<br />Classical case studies (IDRC, IFPRI)<br />Stories of Change (Denning)<br />Innovation Histories (CIAT)<br />HERG Payback Framework (Hanney)<br />Micro-Narratives (Snowden)<br />Impact matrices (Davies)<br />Peer evaluations (CHSRF)<br />Systematic reviews?<br />RCTs?<br />
  28. 28. Conclusions<br />Research to influence policy/practice:<br />Clear objectives<br />Understand the context<br />Theory of change<br />Iterative / learning approach<br />Measuring impact<br />Clear objectives<br />Theory of change<br />5-levels<br />Multiple methods<br />Triangulation<br />Expect the unexpected<br />

×