Your SlideShare is downloading. ×
0
Towards a Research Agenda for ImpactEvaluation of DevelopmentImpact, Innovation and Learning: Towards a Research andPracti...
2
3ResearchAgendaPracticeAgenda
Towards a research agendafor impact evaluation in development1. What it needs to cover2. What is needed to develop it3. Ty...
Filling in the mapof impact evaluation in development
1.What the research agendaneeds to coverTypes of development impact evaluationScale Individual evaluations, evaluations of...
77MANAGEDEFINEFRAMEUNDERSTANDCAUSESSYNTHESIZEREPORT &SUPPORT USEDESCRIBECDC Evaluation Framework with BetterEvaluation com...
1.What the research agendaneeds to cover
1.What the research agendaneeds to cover
2.What is needed to developthe research agendaConsultations with Different parties involved in conducting, managing, using...
3.Types of research neededDocumenting practice Retrospectively, concurrently; good practice, problematicpractice; micro-in...
OVERALL1. How do we do impact evaluation that actuallysupports development?2. How do we support all agents of development,...
MANAGE an evaluation or evaluationsystem4. What are effective ways to support communities to havegenuine involvement in de...
DEFINE what is to be evaluated9. How can a theory of change/program theory effectivelyrepresent complicated aspects of int...
FRAME the boundaries of the evaluation13. What are effective processes for developing goodKey Evaluation Questions – in te...
DESCRIBE activities, outcomes,impacts, context15.When is purposeful sampling most appropriate, and howcan it be used valid...
UNDERSTAND CAUSES of outcomes andimpacts20.What are credible methods and strategies for non-experimental causal inference ...
SYNTHESISE data from one or moreevaluations21.How can different values be accommodated indeveloping an overall evaluative ...
REPORT AND SUPPORT USE23.How can a development impact evaluation respond tosignificant changes in the intended users durin...
5.What is needed to developand enact a research agendaLegitimacyResources, especially complementary to existing resourcesI...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Patricia Rogers
Upcoming SlideShare
Loading in...5
×

IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Patricia Rogers

596

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
596
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Patricia Rogers "

  1. 1. Towards a Research Agenda for ImpactEvaluation of DevelopmentImpact, Innovation and Learning: Towards a Research andPractice Agenda for the Future conferenceInstitute of Development Studies, Brighton, UK26-27 March 2013Professor Patricia RogersRMIT University (Royal Melbourne Institute of Technology), AustraliaPatricia.Rogers@rmit.edu.au
  2. 2. 2
  3. 3. 3ResearchAgendaPracticeAgenda
  4. 4. Towards a research agendafor impact evaluation in development1. What it needs to cover2. What is needed to develop it3. Types of research needed4. Some burning research questions5. What is needed to develop and enact it
  5. 5. Filling in the mapof impact evaluation in development
  6. 6. 1.What the research agendaneeds to coverTypes of development impact evaluationScale Individual evaluations, evaluations of multiple projects in aprogram, evaluation systemsPurpose Identify ‘best buys’, understand how to scale up and translateeffective programs, understand how to improve effectivenessQuestions Does it work? What works? What does it take for it to work? Whatworks for whom in what circumstances? Is it working?Users Donors, implementing agencies, policymakers, regionalassociations, communitiesDone by External evaluators, internal evaluators, managers and staff,communities
  7. 7. 77MANAGEDEFINEFRAMEUNDERSTANDCAUSESSYNTHESIZEREPORT &SUPPORT USEDESCRIBECDC Evaluation Framework with BetterEvaluation components overlaid1.What the research agendaneeds to coverAspects of development impact evaluation
  8. 8. 1.What the research agendaneeds to cover
  9. 9. 1.What the research agendaneeds to cover
  10. 10. 2.What is needed to developthe research agendaConsultations with Different parties involved in conducting, managing, usingand being affected by impact evaluationConsultations about Gaps in knowledge, issues, priorities, opportunitiesReview of Documentation and guidance for development impactevaluation – and impact evaluation generallyIssues and challenges in impact evaluation for developmentPrevious research into impact evaluationPotential methods, tools and approaches from other areas ofevaluation and researchPromising examples and recent innovations in developmentimpact evaluation
  11. 11. 3.Types of research neededDocumenting practice Retrospectively, concurrently; good practice, problematicpractice; micro-interactions; decision-making heuristics;Positive deviance Learning by intended users from success casesTrials To address particular issues through varioustools/methods/strategiesTrials Of possible uses for particular tools/methods/strategiesLongitudinal studies Of the impact of impact evaluationSupporting interdisciplinary communities of practiceKnowledge translation To other contexts (sectors, organisations, roles)
  12. 12. OVERALL1. How do we do impact evaluation that actuallysupports development?2. How do we support all agents of development,including communities, to be reflective and empiricalabout the impact of their work?3. Why does so much development impact evaluationfail to be informed by what has been learned abouteffective evaluation?4. Some burning researchquestions
  13. 13. MANAGE an evaluation or evaluationsystem4. What are effective ways to support communities to havegenuine involvement in decision making aboutevaluations?5. How can an evaluation accommodate different ideas aboutwhat constitutes credible evidence among intendedusers?6. When should different strategies be used for developingan evaluation design (as part of the brief, as part of theproposal, as a separate project)?7. How can an evaluation design best accommodateemerging issues?8. How can organisations working in the same region shareinformation and data collection?8b What are options for funding public interest evaluationsnot under the control of the powerful?
  14. 14. DEFINE what is to be evaluated9. How can a theory of change/program theory effectivelyrepresent complicated aspects of interventions(multiple layers, components and partners andcomplex aspects (adaptability, emergence)?10.How can an organisation support projects to havelocally specific theories of change/program theory thatare still widely coherent?11.What are effective strategies for identifying possiblenegative impacts in advance?12.What investments and activities are the subjects ofevaluation? What examination is made of others?
  15. 15. FRAME the boundaries of the evaluation13. What are effective processes for developing goodKey Evaluation Questions – in terms of likely to beuseful and feasible?14. How can implicit values about results, processes anddistribution of benefits be made explicit?14b How can evaluations deal with ‘undiscussables’ – egactual unstated program objectives, unaddressedpoor performance?
  16. 16. DESCRIBE activities, outcomes,impacts, context15.When is purposeful sampling most appropriate, and howcan it be used validly and effectively?16.How can long-term results be followed up?17.How can unanticipated negative outcomes and impacts beidentified and addressed in data collection and reporting?18.How can reasonable intermediate outcomes be identifiedfor an evaluation that will end before impacts are evident?19.How can Big Data be used effectively for developmentimpact evaluation?19b What standard measures and indicators should be usedfor common outcomes and impacts of interest?
  17. 17. UNDERSTAND CAUSES of outcomes andimpacts20.What are credible methods and strategies for non-experimental causal inference in developmentimpact evaluations?
  18. 18. SYNTHESISE data from one or moreevaluations21.How can different values be accommodated indeveloping an overall evaluative judgement?22.How can systematic reviews which don’t excludematerials in terms of a hierarchy of evidence dealwith the large number of potentially relevant sources?
  19. 19. REPORT AND SUPPORT USE23.How can a development impact evaluation respond tosignificant changes in the intended users during thecourse of an evaluation?24.How can a development impact evaluation provide acoherent message without only focusing on averageeffects?25.What are effective strategies for supporting use ofdevelopment impact evaluation, especially in difficultsituations – eg fragile states, changing decisionmakers?25b What process and structures can be created toprotect those ‘speaking truth to power’?
  20. 20. 5.What is needed to developand enact a research agendaLegitimacyResources, especially complementary to existing resourcesInterdisciplinary co-operation‘Creative abrasion’
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×