Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
February 2013betterevaluation.orgPlanning an Evaluation: Using the Rainbow FrameworkManage an evaluation (or a series of e...
4. DESCRIBE activities, outcomes, impacts and contextCollect and retrieve data to answer descriptive questions about the a...
Upcoming SlideShare
Loading in …5
×

IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Patricia Rogers Handout 2

797 views

Published on

  • Be the first to comment

IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Patricia Rogers Handout 2

  1. 1. February 2013betterevaluation.orgPlanning an Evaluation: Using the Rainbow FrameworkManage an evaluation (or a series of evaluations), including deciding who will conduct the evaluationand who will make decisions about it.Understand and engage stakeholders: Who needs to be involved in the evaluation? How can they beidentified and engaged?Establish decision making processes: Who will have the authority to make what type of decisionsabout the evaluation? Who will provide advice or make recommendations about the evaluation? Whatprocesses will be used for making decisions?Decide who will conduct the evaluation: Who will actually undertake the evaluation?Determine and secure resources: What resources (time, money, and expertise) will be needed for theevaluation and how can they be obtained? Consider both internal (e.g. staff time) and external (e.g.previous participants’ time).Define quality evaluation standards: What will be considered a high quality and ethical evaluation?How should ethical issues be addressed?Document management processes and agreements: How will you document the evaluation’smanagement processes and agreements made?Develop evaluation plan or framework: What is the overall plan for the evaluation? Is there a largerevaluation framework across several related evaluations?Develop evaluation capacity: How can the ability of individuals, groups and organizations to conductand use evaluations be strengthened?1. MANAGE an evaluation or evaluation systemThe BetterEvaluation Rainbow Framework can help you to plan an evaluation by prompting you to think abouta series of key questions. This can be used to develop an evaluation plan, a Terms of Reference, and otherdocuments. It is important to consider these issues, including reporting, at the beginning of an evaluation. Anexpanded version of this framework showing options or methods for each question can be downloaded fromour website: http://betterevaluation.org/plan2. DEFINE what is to be evaluatedDevelop a description (or access an existing version) of what is to be evaluated and how it is understoodto work.Develop initial description: How can you develop a brief description of the project?Develop program theory / logic model: Is there a need to revise or create a logic model (programtheory, theory of change)? How will this be developed? How will it be represented?Identify potential unintended results: How can you identify possible unintended results (both positiveand negative) that will be important?3. FRAME the boundaries for an evaluationSet the parameters of the evaluation – its purposes, key evaluation questions and the criteria andstandards to be used.Decide purpose: What is the purpose of the evaluation?Specify the key evaluation questions: What are the high level questions the evaluation will seek toanswer? How can these be developed?Determine what ‘success’ looks like: What should be the criteria and standards for judgingperformance? Whose criteria and standards matter? What process should be used to developagreement about these?You may use this document under the terms of the Creative Commons Attribution-Non Commercial Unported licence available athttp://creativecommons.org/licenses/by-nc/3.0/.MANAGEFRAMEDEFINE
  2. 2. 4. DESCRIBE activities, outcomes, impacts and contextCollect and retrieve data to answer descriptive questions about the activities of the project/program/policy, the various results it has had, and the context in which it has been implemented.Sample: What sampling strategies will you use for collecting data?Use measures and indicators: What measures or indicators will be used? Are there existing ones thatshould be used or will you need to develop new measures and indicators?Collect and/ or retrieve data: How will you collect and/ or retrieve data about activities, results, contextand other factors?Manage Data: How will you organize and store data and ensure its quality?Combine qualitative and quantitative data: How will you combine qualitative and quantitative data?Analyze data: How will you look for and display patterns in the data?5. UNDERSTAND CAUSES of outcomes and impactsCollect and analyze data to answer causal questions about what has produced outcomes and impactsthat have been observed.Check the results support causal attribution: How will you assess whether the results are consistentwith the theory that the intervention produced them?Compare results to the counterfactual: How will you compare the factual with the counterfactual - whatwould have happened without the intervention?Investigate possible alternative explanations: How will you investigate alternative explanations?6. SYNTHESIZE data from one or more evaluationsCombine data to form an overall assessment of the merit or worth of the intervention, or to summarizeevidence across several evaluations.Synthesize data from a single evaluation: How will you synthesize data from a single evaluation?Synthesize data across evaluations: Do you need to synthesize data across evaluations? If so, howshould this be done?Generalize findings: How can the findings from this evaluation be generalized to the future, to othersites and to other programs?7. REPORT AND SUPPORT USE of findingsDevelop and present findings in ways that are useful for the intended users of the evaluation, andsupport them to make use of them.Identify reporting requirements: Who are the primary intended users of the evaluation? What aretheir primary intended uses of it? Are there secondary intended users whose needs should also beaddressed? Is there a specific timeframe required for reporting - for example, to inform a specificdecision or funding allocations?Develop Reporting Media: What types of reporting formats will be appropriate for the intended users?Ensure accessibility: How can the report be easy to access and use for different users?Review evaluation: How will evaluation reports be reviewed before they are finalized? Will there be areview of the evaluation process to improve this?Develop recommendations: Will the evaluation include recommendations? How will these bedeveloped?Support use: In addition to engaging intended users in the evaluation process, how will you support theuse of evaluation findings?BetterEvaluation - Planning an Evaluation: Using the Rainbow Framework (February 2013)http://betterevaluation.orgDESCRIBEUNDERSTANDCAUSESSYNTHESIZEREPORTANDSUPPORTUSEBetterEvaluation - Planning an Evaluation: Using the Rainbow Framework

×