Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluation Theory Tree: Evaluation Approaches


Published on

During the 2015 American Evaluation Association's Annual Conference in Chicago, Katherine Haugh and Deborah Grodzicki conducted a real time data mini-study to see which evaluation approaches evaluators at #eval15 use most frequently in their work. Basing their mini-study off of Marvin C. Alkin's "Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences," they asked evaluators to vote for the top two approaches they used most often. This handout accompanied the real time data mini-study to provide more information about the formation of the evaluation theory tree, it's three branches, and definitions of the evaluation approaches associated with each branch.

Published in: Government & Nonprofit

Evaluation Theory Tree: Evaluation Approaches

  1. 1. Social accountability Social inquiry Epistemology USE approaches #Eval15 Evaluation Theory Tree Mini Study Katherine Haugh @Katherine_Haugh Deborah Grodzicki @DebGrodzicki Innovation Network Twitter @InnoNet_Eval Email Phone 202-728-0727 Web VALUE approaches METHODS approaches The purpose of this mini study is to measure which evaluation approaches evaluators at #Eval15 use most frequently in their work. During the conference, we will be asking evaluators to indicate the approaches they use most often. Our hope for this mini study is that it will start to bridge the gap between theory and practice in the evaluation field. Stop by our booth to participate! Theory-driven (Rossi & Chen) Experimental & QE (Campbell, Cronbach, Cook) Emergent realist (Henry & Mark) Objective-oriented (Tyler) Goal-free evaluation (Scriven) Cost-analysis (Levin) Responsive evaluation (Stake) Value-engaged (Greene) Fourth generation (Guba & Lincoln) CIPP (Stufflebeam) Utilization-focused (Patton) Developmental evaluation (Patton) Empowerment evaluation (Fetterman) Participatory evaluation (Cousins) Appreciative inquiry (Preskill)
  2. 2. ROOTS:  Social accountability – stems from the need for accounting for actions or resources: goal accountability, process accountability, and outcome accountability.  Social inquiry - emanates from a concern for employing a systematic and justifiable set of methods for determining accountability and measuring social phenomena.  Epistemology - addresses the arguments in the nature of knowledge. Evaluators who have engaged in epistemological discussions typically draw on one of three broad areas of thinking: post-positivism, constructivism, or pragmatism. USE evaluation approaches:  CIPP Evaluation (Stufflebeam) – Context, Input Process, and Product (CIPP) evaluation provides a systematic way of looking at many different aspects of a program’s development process by looking at context, input, process, and product.  Utilization-focused evaluation (Patton) – an approach based on the principle that an evaluation should focus on its usefulness to its intended users.  Developmental evaluation (Patton) – rigorous inquiry for emerging programs in the context of change. It is an intentional approach to using data in meaningful ways to inform innovation in progress.  Empowerment evaluation (Fetterman) –fosters self-determination, to the point where program stakeholders essentially conduct their own evaluations. The outside evaluator often serves as a coach or additional facilitator, providing clients with the knowledge and tools for continuous self-assessment and accountability.  Participatory Evaluation (Cousins) – fosters program personnel participation in the evaluation as a way to achieve buy-in. It focuses on the need for participation to heighten the possibility of utilization.  Appreciative Inquiry (Preskill) – a process that builds on past successes (and peak experiences) in an effort to design and implement future actions. METHODS evaluation approaches:  Theory-driven evaluation (Rossi & Chen) – involves the construction of a detailed program theory, which is then used to guide the evaluation. This approach helps reconcile internal and external validity.  Experimental and Quasi-Experimental Design for Research (Campbell, Cronbach, & Cook) – approaches that include some type of intervention or treatment and provide a comparison. Randomization is the key to true experiments, and lack of randomization is the defining characteristic of quasi-experiments.  Emergent realist evaluations (Henry & Mark) – argues that social betterment, rather than the more popular and pervasive goal of utilization, should motivate evaluation.  Objective-oriented evaluation (Tyler) - focuses on identifying objectives, measuring those objectives, and identifying the extent to which those objectives have been met. VALUING evaluation approaches:  Goal-free evaluation (Scriven) – an evaluation approach in which the evaluator assumes responsibility for determining which program outcomes to examine, rejecting the objectives of the program as a starting point.  Cost analysis evaluation (Levin) – economics-based strategies for determining the value of a program or policy prior to and during implementation  Responsive evaluation (Stake) –the beliefs that there is no true value to anything (i.e. knowledge is context bound), and that stakeholder perspectives are integral elements in evaluations  Value-engaged evaluation (Greene) –evaluation should be democratic and used to determine value, specifically by developing a consensus (through inclusion, dialogue, and deliberation) around a set of criteria used to determine the value of a program.  Fourth generation evaluation (Guba & Lincoln) – individuals “construct” their perceptions of reality. The role of the constructive evaluator is to tease out these constructions and bring them into conjunction with one another. DEFINITIONS The definitions provided below are from Marvin C. Alkin’s Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences, Second Edition.