Developmental evaluation- Michael Quinn Patton a recent concept that has expanded possible roles for evaluators Patton used it to describe certain long-term, partnering relationships with clients who are themselves involved in ongoing program development Evaluator’s role is that of a guide to client; provides feedback, keeps client on target by asking questions of the evaluation, etc. Formative - Michael Scriven - evaluation conducted during the development or improvement of a program or product, with the intent to improve. (analogy to cook tasting the soup vs. guests tasking the soup Summative- implemented to determine the merit, worth, or value of the evaluand - at the completion of a program Monitoring & auditing; outcome eval,(changes in knowledge, attitudes, practices); impact eval,, performance measurement (many gov’t progra Process recently determined another use of eval for organizational learning. When evaluators use a participatory and collaborative approach, the process of being involved leads to participants’ increased understanding and knowledge; impt if eval is integrated into daily work practices, esp used with a team based approach
Example: in text Evaluate the ability of an organization’s website to motivate people to attend a particular training workshop Establishing Criteria What are the criteria of a good web site? Readibility, color, interactivity? Navigability? Ans from subject matter experts or the research lit on websites Constructing Standards How well should the website perform? What constitutes effectiveness? If 1,000 people visited the site each day and 25% of those signed up for the training course, we select this as an acceptable standard. Measuring Performance and Comparing with Standards How well is the web site performing? Measure. Synthesizing and Integrating Evidence into a Judgment of Worth Our data indicate 700 people per day visit and of those, 15% register. The question becomes, Is it worth it for the organization to continue offering and supporting the website? Developing Recommendations We recommend that the website is not worth the resources to maintain it, so it should either be redesigned or eliminated and another method be determined to market our programs to trainees.
Concluding remarks from this chapter Evaluation is A systematic process; a planned and purposeful activity A mechanism for collecting data on questions or issues A process for enhancing knowledge and decision-making A means of judging the worth, merit, or value of an evaluand Not the same as research Evaluations should only be conducted with the intention of using the results Evaluations may be developmental, formative, summative or process. They can also be any combination of these Evaluation can be conducted by either internal or external evaluators.
Adlt 606 class 11 intro to evaluation november 15, 2011
Understanding the Rationale for Developing an Evaluation Plan Selecting an Evaluation Design ADLT 606 Class 11
Developing an Evaluation Plan <ul><li>Developing an evaluation’s purpose statement </li></ul><ul><li>Identifying key stakeholders </li></ul><ul><li>Developing key evaluation questions </li></ul><ul><li>Note: this focusing phase needs to be completed BEFORE determining the design and data collection methods </li></ul>