What is Evaluation and Why Should You Care? Organization Development Network of WNY

1,168 views

Published on

“Evaluation” and “data-based decision making” are common terms that are understood to varying degrees, but not always put into practice. People use evaluative thinking and make “data-based” decisions every day. For example, if you are trying to lose weight, you might look at nutrition information to decide whether to eat an apple or a candy bar. Or, your organization might decide to use one training program over another based on employee feedback and previous results of the program.

However, formal evaluation that elicits information that represents reality and facilitates accurate data-based decisions can be challenging. We will provide an interactive introduction to formal evaluation that will answer:

What is evaluation?
Why would I do evaluation?
How do I do evaluation?

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,168
On SlideShare
0
From Embeds
0
Number of Embeds
719
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Talk about examples of what we have done.School DistrictsHigher Ed Non-profitsFoundationsFederalStateFoundation funded projectsOrganizationally funded projects – need assessments, specific project focused analysis, logic modeling
  • This definition is hardly perfect. There are many types of evaluations that do not necessarily result in an assessment of worth or merit -- descriptive studies, implementation analyses, and formative evaluations, to name a few. Better perhaps is a definition that emphasizes the information-processing and feedback functions of evaluation. Note: What is evaluation?
  • This definition is hardly perfect. There are many types of evaluations that do not necessarily result in an assessment of worth or merit -- descriptive studies, implementation analyses, and formative evaluations, to name a few. Better perhaps is a definition that emphasizes the information-processing and feedback functions of evaluation. 
  • Can answer all or any of these.
  • needs assessment determines who needs the program, how great the need is, and what might work to meet the needevaluability assessment determines whether an evaluation is feasible and how stakeholders can help shape its usefulnessstructured conceptualization helps stakeholders define the program or technology, the target population, and the possible outcomesimplementation evaluation monitors the fidelity of the program or technology deliveryprocess evaluation investigates the process of delivering the program or technology, including alternative delivery procedures
  • Focuses on answering questionsWere your program objectives met?Will you need to improve and modify the overall structure of the program?What is the overall impact of the program?What resources will you need to address the program’s weaknesses?
  • outcome evaluations investigate whether the program or technology caused demonstrable effects on specifically defined target outcomesimpact evaluation is broader and assesses the overall or net effects -- intended or unintended -- of the program or technology as a wholecost-effectiveness and cost-benefit analysis address questions of efficiency by standardizing outcomes in terms of their dollar costs and valuessecondary analysis reexamines existing data to address new questions or use methods not previously employedmeta-analysis integrates the outcome estimates from multiple studies to arrive at an overall or summary judgment on an evaluation question
  • Empowerment evaluation – emphasizes that the evaluation process and methods should be empowering to those who are being evaluation (Fetterman, Kaftarian et al. 1996).Goal-free evaluation – in which the evaluator’s task is to examine all of the outcomes of a program, not just its formal outcomes as identified in its objectives (Scriven 1972).Naturalistic or 4th generation evaluation – emphasizes the qualitative uniqueness of program and is a reaction against the limitation of quantitative evaluation approaches (Guba and Lincoln 1989).Stakeholder evaluation – looks at the differing perspectives of all of a program’s stakeholders (those who have an interest in it) (Greene 1988).Strategic evaluation – emphasizes that evaluation design decisions should be driven by the strategic value of the information they will provide for solving social problems (Duignan 1997).Theory Based Evaluation – puts an emphasis on detailing the assumptions on which a program is based (intervention logic) and follows those steps to see if they occur (Birckmayer and Weiss 2000).Utilization-focused evaluation – determines methods on the basis of what is going to be most useful to different audiences (Patton 1986). It helps make sure that evaluation is not only useful to funders by measuring successes, but also:•Helps staff make real improvements in the program•Helps administrators make decisions about future programs•Helps the community make decisions about resources.
  • Note: why would you do evaluation? What are they currently doing? (get them to talk about what they are doing) talk about when to formalize the process… when it would be necessaryFor OD:Decision makingRetentionEmployee developmentStrategic planningTo measure hiring practices
  • For-profit organizations:ImprovementUnderstanding sales trendsUnderstanding marketing strategies
  • For non profits:Demonstrating Impact EffectivenessImprovementSustainability
  • For schools:Attendance analysisProgram impact on test scoresTeacher effectivenessSelecting service providers
  • Note: why would you do evaluation? Is there anyone currently doing formalevaluation? (get them to talk about what they doing)
  • Note: why would you do evaluation? Is there anyone currently doing formalevaluation? (get them to talk about what they doing)
  • Note: why would you do evaluation? Is there anyone currently doing formalevaluation? (get them to talk about what they doing)
  • Note: why would you do evaluation? Is there anyone currently doing formalevaluation? (get them to talk about what they doing)
  • What is Evaluation and Why Should You Care? Organization Development Network of WNY

    1. 1. What is Evaluation, and whyshould you care?Organization Development Network (ODN) of WNY, May 2013
    2. 2. Agenda• Introduction• What is evaluation?• Why would I do evaluation?• How do I do evaluation?• How do I share results?• Activity• Questions
    3. 3. Introduction• Who we are• What do we do
    4. 4. What is Evaluation?• Definition• Types• Approaches
    5. 5. What is Evaluation?• Definition– Evaluation is the systematic assessment of theworth or merit of some object– Evaluation is the systematic acquisition andassessment of information to provide usefulfeedback about some object
    6. 6. What is Evaluation?NowWhat?SoWhat?What?
    7. 7. What is Evaluation?• Evaluation Types– Two main categories• Formative• Summative
    8. 8. What is Evaluation?• Formative:– On-going process that allows for feedback to beimplemented during a program cycle– Adjustments “on the fly” to help achieve program goals
    9. 9. What is Evaluation?• Formative Types:– Needs Assessment– Evaluability Assessment– Structured Conceptualization– Implementation Evaluation– Process Evaluation
    10. 10. What is Evaluation?• Summative:– occurs at the end of a program cycle and providesan overall description of program effectiveness.– Focuses on answering questions
    11. 11. What is Evaluation?• Summative Types:– Outcome Evaluation– Impact Evaluation– Cost-Effective/Cost-Benefit Analysis– Secondary Analysis– Meta-Analysis
    12. 12. What is Evaluation?• Evaluation Approaches– Empowerment Evaluation– Goal-Free Evaluation– Naturalistic Evaluation– Stakeholder Evaluation– Strategic Evaluation– Theory Based Evaluation– Utilization-Focused Evaluation (UFE)
    13. 13. UpFront Consulting, MN
    14. 14. Why would I do evaluation?
    15. 15. Why would I do evaluation?
    16. 16. Why would I do evaluation?
    17. 17. Why would I do evaluation?
    18. 18. How do I do evaluation?
    19. 19. STEP 1: What question(s) do you want toanswer?
    20. 20. STEP 2: What do you need tocollect to answer the question(s)?
    21. 21. STEP 3: How would you collect data?
    22. 22. STEP 4: How would you report theresults?
    23. 23. Activity• THINK- Think about the following questions:– What question(s) do you want to answer foryour organization??– What do you need to collect to answer the question?– How would you collect data?– How would you report the results?• PAIR- Brainstorm with thepeople around you• SHARE- With everyone
    24. 24. Ideas? Questions?Contact info:Buffalo, NY 14203Office: (716) 362-0627Fax: (716) 362-0712www.ViaEvaluation.comemily@viaeval.comchris@viaeval.comKomani@viaeval.com

    ×