Program Evaluation Handouts

910 views
819 views

Published on

Program Evaluation Material.
Incorporated into the City of Chicago's Intergovernmental Executive Development Program.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
910
On SlideShare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
60
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Program Evaluation Handouts

    1. 1. Program Evaluation George Vukotich, Ph.D. Copyright © 2004 Intergovernmental Executive Development Program
    2. 2. Program Evaluation Measuring Success It all begins with setting goals and knowing where you are going.
    3. 3. SMART GOALS <ul><li>Specific </li></ul><ul><li>Measurable </li></ul><ul><li>Agreed On </li></ul><ul><li>Realistic </li></ul><ul><li>Timed </li></ul>
    4. 4. Relevant Measurement <ul><li>Activity vs. Productivity </li></ul><ul><li>Effectiveness vs. Efficiency </li></ul><ul><li>Direct vs. Indirect </li></ul>Are we measuring the right thing.
    5. 5. Price / Quality / Innovation What is your organization known for. What is your approach to providing services.
    6. 6. What to Evaluate e.g. Evaluating IEDP <ul><li>The course material </li></ul><ul><li>The instructors </li></ul><ul><li>The facility </li></ul><ul><li>Quality of interaction with others </li></ul><ul><li>What was learned </li></ul>
    7. 7. Evaluation Process ADDIE Model <ul><li>Analyze </li></ul><ul><li>Design </li></ul><ul><li>Develop </li></ul><ul><li>Implement </li></ul><ul><li>Evaluate </li></ul>
    8. 8. Evaluate to What Standard <ul><li>Benchmarking </li></ul><ul><li>Based on pre-set goals </li></ul><ul><li>Compare to other similar projects </li></ul><ul><ul><li>Similar industries, organizations, products </li></ul></ul><ul><li>Past / Historical standards </li></ul>
    9. 9. How to evaluate <ul><li>Forms </li></ul><ul><li>Interviews </li></ul><ul><li>Focus Groups </li></ul><ul><li>Self Analysis </li></ul><ul><li>Grades </li></ul><ul><li>Supervisors Review </li></ul><ul><li>Work Productivity </li></ul>
    10. 10. Degree of Success <ul><li>To what extent does something </li></ul><ul><li>need to be effective to. </li></ul>
    11. 11. Formative vs. Summative <ul><li>Conducting the evaluation during the process and making adjustment along the way. </li></ul><ul><li>Waiting until the process is complete and evaluating the results. </li></ul>
    12. 12. Test – Retest Methodology
    13. 13. Sampling
    14. 14. Data – Not just getting it, but using it.
    15. 15. Reliability and Validity <ul><li>VALIDITY: </li></ul><ul><li>Is what we are evaluating the right thing to evaluate to help us be successful. </li></ul><ul><li>RELIABILITY: Will we get consistent measures over time. </li></ul>
    16. 16. Kirkpatrick’s 4 - Levels <ul><li>Level 1 – Reaction </li></ul><ul><li>Level 2 – Learning </li></ul><ul><li>Level 3 – Behavior </li></ul><ul><li>Level 4 -- Results </li></ul>
    17. 17. Balanced Scorecard
    18. 18. Cost / Benefit Analysis <ul><li>Employee job satisfaction </li></ul><ul><li>Students completing a program </li></ul><ul><li>Reduction in processing errors </li></ul><ul><li>Customer service ratings </li></ul>
    19. 19. Unintended Consequences What is the cost of making an error in our evaluation. Risk Analysis
    20. 20. <ul><li>♦ Turnover </li></ul><ul><li>♦ Time to Hire, Cost to Hire </li></ul><ul><li>♦ Reduced Processing Costs </li></ul><ul><li>♦ Improved Safety </li></ul><ul><li>♦ Reduced Insurance Costs </li></ul>C O R R E L A T I O N S Return on Investment (ROI)
    21. 21. Theory into practice How can your team use “program evaluation” to help you be more successful.

    ×