Developing Valid Level 1 Evaluation Forms

4,731 views
4,549 views

Published on

Developing Valid Level 1 Evaluation Forms

Conducting Level 1 evaluations is a ubiquitous part of the workplace learning and performance landscape. If used properly and designed well, they ensure that learning programs are “on track” to achieve key objectives. Yet despite their widespread use and critical role, many questions remain regarding their core purpose and the validity of the results. In this session, you will analyze the true core purpose of Level 1 evaluations – what they should and should not be used for – assess several common mistakes made in creating Level 1 evaluations; and examine six techniques that can be used to eliminate those mistakes and develop valid Level 1 evaluation forms. Learning Objectives:
1. Analyze the true core purpose of Level 1 evaluations. 2. Use six techniques to create valid Level 1 evaluation forms.

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,731
On SlideShare
0
From Embeds
0
Number of Embeds
43
Actions
Shares
0
Downloads
60
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Designing Valid Level 1 Evaluation Forms
  • Developing Valid Level 1 Evaluation Forms

    1. 1. Developing Valid Level 1 Evaluation Forms Presented by: Ken Phillips Phillips Associates May 22, 2008
    2. 2. OBJECTIVES <ul><li>Examine core purpose of Level 1 evaluations </li></ul><ul><li>Analyze strengths and shortcomings of 3 sample Level 1 evaluation forms </li></ul><ul><li>Apply 6 tips to design more valid Level 1 evaluations </li></ul>
    3. 3. QUESTIONS <ul><li>What is the core purpose of Level 1 evaluations? </li></ul><ul><li>How valid are the results? </li></ul>
    4. 4. ANSWERS <ul><li>Improve a </li></ul><ul><li>learning program </li></ul><ul><li>not prove </li></ul><ul><li>something </li></ul>Not very valid &
    5. 5. <ul><li>SAMPLE </li></ul><ul><li>EVALUATION </li></ul><ul><li>FORMS </li></ul>
    6. 6. <ul><li>6 TIPS </li></ul>
    7. 7. TIP 1 <ul><li>Where appropriate, match up qualitative questions with quantitative measures </li></ul>
    8. 8. EXAMPLE <ul><li>In a word, how would you describe this </li></ul><ul><li>session? ____________________________ </li></ul>Using a number, how would you describe this session? No Value Great Value 1 2 3 4 5
    9. 9. TIP 2 <ul><li>When collecting quantitative data using a Likert scale, create a response scale with numbers at regularly spaced intervals and words only at each end* </li></ul><ul><li>* Palmer Morrel-Samuels, “Getting the Truth into Workplace Surveys”, Harvard </li></ul><ul><li>Business Review , 2002. </li></ul>
    10. 10. EXAMPLE <ul><li>How would you rate the overall effectiveness of </li></ul><ul><li>the learning activities used in this session? </li></ul><ul><li>Not Very </li></ul><ul><li>Effective Effective </li></ul><ul><li>1 2 3 4 5 </li></ul>
    11. 11. TIP 3 <ul><li>Use only one response scale with an odd number of points (e.g. 5, 7,9, 11)* </li></ul><ul><li>* Palmer Morrel-Samuels, “Getting the Truth into Workplace Surveys” </li></ul>
    12. 12. ODD vs EVEN SCALE
    13. 13. TIP 4 <ul><li>Use small numbers at the low or left end of the scale and larger numbers at the right or high end of the scale </li></ul>
    14. 14. TIP 5 <ul><li>Write items either as a continuum or as a statement </li></ul>
    15. 15. <ul><li>How effectively did the facilitator use AV </li></ul><ul><li>materials? </li></ul><ul><li>Not Very </li></ul><ul><li>Effectively Effectively </li></ul><ul><li>1 2 3 4 5 </li></ul>EXAMPLES The facilitator effectively used AV materials. Strongly Strongly Disagree Agree 1 2 3 4 5
    16. 16. TIP 6 <ul><li>Include at least one item asking participants how relevant the learning event/material was to them and their job </li></ul>
    17. 17. EXAMPLE <ul><li>How would you rate the overall relevance of the </li></ul><ul><li>session to you and your job? </li></ul><ul><li>Not at all Very </li></ul><ul><li>relevant relevant </li></ul><ul><li>1 2 3 4 5 </li></ul>
    18. 18. REFERENCES <ul><li>Boehle, Sarah, “Remember that mean, crabby teacher in </li></ul><ul><li>high school?”, Training , August 2006, pps. 17-22. </li></ul>Phillips, Ken, “Six Tips on Developing Valid Level 1 Evaluation Forms”, Training Today , Fall 2007, pps. 8 & 14. Morrel-Samuels, Palmer, “Getting the Truth into Workplace Surveys”, Harvard Business Review , February 2002, pps. 111-118. Miner, Nanette, “The Art of Test Creation”, T + D , July 2007, pps. 27-29.
    19. 19. <ul><li>Ken Phillips </li></ul><ul><li>Phillips Associates </li></ul><ul><li>34137 N. Wooded Glen Drive </li></ul><ul><li>Grayslake, Illinois 60030 </li></ul><ul><li>(847) 231-6068 </li></ul><ul><li>www.phillipsassociates.com </li></ul><ul><li>[email_address] </li></ul>

    ×