Four Levels of Learning Evaluation

  • 62 views
Uploaded on

BPAP Training in UMak

BPAP Training in UMak

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
62
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 1 of 11Developed by: BPAP Revision # Information Sheet 4 Levels of Learning Evaluation Learning Objectives: After reading this INFORMATION SHEET, YOU MUST be able to: 1. describe the functions of each level of Kirkpatrick’s training evaluation model; and 2. provide appropriate tools or methodologies for evaluating a training program according to the 4 levels of learning evaluation Conducting training programs entails an investment of money, time and effort. To make sure that this investment is maximized, it is important to determine the effectiveness of any training program. In this lesson, you will understand basic concepts underlying training evaluation, which will aid in measuring the effectiveness of training programs. Evaluation is “the systematic collection of descriptive and judgmental information necessary to make effective training decisions” (Goldstein, 1993, p.147). To provide structure in data gathering to evaluate training, Donald Kirkpatrick (1959) developed a sequence or set of steps (also referred to as levels) that can be followed when evaluating training: 1. Reaction – measures the responses of the participants, or how they felt about the training or activities 2. Learning – measures the extent to which the training objectives were met, or how knowledge and/or skill were increased or improved as a result of training 3. Behavior – measures the transfer of the newly acquired knowledge and/or skill to job performance 4. Results – measures overall impact to the company and can include improvements in productivity, quality, cost savings, or morale. As Kirkpatrick highlights, the 4 levels are in sequential order and the succeeding levels build on the previous ones. As one advances through the levels, obtaining data usually grows more difficult but the information gained also becomes more valuable and useful, especially to stakeholders who make the training decisions. The table lists the 4 levels, what is measured, examples of tools and methods to gather data, timing, advantages and key issues for each of the evaluation levels.
  • 2. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 2 of 11Developed by: BPAP Revision # Level What is measured? Examples of Tools and Methods How information will be gathered Time lag When the informatio n should be gathered Advantages of evaluating at this level Key issues of evaluating at this level 1 Reaction Initial satisfaction with the training experience  Feedback form  Questionnaire / survey  Verbal reaction  Focus Group Discussion Immediate  Low cost  Easy to administ er  Provides insights into participa nts’ opinions  Provides quick feedback regarding trainer presentat ion and facilitatio n  Results are largely opinions and may not be a solid basis for challenging an educational strategy  Not related to trainee learning and transfer  Positive reactions do not guarantee that learning took place 2 Learning New (or improved) knowledge and skills of participants  Pre- and post- test  Interview  Skill practice/ demonstratio n  Participant reflections (oral or written)  Participant portfolios After the program/ 2-3 weeks after the program  Relatively easy to collect  Compare d to Reaction, provides more compellin g evidence whether a training program works  Compared to Reaction, requires more time, funding, and expertise to develop and implement valid measures of learning, especially for complex knowledge or skills  Improved knowledge does not guarantee improved performance 3 Behavior Degree and quality of  Observation and interview 3-6 months  Provides stronger  Compared to Levels 1 and
  • 3. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 3 of 11Developed by: BPAP Revision # implementati on on the actual job over time  Supervisor ratings  Structured interviews with participant and their supervisor  Self- assessment using consistent measurement scales after training evidence that a training investme nt has a desired impact 2, requires more time, funding and expertise to develop and implement valid measures of transfer  Susceptible to environment variables that can change over time  Measuremen t usually requires cooperation and skill of line- managers 4 Results Organization al impact of training (eg. performance, quality, cost savings, absenteeism, turnover, morale, waste, etc.  Performance monitoring (measures are usually already in place, the challenge is relating results to training, eg. attrition, retention, quality ratings, no. of complaints, wastage, growth, etc.) 6-9 months after training  Provides the strongest possible evidence that a training program has the desired impact on an organizat ion  Significant investment and expertise are required to implement successfully across an organization  Measures are distant and can be difficult to correlate  Business results are greatly affected by external factors which obscure the cause of good or poor results
  • 4. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 4 of 11Developed by: BPAP Revision # Self-Check Matching Type: Match the descriptor from Column I against the terminologies in Column II. Write the letter on the space provided. I II 1. The level of evaluation that provides the strongest possible evidence that the training program has achieved the desired impact on an organization A. Reaction 2. This level of evaluation is easy and quick to administer and provides immediate feedback on the delivery of the training B. Learning 3. The level of evaluation that measures the extent to which the knowledge and skills gained in training have been applied to actual job performance C. Behavior 4. This level of evaluation can be measured by comparing the results of a pre- and post-test D. Results
  • 5. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 5 of 11Developed by: BPAP Revision # Answer Key 1. D 2. A 3. C 4. B
  • 6. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 6 of 11Developed by: BPAP Revision # Task Sheet Title: Create a training evaluation plan Performance Objective: Given the qualification you are assigned, you should be able to develop a data collection plan for your training program following the prescribed template. Supplies/Materials: CBLM, Bond papers Equipment: PC, printer with ink Steps/Procedure: 1. Identify the broad program goals for each level of evaluation 2. Determine which appropriate data collection tools or methods will be used for each level 3. Identify the target dates for when the tool or method will be administered 4. Identify who will be responsible for administering each data collection method or tool. 5. Evaluate your work using the Performance Criteria Checklist before submitting your work to your trainer. Be sure that all criteria are complied with. 6. Present your work to your trainer Preparing Your Work Area and Operating the Equipment: Computer 1. Ensure that the computer is turned on and the monitor displays the login page.
  • 7. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 7 of 11Developed by: BPAP Revision # 2. Press CTRL + ALT + DEL on the computer keyboard to login. 3. Enter the username and password To log off from the computer: 1. Close all open programs or files. 2. Press CTRL + ALT + DEL on your computer keyboard. 3. Click “Log Off.” Printer Loading paper to the printer: 1. Pull out the 1st or 2nd tray. 2. Adjust the lateral guides based on the size of paper to be loaded.
  • 8. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 8 of 11Developed by: BPAP Revision # 3. Load the paper into the tray. 4. Close the paper tray. Printing a document: 1. Open the file that you need to print. 2. Click Print. 3. Select the correct printer name. 4. Enter the page numbers and number of copies to be printed. 5. If needed, click the Properties button to change the printer settings. 6. Click the Print button. Safety Procedures Computer 1. Ensure that cables and plugs are secured to avoid tripping or slipping. 2. If equipment is damaged, notify your facilitator. 3. Do not eat or drink beside the computer. 4. Do not handle the computer with wet hands. 5. Do not forget to log off the computer after use. Printer 1. Do not try to dismantle covers and panels which are fixed to the printer. Removing a high-voltage part or laser beam source can cause electric shock or blindness. 2. Do not plug or unplug the power cord with wet hands. 3. Plug the power cord completely into the power socket. Failure to do this could result to fire or electric shock.
  • 9. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 9 of 11Developed by: BPAP Revision # Assessment Method: Portfolio
  • 10. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 10 of 11Developed by: BPAP Revision # EVALUATION PLAN TEMPLATE Qualification: Sales Training Program Date: Prepared By: Level Broad Program Goals Data Collection Tool or Methods Timing Person responsible 1 Reaction/Satisfaction Eg. Average satisfaction score of 4.5 (5.0 scale) from 90% of participants At least 10 recommendations to improve training delivery Feedback form Focus Group Discussion End of each day of class Day after course end Trainer 2 Learning Eg. Acquisition of skills: adherence to 6 steps of customer influence; application of at least 3 closing techniques Demonstration/ role play with performance checklist During program Trainer 3 Application/ implementation Eg. Improved sales conversion rateby 5% Reduced barriers to closing sales Supervisor ratings Self-assessment questionnaire 3 months after program Supervisor 4 Business impact Eg. Increase in revenue from sales to 40% Sales revenue monitoring 6 months after program Manager
  • 11. Trainers Methodology Plus Facilitate Learning Sessions Date Developed: September 2012 Document No: Issued by: Page 11 of 11Developed by: BPAP Revision # Performance Criteria Checklist CRITERIA Did you… YES NO Identify the broad program goals for each level of evaluation? Determine appropriate data collection tools or methods for each level of evaluation? Identify target dates for when the tool or method will be administered? Identify a person to be responsible for administering each data collection method or tool?