Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluation of training

34,306 views

Published on

Learning Resource for HRM , HRD professionals and MBA students on how to evaluate training

Published in: Business, Technology
  • Many thanks. I'd like to share an online training software - ezTalks:Free for up to 100 Participants,Group Video/Audio/IM Chat, Screen Sharing, Whiteboard and Record,etc.(https://www.eztalks.com)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • A comprehensive yet simple and straightforward coverage of the subject matter -both from an academic and practical perspective - kudos to Mr Jayadeva
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Evaluation of training

  1. 2. Evaluation of Training by Jayadeva de Silva Raman K. Attri ©
  2. 3. Formative Evaluation <ul><li>Formative evaluation (also known as internal) is a method of judging the worth of a program while the program activities are forming (in progress). This part of the evaluation focuses on the process. </li></ul>
  3. 5. Summative <ul><li>The summative evaluation (also know as external) is a method of judging the worth of a program at the end of the program activities (summation). The focus is on the outcome. </li></ul>
  4. 8. The Ten Factors of Developing a Training Program <ul><li>1. Determine needs </li></ul><ul><li>2. Set objectives </li></ul><ul><li>3. Determine subject content </li></ul><ul><li>4. Select qualified applicants </li></ul><ul><li>5. Determine the best schedule </li></ul>
  5. 9. The Ten Factors of Developing a Training Program <ul><li>6. Select appropriate facilities </li></ul><ul><li>7. Select qualified instructors </li></ul><ul><li>8. Select and prepare audiovisual aids </li></ul><ul><li>9. Co-ordinate the program </li></ul><ul><li>10. Evaluate the program </li></ul>
  6. 10. Donald Kirkpatrick
  7. 11. Kirkpatrick
  8. 12. Evaluating <ul><li>“ The reason for evaluating is to determine the effectiveness of a training program.” (Kirkpatrick, 1994, ) </li></ul>
  9. 13. Reasons for Evaluating <ul><li>Kirkpatrick gives three reasons ‘why’ there is a need to evaluate training: </li></ul>
  10. 14. Reasons for Evaluating <ul><li>1.“To justify the existence of the training department by showing how it contributes to the organizations’ objectives and goals.” </li></ul>
  11. 15. Reasons for Evaluating <ul><li>2. “To decide whether to continue or discontinue training programs.” </li></ul>
  12. 16. Reasons for Evaluating <ul><li>3. “To gain information on how to improve future training programs.” (Kirkpatrick, 1994, ) </li></ul>
  13. 17. Kirkpatrick: Evaluating Training Programs <ul><li>“ What is quality training?” </li></ul><ul><li>“ How do you measure it?” </li></ul><ul><li>“ How do you improve it?” </li></ul>
  14. 18. 'Training Evaluation <ul><li>Senior management </li></ul><ul><li>The trainers </li></ul><ul><li>Line management </li></ul><ul><li>The training manager </li></ul><ul><li>The trainee </li></ul>
  15. 20. 4-Tier Practical Training Value Measurement Model Training Expectation Measurement Trainee feedback scores on initial impression on to what extent training met their expectations of learning, skills and knowledge. Tool: Survey Form Student feedback Tier-1 Expectations
  16. 21. 4-Tier Practical Training Value Measurement Model Training Expectation Measurement Trainee feedback scores on initial impression on to what extent training met their expectations of learning, skills and knowledge. Tool: Survey Form Student feedback Tier-1 Expectations Pre-Training Vs Post-Training Assessment Tier-2 Improvement Training Improvement Measurement Assessment of trainee’s training exposure and expertise gained on same set of tasks before and after training. Includes comparison of in-training & Post-training test performance, If applicable. Tool: Pre-training and Post-training Survey Forms. Post-training performance tests .
  17. 22. 4-Tier Practical Training Value Measurement Model Training Expectation Measurement Trainee feedback scores on initial impression on to what extent training met their expectations of learning, skills and knowledge. Tool: Survey Form Training effectiveness Measurement Post-training normalized feedback scores and its quarterly trends. Feedback from trainee Manager on visible incremental changes in trainee skills, service parameters and on-job behavior. Tool: Quarterly Follow-on Survey from Managers Student feedback Tier-1 Expectations Post-Training On-Job Behavior Survey Tier-3 Effectiveness Pre-Training Vs Post-Training Assessment Tier-2 Improvement Training Improvement Measurement Assessment of trainee’s training exposure and expertise gained on same set of tasks before and after training. Includes comparison of in-training & Post-training test performance, If applicable. Tool: Pre-training and Post-training Survey Forms. Post-training performance tests .
  18. 23. 4-Tier Practical Training Value Measurement Model Training Expectation Measurement Trainee feedback scores on initial impression on to what extent training met their expectations of learning, skills and knowledge. Tool: Survey Form Training effectiveness Measurement Post-training normalized feedback scores and its quarterly trends. Feedback from trainee Manager on visible incremental changes in trainee skills, service parameters and on-job behavior. Tool: Quarterly Follow-on Survey from Managers Student feedback Tier-1 Expectations Post-Training On-Job Behavior Survey Tier-3 Effectiveness Improvement in business indicators, revenue figures or success parameters Tier-4 Impact Pre-Training Vs Post-Training Assessment Tier-2 Improvement Training Improvement Measurement Assessment of trainee’s training exposure and expertise gained on same set of tasks before and after training. Includes comparison of in-training & Post-training test performance, If applicable. Tool: Pre-training and Post-training Survey Forms. Post-training performance tests . Training Impact Measurement Impact of training for improving revenues, enhancing business or other success factors driving the training needs. Tool: Comparison of Baseline data and Quarterly business data, Impact Factor calculation sheets
  19. 24. Tier-1: how closely does Training meet the expectations of trainees? <ul><li>Feedback score collected from each trainee at end of the class </li></ul><ul><li>Record general impression about the Training on a numerical scale of 1-5 or similar </li></ul><ul><li>Average score reflects how closely training met expectations of field and how close it is aligned with business needs identified earlier which drives the training. </li></ul>
  20. 25. Reactions. <ul><li>Reaction may best be defined as how well the trainees liked a particular training program.&quot; </li></ul><ul><li>Reactions are typically measured at the end of training. </li></ul>
  21. 27. FEEDBACK FORM Please take a few minutes to fill out this feedback form. Your feedback will help us strengthen the course delivery. Course Title - Name of the facilitator - You may provide feedback of the program on the following criteria by putting a (  ) mark in the space provided. 4 – Exceeds Expectation; 3 – Meets Expectation; 2 – Needs Improvement; 1 – Unsatisfactory 1. What did you like most in the workshop? 2. What did you dislike in the workshop? 3. What other changes would you suggest in case the same course is conducted in future? Learning & Development Human Resources Overall Evaluation Supporting Materials Relevance Examples, Cases, Simulation & Exercises Communication Presentation Style Subject Knowledge 1 2 3 4
  22. 28. Verification of Reactions <ul><li>Typically 'happy sheets' </li></ul><ul><li>Feedback forms based on subjective personal reaction to the training experience </li></ul><ul><li>Verbal reaction which can be noted and analyzed </li></ul><ul><li>Post-training surveys or questionnaires </li></ul><ul><li>Subsequent verbal or written reports given by delegates to managers back at their jobs </li></ul>
  23. 40. Tier-2: how well Training improved skills of trainees? <ul><li>Collect Pre-training and post-training data on trainee’s skills before and after the training. A well drafted skill based survey can be used. </li></ul><ul><li>Compare improvements in exposure or expertise gained by the trainee as seen on post-training survey by comparing it with pre-training survey. </li></ul><ul><li>Conduct in-training skill tests as part of the qualification criteria to measure the improvement in the learned skills. </li></ul>
  24. 41. Learning. <ul><li>What principles, facts, and techniques were understood and absorbed by the participants?&quot; </li></ul><ul><li>What the trainees know or can do can be measured during and at the end of training </li></ul>
  25. 44. Verification of Learning <ul><li>WHAT? </li></ul><ul><li>What knowledge was acquired? </li></ul><ul><li>What skills were developed or enhanced? </li></ul><ul><li>What attitudes were changed? </li></ul><ul><li>HOW? </li></ul><ul><li>Tests before and after the training </li></ul><ul><li>Interview or observation can be used before and after training. </li></ul><ul><li>Measurement and analysis is possible and easy on a group scale </li></ul><ul><li>Reliable, clear scoring and measurements need to be established </li></ul>
  26. 45. Tier-3: How effective does the training prove in changing On-job-behavior and skills of trainee? <ul><li>Collect quarterly feedback from trainee’s manager on his observations on improvement of trainee’s on-job performance/ behavior or skills after attending training. </li></ul><ul><li>Analyze the trends in key on-job-performance parameters or indicators as seen on survey with respect to previous quarters. </li></ul><ul><li>Convert delta into normalized scores to indicate the value created by training in the work efficiency of the trainee </li></ul><ul><li>Ensure continual measurement of effectiveness after regular intervals to assess long term value of the training. </li></ul>
  27. 46. Behavior. <ul><li>Changes in on-the-job behavior,Behavior changes are acquired in training and they then transfer (or don't transfer) to the work place. </li></ul><ul><li>What skills did the learner develop, that is, what new information is the learner using on the job? </li></ul>
  28. 47. When evaluating change in behavior, decide: <ul><li>When to evaluate </li></ul><ul><li>How often to evaluate </li></ul><ul><li>How to evaluate </li></ul>
  29. 54. Verification of Behaviour <ul><li>What? </li></ul><ul><li>Whether the trainee is able to transfer the learning to the work environment </li></ul><ul><li>New learning is demonstrated </li></ul><ul><li>Whether the trainee is motivated </li></ul><ul><li>How? </li></ul><ul><li>self-assessment can be useful, using carefully designed criteria and measurements </li></ul><ul><li>cooperation and skill of observers, typically line-managers, are important factors, and difficult to control </li></ul><ul><li>Use of focus groups </li></ul>
  30. 55. Tier-4: To what extent training does impact in improving business / revenue or service parameters specific to trainees or group of trainees? <ul><li>Measure quarterly the business indicators of the trainee’s job or service parameters based on nature of job. Business indicators could be collected based on individual job or group responsible for the said function. </li></ul><ul><li>Ideally business indicator data before the training should be used as baseline. </li></ul><ul><li>Record the business parameters or governing service parameters on quarterly basis </li></ul><ul><li>Calculate qualitative or quantitative impact factor based on normalized delta. </li></ul>
  31. 56. Results. <ul><li>Reduction of costs; </li></ul><ul><li>Reduction of turnover and absenteeism; </li></ul><ul><li>Reduction of grievances; </li></ul><ul><li>increase in quality and quantity or production; </li></ul><ul><li>or Improved morale which, it is hoped, will lead to some of the previously stated results. </li></ul><ul><li>These factors are also measurable in the workplace </li></ul>
  32. 61. The Four Levels <ul><li>Reaction </li></ul><ul><li>Learning </li></ul><ul><li>Behavior </li></ul><ul><li>Results </li></ul>
  33. 63. <ul><li>“ The Four Levels represent a sequence of ways to evaluate (training) programs….As you move from one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information.” (Kirkpatrick, 1994,) </li></ul>
  34. 64. When all conditions are met, the employee must: <ul><li>Realize an opportunity to use the behavioral changes. </li></ul><ul><li>Make the decision to use the behavioral changes. </li></ul><ul><li>Decide whether or not to continue using the behavioral changes. </li></ul>
  35. 65. Return on Impact (RoI) is a new approach which measures the difference training has created on those governing business/ service or revenue parameters which drives training needs.
  36. 66. The four conditions Kirkpatrick identifies for changes to occur: <ul><li>Desire to change </li></ul><ul><li>Knowledge of what to do and how to do it </li></ul><ul><li>Work in the right climate </li></ul><ul><li>Reward for (positive) change </li></ul>
  37. 68. An individual and group exercise <ul><li>Group 1- As an HR Manager,how will you evaluate:- </li></ul><ul><li>T he output effectiveness of a supervisory training programme? </li></ul><ul><li>Group 2- The Outputs to be Evaluated after a Sales Training programme </li></ul><ul><li>Group 3- The Outputs to be Evaluated after a Customer Relations Training programme </li></ul>
  38. 69. Outputs to be Evaluated after Supervisory and Management Training <ul><li>increased output </li></ul><ul><li>reduced absenteeism and tardiness </li></ul><ul><li>reduced cost of new hires </li></ul><ul><li>reduced turnover </li></ul><ul><li>increased number of employee suggestions </li></ul><ul><li>climate survey data (morale and attitudes) </li></ul>
  39. 70. Outputs to be Evaluated after Sales Training <ul><li>sales volume </li></ul><ul><li>average sale size </li></ul><ul><li>add-on sales </li></ul><ul><li>close-to-call ratio </li></ul><ul><li>ratio of new accounts to old accounts </li></ul><ul><li>number of items per order </li></ul>
  40. 71. Outputs to be Evaluated After Customer Relations Training <ul><li>accuracy of orders </li></ul><ul><li>size of orders </li></ul><ul><li>number of transactions per day </li></ul><ul><li>adherence to credit procedures </li></ul><ul><li>number of lost customers </li></ul><ul><li>amount of repeat business </li></ul><ul><li>number of referrals </li></ul><ul><li>number of complaints </li></ul>
  41. 72. Questions & Concerns

×