Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring roi of training

1,170 views

Published on

  • Be the first to comment

Measuring roi of training

  1. 1. 1www.exploreHR.org Measuring ROI of Training
  2. 2. 2www.exploreHR.org You can download these excellent slides at : www.HR-Management-Slides.com
  3. 3. 3www.exploreHR.org Agenda 1. Measuring the Effectiveness of Training Program 2. Measuring ‘Return on Investment’ of Training 3. Enhancing the Effectiveness and ROI of Training
  4. 4. 4www.exploreHR.org Measuring the Effectiveness of Training Program
  5. 5. 5www.exploreHR.org Training Need Analysis Training Objectives Training Delivery Training Evaluation Training Process What are the training needs for this person and/or job? Objective should be measurable and observable Techniques include on- the-job- training, action learning, etc. Measure reaction, learning, behavior, and results
  6. 6. 6www.exploreHR.org The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Four Levels of Training Effectiveness
  7. 7. 7www.exploreHR.org Test the trainees to determine if they learned the principles, skills, and facts they were to learn. Evaluate trainees’ reactions to the program. Did they like the program? Did they think it worthwhile? Level 1 - Reaction Level 2 - Learning The Four Levels of Evaluation
  8. 8. 8www.exploreHR.org What final results were achieved in terms of the training objectives previously set? Did the number of customer complaints about employee drop? Did the reject rate improve? Was turnover reduced, and so forth. Ask whether the trainees’ behavior on the job changed because of the training program. For example, are employees in the store’s complaint department more courteous toward disgruntled customers than previously? Level 3 – Behavior Application Level 4 – Business Impact The Four Levels of Evaluation
  9. 9. 9www.exploreHR.org Value of Information Frequency of Use Difficulty of Assessment Level I. Reaction II. Learning III. Behavior IV. Results The Four Levels of Evaluation Least valuable Most valuable Frequent Infrequent Easy Difficult
  10. 10. 10www.exploreHR.org Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Did they like the facilitators? Did they like the training accommodation and facilities? Level 1 - Reaction
  11. 11. 11www.exploreHR.org Guidelines for Evaluating Reaction 1. Determine what you want to find out 2. Design a form that will quantify reactions 3. Encourage written comments and suggestions 4. Get 100 percent immediate response 5. Get honest response 6. Develop acceptable standards 7. Measure reactions against standards, and take appropriate action 8. Communicate reaction as appropriate
  12. 12. 12www.exploreHR.org Please give us your frank reactions and comments. They will help us to evaluate this program and improve future programs. Program : Facilitator : 1. How do you rate the subject (interest, benefit, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?) a. Excellent b. Very Good c. Good d. Fair e. Poor 3. How do you rate the facilities? (comfort, convinience, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 4. What would have improved the program? ………………………………………………………………………………… ………………………………………………………………………………… ………………………………………………………………………………… Sample of Reaction Form
  13. 13. 13www.exploreHR.org Level 2 - Learning Measuring learning means determining one or more of the following : • What knowledge was learned? • What skills were developed or improved? • What attitudes were changed? Level 2 - Learning
  14. 14. 14www.exploreHR.org Guidelines for Evaluating Learning 1. Use a control group if practical 2. Evaluate knowledge, skills and/or attitudes both before and after the program 3. Use a paper-and-pencil test to measure knowledge 4. Use a performance test to measure skills 5. Get 100 percent immediate response 6. Use the results of evaluation to take appropriate action
  15. 15. 15www.exploreHR.org Guidelines for Evaluating Learning Experimental Group Control Group Pre Test Score 45 46 Post Test Score 55 48 Gain 10 2 Pretest and Posttest Scores on Change Management Training Example :
  16. 16. 16www.exploreHR.org Level 3 – Behavior Application • The frequency of application of new skills/knowledge/ attitudes (on the job) • The effectiveness of the skills/knowledge/ attitudes (as applied on the job) Level 3 – Behavior Application
  17. 17. 17www.exploreHR.org Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for behavior change and application to take place 3. Evaluate both before and after the program if practical 4. Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior 5. Get 100 percent response or a sampling 6. Repeat the evaluation at appropriate times 7. Consider cost versus benefits
  18. 18. 18www.exploreHR.org Example of Survey to Measure Behavior Application Understanding and Motivating 1. Trying to understand my subordinates 5 4 3 2 1 2. Listening to my subordinates 5 4 3 2 1 3. Praising good work 5 4 3 2 1 4. Talking with subordinates about their family and personal interests 5. Asking my subordinates for their ideas 5 4 3 2 1 6. Applying "Management by Walking Around" 5 4 3 2 1 Time and energy spent after the prgram compared to time and energy spent before the program 5 4 3 2 1 Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
  19. 19. 19www.exploreHR.org Level 4 – Business Results • Indicate the extent to which you think this program has influenced each of these measures in your work unit, department, or business unit: • Productivity • Quality • Customer Response Time • Cost Control • Employee Satisfaction • Customer Satisfaction • Other Level 4 – Business Results
  20. 20. 20www.exploreHR.org Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for results to be achieved 3. Measure both before and after the program if practical 4. Repeat the evaluation at appropriate times 5. Consider cost versus benefits 6. Be satisfied with evidence if proof is not possible
  21. 21. 21www.exploreHR.org Performance Indicators HARD DATA INDICATORS • Downtime duration • Number of defect products • Sales volume • Production unit • Customer satisfaction index • Response time to orders • Number of accidents at work • Others
  22. 22. 22www.exploreHR.org Performance Indicators SOFT DATA INDICATORS (intangible impacts) • Job satisfaction • Conducive working relationship • Effective communication • Stress rate • Quality in decision-making
  23. 23. 23www.exploreHR.org Example : Measuring Training Results 0.00 50.00 100.00 150.00 200.00 Program : TQM Training Results after 3 months of training, number of defects dropped to 80 units/day 120 units 80 units Before training After training
  24. 24. 24www.exploreHR.org 0.00 10.00 20.00 30.00 40.00 50.00 Example : Measuring Training Results Program : Sales Training Results after 3 months training, number of sales per salesman increase to 30 units/month. 20 units 30 units Before training After training
  25. 25. 25www.exploreHR.org Measuring Return on Investment of Training
  26. 26. 26www.exploreHR.org Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Training
  27. 27. 27www.exploreHR.org Criteria for Selecting Programs for Levels 4 and 5 Evaluation • Importance of the program in meeting the organization’s goals • Cost of the program • Visibility of the program • Size of the target audience • Extent of management interest
  28. 28. 28www.exploreHR.org Benefits of ROI of Training • Measure contribution • Set priorities • Focus on results • Alter management perceptions of training
  29. 29. 29www.exploreHR.org ROI of Training Model Collect Data Isolate the Effects of Training Convert Data to Monetary Values Calculate ROI of Training Tabulate Program Costs Identify Intangible Benefits
  30. 30. 30www.exploreHR.org Net Program Benefits Program Costs X 100 Return on Investment Formula ROI = Example : • Costs per program (25 participants) $ 88,500 • Benefits per program (1st year) $230,625 $ 230,625 – 88,500 $ 88,500 ROI = X 100 ROI = 161 %
  31. 31. 31www.exploreHR.org Collecting Data • Identify appropriate performance indicators • Develop a collection plan Collect Data
  32. 32. 32www.exploreHR.org Example of Performance Indicators • Units produced • Items sold • Work backlog • New accounts opened • Productivity • Inventory turnover • Etc. Output • Equipment downtime • Overtime • Time to project completion • Processing time • Repair time • Lost time days • Etc Time
  33. 33. 33www.exploreHR.org Example of Hard Indicators • Unit costs • Variable costs • Overhead costs • Operating costs • Number of cost reduction • Etc. Cost • Scrap • Waste • Rejects • Error rates • Rework • Product defects • Product failure Quality
  34. 34. 34www.exploreHR.org Example of Performance Results • Some performance results after training program: • Scrap was reduced from 11 % to 7.4 % • Absenteeism was reduce from 7 % to 3.25 % • The annual turnover rate was reduced from 30 % to 16 % • Lost time accidents were reduced 95 %
  35. 35. 35www.exploreHR.org Isolating the Effects of Training Methods to Isolate the Effects of Training Using Control Group Trend Lines Participants Estimate Supervisors of Participants Estimate
  36. 36. 36www.exploreHR.org Using Control Group • A control group arrangement can be used to isolate training impact. • With this strategy, one group receives training, while another, similar group does not receive training. • The difference in the performance of the two groups is attributed to the training program. Isolating the Effects of Training
  37. 37. 37www.exploreHR.org Trend Lines • Trend lines are used to project the values of specific output variables if training had not been undertaken. • The projection is compared to the actual data after training, and the difference represents the estimate impact of training. Isolating the Effects of Training
  38. 38. 38www.exploreHR.org Trend Lines Analysis Jan Feb Mar Apr JulJunMay Aug At the beginning of May, a Sales training Program session was held VolumeofSales The difference represents the estimate impact of training. Trend Projection Actual sales performance
  39. 39. 39www.exploreHR.org Participants and Supervisors of Participants Estimate of Training’s Impact • This method rests on the assumption that participants (and their supervisors) are capable of estimating how much a performance improvement is related to the training program. • Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue. • They should know how much of the change was caused by applying what they have learned in the program. Isolating the Effects of Training
  40. 40. 40www.exploreHR.org Participants and Supervisors of Participants Estimate of Training’s Impact • Typical Questions to Estimate : • What percent this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program? • What confidence do you have in this estimate, expresses as a percent? • What other factors contributed to this improvement in performance? Isolating the Effects of Training
  41. 41. 41www.exploreHR.org 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - Percent Improvement Caused by Confidence Expressed as a Percent Factors Which Influenced ImprovementNo. Isolating the Effects of Training Example of a Participant’s Estimation The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 %
  42. 42. 42www.exploreHR.org 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - Percent Improvement Caused by Confidence Expressed as a Percent Factors Which Influenced ImprovementNo. Isolating the Effects of Training Example of a Participant’s Estimation • The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % • This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training • The adjusted improvement is now ready for conversion to monetary values, and used in the return on investment
  43. 43. 43www.exploreHR.org Converting Data to Monetary Values Converting Data to Monetary Values • Steps to Convert Data to Monetary Values 1. Focus on a unit of improvement 2. Determine a value of each unit 3. Calculate the change in performance data 4. Determine an annual amount of change 5. Calculate the annual value of improvement
  44. 44. 44www.exploreHR.org Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values 1. Focus on unit improvement • One grievance reaching step two in the four-step grievance resolution process Steps Illustration 2. Determine a value of each unit • Using internal experts, the cost of an average grievance was estimated to be $ 6,500 when considering time and direct costs (V = $ 6,500)
  45. 45. 45www.exploreHR.org Steps to Convert Data to Monetary Values 3. Calculate the change in performance data • Six months after the program was completed, total grievances per month reaching step two declined by ten. • Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training) Steps Illustration 4. Determine an annual amount for the change • Using the six month value, seven per month (grievance reductions), yields an annual improvement of 84 (7 x 12 months)
  46. 46. 46www.exploreHR.org Steps to Convert Data to Monetary Values 5. Calculate the annual value of improvement • Annual value = 84 x $ 6,500 = $ 546,000 Steps Illustration
  47. 47. 47www.exploreHR.org Tabulating cost of the program • Tabulating the costs involves monitoring or developing all of the related costs of the program targeted for the ROI calculation. Tabulating Cost of the Program
  48. 48. 48www.exploreHR.org Tabulating cost of the program • Cost components that should be included are : • The cost to design and develop the program • The cost of all program materials provided to each participant • The cost for facilitator • The cost of the facilities of the training program Tabulating Cost of the Program
  49. 49. 49www.exploreHR.org Tabulating cost of the program • Cost components that should be included are : • Travel, lodging, and meal costs for the participants • Salaries, plus employee benefits of the participants who attend the training • Administrative and overhead costs of the training function, allocated in some convenient way Tabulating Cost of the Program
  50. 50. 50www.exploreHR.org An Example to Illustrate ROI analysis Weeks after training Trained Group Control Group 1 US$ 9,723 9,698 2 9,978 9,720 3 10,424 9,812 13 13,690 11,572 14 11,491 9,683 15 11,044 10,092 Average for weeks 12,075 10,449 13, 14, 15 Post Training Data Average Weekly Sales
  51. 51. 51www.exploreHR.org An Example to Illustrate ROI analysis Data US$ Average weekly sales Trained Group 12,075 Control Group 10,449 Increase 1,626 Profit Contribution from Training 2 % 32.50 (Training Impacts) Total weekly improvement 1,495 (32.5 x 46 participants) Total annual benefit 71,760 ($ 1,495 x 48 weeks) Annualized Program Benefits Note : 46 participants were still in job after 3 months
  52. 52. 52www.exploreHR.org An Example to Illustrate ROI analysis Cost US$ Facilitation fees 11,250 Program materials : $ 35 x 46 1,610 Meals : 3 days x $ 28 x 46 1,288 Participant salaries plus benefit (35 %) 12,442 Coordination and Evaluation 2,500 Total Cost 29,090 Cost Summary : An Illustration ROI (%): $ 71,760 - $ 29,090 $ 29,090 146 % x 100= =
  53. 53. 53www.exploreHR.org Enhancing Training Effectiveness
  54. 54. 54www.exploreHR.org How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance. Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: Addison- Wesley.
  55. 55. 55www.exploreHR.org Source of Barriers to Training Transfer • Lack of reinforcement on the job • Interference from immediate (work) environment • Nonsupportive organizational culture • Trainees’ perception of impractical training programs • Separation from the inspiration or support of the trainer
  56. 56. 56www.exploreHR.org The Transfer Partnership Trainee Trainee recognizes need for new skills Trainee Trainer Trainer designs and/or delivers learning experiences Trainee Trainer Manager supports learning and application on the job Manager
  57. 57. 57www.exploreHR.org The Transfer Matrix Before During After Manager Trainer Trainee Time PeriodsRolePlayers
  58. 58. 58www.exploreHR.org Manager Before Training • Build transfer of training into supervisory performance standards • Collect baseline performance data • Involve supervisors and trainees in needs analysis process • Involve trainees in program planning • Brief trainees on the importance of the training (course objective, content, process, and application on the job) • Review instructional content and materials • Plan to participate in training sessions • Encourage trainees attendance at all sessions
  59. 59. 59www.exploreHR.org Trainer Before Training • Align the training plan with the organization's strategic plan • Systematically design instruction • Provide proactive opportunities • Design a peer coaching component for the program and its follow-up activities
  60. 60. 60www.exploreHR.org Trainee Before Training • Provide input into program planning • Actively explore training options • Participate in advance activities
  61. 61. 61www.exploreHR.org Manager During Training • Prevent interruptions • Transfer work assignment to others • Monitor attendance and attention to training • Recognize trainee participation • Participate in transfer action planning • Review information on employee in training • Plan assessment of transfer of new skills to the job
  62. 62. 62www.exploreHR.org Trainer During Training • Develop application-oriented objectives • Answer the “WIIFM” question • Manage the unlearning process • Provide realistic work-related tasks • Give individualized feedback • Provide job performance aid
  63. 63. 63www.exploreHR.org Trainee During Training • Maintain an ideas and application notebook • Participate actively • Form support groups • Plan for applications • Create behavioral contracts
  64. 64. 64www.exploreHR.org Manager After Training • Plan trainees’ reentry • Provide opportunities to practice new skills • Have trainees participate in transfer-related decisions • Reduce job pressures initially • Give positive reinforcement • Schedule trainee briefings for co-workers • Set mutual expectations for improvement • Arrange proactive (refresher) sessions • Provide and support the use of job aids
  65. 65. 65www.exploreHR.org Trainer After Training • Provide follow-up support • Conduct evaluation surveys and provide feedback • Develop and administer recognition system • Provide refresher/problem-solving sessions
  66. 66. 66www.exploreHR.org Trainee After Training • Practice self-management • Review training content and learned skills • Develop a mentoring relationship • Maintain contact with training ‘buddies’
  67. 67. 67www.exploreHR.org References/Recommended Further Readings 1. Donald Kirkpatrick, Evaluating Training Programs : The Four Levels, Berrett-Koehler Publishers. You can obtain this excellent book at this link : http://www.amazon.com/Evaluating-Training-Programs-Four- Levels/dp/1576753484/ref=sr_1_1?ie=UTF8&s=books&qid=1219800986&sr=1-1 2. Jack J. Phillips and Patricia Phillips, In Action : Measuring Return On Investment, American Society for Training & Development. You can obtain this excellent book at this link : http://www.amazon.com/Action- Measuring-Return-Investment/dp/1562860089/ref=sr_1_3?ie=UTF8&s=books&qid=1219801000&sr=1-3 3. Mary Broad and John Newstrom, Transfer of training: Action Packed Strategies to Ensure High Payoff from Training Investments, Addison-Wesley. You can obtain this excellent book at this link : http://www.amazon.com/Transfer-Training-Action-Packed-Strategies- Investments/dp/0738205672/ref=sr_1_1?ie=UTF8&s=books&qid=1219801069&sr=1-1
  68. 68. 68www.exploreHR.org End of Material
  69. 69. 69www.exploreHR.org You can download these excellent slides at : www.HR-Management-Slides.com

×