Measuring roi of training

6,111 views
5,832 views

Published on

0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,111
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
954
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Notes
  • Measuring roi of training

    1. 1. Measuring ROI of Trainingwww.exploreHR.org 1
    2. 2. Agenda 1. Measuring the Effectiveness of Training Program 2. Measuring ‘Return on Investment’ of Training 3. Enhancing the Effectiveness and ROI of Training If you find this presentation useful, please consider telling others about our site (www.exploreHR.org)www.exploreHR.org 2
    3. 3. Measuring the Effectiveness of Training Programwww.exploreHR.org 3
    4. 4. Training Process Training Training Training Training Need Objectives Evaluation Delivery Analysis What are Objective Techniques Measure the training should be include on- reaction, needs for measurable the-job- learning, this person and training, behavior, and/or job? observable action and results learning, etc.www.exploreHR.org 4
    5. 5. The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Four Levels of Training Level 3 – Behavior Effectiveness Application Level 4 – Business Impactwww.exploreHR.org 5
    6. 6. The Four Levels of Evaluation Evaluate trainees’ reactions to Level 1 - the program. Did they like the Reaction program? Did they think it worthwhile? Test the trainees to determine if Level 2 - they learned the principles, Learning skills, and facts they were to learn.www.exploreHR.org 6
    7. 7. The Four Levels of Evaluation Ask whether the trainees’ behavior on the job Level 3 – changed because of the training program. For Behavior example, are employees in the store’s complaint Application department more courteous toward disgruntled customers than previously? What final results were achieved in terms of the Level 4 – training objectives previously set? Did the number Business of customer complaints about employee drop? Did Impact the reject rate improve? Was turnover reduced, and so forth.www.exploreHR.org 7
    8. 8. The Four Levels of Evaluation Value of Frequency of Difficulty of Information Use Assessment Level Least Frequent Easy valuable I. Reaction II. Learning III. Behavior Most Infrequent Difficult IV. Results valuablewww.exploreHR.org 8
    9. 9. Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Level 1 - Did they like the Reaction facilitators? Did they like the training accommodation and facilities?www.exploreHR.org 9
    10. 10. Guidelines for Evaluating Reaction 1. Determine what you want to find out 2. Design a form that will quantify reactions 3. Encourage written comments and suggestions 4. Get 100 percent immediate response 5. Get honest response 6. Develop acceptable standards 7. Measure reactions against standards, and take appropriate action 8. Communicate reaction as appropriatewww.exploreHR.org 10
    11. 11. Please give us your frank reactions and comments. They will help us to evaluate this program and improve future programs. Sample of Program : Reaction Facilitator : Form 1. How do you rate the subject (interest, benefit, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?) a. Excellent b. Very Good c. Good d. Fair e. Poor 3. How do you rate the facilities? (comfort, convinience, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 4. What would have improved the program? ………………………………………………………………………………… ………………………………………………………………………………… …………………………………………………………………………………www.exploreHR.org 11
    12. 12. Level 2 - Learning Measuring learning means determining one or more of the following : Level 2 - • What knowledge was Learning learned? • What skills were developed or improved? • What attitudes were changed?www.exploreHR.org 12
    13. 13. Guidelines for Evaluating Learning 1. Use a control group if practical 2. Evaluate knowledge, skills and/or attitudes both before and after the program 3. Use a paper-and-pencil test to measure knowledge 4. Use a performance test to measure skills 5. Get 100 percent immediate response 6. Use the results of evaluation to take appropriate actionwww.exploreHR.org 13
    14. 14. Guidelines for Evaluating Learning Pretest and Posttest Scores on Change Management Training Example : Experimental Group Control Group Pre Test Score 45 46 Post Test Score 55 48 Gain 10 2www.exploreHR.org 14
    15. 15. Level 3 – Behavior Application • The frequency of application of new skills/knowledge/ Level 3 – attitudes (on the job) Behavior Application • The effectiveness of the skills/knowledge/ attitudes (as applied on the job)www.exploreHR.org 15
    16. 16. Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for behavior change and application to take place 3. Evaluate both before and after the program if practical 4. Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior 5. Get 100 percent response or a sampling 6. Repeat the evaluation at appropriate times 7. Consider cost versus benefitswww.exploreHR.org 16
    17. 17. Example of Survey to Measure Behavior Application Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less Time and energy spent after the prgram compared to time and energy spent before the program Understanding and Motivating 1. Trying to understand my subordinates 5 4 3 2 1 2. Listening to my subordinates 5 4 3 2 1 3. Praising good work 5 4 3 2 1 4. Talking with subordinates about 5 4 3 2 1 their family and personal interests 5. Asking my subordinates for their ideas 5 4 3 2 1 6. Applying "Management by Walking Around" 5 4 3 2 1www.exploreHR.org 17
    18. 18. Level 4 – Business Results • Indicate the extent to which you think this program has influenced each of these measures in your work unit, Level 4 – department, or business unit: Business • Productivity Results • Quality • Customer Response Time • Cost Control • Employee Satisfaction • Customer Satisfaction • Otherwww.exploreHR.org 18
    19. 19. Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for results to be achieved 3. Measure both before and after the program if practical 4. Repeat the evaluation at appropriate times 5. Consider cost versus benefits 6. Be satisfied with evidence if proof is not possiblewww.exploreHR.org 19
    20. 20. Performance Indicators HARD DATA INDICATORS • Downtime duration • Number of defect products • Sales volume • Production unit • Customer satisfaction index • Response time to orders • Number of accidents at work • Otherswww.exploreHR.org 20
    21. 21. Performance Indicators SOFT DATA INDICATORS (intangible impacts) • Job satisfaction • Conducive working relationship • Effective communication • Stress rate • Quality in decision-makingwww.exploreHR.org 21
    22. 22. Example : Measuring Training Results 200.00 Program : TQM Training Results after 3 150.00 120 units months of training, number 100.00 80 units of defects dropped to 80 units/day 50.00 0.00 Before training After trainingwww.exploreHR.org 22
    23. 23. Example : Measuring Training Results Program : 50.00 Sales Training 40.00 Results after 3 months training, training 30 units number of sales 30.00 per salesman 20 units increase to 30 20.00 units/month. 10.00 0.00 Before training After trainingwww.exploreHR.org 23
    24. 24. Measuring Return on Investment of Trainingwww.exploreHR.org 24
    25. 25. Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Trainingwww.exploreHR.org 25
    26. 26. Criteria for Selecting Programs for Levels 4 and 5 Evaluation • Importance of the program in meeting the organization’s goals • Cost of the program • Visibility of the program • Size of the target audience • Extent of management interestwww.exploreHR.org 26
    27. 27. Benefits of ROI of Training • Measure contribution • Set priorities • Focus on results • Alter management perceptions of trainingwww.exploreHR.org 27
    28. 28. ROI of Training Model Isolate the Convert Data Collect Data Effects of to Monetary Training Values Identify Intangible Calculate ROI Benefits of Training Tabulate Program Costswww.exploreHR.org 28
    29. 29. Return on Investment Formula Net Program Benefits ROI = X 100 Program Costs Example : • Costs per program (25 participants) $ 88,500 • Benefits per program (1st year) $230,625 $ 230,625 – 88,500 ROI = X 100 $ 88,500 ROI = 161 %www.exploreHR.org 29
    30. 30. Collecting Data • Identify appropriate Collect performance indicators Data • Develop a collection planwww.exploreHR.org 30
    31. 31. Example of Performance Indicators Output Time • Units produced • Equipment downtime • Items sold • Overtime • Work backlog • Time to project • New accounts opened completion • Productivity • Processing time • Inventory turnover • Repair time • Etc. • Lost time days • Etcwww.exploreHR.org 31
    32. 32. Example of Hard Indicators Cost Quality • Unit costs • Scrap • Variable costs • Waste • Overhead costs • Rejects • Operating costs • Error rates • Number of cost • Rework reduction • Product defects • Etc. • Product failurewww.exploreHR.org 32
    33. 33. Example of Performance Results • Some performance results after training program: • Scrap was reduced from 11 % to 7.4 % • Absenteeism was reduce from 7 % to 3.25 % • The annual turnover rate was reduced from 30 % to 16 % • Lost time accidents were reduced 95 %www.exploreHR.org 33
    34. 34. Isolating the Effects of Training Using Control Group Trend Methods to Lines Isolate the Effects of Participants Training Estimate Supervisors of Participants Estimatewww.exploreHR.org 34
    35. 35. Isolating the Effects of Training Using • A control group arrangement can be Control used to isolate training impact. Group • With this strategy, one group receives training, while another, similar group does not receive training. • The difference in the performance of the two groups is attributed to the training program.www.exploreHR.org 35
    36. 36. Isolating the Effects of Training Trend • Trend lines are used to project the Lines values of specific output variables if training had not been undertaken. • The projection is compared to the actual data after training, and the difference represents the estimate impact of training.www.exploreHR.org 36
    37. 37. Trend Lines Analysis Actual sales performance At the beginning of The May, a Sales training difference Volume of Sales Program session was represents held the estimate impact of training. Trend Projection Jan Feb Mar Apr May Jun Jul Augwww.exploreHR.org 37
    38. 38. Isolating the Effects of Training • This method rests on the assumption that Participants and participants (and their supervisors) are Supervisors of capable of estimating how much a Participants Estimate of performance improvement is related to the Training’s Impact training program. • Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue. • They should know how much of the change was caused by applying what they have learned in the program.www.exploreHR.org 38
    39. 39. Isolating the Effects of Training • Typical Questions to Estimate : Participants and Supervisors of • What percent this improvement Participants can be attributed to the application Estimate of of skills/techniques/knowledge Training’s Impact gained in the training program? • What confidence do you have in this estimate, expresses as a percent? • What other factors contributed to this improvement in performance?www.exploreHR.org 39
    40. 40. Isolating the Effects of Training Example of a Participant’s Estimation Percent Confidence Factors Which Influenced Improvement Expressed as No. Improvement Caused by a Percent 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 %www.exploreHR.org 40
    41. 41. Isolating the Effects of Training Example of a Participant’s Estimation Percent Confidence Factors Which Influenced Improvement Expressed as No. Improvement Caused by a Percent 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - • The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % • This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training • The adjusted improvement is now ready for conversion to monetary values, and used in the return on investmentwww.exploreHR.org 41
    42. 42. Converting Data to Monetary Values • Steps to Convert Data to Monetary Converting Values Data to Monetary 1. Focus on a unit of improvement Values 2. Determine a value of each unit 3. Calculate the change in performance data 4. Determine an annual amount of change 5. Calculate the annual value of improvementwww.exploreHR.org 42
    43. 43. Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values Steps Illustration 1. Focus on unit • One grievance reaching step two in the improvement four-step grievance resolution process 2. Determine a • Using internal experts, the cost of an value of each average grievance was estimated to be $ unit 6,500 when considering time and direct costs (V = $ 6,500)www.exploreHR.org 43
    44. 44. Steps to Convert Data to Monetary Values Steps Illustration 3. Calculate the • Six months after the program was completed, change in total grievances per month reaching step two performance declined by ten. data • Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training) 4. Determine an • Using the six month value, seven per month annual amount (grievance reductions), yields an annual for the change improvement of 84 (7 x 12 months)www.exploreHR.org 44
    45. 45. Steps to Convert Data to Monetary Values Steps Illustration 5. Calculate the • Annual value = 84 x $ 6,500 = $ 546,000 annual value of improvementwww.exploreHR.org 45
    46. 46. Tabulating Cost of the Program Tabulating • Tabulating the costs involves cost of the monitoring or developing all of the program related costs of the program targeted for the ROI calculation.www.exploreHR.org 46
    47. 47. Tabulating Cost of the Program • Cost components that should be Tabulating included are : cost of the program • The cost to design and develop the program • The cost of all program materials provided to each participant • The cost for facilitator • The cost of the facilities of the training programwww.exploreHR.org 47
    48. 48. Tabulating Cost of the Program • Cost components that should be Tabulating included are : cost of the program • Travel, lodging, and meal costs for the participants • Salaries, plus employee benefits of the participants who attend the training • Administrative and overhead costs of the training function, allocated in some convenient waywww.exploreHR.org 48
    49. 49. An Example to Illustrate ROI analysis Average Weekly Sales Post Training Data Weeks after training Trained Group Control Group 1 US$ 9,723 9,698 2 9,978 9,720 3 10,424 9,812 13 13,690 11,572 14 11,491 9,683 15 11,044 10,092 Average for weeks 12,075 10,449 13, 14, 15www.exploreHR.org 49
    50. 50. An Example to Illustrate ROI analysis Annualized Program Benefits Data US$ Average weekly sales Trained Group 12,075 Control Group 10,449 Increase 1,626 Profit Contribution from Training 2 % 32.50 (Training Impacts) Total weekly improvement 1,495 (32.5 x 46 participants) Total annual benefit 71,760 ($ 1,495 x 48 weeks) Note : 46 participants were still in job after 3 monthswww.exploreHR.org 50
    51. 51. An Example to Illustrate ROI analysis Cost Summary : An Illustration Cost US$ ROI (%): Facilitation fees 11,250 Program materials : $ 35 x 46 1,610 $ 71,760 - $ 29,090 Meals : 3 days x $ 28 x 46 1,288 = x 100 $ 29,090 Participant salaries plus benefit (35 %) 12,442 Coordination and Evaluation 2,500 = 146 % Total Cost 29,090www.exploreHR.org 51
    52. 52. Enhancing Training Effectivenesswww.exploreHR.org 52
    53. 53. How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance. performance Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: Addison- Wesley.www.exploreHR.org 53
    54. 54. Source of Barriers to Training Transfer • Lack of reinforcement on the job • Interference from immediate (work) environment • Nonsupportive organizational culture • Trainees’ perception of impractical training programs • Separation from the inspiration or support of the trainerwww.exploreHR.org 54
    55. 55. The Transfer Partnership Trainee Trainee recognizes need for new skills Trainer designs and/or Trainee Trainer delivers learning experiences Manager supports learning Trainee Trainer and application on the job Managerwww.exploreHR.org 55
    56. 56. The Transfer Matrix Time Periods Before During After Manager Role Players Trainer Traineewww.exploreHR.org 56
    57. 57. Before Training • Build transfer of training into supervisory Manager performance standards • Collect baseline performance data • Involve supervisors and trainees in needs analysis process • Involve trainees in program planning • Brief trainees on the importance of the training (course objective, content, process, and application on the job) • Review instructional content and materials • Plan to participate in training sessions • Encourage trainees attendance at all sessionswww.exploreHR.org 57
    58. 58. Before Training • Align the training plan with the organizations Trainer strategic plan • Systematically design instruction • Provide proactive opportunities • Design a peer coaching component for the program and its follow-up activitieswww.exploreHR.org 58
    59. 59. Before Training • Provide input into program planning Trainee • Actively explore training options • Participate in advance activitieswww.exploreHR.org 59
    60. 60. During Training • Prevent interruptions Manager • Transfer work assignment to others • Monitor attendance and attention to training • Recognize trainee participation • Participate in transfer action planning • Review information on employee in training • Plan assessment of transfer of new skills to the jobwww.exploreHR.org 60
    61. 61. During Training • Develop application-oriented objectives Trainer • Answer the “WIIFM” question • Manage the unlearning process • Provide realistic work-related tasks • Give individualized feedback • Provide job performance aidwww.exploreHR.org 61
    62. 62. During Training • Maintain an ideas and application notebook Trainee • Participate actively • Form support groups • Plan for applications • Create behavioral contractswww.exploreHR.org 62
    63. 63. After Training • Plan trainees’ reentry Manager • Provide opportunities to practice new skills • Have trainees participate in transfer-related decisions • Reduce job pressures initially • Give positive reinforcement • Schedule trainee briefings for co-workers • Set mutual expectations for improvement • Arrange proactive (refresher) sessions • Provide and support the use of job aidswww.exploreHR.org 63
    64. 64. After Training • Provide follow-up support Trainer • Conduct evaluation surveys and provide feedback • Develop and administer recognition system • Provide refresher/problem-solving sessionswww.exploreHR.org 64
    65. 65. After Training • Practice self-management Trainee • Review training content and learned skills • Develop a mentoring relationship • Maintain contact with training ‘buddies’www.exploreHR.org 65
    66. 66. Recommended Further Readings 1. Donald Kirkpatrick, Evaluating Training Programs : The Four Levels, Berrett-Koehler Publishers 2. Jack J. Phillips and Patricia Phillips, In Action : Measuring Return On Investment, American Society for Training & Development 3. Mary Broad and John Newstrom, Transfer of training: Action Packed Strategies to Ensure High Payoff from Training Investments, Addison- Wesley.www.exploreHR.org 66
    67. 67. End of Materialwww.exploreHR.org 67

    ×