Measuring ROI of Training

Slide 1: Measuring ROI of Training 1 www.exploreHR.org

Slide 2: Agenda 1. Measuring the Effectiveness of Training Program 2. Measuring ‘Return on Investment’ of Training 3. Enhancing the Effectiveness and ROI of Training 2 www.exploreHR.org

Slide 3: You can download this presentation at: www.exploreHR.org Please visit www.exploreHR.org for more presentations on leadership, personal development, and HR management. 3 www.exploreHR.org

Slide 4: Measuring the Effectiveness of Training Program 4 www.exploreHR.org

Slide 5: Training Process Training Training Training Training Need Evaluation Objectives Delivery Analysis What are Objective Techniques Measure the training should be include on- reaction, needs for measurable the-job- learning, this person and training, behavior, and/or job? observable action and results learning, etc. 5 www.exploreHR.org

Slide 6: The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Four Levels of Training Level 3 – Behavior Effectiveness Application Level 4 – Business Impact 6 www.exploreHR.org

Slide 7: The Four Levels of Evaluation Evaluate trainees’ reactions to Level 1 - Reaction the program. Did they like the program? Did they think it worthwhile? Test the trainees to determine if Level 2 - Learning they learned the principles, skills, and facts they were to learn. If you find this presentation useful, please consider telling others about our site (www.exploreHR.org) 7 www.exploreHR.org

Slide 8: The Four Levels of Evaluation Ask whether the trainees’ behavior on the job Level 3 – changed because of the training program. For Behavior example, are employees in the store’s complaint Application department more courteous toward disgruntled customers than previously? What final results were achieved in terms of the Level 4 – training objectives previously set? Did the number Business of customer complaints about employee drop? Did Impact the reject rate improve? Was turnover reduced, and so forth. 8 www.exploreHR.org

Slide 9: The Four Levels of Evaluation Difficulty of Value of Frequency of Assessment Information Use Level Frequent Easy Least valuable I. Reaction II. Learning III. Behavior Difficult Infrequent Most IV. Results valuable 9 www.exploreHR.org

Slide 10: Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Level 1 - Did they like the Reaction facilitators? Did they like the training accommodation and facilities? 10 www.exploreHR.org

Slide 11: Guidelines for Evaluating Reaction 1. Determine what you want to find out 2. Design a form that will quantify reactions 3. Encourage written comments and suggestions 4. Get 100 percent immediate response 5. Get honest response 6. Develop acceptable standards 7. Measure reactions against standards, and take appropriate action 8. Communicate reaction as appropriate 11 www.exploreHR.org

Slide 12: Please give us your frank reactions and comments. They will help us to evaluate this program and improve future programs. Sample of Program : Reaction Facilitator : Form 1. How do you rate the subject (interest, benefit, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?) a. Excellent b. Very Good c. Good d. Fair e. Poor 3. How do you rate the facilities? (comfort, convinience, etc) a. Excellent b. Very Good c. Good d. Fair e. Poor 4. What would have improved the program? ………………………………………………………………………………… ………………………………………………………………………………… ………………………………………………………………………………… 12 www.exploreHR.org

Slide 13: Level 2 - Learning Measuring learning means determining one or more of the following : • What knowledge was Level 2 - Learning learned? • What skills were developed or improved? • What attitudes were changed? 13 www.exploreHR.org

Slide 14: Guidelines for Evaluating Learning 1. Use a control group if practical 2. Evaluate knowledge, skills and/or attitudes both before and after the program 3. Use a paper-and-pencil test to measure knowledge 4. Use a performance test to measure skills 5. Get 100 percent immediate response 6. Use the results of evaluation to take appropriate action 14 www.exploreHR.org

Slide 15: Guidelines for Evaluating Learning Pretest and Posttest Scores on Change Management Training Example : Experimental Group Control Group Pre Test Score 45 46 Post Test Score 55 48 Gain 10 2 15 www.exploreHR.org

Slide 16: Level 3 – Behavior Application • The frequency of application of new skills/knowledge/ Level 3 – attitudes (on the job) Behavior • The effectiveness of the Application skills/knowledge/ attitudes (as applied on the job) 16 www.exploreHR.org

Slide 17: Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for behavior change and application to take place 3. Evaluate both before and after the program if practical 4. Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior 5. Get 100 percent response or a sampling 6. Repeat the evaluation at appropriate times 7. Consider cost versus benefits 17 www.exploreHR.org

Slide 18: Example of Survey to Measure Behavior Application Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less Time and energy spent after the prgram compared to time and energy spent before the program Understanding and Motivating 1. Trying to understand my subordinates 5 4 3 2 1 2. Listening to my subordinates 5 4 3 2 1 3. Praising good work 5 4 3 2 1 4. Talking with subordinates about 5 4 3 2 1 their family and personal interests 5. Asking my subordinates for their ideas 5 4 3 2 1 6. Applying \"Management by Walking Around\" 5 4 3 2 1 18 www.exploreHR.org

Slide 19: Level 4 – Business Results • Indicate the extent to which you think this program has influenced each of these measures in your work unit, Level 4 – department, or business unit: Business • Productivity Results • Quality • Customer Response Time • Cost Control • Employee Satisfaction • Customer Satisfaction • Other 19 www.exploreHR.org

Slide 20: Guidelines for Evaluating Learning 1. Use a control group if practical 2. Allow time for results to be achieved 3. Measure both before and after the program if practical 4. Repeat the evaluation at appropriate times 5. Consider cost versus benefits 6. Be satisfied with evidence if proof is not possible 20 www.exploreHR.org

Slide 21: Performance Indicators HARD DATA INDICATORS • Downtime duration • Number of defect products • Sales volume • Production unit • Customer satisfaction index • Response time to orders • Number of accidents at work • Others 21 www.exploreHR.org

Slide 22: Performance Indicators SOFT DATA INDICATORS (intangible impacts) • Job satisfaction • Conducive working relationship • Effective communication • Stress rate • Quality in decision-making 22 www.exploreHR.org

Slide 23: Example : Measuring Training Results 200.00 Program : TQM Training Results after 3 150.00 120 units months of training, number 80 units 100.00 of defects dropped to 80 units/day 50.00 0.00 After training Before training 23 www.exploreHR.org

Slide 24: Example : Measuring Training Results Program : 50.00 Sales Training 40.00 Results after 3 30 units months training, training number of sales 30.00 per salesman 20 units increase to 30 20.00 units/month. 10.00 0.00 Before training After training 24 www.exploreHR.org

Slide 25: Measuring Return on Investment of Training 25 www.exploreHR.org

Slide 26: Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Training 26 www.exploreHR.org

Slide 27: Criteria for Selecting Programs for Levels 4 and 5 Evaluation • Importance of the program in meeting the organization’s goals • Cost of the program • Visibility of the program • Size of the target audience • Extent of management interest 27 www.exploreHR.org

Slide 28: Benefits of ROI of Training • Measure contribution • Set priorities • Focus on results • Alter management perceptions of training 28 www.exploreHR.org

Slide 29: ROI of Training Model Isolate the Convert Data Collect Effects of to Monetary Data Training Values Identify Calculate ROI Intangible of Training Benefits Tabulate Program Costs 29 www.exploreHR.org

Slide 30: Return on Investment Formula Net Program Benefits ROI = X 100 Program Costs Example : • Costs per program (25 participants) $ 88,500 • Benefits per program (1st year) $230,625 $ 230,625 – 88,500 ROI = X 100 $ 88,500 ROI = 161 % 30 www.exploreHR.org

Slide 31: Collecting Data • Identify appropriate Collect performance indicators Data • Develop a collection plan 31 www.exploreHR.org

Slide 32: Example of Performance Indicators Output Time • Equipment downtime • Units produced • Overtime • Items sold • Time to project • Work backlog completion • New accounts opened • Processing time • Productivity • Repair time • Inventory turnover • Lost time days • Etc. • Etc 32 www.exploreHR.org

Slide 33: Example of Hard Indicators Cost Quality • Scrap • Unit costs • Waste • Variable costs • Rejects • Overhead costs • Error rates • Operating costs • Rework • Number of cost • Product defects reduction • Product failure • Etc. 33 www.exploreHR.org

Slide 34: Example of Performance Results • Some performance results after training program: • Scrap was reduced from 11 % to 7.4 % • Absenteeism was reduce from 7 % to 3.25 % • The annual turnover rate was reduced from 30 % to 16 % • Lost time accidents were reduced 95 % 34 www.exploreHR.org

Slide 35: Isolating the Effects of Training Using Control Group Trend Methods to Lines Isolate the Effects of Participants Training Estimate Supervisors of Participants Estimate 35 www.exploreHR.org

Slide 36: Isolating the Effects of Training • A control group arrangement can be Using used to isolate training impact. Control Group • With this strategy, one group receives training, while another, similar group does not receive training. • The difference in the performance of the two groups is attributed to the training program. 36 www.exploreHR.org

Slide 37: Isolating the Effects of Training • Trend lines are used to project the Trend values of specific output variables if Lines training had not been undertaken. • The projection is compared to the actual data after training, and the difference represents the estimate impact of training. 37 www.exploreHR.org

Slide 38: Trend Lines Analysis Actual sales performance At the beginning of The May, a Sales training difference Volume of Sales Program session was represents held the estimate impact of training. Trend Projection Jan Feb Mar Apr May Jun Jul Aug 38 www.exploreHR.org

Slide 39: Isolating the Effects of Training • This method rests on the assumption that Participants and participants (and their supervisors) are Supervisors of capable of estimating how much a Participants performance improvement is related to the Estimate of training program. Training’s Impact • Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue. • They should know how much of the change was caused by applying what they have learned in the program. 39 www.exploreHR.org

Slide 40: Isolating the Effects of Training • Typical Questions to Estimate : Participants and Supervisors of • What percent this improvement can Participants be attributed to the application of Estimate of skills/techniques/knowledge gained Training’s Impact in the training program? • What confidence do you have in this estimate, expresses as a percent? • What other factors contributed to this improvement in performance? 40 www.exploreHR.org

Slide 41: Isolating the Effects of Training Example of a Participant’s Estimation Percent Confidence Factors Which Influenced Improvement Expressed as No. Improvement Caused by a Percent 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % 41 www.exploreHR.org

Slide 42: Isolating the Effects of Training Example of a Participant’s Estimation Percent Confidence Factors Which Influenced Improvement Expressed as No. Improvement Caused by a Percent 1 Training Program 50% 70% 2 Change in Procedures 10% 80% 3 Adjustment in Standards 10% 50% 4 Revision to Incentive Plan 20% 90% 5 Increased Management Attention 10% 50% 6 Other - - • The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 % • This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training • The adjusted improvement is now ready for conversion to monetary values, and used in the return on investment 42 www.exploreHR.org

Slide 43: Converting Data to Monetary Values • Steps to Convert Data to Monetary Converting Values Data to 2. Focus on a unit of improvement Monetary Values 3. Determine a value of each unit 4. Calculate the change in performance data 5. Determine an annual amount of change 6. Calculate the annual value of improvement 43 www.exploreHR.org

Slide 44: Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values Steps Illustration • One grievance reaching step two in the 1. Focus on unit improvement four-step grievance resolution process • Using internal experts, the cost of an 2. Determine a value of each average grievance was estimated to be $ unit 6,500 when considering time and direct costs (V = $ 6,500) 44 www.exploreHR.org

Slide 45: Steps to Convert Data to Monetary Values Steps Illustration 3. Calculate the • Six months after the program was completed, change in total grievances per month reaching step two performance declined by ten. data • Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training) 4. Determine an • Using the six month value, seven per month annual amount (grievance reductions), yields an annual for the change improvement of 84 (7 x 12 months) 45 www.exploreHR.org

Slide 46: Steps to Convert Data to Monetary Values Steps Illustration 5. Calculate the • Annual value = 84 x $ 6,500 = $ 546,000 annual value of improvement 46 www.exploreHR.org

Slide 47: Tabulating Cost of the Program • Tabulating the costs involves Tabulating monitoring or developing all of the cost of the related costs of the program targeted program for the ROI calculation. 47 www.exploreHR.org

Slide 48: Tabulating Cost of the Program • Cost components that should be Tabulating included are : cost of the • The cost to design and develop program the program • The cost of all program materials provided to each participant • The cost for facilitator • The cost of the facilities of the training program 48 www.exploreHR.org

Slide 49: Tabulating Cost of the Program • Cost components that should be Tabulating included are : cost of the • program Travel, lodging, and meal costs for the participants • Salaries, plus employee benefits of the participants who attend the training • Administrative and overhead costs of the training function, allocated in some convenient way 49 www.exploreHR.org

Slide 50: An Example to Illustrate ROI analysis Average Weekly Sales Post Training Data Weeks after training Trained Group Control Group 1 US$ 9,723 9,698 2 9,978 9,720 3 10,424 9,812 13 13,690 11,572 14 11,491 9,683 15 11,044 10,092 Average for weeks 12,075 10,449 13, 14, 15 50 www.exploreHR.org

Slide 51: An Example to Illustrate ROI analysis Annualized Program Benefits Data US$ Average weekly sales Trained Group 12,075 Control Group 10,449 Increase 1,626 Profit Contribution from Training 2 % 32.50 (Training Impacts) Total weekly improvement 1,495 (32.5 x 46 participants) Total annual benefit 71,760 ($ 1,495 x 48 weeks) Note : 46 participants were still in job after 3 months 51 www.exploreHR.org

Slide 52: An Example to Illustrate ROI analysis Cost Summary : An Illustration Cost US$ ROI (%): Facilitation fees 11,250 $ 71,760 - $ 29,090 Program materials : $ 35 x 46 1,610 = x 100 Meals : 3 days x $ 28 x 46 1,288 $ 29,090 Participant salaries plus benefit (35 %) 12,442 = 146 % Coordination and Evaluation 2,500 Total Cost 29,090 52 www.exploreHR.org

Slide 53: Enhancing Training Effectiveness 53 www.exploreHR.org

Slide 54: How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance. performance Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: Addison- Wesley. 54 www.exploreHR.org

Slide 55: Source of Barriers to Training Transfer • Lack of reinforcement on the job • Interference from immediate (work) environment • Nonsupportive organizational culture • Trainees’ perception of impractical training programs • Separation from the inspiration or support of the trainer 55 www.exploreHR.org

Slide 56: The Transfer Partnership Trainee recognizes need for Trainee new skills Trainer designs and/or delivers Trainee Trainer learning experiences Manager supports learning Trainee Trainer and application on the job Manager 56 www.exploreHR.org

Slide 57: The Transfer Matrix Time Periods Before During After Manager Role Players Trainer Trainee 57 www.exploreHR.org

Slide 58: Before Training • Build transfer of training into supervisory Manager performance standards • Collect baseline performance data • Involve supervisors and trainees in needs analysis process • Involve trainees in program planning • Brief trainees on the importance of the training (course objective, content, process, and application on the job) • Review instructional content and materials • Plan to participate in training sessions • Encourage trainees attendance at all sessions 58 www.exploreHR.org

Slide 59: Before Training • Align the training plan with the organization's Trainer strategic plan • Systematically design instruction • Provide proactive opportunities • Design a peer coaching component for the program and its follow-up activities 59 www.exploreHR.org

Slide 60: Before Training • Provide input into program planning Trainee • Actively explore training options • Participate in advance activities 60 www.exploreHR.org

Slide 61: During Training • Prevent interruptions Manager • Transfer work assignment to others • Monitor attendance and attention to training • Recognize trainee participation • Participate in transfer action planning • Review information on employee in training • Plan assessment of transfer of new skills to the job 61 www.exploreHR.org

Slide 62: During Training • Develop application-oriented objectives Trainer • Answer the “WIIFM” question • Manage the unlearning process • Provide realistic work-related tasks • Give individualized feedback • Provide job performance aid 62 www.exploreHR.org

Slide 63: During Training • Maintain an ideas and application notebook Trainee • Participate actively • Form support groups • Plan for applications • Create behavioral contracts 63 www.exploreHR.org

Slide 64: After Training • Plan trainees’ reentry Manager • Provide opportunities to practice new skills • Have trainees participate in transfer-related decisions • Reduce job pressures initially • Give positive reinforcement • Schedule trainee briefings for co-workers • Set mutual expectations for improvement • Arrange proactive (refresher) sessions • Provide and support the use of job aids