Measuring ROI of Training

Agenda
Measuring the Effectiveness of Training Program
Measuring ‘Return on Investment’ of Training
Enhancing the Effectiveness and ROI of Training

www.exploreHR.org You can download this presentation at: Please visit www.exploreHR.org for more presentations on leadership, personal development, and HR management.

Measuring the Effectiveness of Training Program

Training Need Analysis Training Objectives Training Delivery Training Evaluation Training Process What are the training needs for this person and/or job? Objective should be measurable and observable Techniques include on-the-job-training, action learning, etc. Measure reaction, learning, behavior, and results

The Four Levels of Evaluation Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Four Levels of Training Effectiveness

Test the trainees to determine if they learned the principles, skills, and facts they were to learn. Evaluate trainees’ reactions to the program. Did they like the program? Did they think it worthwhile? Level 1 - Reaction Level 2 - Learning The Four Levels of Evaluation If you find this presentation useful, please consider telling others about our site (www.exploreHR.org)

What final results were achieved in terms of the training objectives previously set? Did the number of customer complaints about employee drop? Did the reject rate improve? Was turnover reduced, and so forth. Ask whether the trainees’ behavior on the job changed because of the training program. For example, are employees in the store’s complaint department more courteous toward disgruntled customers than previously? Level 3 – Behavior Application Level 4 – Business Impact The Four Levels of Evaluation

Level
I. Reaction
II. Learning
III. Behavior
IV. Results
Value of Information Frequency of Use Difficulty of Assessment The Four Levels of Evaluation Least valuable Most valuable Frequent Infrequent Easy Difficult

Level 1 - Reaction Evaluate trainees’ reactions to the program: Did they like the program? Did they like the facilitators? Did they like the training accommodation and facilities? Level 1 - Reaction

Guidelines for Evaluating Reaction
Determine what you want to find out
Design a form that will quantify reactions
Encourage written comments and suggestions
Get 100 percent immediate response
Get honest response
Develop acceptable standards
Measure reactions against standards, and take appropriate action
Communicate reaction as appropriate

Sample of Reaction Form

Level 2 - Learning
Measuring learning means determining one or more of the following :
What knowledge was learned?
What skills were developed or improved?
What attitudes were changed?
Level 2 - Learning

Guidelines for Evaluating Learning
Use a control group if practical
Evaluate knowledge, skills and/or attitudes both before and after the program
Use a paper-and-pencil test to measure knowledge
Use a performance test to measure skills
Get 100 percent immediate response
Use the results of evaluation to take appropriate action

Guidelines for Evaluating Learning Pretest and Posttest Scores on Change Management Training Example :

Level 3 – Behavior Application
The frequency of application of new skills/knowledge/ attitudes (on the job)
The effectiveness of the skills/knowledge/ attitudes (as applied on the job)
Level 3 – Behavior Application

Allow time for behavior change and application to take place
Evaluate both before and after the program if practical
Survey and/or interview one or more of the following : trainees, their immediate supervisor, their subordinates, and others who often observe their behavior
Get 100 percent response or a sampling
Repeat the evaluation at appropriate times
Consider cost versus benefits

Example of Survey to Measure Behavior Application Instruction: The objective of this questionnaire is to determine the extent to which those who attended the recent program on Leadership have applied the principles and techniques that they learned there to the job. Circle the answer that you consider appropriate for each question. 5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less

Level 4 – Business Results
Indicate the extent to which you think this program has influenced each of these measures in your work unit, department, or business unit:
Productivity
Quality
Customer Response Time
Cost Control
Employee Satisfaction
Customer Satisfaction
Other
Level 4 – Business Results

Allow time for results to be achieved
Measure both before and after the program if practical
Repeat the evaluation at appropriate times
Consider cost versus benefits
Be satisfied with evidence if proof is not possible

Performance Indicators
HARD DATA INDICATORS
Downtime duration
Number of defect products
Sales volume
Production unit
Customer satisfaction index
Response time to orders
Number of accidents at work
Others

Performance Indicators
SOFT DATA INDICATORS (intangible impacts)
Job satisfaction
Conducive working relationship
Effective communication
Stress rate
Quality in decision-making

Example : Measuring Training Results Program : TQM Training Results after 3 months of training, number of defects dropped to 80 units/day 120 units 80 units Before training After training

Example : Measuring Training Results Program : Sales Training Results after 3 months training , number of sales per salesman increase to 30 units/month. 20 units 30 units Before training After training

Measuring Return on Investment of Training

Level 5 : Return on Investment of Training Level 1 - Reaction Level 2 - Learning Level 3 – Behavior Application Level 4 – Business Impact Level 5 – Return on Investment of Training

Criteria for Selecting Programs for Levels 4 and 5 Evaluation
Importance of the program in meeting the organization’s goals
Cost of the program
Visibility of the program
Size of the target audience
Extent of management interest

Benefits of ROI of Training
Measure contribution
Set priorities
Focus on results
Alter management perceptions of training

ROI of Training Model Collect Data Isolate the Effects of Training Convert Data to Monetary Values Calculate ROI of Training Tabulate Program Costs Identify Intangible Benefits

Net Program Benefits Program Costs X 100 Return on Investment Formula ROI =
Example :
Costs per program (25 participants) $ 88,500
Benefits per program (1st year) $230,625
$ 230,625 – 88,500 $ 88,500 ROI = X 100 ROI = 161 %

Collecting Data
Identify appropriate performance indicators
Develop a collection plan
Collect Data

Example of Performance Indicators
Units produced
Items sold
Work backlog
New accounts opened
Productivity
Inventory turnover
Etc.
Output
Equipment downtime
Overtime
Time to project completion
Processing time
Repair time
Lost time days
Etc
Time

Example of Hard Indicators
Unit costs
Variable costs
Overhead costs
Operating costs
Number of cost reduction
Etc.
Cost
Scrap
Waste
Rejects
Error rates
Rework
Product defects
Product failure
Quality

Example of Performance Results
Some performance results after training program:
Scrap was reduced from 11 % to 7.4 %
Absenteeism was reduce from 7 % to 3.25 %
The annual turnover rate was reduced from 30 % to 16 %
Lost time accidents were reduced 95 %

Isolating the Effects of Training Methods to Isolate the Effects of Training Using Control Group Trend Lines Participants Estimate Supervisors of Participants Estimate

Using Control Group
A control group arrangement can be used to isolate training impact.
With this strategy, one group receives training, while another, similar group does not receive training.
The difference in the performance of the two groups is attributed to the training program.
Isolating the Effects of Training

Trend Lines
Trend lines are used to project the values of specific output variables if training had not been undertaken.
The projection is compared to the actual data after training, and the difference represents the estimate impact of training.

Trend Lines Analysis Jan Feb Mar Apr Jul Jun May Aug At the beginning of May, a Sales training Program session was held Volume of Sales The difference represents the estimate impact of training. Trend Projection Actual sales performance

Participants and Supervisors of Participants Estimate of Training’s Impact
This method rests on the assumption that participants (and their supervisors) are capable of estimating how much a performance improvement is related to the training program.
Because their actions have produced the improvement, participants (and their supervisors) may have very accurate input on the issue.
They should know how much of the change was caused by applying what they have learned in the program.

Participants and Supervisors of Participants Estimate of Training’s Impact
Typical Questions to Estimate :
What percent this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program?
What confidence do you have in this estimate, expresses as a percent?
What other factors contributed to this improvement in performance?

Isolating the Effects of Training Example of a Participant’s Estimation The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 %

Isolating the Effects of Training Example of a Participant’s Estimation
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 %
This adjusted percentage is then multiplied by the actual amount of improvement (post-program minus pre-program value) to isolate the portion attributed to training
The adjusted improvement is now ready for conversion to monetary values, and used in the return on investment

Converting Data to Monetary Values Converting Data to Monetary Values
Steps to Convert Data to Monetary Values
Focus on a unit of improvement
Determine a value of each unit
Calculate the change in performance data
Determine an annual amount of change
Calculate the annual value of improvement

Steps to Convert Data to Monetary Values An example to illustrate the steps to convert data to monetary values 1. Focus on unit improvement
One grievance reaching step two in the four-step grievance resolution process
Steps Illustration 2. Determine a value of each unit
Using internal experts, the cost of an average grievance was estimated to be $ 6,500 when considering time and direct costs (V = $ 6,500)

Steps to Convert Data to Monetary Values 3. Calculate the change in performance data
Six months after the program was completed, total grievances per month reaching step two declined by ten.
Seven of the then grievance reductions were related to the program as determined by supervisors (isolating the effects of training)
Steps Illustration 4. Determine an annual amount for the change
Using the six month value, seven per month (grievance reductions), yields an annual improvement of 84 (7 x 12 months)

Steps to Convert Data to Monetary Values 5. Calculate the annual value of improvement
Annual value = 84 x $ 6,500 = $ 546,000
Steps Illustration

Tabulating cost of the program
Tabulating the costs involves monitoring or developing all of the related costs of the program targeted for the ROI calculation.
Tabulating Cost of the Program

Cost components that should be included are :
The cost to design and develop the program
The cost of all program materials provided to each participant
The cost for facilitator
The cost of the facilities of the training program

Cost components that should be included are :
Travel, lodging, and meal costs for the participants
Salaries, plus employee benefits of the participants who attend the training
Administrative and overhead costs of the training function, allocated in some convenient way

An Example to Illustrate ROI analysis Average Weekly Sales

An Example to Illustrate ROI analysis Annualized Program Benefits Note : 46 participants were still in job after 3 months

An Example to Illustrate ROI analysis Cost Summary : An Illustration ROI (%): $ 71,760 - $ 29,090 $ 29,090 146 % x 100 = =

Enhancing Training Effectiveness

How Effective is Your Training Program? Broad and Newstrom (1992) report studies have shown less than 30% of what is actually taught transfers to the job in a way that enhances performance . Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: Addison-Wesley.

Source of Barriers to Training Transfer
Lack of reinforcement on the job
Interference from immediate (work) environment
Nonsupportive organizational culture
Trainees’ perception of impractical training programs
Separation from the inspiration or support of the trainer

The Transfer Partnership Trainee Trainee recognizes need for new skills Trainee Trainer Trainer designs and/or delivers learning experiences Trainee Trainer Manager supports learning and application on the job Manager

The Transfer Matrix Before During After Manager Trainer Trainee Time Periods Role Players

Manager Before Training
Build transfer of training into supervisory performance standards
Collect baseline performance data
Involve supervisors and trainees in needs analysis process
Involve trainees in program planning
Brief trainees on the importance of the training (course objective, content, process, and application on the job)
Review instructional content and materials
Plan to participate in training sessions
Encourage trainees attendance at all sessions

Trainer Before Training
Align the training plan with the organization's strategic plan
Systematically design instruction
Provide proactive opportunities
Design a peer coaching component for the program and its follow-up activities

Trainee Before Training
Provide input into program planning
Actively explore training options
Participate in advance activities

Manager During Training
Prevent interruptions
Transfer work assignment to others
Monitor attendance and attention to training
Recognize trainee participation
Participate in transfer action planning
Review information on employee in training
Plan assessment of transfer of new skills to the job

Trainer During Training
Develop application-oriented objectives
Answer the “WIIFM” question
Manage the unlearning process
Provide realistic work-related tasks
Give individualized feedback
Provide job performance aid

Trainee During Training
Maintain an ideas and application notebook
Participate actively
Form support groups
Plan for applications
Create behavioral contracts

Manager After Training
Plan trainees’ reentry
Provide opportunities to practice new skills
Have trainees participate in transfer-related decisions
Reduce job pressures initially
Give positive reinforcement
Schedule trainee briefings for co-workers
Set mutual expectations for improvement
Arrange proactive (refresher) sessions
Provide and support the use of job aids

Trainer After Training
Provide follow-up support
Conduct evaluation surveys and provide feedback
Develop and administer recognition system
Provide refresher/problem-solving sessions

Trainee After Training
Practice self-management
Review training content and learned skills
Develop a mentoring relationship
Maintain contact with training ‘buddies’

Recommended Further Readings
Donald Kirkpatrick, Evaluating Training Programs : The Four Levels , Berrett-Koehler Publishers
Jack J. Phillips and Patricia Phillips, In Action : Measuring Return On Investment , American Society for Training & Development
Mary Broad and John Newstrom, Transfer of training: Action P acked S trategies to E nsure H igh P ayoff from T raining I nvestments , Addison-Wesley.

End of Material

http://www.explorehr.org	18370
http://marketinghrdpresentation.com	1163
http://www.slideshare.net	548
http://thetrainingworld.com	192
http://translate.googleusercontent.com	68
http://edtech2.boisestate.edu	38
http://gestaoprojectoseducacionais.blogspot.com	31
http://ashahr.blogspot.com	28
http://www.epi-learning.com	28
http://www.cplpcoach.com	24
http://www.elearningcouncil.com	16
http://www.e-presentations.us	15
http://www.mbaclubindia.com	11
http://dorn-competentum.blogspot.com	10
http://www.brijj.com	9
http://static.slidesharecdn.com	8
http://www.comba.en.conform.it	7
http://togaranko.blogspot.com	7
https://online.lasalle.edu	7
http://cotradeco.com	6
http://chrischewcorner.blogspot.com	6
http://journeyofhrd.blogspot.com	5
http://www.dominicrajesh.com	5
http://www.techgig.com	4
http://getsortednow.com	4
http://intranet.optineo.com	4
http://webcache.googleusercontent.com	4
http://valuelearning.blogspot.com	3
http://ashahr.blogspot.in	2
http://www.lmodules.com	2
http://server8.kproxy.com	2
http://moodletest.hsw.wales.nhs.uk	2
file://	2
http://ig.gmodules.com	2
http://www.filescon.com	2
http://murtza.blogspot.com	2
http://humanflow.nazwa.pl	2
http://www.epi-learning.com.au	2
http://feeds.feedburner.com	1
http://www.danonecommunities.com	1
http://amurillo.typepad.com	1
http://explorehr.org	1
http://tubemote.com	1
http://blogs.salesforce.com	1
https://si0.twimg.com	1
http://207.46.192.232	1
http://www.hospitalityeducators.com	1
http://64.233.179.104	1
http://www.togaranko.blogspot.com	1
http://caeluajoaquimpinto.webnode.pt	1
http://weeadrian.blogspot.com	1

Measuring ROI of Training

by Yodhia Antariksa on May 19, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

61 Embeds 20,664

Statistics

Measuring ROI of Training — Presentation Transcript