Presented at the Montana Hospital Association's Spring 2009 Conference.
See more at: http://www.integratedhealthcarestrategies.com/knowledgecenter.aspx.
Presented at the Montana Hospital Association's Spring 2009 Conference.
See more at: http://www.integratedhealthcarestrategies.com/knowledgecenter.aspx.
Balanced scorecard and policy deploymentRamesh P.R.
Greetings from Seven Steps!
As another April is around the corner, we are sure that you are busy with the ambitious plans and goals for the next year. As you may agree, one of the biggest challenge for any organization is the gap in the execution of this strategies and plans.
A Study Says 9 of 10 Companies Fail to Execute Strategy .
Companies are struggling hard to align individual’s goals and execution plans in line with the organizational goals. Seven Steps Academy of Excellence developed a unique and powerful experiential learning programme to help the organizations in achieving their Business and Operational goals . More than 30 organizations enjoying the benefits of this programme
Welcome to the 2 day Experiential Learning Program on
“Effective Goal deployment using Balance Score Card and Policy Deployment Tools”
Human Resource Metrics PowerPoint Presentation Slides SlideTeam
Help your company get empowered with the best human capital with the help of your analysis presented through these impactful Human Resource Metrics PowerPoint presentation slides. The slides cover all the Key Performance Indicators (KPIs) of a HR Metrics report like employee turnover, training, return on human capital, absenteeism, ratio of HR professionals to employees, ratio of business partners per employee, billable hours, costs of labor, returns and expenses per employee. The best KPIs should be able to reflect the human capital performance, such as financial outcomes, performance drivers, etc. The best way to design a good KPI is to communicate with the company business managers who know the jobs the best in their own divisions. You can compare the efficiency and effectiveness of the team through a dedicated slide provided. Quantifying the cost and the impact of talent management programs and HR processes, and measuring the success of HR initiatives can be done conveniently through the various slides provided. Your research based on theories such as Maslow’s hierarchy of needs can be displayed well in the pyramid diagram provided for it. Additional templates for details like agenda, mission, team, goal, finance, timeline, location, lego and magnifying glass have been provided. These slides can be used according to your presentation’s requirement. Set them up with our Human Resource Metrics PowerPoint Presentation Slides. Your team will take over from there.
Balanced scorecard and policy deploymentRamesh P.R.
Greetings from Seven Steps!
As another April is around the corner, we are sure that you are busy with the ambitious plans and goals for the next year. As you may agree, one of the biggest challenge for any organization is the gap in the execution of this strategies and plans.
A Study Says 9 of 10 Companies Fail to Execute Strategy .
Companies are struggling hard to align individual’s goals and execution plans in line with the organizational goals. Seven Steps Academy of Excellence developed a unique and powerful experiential learning programme to help the organizations in achieving their Business and Operational goals . More than 30 organizations enjoying the benefits of this programme
Welcome to the 2 day Experiential Learning Program on
“Effective Goal deployment using Balance Score Card and Policy Deployment Tools”
Human Resource Metrics PowerPoint Presentation Slides SlideTeam
Help your company get empowered with the best human capital with the help of your analysis presented through these impactful Human Resource Metrics PowerPoint presentation slides. The slides cover all the Key Performance Indicators (KPIs) of a HR Metrics report like employee turnover, training, return on human capital, absenteeism, ratio of HR professionals to employees, ratio of business partners per employee, billable hours, costs of labor, returns and expenses per employee. The best KPIs should be able to reflect the human capital performance, such as financial outcomes, performance drivers, etc. The best way to design a good KPI is to communicate with the company business managers who know the jobs the best in their own divisions. You can compare the efficiency and effectiveness of the team through a dedicated slide provided. Quantifying the cost and the impact of talent management programs and HR processes, and measuring the success of HR initiatives can be done conveniently through the various slides provided. Your research based on theories such as Maslow’s hierarchy of needs can be displayed well in the pyramid diagram provided for it. Additional templates for details like agenda, mission, team, goal, finance, timeline, location, lego and magnifying glass have been provided. These slides can be used according to your presentation’s requirement. Set them up with our Human Resource Metrics PowerPoint Presentation Slides. Your team will take over from there.
Key Performance Indicators (KPIs) should be used to understand performance and drive better decision-making. However, most companies struggle to find the vital few KPIs. Here is a list of the 25 top KPIs is use today.
There's more to learning evaluation than surveys and smile sheets. In this recent webinar, Andrew Downes laid down practical, straightforward advice on how to take your learning evaluation further and measure whether your learning programs are having the impact they were designed to achieve.
Here's the slides!
Evaluation is a planned process which provides specific information about a selected session, program for the purpose of determining value or decision making.
Measuring Roi Of Training & Development Ravinder TulsianiRavinder Tulsiani
How to calcluate the return on learning investment (ROI). Companies allocate certain amount of funds and resources to the training budget, what they want to see is how the training impacts their core business objectives (eg. growth, reduce risk etc...) Learn how…
If you are responsible for managing your nonprofit's training, then you know that providing courses and classes is only part of the challenge. You also need to investigate, plan, coordinate, communicate, budget, and persuade. All of these management functions become easier when you have a solid set of training metrics to work from. A "training scorecard" gives you a tool to track how things are going, and gives you the data to stand on equal footing with other leaders in your organization.
Curated by the Cornerstone OnDemand Foundation and Steve Semler, Senior Training Manager at MoneyGram International, this special presentation for nonprofits focuses on Learning Metrics: Building Your Training Scorecard. You will learn:
• The four ascending categories of learning metrics
• How to capture and present qualitative and quantitative training evaluation data
• Which metrics to include on a training scorecard
• How to establish a rhythm of evaluation and reporting that supports your organization's training and learning needs
Training needs analysis, skills auditing, training evaluation, calculating training ROI and strategic learning and development best practice principles and processes
3. 3www.exploreHR.org
Agenda
1. Measuring the Effectiveness of Training
Program
2. Measuring ‘Return on Investment’ of Training
3. Enhancing the Effectiveness and ROI of
Training
6. 6www.exploreHR.org
The Four Levels of Evaluation
Level 1 - Reaction
Level 2 - Learning
Level 3 – Behavior
Application
Level 4 – Business
Impact
Four Levels
of Training
Effectiveness
7. 7www.exploreHR.org
Test the trainees to determine if
they learned the principles,
skills, and facts they were to
learn.
Evaluate trainees’ reactions to
the program. Did they like the
program? Did they think it
worthwhile?
Level 1 -
Reaction
Level 2 -
Learning
The Four Levels of Evaluation
8. 8www.exploreHR.org
What final results were achieved in terms of the
training objectives previously set? Did the number
of customer complaints about employee drop? Did
the reject rate improve? Was turnover reduced,
and so forth.
Ask whether the trainees’ behavior on the job
changed because of the training program. For
example, are employees in the store’s complaint
department more courteous toward disgruntled
customers than previously?
Level 3 –
Behavior
Application
Level 4 –
Business
Impact
The Four Levels of Evaluation
10. 10www.exploreHR.org
Level 1 - Reaction
Evaluate trainees’
reactions to the program:
Did they like the program?
Did they like the
facilitators?
Did they like the training
accommodation and
facilities?
Level 1 -
Reaction
11. 11www.exploreHR.org
Guidelines for Evaluating Reaction
1. Determine what you want to find out
2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get 100 percent immediate response
5. Get honest response
6. Develop acceptable standards
7. Measure reactions against standards, and take
appropriate action
8. Communicate reaction as appropriate
12. 12www.exploreHR.org
Please give us your frank reactions and comments. They will help us to evaluate
this program and improve future programs.
Program :
Facilitator :
1. How do you rate the subject (interest, benefit, etc)
a. Excellent
b. Very Good
c. Good
d. Fair
e. Poor
2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?)
a. Excellent
b. Very Good
c. Good
d. Fair
e. Poor
3. How do you rate the facilities? (comfort, convinience, etc)
a. Excellent
b. Very Good
c. Good
d. Fair
e. Poor
4. What would have improved the program?
…………………………………………………………………………………
…………………………………………………………………………………
…………………………………………………………………………………
Sample of
Reaction
Form
13. 13www.exploreHR.org
Level 2 - Learning
Measuring learning means
determining one or more of
the following :
• What knowledge was
learned?
• What skills were
developed or improved?
• What attitudes were
changed?
Level 2 -
Learning
14. 14www.exploreHR.org
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Evaluate knowledge, skills and/or attitudes both
before and after the program
3. Use a paper-and-pencil test to measure knowledge
4. Use a performance test to measure skills
5. Get 100 percent immediate response
6. Use the results of evaluation to take appropriate
action
15. 15www.exploreHR.org
Guidelines for Evaluating Learning
Experimental Group Control Group
Pre Test Score 45 46
Post Test Score 55 48
Gain 10 2
Pretest and Posttest Scores on Change Management Training
Example :
16. 16www.exploreHR.org
Level 3 – Behavior Application
• The frequency of
application of new
skills/knowledge/
attitudes (on the job)
• The effectiveness of the
skills/knowledge/
attitudes (as applied on
the job)
Level 3 –
Behavior
Application
17. 17www.exploreHR.org
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for behavior change and application to take
place
3. Evaluate both before and after the program if practical
4. Survey and/or interview one or more of the following :
trainees, their immediate supervisor, their subordinates,
and others who often observe their behavior
5. Get 100 percent response or a sampling
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits
18. 18www.exploreHR.org
Example of Survey to Measure Behavior Application
Understanding and Motivating
1. Trying to understand my subordinates 5 4 3 2 1
2. Listening to my subordinates 5 4 3 2 1
3. Praising good work 5 4 3 2 1
4. Talking with subordinates about
their family and personal interests
5. Asking my subordinates for their ideas 5 4 3 2 1
6. Applying "Management by Walking Around" 5 4 3 2 1
Time and energy spent after the prgram
compared to time and energy spent before
the program
5 4 3 2 1
Instruction:
The objective of this questionnaire is to determine the extent to which those who
attended the recent program on Leadership have applied the principles and techniques
that they learned there to the job.
Circle the answer that you consider appropriate for each question.
5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
19. 19www.exploreHR.org
Level 4 – Business Results
• Indicate the extent to which you
think this program has
influenced each of these
measures in your work unit,
department, or business unit:
• Productivity
• Quality
• Customer Response Time
• Cost Control
• Employee Satisfaction
• Customer Satisfaction
• Other
Level 4 –
Business
Results
20. 20www.exploreHR.org
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for results to be achieved
3. Measure both before and after the program if practical
4. Repeat the evaluation at appropriate times
5. Consider cost versus benefits
6. Be satisfied with evidence if proof is not possible
21. 21www.exploreHR.org
Performance Indicators
HARD DATA INDICATORS
• Downtime duration
• Number of defect products
• Sales volume
• Production unit
• Customer satisfaction index
• Response time to orders
• Number of accidents at work
• Others
23. 23www.exploreHR.org
Example : Measuring Training Results
0.00
50.00
100.00
150.00
200.00
Program : TQM
Training
Results after
3 months of
training, number
of defects
dropped to 80
units/day
120 units
80 units
Before training After training
26. 26www.exploreHR.org
Level 5 : Return on Investment of Training
Level 1 - Reaction
Level 2 - Learning
Level 3 – Behavior
Application
Level 4 –
Business Impact
Level 5 – Return
on Investment of
Training
27. 27www.exploreHR.org
Criteria for Selecting Programs for Levels
4 and 5 Evaluation
• Importance of the program in meeting the
organization’s goals
• Cost of the program
• Visibility of the program
• Size of the target audience
• Extent of management interest
28. 28www.exploreHR.org
Benefits of ROI of Training
• Measure contribution
• Set priorities
• Focus on results
• Alter management perceptions of training
29. 29www.exploreHR.org
ROI of Training Model
Collect
Data
Isolate the
Effects of
Training
Convert Data
to Monetary
Values
Calculate ROI
of Training
Tabulate
Program
Costs
Identify
Intangible
Benefits
30. 30www.exploreHR.org
Net Program Benefits
Program Costs
X 100
Return on Investment Formula
ROI =
Example :
• Costs per program (25 participants) $ 88,500
• Benefits per program (1st year) $230,625
$ 230,625 – 88,500
$ 88,500
ROI = X 100
ROI = 161 %
32. 32www.exploreHR.org
Example of Performance Indicators
• Units produced
• Items sold
• Work backlog
• New accounts opened
• Productivity
• Inventory turnover
• Etc.
Output
• Equipment downtime
• Overtime
• Time to project
completion
• Processing time
• Repair time
• Lost time days
• Etc
Time
33. 33www.exploreHR.org
Example of Hard Indicators
• Unit costs
• Variable costs
• Overhead costs
• Operating costs
• Number of cost
reduction
• Etc.
Cost
• Scrap
• Waste
• Rejects
• Error rates
• Rework
• Product defects
• Product failure
Quality
34. 34www.exploreHR.org
Example of Performance Results
• Some performance results after training
program:
• Scrap was reduced from 11 % to 7.4 %
• Absenteeism was reduce from 7 % to 3.25 %
• The annual turnover rate was reduced from 30
% to 16 %
• Lost time accidents were reduced 95 %
35. 35www.exploreHR.org
Isolating the Effects of Training
Methods to
Isolate the
Effects of
Training
Using
Control Group
Trend
Lines
Participants
Estimate
Supervisors of
Participants
Estimate
36. 36www.exploreHR.org
Using
Control
Group
• A control group arrangement can be
used to isolate training impact.
• With this strategy, one group
receives training, while another,
similar group does not receive
training.
• The difference in the performance of
the two groups is attributed to the
training program.
Isolating the Effects of Training
37. 37www.exploreHR.org
Trend
Lines
• Trend lines are used to project the
values of specific output variables if
training had not been undertaken.
• The projection is compared to the
actual data after training, and the
difference represents the estimate
impact of training.
Isolating the Effects of Training
38. 38www.exploreHR.org
Trend Lines Analysis
Jan Feb Mar Apr JulJunMay Aug
At the beginning of
May, a Sales training
Program session was
held
VolumeofSales
The
difference
represents
the estimate
impact of
training.
Trend Projection
Actual sales performance
39. 39www.exploreHR.org
Participants and
Supervisors of
Participants
Estimate of
Training’s Impact
• This method rests on the assumption that
participants (and their supervisors) are
capable of estimating how much a
performance improvement is related to the
training program.
• Because their actions have produced the
improvement, participants (and their
supervisors) may have very accurate input
on the issue.
• They should know how much of the
change was caused by applying what they
have learned in the program.
Isolating the Effects of Training
40. 40www.exploreHR.org
Participants and
Supervisors of
Participants
Estimate of
Training’s Impact
• Typical Questions to Estimate :
• What percent this improvement
can be attributed to the application
of skills/techniques/knowledge
gained in the training program?
• What confidence do you have in
this estimate, expresses as a
percent?
• What other factors contributed to
this improvement in performance?
Isolating the Effects of Training
41. 41www.exploreHR.org
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%
4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
Percent
Improvement
Caused by
Confidence
Expressed as
a Percent
Factors Which Influenced
ImprovementNo.
Isolating the Effects of Training
Example of a Participant’s Estimation
The confidence percentage is multiplied by the
estimate (50 % x 70 %) to produce a usable training
factor value of 35 %
42. 42www.exploreHR.org
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%
4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
Percent
Improvement
Caused by
Confidence
Expressed as
a Percent
Factors Which Influenced
ImprovementNo.
Isolating the Effects of Training
Example of a Participant’s Estimation
• The confidence percentage is multiplied by the estimate (50 % x 70 %) to
produce a usable training factor value of 35 %
• This adjusted percentage is then multiplied by the actual amount of
improvement (post-program minus pre-program value) to isolate the portion
attributed to training
• The adjusted improvement is now ready for conversion to monetary values,
and used in the return on investment
43. 43www.exploreHR.org
Converting Data to Monetary Values
Converting
Data to
Monetary
Values
• Steps to Convert Data to Monetary
Values
1. Focus on a unit of improvement
2. Determine a value of each unit
3. Calculate the change in performance
data
4. Determine an annual amount of change
5. Calculate the annual value of
improvement
44. 44www.exploreHR.org
Steps to Convert Data to Monetary Values
An example to illustrate the steps to convert data to monetary values
1. Focus on unit
improvement
• One grievance reaching step two in the
four-step grievance resolution process
Steps Illustration
2. Determine a
value of each
unit
• Using internal experts, the cost of an
average grievance was estimated to be $
6,500 when considering time and direct
costs (V = $ 6,500)
45. 45www.exploreHR.org
Steps to Convert Data to Monetary Values
3. Calculate the
change in
performance
data
• Six months after the program was completed,
total grievances per month reaching step two
declined by ten.
• Seven of the then grievance reductions were
related to the program as determined by
supervisors (isolating the effects of training)
Steps Illustration
4. Determine an
annual amount
for the change
• Using the six month value, seven per
month (grievance reductions), yields an
annual improvement of 84 (7 x 12 months)
46. 46www.exploreHR.org
Steps to Convert Data to Monetary Values
5. Calculate the
annual value of
improvement
• Annual value = 84 x $ 6,500 = $ 546,000
Steps Illustration
47. 47www.exploreHR.org
Tabulating
cost of the
program
• Tabulating the costs involves
monitoring or developing all of the
related costs of the program
targeted for the ROI calculation.
Tabulating Cost of the Program
48. 48www.exploreHR.org
Tabulating
cost of the
program
• Cost components that should be
included are :
• The cost to design and develop
the program
• The cost of all program materials
provided to each participant
• The cost for facilitator
• The cost of the facilities of the
training program
Tabulating Cost of the Program
49. 49www.exploreHR.org
Tabulating
cost of the
program
• Cost components that should be
included are :
• Travel, lodging, and meal costs
for the participants
• Salaries, plus employee benefits
of the participants who attend
the training
• Administrative and overhead
costs of the training function,
allocated in some convenient
way
Tabulating Cost of the Program
50. 50www.exploreHR.org
An Example to Illustrate ROI analysis
Weeks after
training Trained Group Control Group
1 US$ 9,723 9,698
2 9,978 9,720
3 10,424 9,812
13 13,690 11,572
14 11,491 9,683
15 11,044 10,092
Average for weeks 12,075 10,449
13, 14, 15
Post Training Data
Average Weekly Sales
51. 51www.exploreHR.org
An Example to Illustrate ROI analysis
Data US$
Average weekly sales
Trained Group 12,075
Control Group 10,449
Increase 1,626
Profit Contribution from Training 2 % 32.50
(Training Impacts)
Total weekly improvement 1,495
(32.5 x 46 participants)
Total annual benefit 71,760
($ 1,495 x 48 weeks)
Annualized Program Benefits
Note : 46 participants were still in job after 3 months
52. 52www.exploreHR.org
An Example to Illustrate ROI analysis
Cost US$
Facilitation fees 11,250
Program materials : $ 35 x 46 1,610
Meals : 3 days x $ 28 x 46 1,288
Participant salaries plus benefit (35 %) 12,442
Coordination and Evaluation 2,500
Total Cost 29,090
Cost Summary : An Illustration
ROI (%):
$ 71,760 - $ 29,090
$ 29,090
146 %
x 100=
=
54. 54www.exploreHR.org
How Effective is Your Training Program?
Broad and Newstrom (1992) report studies have
shown less than 30% of what is actually taught
transfers to the job in a way that enhances
performance.
Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed
strategies to ensure high payoff from training investments. Reading, MA: Addison-
Wesley.
55. 55www.exploreHR.org
Source of Barriers to Training Transfer
• Lack of reinforcement on the job
• Interference from immediate (work) environment
• Nonsupportive organizational culture
• Trainees’ perception of impractical training programs
• Separation from the inspiration or support of the trainer
56. 56www.exploreHR.org
The Transfer Partnership
Trainee Trainee recognizes need for
new skills
Trainee Trainer
Trainer designs and/or
delivers learning experiences
Trainee Trainer
Manager supports learning
and application on the job
Manager
58. 58www.exploreHR.org
Manager
Before Training
• Build transfer of training into supervisory
performance standards
• Collect baseline performance data
• Involve supervisors and trainees in needs
analysis process
• Involve trainees in program planning
• Brief trainees on the importance of the training
(course objective, content, process, and
application on the job)
• Review instructional content and materials
• Plan to participate in training sessions
• Encourage trainees attendance at all sessions
59. 59www.exploreHR.org
Trainer
Before Training
• Align the training plan with the organization's
strategic plan
• Systematically design instruction
• Provide proactive opportunities
• Design a peer coaching component for the
program and its follow-up activities
61. 61www.exploreHR.org
Manager
During Training
• Prevent interruptions
• Transfer work assignment to others
• Monitor attendance and attention to training
• Recognize trainee participation
• Participate in transfer action planning
• Review information on employee in training
• Plan assessment of transfer of new skills to the
job
62. 62www.exploreHR.org
Trainer
During Training
• Develop application-oriented objectives
• Answer the “WIIFM” question
• Manage the unlearning process
• Provide realistic work-related tasks
• Give individualized feedback
• Provide job performance aid
64. 64www.exploreHR.org
Manager
After Training
• Plan trainees’ reentry
• Provide opportunities to practice new skills
• Have trainees participate in transfer-related
decisions
• Reduce job pressures initially
• Give positive reinforcement
• Schedule trainee briefings for co-workers
• Set mutual expectations for improvement
• Arrange proactive (refresher) sessions
• Provide and support the use of job aids
65. 65www.exploreHR.org
Trainer
After Training
• Provide follow-up support
• Conduct evaluation surveys and provide
feedback
• Develop and administer recognition system
• Provide refresher/problem-solving sessions
67. 67www.exploreHR.org
References/Recommended Further Readings
1. Donald Kirkpatrick, Evaluating Training Programs : The Four
Levels, Berrett-Koehler Publishers. You can obtain this excellent book at this link :
http://www.amazon.com/Evaluating-Training-Programs-Four-
Levels/dp/1576753484/ref=sr_1_1?ie=UTF8&s=books&qid=1219800986&sr=1-1
2. Jack J. Phillips and Patricia Phillips, In Action : Measuring
Return On Investment, American Society for Training &
Development. You can obtain this excellent book at this link : http://www.amazon.com/Action-
Measuring-Return-Investment/dp/1562860089/ref=sr_1_3?ie=UTF8&s=books&qid=1219801000&sr=1-3
3. Mary Broad and John Newstrom, Transfer of training: Action
Packed Strategies to Ensure High Payoff from Training
Investments, Addison-Wesley. You can obtain this excellent book at this link :
http://www.amazon.com/Transfer-Training-Action-Packed-Strategies-
Investments/dp/0738205672/ref=sr_1_1?ie=UTF8&s=books&qid=1219801069&sr=1-1