This document discusses how to better measure and demonstrate the value of training programs. It suggests moving beyond simply measuring inputs and reactions to training, and instead focusing on outcomes like learning, application of skills on the job, impact on business metrics, and financial return on investment. The document provides examples of different levels of evaluation for training programs from reaction to ROI. It emphasizes the importance of aligning training objectives and evaluation with organizational needs and goals to demonstrate how training contributes to business performance.

1. Does learning solve business
performance problems?
By Javier Amaro

2. What is value?
• Think about a training program that you
recently implemented.
• How would you define the value delivered by
the program?

3. What is the value of training?

4. New definition of value
Be balanced, with qualitative and quantitative
data
Contain financial and nonfinancial perspectives
Reflect strategic and tactical issues
Satisfy all key stakeholders
Be consistent in collection and analysis
Be grounded in conservative standards
Come from credible sources

5. Do we have a place in the table?

6. Do we have a place in the table?
• Do training and development professionals
participate in business decisions?
• Do training and development professionals
participate in government strategic decisions?

7. Shifting Paradigms
Activity Based Results Based
 No business need for the program  Program linked to specific business needs
 No assessment of performance issues  Assessment of performance effectiveness
 No specific measurable objectives  Specific objectives for behavior and business impact
 No effort to prepare program participants to
achieve results
 Results expectations communicated to participants
 No effort to prepare the work environment to
support transfer
 Environment prepared to support transfer
 No efforts to build partnerships with key
managers
 Partnerships established with key managers and
clients
 No measurement of results or benefit-cost
analysis
 Measurement of results and benefit-cost analysis
 Planning and reporting is input focused  Planning and reporting is outcome focused

8. Training Solutions Provider?

9. Become a business partner

10. Levels of Evaluation
10
Levels of Evaluation Measurement Focus
0. Inputs and Indicators The input into the project in terms of
scope, volume, efficiencies, costs
Participants, Hours, Costs, Timing
1. Reaction & Perceived
Value
Reaction to the project or program,
including the perceived value
Relevance, Importance, Usefulness,
Appropriateness, Intent to use,
Motivation to take action
2. Learning
Learning to use the content and
materials, including the confidence to
use what was learned
Skills, Knowledge, Capacity,
Competencies, Confidence, Contacts
3. Application &
Implementation
Use of content and materials in the work
environment, including progress with
actual items and implementation
Extent of use, Task completion,
Frequency of use, Actions completed,
Success with use, Barriers to use,
Enablers to use
4. Business Impact
The consequences of the use of the
content and materials expressed as
business impact measures
Productivity, Revenue, Quality, Time,
Efficiency, Customer Satisfaction,
Employee Engagement
Typical Measures
5. ROI
Comparison of monetary benefits from
program to program costs
Benefit-Cost Ratio (BCR), ROI%,
Payback Period

11. Executive view – the report
Measure
We currently
measure this
We should
measure this in
the future
My ranking of the
importance of this
measure
1. Inputs: “Last year, 78,000 employees received formal learning.” 94%
2. Efficiency: “Formal learning costs $2.15 per hour of learning
consumed.”
78%
3. Reaction: “Employees rated our training very high, averaging 4.2 out
of 5.”
53%
4. Learning: “92% of participants increased knowledge and skills” 32%
5. Application: “At least 78% of employees are using the skills on the
job.”
11%
6. Impact: “Our programs are driving our top 5 business measures in
the organizations.”
8%
7. ROI: “Five ROI studies were conducted on major programs yielding an
average of 68% ROI.”
4%
8. Awards: “Our learning and development program won an award from
American Society of Training and Development.”
40%

12. Executive view – the need
Measure
We currently
measure this
We should
measure this in
the future
My ranking of the
importance of this
measure
1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 85%
2. Efficiency: “Formal learning costs $2.15 per hour of learning
consumed.”
78% 82%
3. Reaction: “Employees rated our training very high, averaging 4.2 out
of 5.”
53% 22%
4. Learning: “92% of participants increased knowledge and skills” 32% 28%
5. Application: “At least 78% of employees are using the skills on the
job.”
11% 61%
6. Impact: “Our programs are driving our top 5 business measures in
the organizations.”
8% 96%
7. ROI: “Five ROI studies were conducted on major programs yielding an
average of 68% ROI.”
4% 74%
8. Awards: “Our learning and development program won an award from
American Society of Training and Development.”
40% 44%

13. Executive view – the gap
Measure
We currently
measure this
We should
measure this in
the future
My ranking of the
importance of this
measure
1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 86% 6
2. Efficiency: “Formal learning costs $2.15 per hour of learning
consumed.”
78% 82% 7
3. Reaction: “Employees rated our training very high, averaging 4.2 out
of 5.”
53% 22% 8
4. Learning: “92% of participants increased knowledge and skills” 32% 28% 5
5. Application: “At least 78% of employees are using the skills on the
job.”
11% 61% 4
6. Impact: “Our programs are driving our top 5 business measures in
the organizations.”
8% 96% 1
7. ROI: “Five ROI studies were conducted on major programs yielding an
average of 68% ROI.”
4% 74% 2
8. Awards: “Our learning and development program won an award from
American Society of Training and Development.”
40% 44% 3

14. Fear of negative ROI

15. Chain of Impact
Reaction & Planned Action
Learning
Application & Implementation
Isolate the Effects of the Program
Intangible Benefits
Impact

16. Program Alignment: The V Model

17. Matching evaluation levels
Improve work group productivity by 20% in 6 months 4
Initiate at least three cost reduction projects in 15 days 3
Achieve an average cost reduction of $20,000 per project 4
Use counseling skills in 90% of situations where work habits are unacceptable 3
Be able to describe the four elements of the employee assistance program 2
Provide a 4 out of 5 rating on relevance of course content 1
Achieve a 2:1 benefit-to-cost ratio one year after the new leadership program
is implemented
5
For each objective listed below, indicate the level of evaluation at which the objective is
aimed.
1. Reaction and planned action
2. Learning
3. Application
4. Impact
5. Return on Investment (ROI)

18. Program Alignment V Model
Decrease
absenteeism by 5%

19. Key Alignment Questions
Potential ROI ROI
Payoffs Objectives5 5
Is this a problem worth solving?
Is there a potential pay off?
What is the actual ROI?
What is the BCR?
Needs Program
Assessment Objectives Evaluation

20. Needs Program
Assessment Objectives Evaluation
Business Impact Business
Needs Objectives Impact
Key Alignment Questions
4 4
What is the specific measure?
What happens if we do nothing?
Which business measure improved?
How much is related to the program?

21. Needs Program
Assessment Objectives Evaluation
Job Performance Application Application
Needs Objectives
Key Alignment Questions
3 3
What is occurring or not occurring on the
job that influences the business
measure?
What has changed?
Which skills/knowledge has been applied?

22. Skills/Knowledge Learning Learning
Needs Objectives
Needs Program
Assessment Objectives Evaluation
Key Alignment Questions
2 2
What skills or knowledge is needed to
support the job performance need?
What did they learn?
What are the takeaways?

23. Preferences Reaction Reaction &
Objectives Planned Action
Needs Program
Assessment Objectives Evaluation
Key Alignment Questions
1 1
How should the solution be
structured?
What was the reaction to the program?
How will the program be implemented?

24. Business Alignment process
1. Determine payoff needs
2. Identify business needs
3. Identify performance needs
4. Identify learning needs
5. Set Objectives
6. Design for results
7. Measure the business impact
8. Isolate the effects of the program

25. Training with the end in mind
• What is the value of your training?
– ROI
– Impact
– Application
– Learning
– Reaction
– Input
• Report the whole picture

26. Build up Capabilities in ROI
• ROI Certification program
– Sydney from 7th to 11th December 2015
– http://insources.com.au/index.php/roi/roi-
certification

27. Feedback Counts!
Your feedback helps ATD 2015 APC continue to
provide top-notch educational programs that
help you stay on top of a changing profession.
Evaluation forms for this session are available
via the mobile app and at the following link:
https://zh.surveymonkey.com/r/ATD2015APC