Training Evaluation

1. Training Evaluation

2. • Models of Evaluation of Training Programs
• Kirkpatrick Model
• CIRO Model
• Kaufman’s Five Levels of Evaluation
• CIPP Model
• Phillips Five-Level ROI Framework

3. Evaluation of training
• It is the collection of analysis and
interpretation of information about any
aspect of a programme of education or
training as part of a recognized process of
judging its effectiveness, efficiency and
any other outcomes it may have.
• Assessment of value or worth

4. • Training effectiveness – the benefits that the company
and the trainees receive from training
• Training outcomes or criteria – that the trainer and the
company use to evaluate training programs
• Training evaluation – the process of collecting the
outcomes needed to determine if training is effective
• Evaluation design – from whom, what, when, and how
information needed for determining the effectiveness
of the training program will be collected

5. Reasons for Evaluating Training
• Companies are investing millions of dollars in
training programs to help gain a competitive
advantage
• Training investment is increasing because learning
creates knowledge
– This differentiates between those companies and
employees who are successful and those who are not
– Because companies have made large dollar
investments in training and education and view
training as a strategy to be successful, they expect the
outcomes or benefits related to training to be
measurable

6. Training evaluation provides the
data needed to demonstrate that
training does provide benefits to the
company.

7. Formative Evaluation
• The evaluation of training that takes place during
program design and development
• Helps to ensure that:
– the training program is well organized and runs
smoothly
– trainees learn and are satisfied with the program
• Provides information about how to make the
program better

8. Summative Evaluation
• Evaluation conducted to determine the
extent to which trainees have changed as
a result of participating in the training
program
• May also measure the return on
investment (ROI) that the company
receives from the training program

9. Why Should A Training Program Be
Evaluated? (1 of 2)
• To identify the program’s strengths and
weaknesses
• To assess whether content, organization,
and administration of the program
contribute to learning and the use of
training content on the job
• To identify which trainees benefited most
or least from the program

10. Why Should A Training Program Be
Evaluated? (2 of 2)
• To gather data to assist in marketing
training programs
• To determine the financial benefits and
costs of the programs
• To compare the costs and benefits of
training versus non-training investments
• To compare the costs and benefits of
different training programs to choose the
best program

11. Kirkpatrick Model
Four-Level Model of Evaluation (1959):

12. Training Outcomes: Kirkpatrick’s Four-
Level Framework of Evaluation Criteria
Level Criteria Focus
1 Reactions Trainee satisfaction
2 Learning Acquisition of knowledge, skills, attitudes,
behavior
3 Behavior Improvement of behavior on the job
4 Results Business results achieved by trainees

13. Outcomes Used in Evaluating
Training Programs: (1 of 4)
Affective
Outcomes
Results
Return on
Investment
Cognitive
Outcomes
Skill-Based
Outcomes

14. Outcomes Used in Evaluating
Training Programs: (2 of 4)
• Cognitive outcomes
– determine the degree to which trainees are familiar
with the principles, facts, techniques, procedures, or
processes emphasized in the training program
– measure what knowledge trainees learned in the
program
• Skill-based outcomes
– assess the level of technical or motor skills
– include acquisition or learning of skills and use of skills
on the job

15. Outcomes Used in Evaluating
Training Programs: (3 of 4)
• Affective outcomes
–include attitudes and motivation
–reaction outcomes – trainees’ perceptions of
the program including the facilities, trainers,
and content
• Results
–determine the training program’s payoff for the
company

16. Outcomes Used in Evaluating
Training Programs: (4 of 4)
• Return on Investment (ROI)
–comparing the training’s monetary benefits
with the cost of the training
• direct costs
• indirect costs
• benefits

17. ROI = Total Program Benefits – Total
Program Costs x 100%
Total Program Costs or,
ROI = Net Program Benefits x 100%
Total Program Costs

18. CIPP Evaluation Model (1970s):
• Also known as the Context, Input, Process,
Product model.
• Evaluation focuses on measuring the :
context for training (need analysis),
inputs (resources such as budgets,
schedules),
the process of conducting training (for
feedback) and
the product or outcome (success in
meeting objectives)

19. CIPP Evaluation Model (1970s):
D. L. Stufflebeam

20. S. No. Variables Brief Description
1 Context is about the environment in which a program would be
used. This context analysis is called a needs
assessment.
2 Input analysis is about the resources that will be used to
develop the program, such as people, funds, space and
equipment
3 Process evaluation examines the status during the development
of the program (formative)
4 Product evaluation that assessments on the success of the
program (summative)

21. CIRO Model
• Also known as the Context, Input,
Reaction, Outcome model
• The context, input and product
evaluation same as CIPP
• Emphasizes trainee reaction to improve
the training program

22. Context
Evaluation
it involves collecting of information about performance
deficiency and setting objective
Input evaluation answer of question like :
a) Relative merit & demerit of different training & development?
b) Is it feasible for outsourcing the training programme?
c) Should it be organized with internal resources?
d) How much time required for preparation?
e) What result were obtained when a similar programme was
conducted in the past?
Reaction
Evaluation
This includes subjective reports from the particulars about the
whole programme and contains question about training and
development materials, instructors, pedagogy, presentation,
facilities etc
Outcome
Evaluation
his includes the following aspects –
1) Defining training and development objective
2) Constructing measures of these objectives
3) Assessing the result and using them to improve future
programmes.

23. Phillips Five-Level ROI Framework

24. Reaction, Satisfaction, &
Planned Action
Measures participant reaction to and
satisfaction with the training program and
participant’s plans for action
Learning Measures skills and knowledge gains
Application and
Implementation
Measures changes in on-the-job application,
behaviour change, and implementation.
Business Impact Measures business impact
Return on Investment
(ROI)
Compares the monetary value of the business
outcomes with the costs of the training program

25. • Level 1 - Reaction, Satisfaction, and Planned Action: Level 1 measures participants’ satisfaction with
a program as well as their plans to use what they have learned. Although most organizations
evaluate at Level 1 exclusively, it should be noted that this level of evaluation does not guarantee
that participants have learned new skills or knowledge or will use them on the job.
• Level 2 – Learning: Using tests, skill practices, role playing, simulations, group evaluations, and
other assessment tools, level 2 evaluations assess how much participants have earned. Again,
although it is useful to know that participants have absorbed the new skills and knowledge, a positive
outcome here does not mean that participants will use the new learning when they are back on the
job. Level 3 - Behaviour, Application and Implementation:
• Level 3 evaluation assesses whether (and how much) participants applied the new knowledge and
skills on the job. The extent to which the new learning is applied in the workplace (or changes
behaviours) determines Level 3 success. Here too, it should be remembered, a favourable Level 3
evaluation does not guarantee that business outcomes will be positive.
• Level 4 - Business Impact: Level 4 measures the extent to which business measures have improved
after training. Typical Level 4 measures are output, quality, costs, and time. It is important to go
beyond Level 4 business results, however, because even if the training program results in substantial
business improvement, there is still a concern that the program’s costs may outweigh its business
benefits.
• Level 5 - Return on Investment (ROI): ROI is the ultimate level of evaluation. It compares the
monetary benefits from the program with the program costs. Although the ROI can be expressed in
several ways, it is usually presented as a percentage or cost/benefit ratio. (Phillips 2005) The
Phillips’ model evolves from, and can be distinguished from, the earlier Kirkpatrick model by the
adoption of return on investment to yield additional, critical insight. ROI allows decision makers to
compare the ultimate value of a training investment with other potential investment opportunities.

26. Kaufman’s Five Levels of
Evaluation

27. Level Evaluation Area
1 (a) Enabling Enabling” evaluation, designed to evaluate the quality and
availability of physical, financial and human resource
1(b) Reaction Reaction,” evaluates the efficiency and acceptability of the means,
methods and processes of the proposed training program. Test
subjects are asked how they feel about the instruction.
2 Acquisition evaluates the competency and mastery of the test group/individual in
a classroom setting
3 Application evaluates the success of the test group/individual’s utilization of the
training program. Test subjects are monitored to determine how
much and how well they implement the knowledge they gained
within the organization.
4 Organizational Outputs This level is designed to evaluate the results of the contributions and
payoffs of the organization as a whole as a result of the proposed
training program. Success is measured in terms of the organization's
overall performance and the return on investments.
5 Societal Outcomes the contributions to and from the client and society as a whole are
evaluated. Responsiveness, potential consequences and payoffs are
gauged to determine the success of implementing the proposed
training program.