Training evaluation is the systematic process of collecting information and using that information to improve your training. Evaluation provides feedback to help you identify if your training achieved your intended outcomes, and helps you make decisions about future training.

1. Training
Evaluation
Chapter 11
D R N A Z R U L I S L A M
N A Z R U L K U @ G M A I L . C O M

2. Lecture Learning Objectives
• After reading this chapter, you will be able:
– Define training evaluation and the main reasons for conducting
evaluations
– Discuss the barriers to evaluation and the factors that affect whether
or not an evaluation is conducted
– Describe the different types of evaluations
– Describe the models of training evaluation and the relationship
between them
– Describe the main variables to measure in a training evaluation and how
they are measured.
– Discuss the different types of designs for training evaluation as well as
their requirements, limits, and when they should be used.

3. 3
INSTRUCTIONAL SYSTEMS DESIGN MODEL

4. 4
INSTRUCTIONAL SYSTEMS DESIGN MODEL
Training evaluation is the third step of the ISD
model and consists of two parts:
1. The evaluation criteria (what is being measured)
2. Evaluation design (how it will be measured)
These concepts are covered in the next two chapters
 Each has a specific and important role to play in the
effective evaluation of training and the completion of
the ISD model

5. Training Evaluation
Training Evaluation is a process to assess the value -the
worthiness - of training programs to employees and to
organizations.
Training evaluation assesses this value by analyzing data
collected from trainees, supervisors, or others familiar with
the trainees and with the job context.
Training evaluation is not a single procedure. Rather, it is a
continuum of techniques, methods, and measures. In one
end, simple evaluations that focus on trainee satisfaction &
in other end, following more elaborate procedures and more
complete questionnaires and interviews for in-depth
evaluation.

6. Training Evaluation, Contd.,
Training evaluation helps to answer the following questions:
1. Is the training program effective or achieved the training
goals?
2. Should the organization cancel or continue a training
program?
3. Should an existing training program be modified?
4. How should it be modified?

7. Why Conduct Training Evaluation?
Management has a stake in ensuring that the resources invested in
training bear fruit. Therefore, the reasons for training evaluation
are:
1. Help fulfill the managerial responsibility to improve training;
2. Assist managers in identifying the training programs most useful to
employees and to assist management in the determination of who
should be trained;
3. Determine the cost benefits of a program and to help ascertain in which
program or training technique is most cost-effective;
4. Determine whether the training program has achieved the expected
results or solved the problem for which training was the anticipated
solution;
5. Diagnose the strengths and weaknesses of a program and pinpoint
needed improvements;
6. Use the evaluation information to justify and reinforce, of merited, the
value and credibility of the training function to the organization.

8. Reasons for Conducting Training Evaluation
suggested by Kirkpatrik
1.“To justify the existence of the training department by showing
how it contributes to the organizations’ objectives and goals.”
2. “To decide whether to continue or discontinue training
programs.”
3. 3. “To gain information on how to improve future training
programs.” (Kirkpatrick, 1994, pg. 18)
“The reason for evaluating is to determine the effectiveness
of a training program.” (Kirkpatrick, 1994, pg. 3)

9. Barriers To Training Evaluation
• Many organizations do not conduct training
evaluation because they are perceived to be
– too complicated to implement,
– too time consuming, and/or
– too expensive.
• Some training managers do not conduct evaluations
because top management does not demand them
and because it is difficult to isolate the effects of
training among many other variables that might also
be having an effect on employees and organizations.

10. Barriers To Training Evaluation, Contd.,
• So, barriers to training fall into two
categories:
- Pragmatic barriers &
- Political barriers

11. Pragmatic Barriers to Training Evaluation
1. Evaluation requires knowledge about research design,
measurement and data analysis for which some training
manager feel insecure about taking on such a task.
2. It is claimed that training evaluation process has been unduly
mystified/made it confused.
However,
1. The principles, techniques, and procedures involved in
training evaluation are logical and straightforward, and most
can easily be implemented.
2. Web-based questionnaires and computerized work
performance data made it more helpful for organizations.

12. Political Barriers to Training Evaluation
1. Evaluations are conducted when there is pressure from
management to do so. Training managers usually forgo this
exercise. Training evaluation might demonstrate that part or
the entire program is not effective.
2. Some trainers fear that negative evaluation results might
reflect poorly on them.
3. May be perceived as a conflict of interest. Therefore, many
feel that evaluation should be conducted by third party
professional.
4. Many managers do not want to divert training money for
evaluation.
However, as long as training managers use established method
of evaluation and document them, there is little ground for
concern about conflict of interest.

13. Types of Training Evaluation
Evaluations differ with respect to the type of information that
is gathered and how that is accomplished. Evaluations also
differ with respect to their purposes. Worthen and Sanders
distinguished between formative and summative evaluations.
1. Formative Evaluation
This is designed to help evaluators assess the value of the
training materials and processes with the key goal of
identifying improvements to the instructional experience
(the clarity, complexity, relevance of the content, how they
are presented, and context of the training). Hence, it provides
data that are of special interest to training program designers
and instructors.

14. Types of Training Evaluation, Contd.,
2. Summative Evaluation
Summative evaluations are designed to provide data
about the worthiness or effectiveness of a training
program. Has the training program resulted in payoffs
for the organization? Cost benefit analysis are usually
summative. Economic indices are often an integral and
important part of these types of evaluations.
Consequently, organizational managers show great
interest in these results.

15. Types of Training Evaluation, Contd.,
A further distinction can be made between descriptive
and causal evaluations.
1. Descriptive Evaluation
Descriptive evaluations provide information that
describes the trainee once they have completed a
training program. What has the trainee learned in
training? Is the trainee more confident about using the
skill? Is it used on the job? Most evaluation designs have
descriptive components.

16. Types of Training Evaluation, Contd.,
2. Casual Evaluations
Causal evaluations are used to determine whether the
training caused the post-training behaviors. Was the
performance improvement caused by the training
program? Causal evaluations require more
sophisticated experimental and statistical procedures.

17. 1. Kirkpatrick’s Hierarchical Model
2. COMA Model (Cognitive, Organizational,
Motivational, & Attitudinal)
3. Decision-based Evaluation Model
Models of Training Evaluation

18. Donald Kirkpartick has developed a very popular evaluation model
that has been used since the late 1950s by the training community.
The focus is on measuring four kinds of outcomes that should result
from a highly effective training program. Oldest, best known, and
most frequently used model. Kirkpatrick’s model includes four
levels or steps of outcome evaluation:
1. Level 1 Evaluation—Reaction
2. Level 2 Evaluation—Learning
3. Level 3 Evaluation—Behavior
4. Level 4 Evaluation—Results
Kirkpatrick’s Hierarchical Model

19. • The model was then updated in 1975, and again in 1994,
when he published his best-known work, "Evaluating
Training Programs“
• “The Four Levels represent a sequence of ways to
evaluate (training) programs….As you move from one
level to the next, the process becomes more difficult and
time-consuming, but it also provides more valuable
information.” (Kirkpatrick, 1994, pg. 21)
• Level 1: Reaction
– is the measuring of the reaction of the participants in
the training program.
– is “a measure of customer satisfaction.” (Kirkpatrick,
1994, pg. 21)
Kirkpatrick’s Hierarchical Model, Contd.,

20. • Level 1: Reaction
• This level measures how your trainees (the people being trained),
reacted to the training. Obviously, you want them to feel that the
training was a valuable experience, and you want them to feel good
about the instructor, the topic, the material, its presentation, and the
venue.
• It's important to measure reaction, because it helps you understand
how well the training was received by your audience. It also helps
you improve the training for future trainees, including identifying
important areas or topics that are missing from the training. In a
nutshell,
– is the measuring of the reaction of the participants in the training
program.
– is “a measure of customer satisfaction.” (Kirkpatrick, 1994, pg.
21)
Kirkpatrick’s Hierarchical Model, Contd.,

21. • Level 2: Learning
– is the change in the participants’ attitudes, or an increase
in knowledge, or greater skills received, as a result of the
participation of the program.
– At level 2, you measure what your trainees have learned.
How much has their knowledge increased as a result of
the training?
– The measuring of learning in any training program is the
determination of at least one of these measuring
parameters:
• Did the attitudes change positively?
• Is the knowledge acquired related and helpful to the
task?
• Is the skill acquired related and helpful to the task?
Kirkpatrick’s Hierarchical Model, Contd.,

22. • Level 3: Behavior
At this level, you evaluate how far your trainees have changed their
behavior, based on the training they received. Specifically, this
looks at how trainees apply the information.
– attempts to evaluate how much transfer of knowledge, skills,
and attitude occurs after the training.
– The four conditions Kirkpatrick identifies for changes to occur:
• Desire to change
• Knowledge of what to do and how to do it
• Work in the right climate
• Reward for (positive) change
– When all conditions are met, the employee must:
• Realize an opportunity to use the behavioral changes.
• Make the decision to use the behavioral changes.
• Decide whether or not to continue using the behavioral
changes.
Kirkpatrick’s Hierarchical Model, Contd.,

23. • Level 4: Results
Of all the levels, measuring the final results of the training is likely to be the
most costly and time consuming. The biggest challenges are identifying
which outcomes, benefits, or final results are most closely linked to the
training, and coming up with an effective way to measure these outcomes
over the long term.
– is the most important and difficult of all - determining final results after
training.
– Evaluation Questions:
• Increased production?
• Improved quality?
• Decreased costs?
• Improved safety numbers?
• Increased sales?
• Reduced turnover?
• Higher profits?
Kirkpatrick’s Hierarchical Model, Contd.,

24. Critique of Kirkpatrick's Model
• Kirkpatrick's model provides a systematic framework that has
guided countless training evaluations.
1. Learning not necessarily translate into on-the-job
behavior changes.
2. Lack of precision : what exactly is meant by “reactions”
or “knowledge” .
3. Kirkpatrick requires all training evaluations to rely on
the same variables and outcome measures.
The current view is that the type of evaluation, as well
as the measures and procedures, should be selected as
a function of the organizational situation and the
purposes of the evaluation.

25. COMA Model
A training evaluation model that involves the
measurement of Cognitive, Organizational, Motivational
and Attitudinal variables.
• Instead of relying exclusively on reaction and declarative
learning measures, COMA suggests the measurement of
variables that research has shown to be important for
transfer of learning.

26. 26
COMA Model, Contd.,
A training evaluation model that involves
the measurement of four types of
variables
1. Cognitive
2. Organizational Environment
3. Motivation
4. Attitudes

27. COMA Model, Contd.,
1. Cognitive variables refer to the level of learning that the trainee has
gained from a training program.
2. Organizational Environment refers to a cluster of variables that
are generated by the work environment and that impact transfer of
training. (opportunity to practice, degree of support)
3. Motivation to Transfer refers to the desire to transfer on the job
what was learned in the training
4. Attitudes refer to individuals’ feeling and thinking processes.
(self efficacy, perceptions of control, expectations about self and
environment.)

28. COMA Model, Contd.,
The COMA model improves on Kirkpatrick’s model in four
ways:
1. Transforms the typical reaction by incorporating
greater number of measures
2. Useful for formative evaluations
3. The measures are known to be causally related to
training success
4. Defines new variables with greater precision

29. Decision-Based Evaluation Model
A training evaluation model that specifies the target, focus,
and methods of evaluation.
target: What do we wish to find out from this evaluation?
focus: What are the variables we will measure?
methods: What method may be appropriate for conducting
the evaluation?

30. Conclusions on the Evaluation Methods
The Kirkpatrick, COMA, DBE models of training
evaluation all require that for each training course
people with somewhat specialized skills develop the
evaluation measures and questionnaires, and
analyze and interpret the data.

31. Training Evaluation Variables
Training evaluation requires the measurement of variables,
many of which are specified in the training evaluation
models.
1. Reactions
2. Learning (e.g., declarative - yes or no questions &
procedural – procedural mastery)
3. Behavior – self report
4. Motivation
5. Self-efficacy – self confidence
6. Perceived and/anticipated support
7. Organizational perceptions
8. Organizational results

32. Training Evaluation

33. Organizational Results
Hard Data and Soft Data
• Hard Data
Results that can be measured objectively
( output, quality, time, cost)
• Soft Data
Results measures that consists of employees’ perceptions
and attitudes of organizational processes.
(work habits, work climate, management)

34. Training Evaluation Design
1. Non-experimental Designs: When the comparison is made
to a standard and not to another group of (untrained
people).
2. Experimental Design: When the trained group is compared
to another group that does not receive the training and
when assignment of people to the training group and the
non training group is random.
3. Quasi-experimental Design: When the trained group is
compared to another group that does not receive the
training but when the assignment of people to the training
and the non training group is not random.

35. Questions
1. Define training evaluation. What are the benefits
and barriers of training evaluation?
2. What are the types of Training evaluation?
Explain Kirkpatrick’s Hierarchical Model of
training evaluation.
3. Discuss Decision-Based Evaluation and COMA
Model of training evaluation.