The document discusses various models for evaluating training programs, including Kirkpatrick's four-level model, CIRO model, and Phillips ROI model. It describes the levels or stages of each model, what they evaluate (e.g. reactions, learning, behavior, results), and methods for measuring outcomes. The models provide frameworks to investigate the effectiveness of training and determine if goals were achieved.

1. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 1
Module 3
Training Evaluation

2. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 2
• Evaluating a training program-
approaches
• Methods of Training: Job Instructed
Training , Coaching, Mentoring, Job
Rotation, Apprenticeship training
• Learning Theories – Jaen Piaget’s
theory, Social Learning theory,
Kolb’s model on Learning style of
individuals

3. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 3
3
INSTRUCTIONAL SYSTEMS DESIGN
MODEL

4. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 4
4
INSTRUCTIONAL SYSTEMS DESIGN
MODEL
Training evaluation is the third step of the ISD model and
consists of two parts:
 The evaluation criteria (what is being measured)
 Evaluation design (how it will be measured)
 Each has a specific and important role to play in the
effective evaluation of training and the completion of
the ISD model

5. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 5
Introduction (1 of 2)
Training effectiveness refers to the benefits that
the company and the trainees receive from
training
Training outcomes or criteria refer to measures
that the trainer and the company use to evaluate
training programs

6. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 6
Introduction (2 of 2)
Training evaluation refers to the process of
collecting the outcomes needed to determine if
training is effective
Evaluation design refers to from whom, what,
when, and how information needed for
determining the effectiveness of the training
program will be collected

7. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 7
Reasons for Evaluating Training (1 of 2)
Companies are investing millions of dollars in
training programs to help gain a competitive
advantage
Training investment is increasing because
learning creates knowledge which differentiates
between those companies and employees who are
successful and those who are not

8. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 8
Reasons for Evaluating Training (2 of 2)
Because companies have made large dollar
investments in training and education and view
training as a strategy to be successful, they
expect the outcomes or benefits related to
training to be measurable.

9. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 9
Formative Evaluation
Types of Evaluation
1.Formative evaluation – evaluation conducted
to improve the training process
Helps to ensure that:
the training program is well organized and runs
smoothly
trainees learn and are satisfied with the program
Provides information about how to make the
program better

10. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 10
Summative Evaluation
2. Summative evaluation – evaluation
conducted to determine the extent to which
trainees have changed as a result of participating
in the training program
May also measure the return on investment (ROI)
that the company receives from the training
program

11. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 11
Why Should A Training Program Be
Evaluated? (1 of 2)
To identify the program’s strengths and
weaknesses
To assess whether content, organization, and
administration of the program contribute to
learning and the use of training content on the job
To identify which trainees benefited most or least
from the program

12. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 12
Why Should A Training Program Be
Evaluated? (2 of 2)
To gather data to assist in marketing training
programs
To determine the financial benefits and costs of
the programs
To compare the costs and benefits of training
versus non-training investments
To compare the costs and benefits of different
training programs to choose the best program

13. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 13
The Evaluation Process
Conduct a Needs Analysis
Develop Measurable Learning Outcomes
and Analyze Transfer of Training
Develop Outcome Measures
Choose an Evaluation Strategy
Plan and Execute the Evaluation

14. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 14
What are training evaluation models?
 Training evaluation models are systematic frameworks for investigating and
analyzing the effectiveness of training or learning journeys. Different models
target different things but in general, they look at things such as:
 Was the training successful?
 What did the participants learn?
 Did the participants use what they learned on-the-job?
 What was the impact on the organization?
 Was the training a good investment?
 Did the training offer value for money?
 Could the training be improved?

15. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 15
15

16. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 16
Training Outcomes: Kirkpatrick’s Four-Level
Framework of Evaluation Criteria
Level Criteria Focus
1 Reactions Trainee satisfaction
2 Learning Acquisition of knowledge, skills, attitudes, behavior
3 Behavior Improvement of behavior on the job
4 Results Business results achieved by trainees

17. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 17
Level 1: Reaction
This level helps you determine how the participants responded to the training. This helps identify whether the
conditions for learning were present in the training.
Level 2: Learning
Through the use of short quizzes or practical tests, this stage helps you determine what,( if anything,) the
participants learned from the training.
Level 3: Behavior
The third stage takes place a while after the training has finished. Using various assessment methods, you try to
assess whether the course participants put what they learned into practice on-the-job.
Level 4: Results
The Kirkpatrick model measures results against stakeholder’s expectations. This is known as ROE (Return on
Expectations).
In summary, the Kirkpatrick Model is the most common training evaluation model in use worldwide today. As
we’ll see, it has provided the base for many other training evaluation models.

18. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 18
The CIRO Model
In 1970, Peter Warr, Michael Bird, and Neil
Rackham published their book, Evaluation of
management training. Their framework for
evaluating training became known as the ‘CIRO
model’ and offers businesses a way of evaluating
training needs and results.
Unlike other models such as Kirkpatrick’s Model
that can be applied to a broad range of training
and evaluation programs, the CIRO model is
specifically aimed at evaluating management

19. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 19
CIRO - Acronym
Context
Input
Reaction
Output
The CIRO model is hierarchical, meaning that
practitioners must start by studying ‘Context’,
before moving through ‘Input’, ‘Reaction’ and
‘Output’.

20. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 20
CIRO model contd..

21. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 21
CIRO contd..

22. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 22
CIRO contd..

23. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 23
CIRO contd..

24. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 24
The Phillips ROI Model
As the Kirkpatrick Model grew in popularity
during the 1970s, many academics and business
practitioners wanted to build and expand on it.
Among them was Jack Phillips, who published
his own book, Return on Investment in Training
and Performance, in 1980. Phillips wanted to
build on Don Kirkpatrick’s work and address
several of what he perceived to be its
shortcomings. Over time, Phillip’s approach to
training evaluation became known as the Phillips
ROI Model.

25. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 25
The most commonly quoted aspects of the
Phillips ROI Model is the addition of a fifth level.
This expands upon the Kirkpatrick Model and
offers organizations a way of calculating the ROI
of their training. The Phillips model has five
levels that broadly follow the scope and sequence
of the Kirkpatrick model.

26. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 26
Level 1: Reaction
In common with the Kirkpatrick Model, the Phillips ROI Model begins by evaluating the participants’ reaction to the
training they received.
Level 2: Learning
The second level of the Philips ROI Model evaluates what, if any, learning took place during the training.
Level 3: Application and Implementation
Like the Kirkpatrick Model, the Phillips ROI Model looks at whether the participants used what they learned during the
training when they returned to the workplace. However, Phillips approach helps an organization determine whether an issue
(if there is one) lies with the application of the learning or its implementation. This represents a subtle but crucial
improvement over the Kirkpatrick Model.
Level 4: Impact
While the fourth level of the Kirkpatrick taxonomy focuses purely on results, the Phillips ROI model is much broader and
looks at the impact of the training. This helps identify whether factors other than training were responsible for delivering
the outcomes.
Level 5: Return on investment (ROI)
Unlike the Kirkpatrick Model that simply measures training results again stakeholder expectations (ROE), the Phillips ROI
model contains a fifth level. This is designed to measure ‘return on investment’, or ROI. This level uses cost-benefit
analysis to determine the value of training programs.

27. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 27

28. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 28
The Phillips ROI Model found favor with
organizations who wanted to assign a monetary
value to the results of the training.
Given the cost and complexity of applying an
ROI analysis, however, only around five to ten
percent of training courses actually require a full,
five-level evaluation.

29. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 29
29
EVALUATION - VARIABLES
A. Reactions
B. Learning
C. Behaviour
D. Motivation
E. Self-efficacy
F. Perceived/anticipated support
G. Organizational perceptions
H. Organizational results

30. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 30
30
VARIABLES
A. Reactions
1. Affective reactions: Measures that assess trainees’
likes and dislikes of a training program
2. Utility reactions: Measures that assess the
perceived usefulness of a training program

31. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 31
31
VARIABLES
B. Learning
Learning outcomes can be measured by:
1. Declarative learning: Refers to the acquisition of
facts and information, and is
by far the most frequently assessed learning
measure
2. Procedural learning: Refers to the organization
of facts and information into a smooth
behavioural sequence

32. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 32
32
VARIABLES
C. Behaviour
Behaviours can be measured using three
approaches:
1. Self-reports
2. Observations
3. Production indicators

33. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 33
33
VARIABLES
D. Motivation
Two types of motivation in the training
context:
1. Motivation to learn
2. Motivation to apply the skill on-the-job (transfer)
E. Self-Efficacy
Beliefs that trainees have about their ability to
perform the behaviours that were taught in a
training program

34. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 34
34
VARIABLES
F. Perceived and/or Anticipated Support
Two important measures of support:
1. Perceived support: The degree to which the trainee
reports receiving support in attempts to transfer the
learned skills
2. Anticipated support: The degree to which the
trainee expects to supported in attempts to transfer
the learned skills

35. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 35
35
VARIABLES
G. Organizational Perceptions
Two scales designed to measure perceptions:
1. Transfer climate: Can be assessed via a
questionnaire that identifies eight sets of “cues”
2. Continuous learning culture: Can be assessed via
questionnaire

36. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 36
36
VARIABLES
G. Organizational Perceptions (cont'd)
Transfer climate cues include:
 Goal cues
 Social cues
 Task and structural cues
 Positive feedback
 Negative feedback
 Punishment
 No feedback
 Self-control

37. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 37
37
VARIABLES
H. Organizational Results
Results information includes:
1. Hard data: Results measured objectively (e.g.,
number of items sold)
2. Soft data: Results assessed through perceptions
and judgments (e.g., attitudes)
3. Return on expectations: Measurement of a
training program’s ability to meet managerial
expectations

38. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 38
Outcomes Used in Evaluating Training
Programs: (1 of 4)
Affective
Outcomes
Results
Return on
Investment
Cognitive
Outcomes
Skill-Based
Outcomes

39. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 39
Outcomes Used in Evaluating Training
Programs: (2 of 4)
Cognitive Outcomes
Determine the degree to which trainees are familiar
with the principles, facts, techniques, procedures, or
processes emphasized in the training program
Measure what knowledge trainees learned in the
program
Skill-Based Outcomes
Assess the level of technical or motor skills
Include acquisition or learning of skills and use of
skills on the job

40. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 40
Outcomes Used in Evaluating Training
Programs: (3 of 4)
Affective Outcomes
Include attitudes and motivation
Trainees’ perceptions of the program including the
facilities, trainers, and content
Results
Determine the training program’s payoff for the
company

41. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 41
Outcomes Used in Evaluating Training
Programs: (4 of 4)
Return on Investment (ROI)
Comparing the training’s monetary benefits with the
cost of the training
direct costs
indirect costs
benefits

42. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 42

43. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 43

44. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 44

45. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 45

46. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 46
Methods of Training:
Job Instructed Training
Coaching
Mentoring
Job Rotation
Apprenticeship training

47. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 47
Job Rotation: banking sector for the probationary
officers
Advantages and Disadvantages of job rotation
Coaching : The trainee is placed under a
particular supervisor who functions as a coach in
training the individual.
Job Instruction: training step by step. trainer explains
the trainee the way of doing the jobs, job knowledge and
skills and allows him to do the job. The trainer appraises
the performance of the trainee, provides feedback
information and corrects the trainee.

48. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 48
Apprenticeship: a formalized method of training
curriculum program that combines classroom
education with on-the-job work under close
supervision.
Appropriate for training in crafts, trades and
technical areas, especially when proficiency in a
job is the result of a relatively long training or
apprenticeship period, e.g., job of a craftsman, a
machinist, a printer, a tool maker, a pattern
designer, a mechanic, etc.

49. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 49
Mentoring
Mentoring is the Employee training system under
which a senior or more experienced person (the
mentor) is assigned to act as an advisor,
counselor, or guide to a junior or trainee -
providing support and feedback
Mentoring is the process of sharing knowledge
and experience with a junior employee.

50. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 50
Learning Theories – Jaen Piaget’s theory, Social
Learning theory, Kolb’s model on Learning style
of individuals

51. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 51
Piaget’s Psychological
Development

52. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 52
Piaget (1896 - 1980)
Swiss Psychologist, worked for
several decades on understanding
children’s cognitive development
Most widely known theory of cognitive
development.
Was intrigued by kids’ thoughts & behavior,
& worked to understand their cognitive
development

53. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 53
Constructivism
Assumption that learning is an active process of
construction rather than a passive assimilation
of information or rote memorization.

54. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 54
Piaget & Learning
Two main states – equilibrium & disequilibrium
Believed that we are driven or motivated to learn
when we are in disequilibrium
We want to understand things

55. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 55
Piaget & Learning
• Equilibration: assimilation & accommodation
• We adjust our ideas to make sense of reality
• Assimilation:
• process of matching external reality to an
existing cognitive structure.
• Accommodation:
• When there’s an inconsistency between the
learner’s cognitive structure & the thing being
learned the child will reorganize her thoughts

56. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 56
Example of Learning….

57. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 57
Piaget’s Four Stages
Believed that all children develop according to four
stages based on how they see the world.
He thought the age may vary some, but that we all go
through the stages in the same order.
1. Sensori-motor (birth –2 years)
2. Preoperational (~2-7)
3. Concrete operational (~7-11)
4. Formal operations (~12-15)

58. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 58
Sensorimotor Stage
Birth to about 2 years, rapid change is seen
throughout
The child will:
Explore the world through senses & motor
activity
Early on, baby can’t tell difference between
themselves & the environment
If they can’t see something then it doesn’t exist
Begin to understand cause & effect
Can later follow something with their eyes

59. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 59
Preoperational Stage
About 2 to about 7
Better speech communication
Can imagine the future & reflect on the past
Develop basic numerical abilities
Still pretty egocentric, but learning to be able to
delay gratification
Can’t understand conservation of matter
Has difficulty distinguishing fantasy from reality
(ex: cartoon characters are real people).

60. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 60
Concrete Operational Stage
From about 7 to about 11
Abstract reasoning ability & ability to generalize
from the concrete increases
Understands conservation of matter

61. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 61
Formal Operations
From about 12 to about 15
Be able to think about hypothetical situations
Form & test hypotheses
Organize information
Reason scientifically

62. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 62
… Piaget’s Development
Development happens from one stage to
another through interaction with the
environment.
Changes from stage to stage may occur
abruptly and kids will differ in how long they
are in each stage.
Cognitive development can only happen after
genetically controlled biological growth occurs.

63. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 63
…Piaget’s Development
Development leads to learning
Drive for development is internal
The child can only learn certain things when she is at
the right developmental stage
Environmental factors can influence but not direct
development
Development will happen naturally through regular
interaction with social environment

64. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 64
Piaget & Education
Piaget did not think it was possible to hurry along
or skip stages through education

65. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 65
Social Learning theory - Alan
Bandura
Social learning theory is based upon the work of
Albert Bandura. It is also referred to as Social
Cognitive Theory (SCT)
This theory attempts to understand the process
that is involved in explaining how we learn from
each other
It focuses on learning that occurs by direct
experience by observing, imitating, and modeling
It provides a framework for understanding,
predicting and potentially changing human
behaviour.
Aspects of Bandura’s theory are that:
o children learn by observing others,
o the same set of stimuli may provoke different
responses from different people, or from the
same people at different times
o the environment and a person’s behaviour are
interlinked;
o personality is an interaction between three
factors: the environment, behaviour, and a
person’s psychological processes.
All 3 play an important role in the learning
process. They are constantly influencing each
other
Environmental factors would include
Social aspects - peers, family and teachers
and Physical factors - the layout or size of
the class room or the temperature of the
room

66. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 66
What is effective modeling?
Four conditions are necessary for effective modeling to occur
Attention: the person must first pay attention to the model.
Retention: the observer must be able to remember the behavior
that has been observed. One way of increasing this is using the
technique of rehearsal.
Motor reproduction: the third condition is the ability to
replicate the behavior that the model has just demonstrated.
Motivation: the final condition for modeling to occur is
motivation, learners must want to demonstrate what they have
learned.

67. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 67
Kolb's Experiential Learning style theory - the
transformation of experience to knowledge
The process of going through
the cycle results in the
formation of increasingly
complex and abstract ‘mental
models’ of whatever the
learner is learning about.

68. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 68
Kolb's learning theory
(1984) -four distinct
learning styles, based
on a four-stage
learning cycle

69. McGraw-Hill/Irwin © 2005 The McGraw-Hill Companies, Inc. All rights reserved. 6 - 69
1. Concrete Experience - the learner encounters a concrete experience. This
might be a new experience or situation, or a reinterpretation of existing
experience in the light of new concepts.
2. Reflective Observation of the New Experience - the learner reflects on the
new experience in the light of their existing knowledge. Of particular
importance are any inconsistencies between experience and understanding.
3. Abstract Conceptualization - reflection gives rise to a new idea, or a
modification of an existing abstract concept (the person has learned from their
experience).
4. Active Experimentation - the newly created or modified concepts give rise to
experimentation. The learner applies their idea(s) to the world around them to
see what happens.