The document provides an overview of Donald Kirkpatrick's four levels of training evaluation: reaction, learning, behavior, and results. It describes each level and gives examples of how to measure impact at each level. It also presents two case studies, including how Intel applied the four levels both forwards and backwards in evaluating its training programs, and how St. Luke's Hospital used statistical analysis to evaluate the impact of an outdoor training program across all four levels over time.

1. Kirkpatrick

2. The Four Levels
 Reaction
 Learning
 Behavior
 Results

3. All about Kirkpatrick
In 1959, Kirkpatrick wrote four articles describing
the four levels for evaluating training programs.
He was working on his dissertation for a Ph.D.
when he came up with the idea of defining
evaluation.
Evaluation, as according to Kirkpatrick, seems to
have multiple meanings to training and
developmental professionals. Some think
evaluation is a change in behavior, or the
determination of the final results.

4. All about Kirkpatrick
(continued)
 Kirkpatrick says they are all right, and
yet all wrong. All four levels are
important in understanding the basic
concepts in training. There are
exceptions, however.

5. Kirkpatrick: Evaluating
Training Programs
 “What is quality training?”
 “How do you measure it?”
 “How do you improve it?”

6. Evaluating
“The reason for evaluating is to
determine the effectiveness of a
training program.” (Kirkpatrick,
1994, pg. 3)

7. The Ten Factors of Developing
a Training Program
1. Determine needs
2. Set objectives
3. Determine subject content
4. Select qualified applicants
5. Determine the best schedule

8. The Ten Factors of Developing
a Training Program
6. Select appropriate facilities
7. Select qualified instructors
8. Select and prepare audiovisual
aids
9. Co-ordinate the program
10. Evaluate the program

9. Reasons for Evaluating
Kirkpatrick gives three reasons ‘why’
there is a need to evaluate training:
1.“To justify the existence of the training
department by showing how
it contributes to the organizations’
objectives and goals.”

10. Reasons for Evaluating
2. “To decide whether to continue or
discontinue training programs.”
3. “To gain information on how to
improve future training programs.”
(Kirkpatrick, 1994, pg. 18)

11. The Four Levels
 Reaction
 Learning
 Behavior
 Results

12. “The Four Levels represent a
sequence of ways to evaluate
(training) programs….As you move
from one level to the next, the
process becomes more difficult and
time-consuming, but it also provides
more valuable information.”
(Kirkpatrick, 1994, pg. 21)

13. Reaction:
is the measuring of the reaction of
the participants in the training
program.
is “a measure of customer
satisfaction.” (Kirkpatrick, 1994,
pg. 21)

14. Learning:
is the change in the participants’
attitudes, or an increase in
knowledge, or greater skills
received, as a result of the
participation of the program.

15. Learning
The measuring of learning in any training
program is the determination of at least one
of these measuring parameters:
 Did the attitudes change positively?
 Is the knowledge acquired related and
helpful to the task?
 Is the skill acquired related and helpful to
the task?

16. Behavior
Level 3 attempts to evaluate how
much transfer of knowledge, skills,
and attitude occurs after the
training.

17. The four conditions Kirkpatrick
identifies for changes to occur:
 Desire to change
 Knowledge of what to do and
how to do it
 Work in the right climate
 Reward for (positive) change

18. When all conditions are met,
the employee must:
 Realize an opportunity to use the
behavioral changes.
 Make the decision to use the
behavioral changes.
 Decide whether or not to continue
using the behavioral changes.

19. When evaluating change in
behavior, decide:
 When to evaluate
 How often to evaluate
 How to evaluate

20. Guidelines for evaluating
behavior:
 Use a control group
 Allow time for change to occur
 Evaluate before and after
 Survey/interview observers
 Get 100% response or sampling
 Repeat evaluation, as appropriate
 Consider cost versus benefits

21. Results
Level 4 is the most important and
difficult of all - determining final
results after training.

22. Evaluation Questions:
 Increased production?
 Improved quality?
 Decreased costs?
 Improved safety numbers?
 Increased sales?
 Reduced turnover?
 Higher profits?

23. Guidelines for evaluating
results:
 Use a control group.
 Allow time for results to be achieved.
 Measure before and after the program.
 Repeat the measurements, as needed.
 Consider cost versus benefits.
 Be satisfied with evidence if proof is not
possible.

24. Case Study #1
INTEL CORPORATION

25. Intel’s Compromise
of the Kirkpatrick Model
Intel uses the four-level model as
an analysis instrument to
determine the initial training needs
and design of its training program;
as well as using the model for
evaluations.

26. Intel’s Compromise
of the Kirkpatrick Model
Their uniqueness of using the model
is in the fact that the designers of
the training program worked
backwards in the analysis of the
training, starting with Level Four.

27. The Model
This implementation of the
Kirkpatrick Model stands as vivid
testimony to the versatility of the
model as a training tool, and in
developing fledgling training
programs.

28. The Model
It also reflects the open-mindedness
of the senior executives at Intel for
their infinite use of the model and
the use of the genius and visions of
Kirkpatrick.

29. How Intel applies the analysis
to their training program
Level Four …”Determine the
organizations’ structure and future
needs.”
Level Three. Change the
environmental conditions and
employee conditions to improve
business indicators.

30. How Intel applies the analysis
to their training program
Level Two. “Design a training
program that would ensure a transfer
of deficient skills and knowledge.”
Level One. Use a questionnaire,
according to their skill level, that
would instruct and inspire training
participants.

31. How Intel applies evaluation
to their training program
Level One - Questionnaire.
Level Two - Demonstrate competency,
create action plans through group
simulations.
Level Three - Follow-up to determine if
action plans were met (specific steps to
implement concepts of what was learned).
Level Four - Ongoing process of tracking
business indicators.

32. Case Study #2
ST. LUKE’S HOSPITAL

33. St. Luke’s is unique -
 Evaluation of outdoor-based
training program, not classroom.
 Results analyzed statistically to
determine the significance of any
change.
 Evaluation led to recommendations
for future programs.

34. The New Questionnaire
 Used before attendance in the program.
 Used 3 months after completion of the
program.
 Used again 6 months after completion of the
program.
(Communication showed statistically significant
improvement, and Group Effectiveness showed
statistically significant change.)

35. Kirkpatrick’s 4 Levels of
Evaluation are:
 Level 1 - Reaction: how participants reacted
to the program.
 Level 2 - Learning: what participants
learned from the program.
 Level 3 - Behavior: whether what was
learned is being applied on the job.
 Level 4 - Results: whether that application
is achieving results.

36. Post-test Questions
(1) Name three ways evaluation results can
be measured.
(2) Do all 4 Levels have to be used?
(3) Do they have to be used in 1,2,3,4 order?
(4) Is Kirkpatrick’s method of evaluation
summative or formative?
(5) Which developmental “view” does
Kirkpatrick use? (discrepancy,
democratic, analytical, diagnostic)

37. “IF YOU THINK TRAINING IS
EXPENSIVE, TRY IGNORANCE.”
and, remember, the definition of
ignorance is
repeating the same behavior, over and
over, and expecting different results!