This document discusses various models and methods for evaluating training programs to determine their effectiveness. It describes several frameworks for training evaluation, including Kirkpatrick's four-level model of reaction, learning, behavior, and results. It also discusses other models that build on Kirkpatrick's, such as Phillips' addition of return on investment and Kaufman's five levels. The document advocates regular evaluation of training programs to identify areas for improvement and determine whether programs should be continued or cancelled based on whether they meet organizational goals and objectives.

2.  Many companies implement training to
improve their employee productivity and
work performance. But how can you
actually tell that it works? And how do
you know what you should fix in your
courses to make them more effective?
The best way is to start
evaluating training programs.

3.  Training program evaluation is the
regular analysis of how efficient and
effective training programs are. It
involves feedback from learners,
comparing their job performance before
and after the program, accomplishing the
company’s strategic priorities, and much
more

4.  Evaluation of training helps
companies find out whether training
programs meet their goals and
objectives. The results of training
evaluation are used to improve
training programs, or cancel them if
they prove unsuccessful

5.  Discover loopholes in courses. A part of
training evaluation is to assess the quality of
employees’ work, which might reveal some
loopholes in training programs. Let’s say the
quiz results show that almost all of your
workers failed the question on data security.
Then, upon checking the course, you discover
that this issue is not addressed adequately. As
a result, you might add a module on data
security to the training program.

6.  Save money. By comparing
training costs to results achieved
by training, companies can see
how profitable courses are,
which training programs are
worth investing in, and which
are a waste of money and time.

7.  Improve materials and tools. Without training
evaluation, it’s impossible to gauge the quality
of the learning content and the effectiveness of
the tools they use to build courses. Do learners
drop courses after a few slides? Your
introduction might not be engaging enough.
Try shooting a motivational video to trigger
their interest and explain why the course is
important. By analyzing the evaluation results,
you can see both weak and strong points, and
adjust training programs accordingly.

8.  Kirkpatrick’s Four-Level Training Evaluation
Model
 One of the oldest and, without a doubt, the
most famous one is Kirkpatrick’s model. It is
still used more than sixty years after its
development.
 According to this model, training programs
should be evaluated on 4 levels: Reaction,
Learning, Impact, and Results.

10.  Level 1: Reaction
 When learners complete your course, assess their
reactions. Ask them to complete a survey with
questions like:
 How satisfied are you with the learning
experience?
 Did the training content meet your expectations?
 Did you learn anything new?
 How would you rate the quality of the training?
 Do you find this training useful?

11.  Level 2: Learning
 Measure how much was learned in the course.
For example, create online quizzes to see what
knowledge and skills they have and haven’t
acquired during the training. In order to gain
the most precise results, some companies also
choose to create pre-quizzes. In this way, you
will know for sure what your learners knew
before the course, and what they learned from
taking it

12.  Level 3: Behavior
 See if an employee’s behavior has
changed after taking the training
program. And if it has, how much? The
most effective way to do this is to
compare 360-degree reviews (feedback
from the employee’s colleagues,
supervisors, subordinates, and others) on
the employee both before and after they
took the course.

13.  Level 4: Results
 This is the most important level of
training evaluation. After all, better
results are the primary goal of corporate
training. Evaluate what impact your
course has had on business by analyzing
quality, efficiency, productivity, and
customer satisfaction ratings.

14.  The Phillips ROI Model
 The Phillips ROI model can be thought of as
an extended version of Kirkpatrick’s
taxonomy. They both have the same levels,
except for an extra one in the end – Return on
investment (ROI). While Kirkpatrick’s model
evaluates only a return on expectations (ROE),
the ROI model can actually tell you if it was
the right decision to invest in a training
program.

16.  How to measure ROI with the Phillips model:
 Gather business data from before, during, and
after the course, and measure the impact that a
training had on your company’s profit,
productivity, performance, etc. – depending on
what the program was aiming for. Then,
compare the training cost to the value it
provided. If the value exceeds the training cost,
you’re doing great. If not, you need to find
which level or levels turned out to be
ineffective and improve your training.

17.  Kaufman’s Five Levels of Evaluation
 Another method building off of
Kirkpatrick’s model is Kaufman’s five
levels of evaluation. He divided the first
level into two, grouped Kirkpatrick’s
levels 2 and 3 as ‘micro’ levels, and
added a fifth level to evaluate results for
both customers and society

19.  How to use Kaufman’s five levels of
evaluation:
 Level 1a: Input
 Measure the amount of resources, such
as time and money, that were invested in
your training program.
 Level 1b: Process
 Assess learners’ reactions to the course.

20.  Level 2: Acquisition
 Evaluate individual or micro benefits of your
training by stating whether training objectives
for individual learners or small groups were
met. This means you should find out if your
students acquired new knowledge and skills
and applied them in their work tasks.
 Level 3: Application
 Gauge how well learners utilize new
knowledge and skills in their on-the-job
performance.

21.  Level 4: Organizational payoffs
 Measure payoffs for your company as a whole.
A payoff can be an improvement in
performance, a reduction of costs, or increased
profitability.
 Level 5: Societal Outcomes
 At the final level, you are to evaluate the
impact that your course had on what Kaufman
termed ‘mega-level clients.’ By these, he means
business clients or society as a whole.

22.  Kaufman’s model isn’t entirely
applicable to real life. Measuring how
much impact your training had on
society is far too expensive, complicated,
and impractical. Still, Kaufman offered
some useful ideas, such as splitting the
first level into two and evaluating the
content you provide and the resources
you invest separately

23.  The CIRO Model
 CIRO stands for Context, Input,
Reaction, and Output. Unlike other
models, it is designed specifically for
management training evaluation. So,
if you need to assess management
courses, this model might be your
best option.

25.  Stage 1: Context
 Evaluate your company’s current
situation. Try to list all the factors
that may affect the results of a
training. Meanwhile, during this
stage, find out what your
organization is lacking in terms of
performance.

26.  Stage 2: Input
 Your goal at this stage is to identify the best
training intervention option. Gather
information about all potential methods and
training techniques. Also, think about how
you will design, manage, and deliver your
course to your learners. Analyze your
company’s resources and determine the best
way to use them to achieve your objectives

27.  Stage 3: Reaction
 At this stage, ask your learners’ opinions of the
course. Focus on three aspects:
1)Program content
2)Approach
3)Value addition
 What you’re interested in at this point is not
only whether learners liked or disliked the
course, but also what changes they would like
to see in this training program.

28.  Stage 4: Output
 At this stage, you present the results of the
training. There are four levels of measurement:
1)Learner
2)Workplace
3)Team or department
4)Business
 Choose one of them in accordance with the
purpose of the evaluation and available
resources.

29.  Anderson’s Model of Learning Evaluation
 Imagine there’s a company that produces wooden
furniture. In order to increase sales, they invested in
salesperson training programs. The training turned
out to be highly effective – sales increased. But here’s
the problem: the factory workers didn’t receive the
training, so they couldn’t build furniture fast enough
to finish new orders. As a result, clients’ waiting time
increased, and it affected the company’s business
negatively. Anderson’s model helps to prevent such
situations.

30.  Unlike other models, Anderson’s model
of learning evaluation, also known as
Anderson’s Value of Learning Model,
focuses on a company’s entire learning
strategy, rather than a particular training
program. It consists of three stages that
help to determine the best learning
strategy for an organization’s needs.

32.  Stage 1
 Determine whether the current training
programs target the strategic priorities of
your company. Let’s return to our
example. Let’s say the company’s
strategic priority is to increase the
number of sales and expand its position
on the market. Does salesperson training
aim for these goals? Yes.

33.  Stage 2
 Measure the contribution of learning to strategic
results. At this point, our company uses different
measures to assess how much effect the training
has had on accomplishing the primary goals.
Collecting the research results, we can see that the
program helped increase the number of sales, but
it didn’t lead to an increase in the company’s
market share. Moreover, over time, the number of
customers started decreasing because the waiting
time was too long.

34. Stage 3
Set the most relevant approaches for your company.
The choice of an approach depends on stakeholders’
goals and values. Anderson offered 4 categories of
measure:
 Emphasis on short-term benefits
 Emphasis on long-term benefits
 Senior management trust in learning contribution
 The organization requires learning value metrics
 Here’s a table that might help you find the best
approach for your organization

35.  Training evaluation tools are the means
you use to evaluate training programs.
They are usually divided into 4
categories: questionnaires, interviews,
focus groups, and observations. We’ll
add one more – LMS Reporting. To
obtain the deepest and most accurate
assessment, these methods are often used
collectively

36. Questionnaires
 Questionnaires are the training
evaluation method most often used. They
consist of a set of questions that aim to
obtain useful information from the
participants. This tool is great for
assessing the reaction of learners after a
program

37. Questionnaires
Pros:
 Allows accumulation of a large amount of information
 Cost-efficient
 Time-efficient
 Covers a large group of participants
Cons:
 Low response rate
 Some unreliable responses
 Cannot clarify vague answers
 Subjective interpretation of questions by employees

38. Interviews
 The aim of interviews is to gather both
opinions and facts. Unlike
questionnaires, interviews can reveal
some insights into employees’ attitudes,
behaviors, and mindsets. Apart from a
classical face-to-face session, you can also
conduct an interview by telephone or via
the internet.

39. Interviews
 Pros:
 A better understanding of employees’
perspectives
 Allows asking clarifying questions
 Flexible
 Cons:
 Time-consuming
 Doesn’t cover a lot of learners – one at a time

40. Focus groups
 One can think of this method as a mixture of a
questionnaire and an interview, because it
enables you to both cover a large group of
people and get insights. If you’re looking for
qualitative data, in an endeavor to get a clear
picture of employees’ perspectives, but don’t
have enough resources to conduct interviews
with all workers, focus groups may be what
you need

41. Focus groups
Pros:
 Receive detailed feedback from many people at the
same time
 Ability to ask determining questions
Cons:
 Time-consuming
 Several people are required to manage (moderator
and assistant)
 Comfortable environment required for people to
speak freely and honestly

42. Observations
 An observation is perhaps the best way to see
changes in behavior and attitude after training.
Unlike all other methods, observation doesn’t
rely on what employees say about themselves
or each other. By just watching a person
working, you can see with your own eyes
whether they apply new skills and knowledge
in their job. Nevertheless, it has its limitations.

43. Observations
Pros:
 Cost-saving
 Gives a more realistic view, not being based on opinions
 Gathers valuable non-verbal information
 Can be conducted while a person is completing the course
Cons:
 Time-consuming (one person at a time)
 May provide unreliable information, as people tend to
behave better when someone’s watching
 An observer may misinterpret what they see
 Doesn’t lead to better understanding reasons for an
employee’s attitude or behavior

44.  LMS reporting
 A learning management system (LMS) is software
for delivering online programs to your learners.
Within this system, there is another system, LMS
Reporting, that collects and analyzes data from
your online programs. With its help, you can
reveal the weak points of your courses. For
example, you’ve designed a training program and
assigned it to your employees. Time goes by, but
you can’t see any results. This is when you might
want to take a look at LMS reports

45.  LMS reporting
 Pros:
 Time-saving
 Automatic process
 Objective feedback provided by the system
 Helps to find weaknesses in training programs
easily
 Works 24/7
 Cons:
 Can only be applied to online programs
 Doesn’t lead to better understanding reasons
behind an employee’s attitude or behavior

46. THANK YOU