This document discusses different approaches to measuring the effectiveness of training programs. It describes Ralph Tyler's early work emphasizing the importance of training objectives. It then explains Kirkpatrick's four-level model for evaluating training effectiveness, including reaction, learning, behavior, and results. Finally, it discusses the input-output model for evaluating whether training delivers short-term and long-term benefits to organizations.

1. How to measure training effectiveness
Each year globally many millions of dollars are spent on training. It would seem reasonable to establish
just how effective this training has been if only to help justify the budget. Evaluation techniques are not
new, indeed the most influential early work on evaluation was performed in the USA during the 1940s by
Tyler. Probably the most important aspect of Ralph W Tyler’s work was the realization of the importance
of objectives in designing an effective school curriculum. This early methodology can be defined as the
scientific/experimental approach.
The model emphasized the importance of knowing the educational standards of the individuals in the
control group before the new education initiative took place and then determining the change that had
taken place by measuring the difference in attainment after the educational intervention was completed.
In other words the aim was to determine the effectiveness of the training by scientific means on the level
of performance of the individual. Although there have been some critics of this methodology it has proved
to be valuable in that organizations have been able to quantify the relationship between their corporate
objectives (often expressed as key performance indicators (KPIs) today) and the training aims and
objectives.
The main criticism of the scientific/experimental approach is that the methodology is unable to take into
account the unexpected or unintentional learning that can take place within a dynamic and changing
learning environment. Those readers who can remember reading about the original, and now famous
Hawthorne experiments to determine the effects of illumination levels on production held at the Western
Electric plant in Cicero, Illinois in the 1920’s and the later experiments also conducted by the National
Research Council showed that human behavior is indeed difficult to quantify exactly. Nevertheless the
greater attention that was given to training methodology and in particular the effectiveness of
management development during the 1960s and 1970s has led to much better understanding and the
emergence of systems evaluation methodology.
As we will see this approach can be divided into two main schools; the narrow focused approach being
attributed to Donald L Kirkpatrick in the early 1960s and the somewhat broader model advocated by P.
Warr et al (The Evaluation of Management Training) known as the CIRO model. The CIRO model looked
at context evaluation, input evaluation, reaction evaluation and outcome evaluation.
Donald L Kirkpatrick’s "Four steps to measuring training effectiveness”
This model uses four separate stages for the evaluation of the effectiveness of a training program.
The four stages are:-
 Reaction
 Learning
 Behavior
 Results
Level 1
The first stage is about the reaction of the trainee to the training. This sort of measurement is concerned
with how the trainees "feel” about the course. The usual course feedback sheets are an example of the
Kirkpatrick level 1 evaluation. Most organizations do not do any more than this type of measurement and
analysis. The drawback is that we do not really know if the trainee has actually learnt anything. What
really seems to be being asked of the trainee was how "happy” were you with the course; hence the

2. somewhat derogatory description that is often applied to the "happy sheets”! We will now look at the next
level of evaluation concerned with Learning.
Level 2
Things can be improved by using a pre-test and post-test and comparing the results. The questions need
to be objective and closely related to the course objectives (more about that later). In this way we can
determine if the training actually delivered knowledge and this was understood by the trainees at the time.
An organization that does this can be confident that the trainee has actually learnt something at that time.
Why do I make the point that we have to make the measurement and consider the learning at a certain
time? Well because we do not know if the learning has had time to be internalized and become
"concrete”. All too often trainees (and delegates at conferences for example) will have difficulty
remembering what was in the course or seminar they attended a few days later let alone months later.
There are various techniques that we can employ to improve the level of recall but basically "if we don’t
use it we will lose it”!
Level 3
This is concerned with "behavior”. By that we mean the measurable change in an individual as a result of
their attendance on the training course. This is, in my opinion, the least we should be expecting from any
training program. After all what is the point of spending money and using resources if the training does
not affect some measurable change in the behavior of the trainee?
Level 4
Kirkpatrick is now concerned with the training to determine if it has actually been translated into tangible
benefits to the organization. Quite simply has productivity and or quality been improved? Have the
number of accidents or incidents been reduced? Has plant availability and or plant utilization been
improved? Has the morale of the workforce changed for the better? These are metrics which really have
an impact on the "bottom line” and for that reason feature in the companies balance sheets and KPIs. We
have to ask ourselves is this not the real reason for training? Training has to make a real difference in
performance and effectiveness; this is tied closely to competence. Training has be proven to deliver
results and be cost effective. It might sound simple but it is not for most organizations. The reason is that
most do not have in place any system for measuring the improvement in competence of the individual; let
alone a systematic approach to identifying the most effective means of assisting the individual to
becoming competent.
Cost effective training
If we all now accept that the majority of organizations carry out training to change the behavior of the workforce
and improve their performance in the workplace we also must accept that this cannot be achieved without a clear
Plan and strategy.
This rational approach to training and development is best demonstrated by the Competence Profile(s) or
Competence Map(s) for a particular job(s) in the organization(s). If the approach in constructing these has
Been methodical we will have at our disposal a database of learning objectives.
I call these TCOs or terminal competence objectives. These TCOs must relate to the needs of the organization
and will contain the "underpinning” knowledge required to understand the tasks or duties the employee must
Perform as part of his/her duties as well as the practical skills necessary.
In this way we are able to ensure that the training is appropriate to the individual at that time in their development

3. And truly meets both their needs and those of the organization.
If we use these TCOs to develop the training program (whether it is a classroom based course or a workshop activity)
we will have a course "tailored” to the needs of the organization and furthermore we will be able to measure the
Improvement in performance and contribution to the team’s KPIs.
Evidence or proof?
A goal can be considered to be something that the organization strives to achieve through the meeting of
specific objectives. Achieving the individual objectives in a methodical and logical way effectively maps
out the process towards the eventual achievement of the goal. If we look at an important KPI for any
organization it might be expressed as "zero lost time injuries”; in other words no employee being away
from work for more than 3 days because of an injury sustained while in the company or performing work
related duties (like driving a truck for example). If a company with over a 1,000 employees for example
achieves a figure of say 6,000,000 man hours without a LTI then clearly their HSE systems must be
working. This can be translated into real company benefits and major savings to the organization. There
are many tangible benefits and other benefits which are very important but more difficult to quantify.
Benefits can include:-
 Better productivity
 Lower insurance premiums
 Improved morale
 Enhanced reputation
 Reduced stress
 No legal costs
 No compensation cases
 Lower staff turnover
 Employer of choice
Input, Process and Output
I have presented an argument for mapping out the competences required to perform a job and to linking
these competences to the delivery of training interventions. In effect this means that the strategic goals of
the organization must be cascaded down to team level within the organizations. In time new competences
will be required for team members as the industry "drivers” like new technology and competition
encourages or perhaps forces the organization to adopt new work systems and processes. Some leading
companies recognized this change model and one (IBM) introduced the IPO paradigm. The concern
being is training delivering the desired results within the organization? The IPO model was designed to
confront and solve problem. In this model a distinction is made between output (short term benefits) and

4. outcomes (benefits which are longer term but actually determine the availability of future training
resources). The following quote is particularly relevant here "The ultimate payoff or added value of an
employee’s learning experience is how well he or she performs on the job.” David S Bushnell writing
about the "Input, Process, and Output: A model for Evaluating Training” 1990 Training and Development
Journal.
Flow chart of input-output approach to training evaluation
Conclusion
We have seen that evaluation of training interventions such as conventional courses are generally a
neglected area as regards evaluation of their effectiveness. In general the "happy sheet” type of
evaluation is ineffective as it is inclined to be subjective, and influenced by considerations which are
actually outside the course itself. That is the course content could be poor or inappropriate but was
delivered by a good instructor who managed the group well and they actually "enjoyed” the experience
but learnt little!
Instead I have suggested that we owe it to the organization to identify the competences that are really
required by the individual to do his or her job well. These competences are the skills and underpinning
knowledge which should be part of the company database and accessible through its Competence
Management System.
The training intervention has to be designed to meet the needs of the organization by satisfying the
competence development requirements of the individual. By achieving this goal we have a rational and
justifiable case for training and we are able to prove that training really does deliver tangible benefits, not
least to the "bottom line”.
In this global marketplace it is essential to maintain a competitive advantage through the workforce; this is
best achieved by an effective Performance Management system underpinned by a robust Competence
Management System.