1. 26 www.martupdate.com November 2014 November 2014 www.martupdate.com 27
Measuring Training Impact
The Holy Grail of Learning
Continuous learning is essential to the ongoing success of any organization. But it comes as some sur-prise
to find that many mid-tier organizations don’t measure the impact of their employee
training. That’s right – fewer than 15 % measure learning impact. Yet, behaviour changes and ROI are
the very things almost every organization wants from their training department. How can you turn it
into something more concrete?
TALENT MANAGMENT
Pavan Sriram
More than 240 books on
Amazon.com and 90 articles
on the ASTD website are
devoted to the measurement
and evaluation of training.
There are also more than
2,000 websites featuring
consultants and tools to help
the measurement process.
Yet, despite all these extensive
resources, most training
departments in my opinion
do not have an actionable
measurement program.
As many of you are aware,
sustainable behaviour change
is a good indication of the
success of a learning and
development program. But it
is equally important to ensure
that change in behaviour is
aligned with the business
outcomes - whether that’s to
develop sales leadership skills
or improve productivity among
staff for greater return on
investment, linking behaviour
with business outcomes will
enable you to measure and
track any impact that change
has had on the business and
therefore demonstrate the true
value of training. Companies
that can demonstrate
behaviour change, maintain
this change and measure its
effect on business results are
the ones that perform the best.
The key is to focus 90% of
time and resource on putting
in place steps that will align
learning with the goals of the
organization and sustain these
skills. The other 10% should
be spent on equipping staff
with skills through training
programmes.
Continuous learning is essential
to the ongoing success of
any organization. It allows
employees to cope with and
also excel at increased job
demands - allowing businesses
to achieve excellence
and maintain competitive
advantage. So it comes as
some surprise to find that
many organizations don’t
measure the impact of their
staff training and development
on performance.
The reasons include lack of
resources, confusion about
what should be measured and
lack of staff who understand
measurement methodologies.
Many simply don’t think it
matters to know which training
is working best or having most
impact. Also, many managers
in charge of training simply
don’t see measurement as a
priority. However, as the saying
goes; “if you can’t measure it,
you can’t manage it”. And if
you can’t measure or manage
it, you may well ask yourself
exactly what the point is in
investing in it.
A research conducted by
Deloitte in 2012 surveyed
over 600 Training Managers
asking them what their top
two challenges were - 92%
percent of respondents rated
measurement as the top
challenge.
So the biggest challenge of
Training Managers and Vendors
is not to deliver appropriate
training, but to measure the
effectiveness of the training
imparted. Unless the value or
the impact of training can be
accurately measured, there
is danger of it becoming
an enjoyable activity, but
producing very little value to
the organization. While it is
important for participants to
enjoy the learning experience,
that itself is not the objective
of training. The objective is to
deliver high impact training
that will raise the bottom line
by increasing productivity.
Are We Going About
Measurement the Wrong Way?
Why is this? One reason
organizations fall short
in meeting their own
expectations is that many
organizations tend to approach
the measurement problem
the wrong way. Rather than
establish a measurement
process which delivers
actionable information to help
them improve the effectiveness
and efficiency of their
operation, they overly focus
on measuring the four levels
specified by the Kirkpatrick
model (1=satisfaction,
2=learning, 3=job impact,
4=business impact).
Donald Kirkpatrick’s Four
Levels of Training Evaluation
has been the industry standard
for measuring training success
since 1979. Level-1 measures
the participant’s reaction to
the training program (smile
sheets or program feedback
form) and Level-2 measures
the learning that has occurred,
usually through pre- and post-testing.
But Level-3 (behaviour changes
back on the job) and Level-4
(ROI) are rarely, if ever,
measured. In fact, according
to the Learning Resources
Network, 77% of organizations
use reaction measures; 36%
use learning evaluations; 15%
measure behavior change; ‘and
only 8% measure results’.
While the Kirkpatrick model
is a good taxonomy to help
understand the basics concepts
of learning measurement, it is
limiting. Organizations which
implement the most effective
and complete measurement
programs do not overly dwell
on the measurement of job
and business impact: they
focus on capturing actionable,
operational information which
is relevant to their business and
their key training programs.
They set operational targets
through key performance
indicators and they measure
them consistently and
regularly.
At ITTIGE Learning where
we work with companies
on learning engagements
to ensure effective business
outcomes – we suggest these
five strategies to improve a
training program’s overall
efficiency during the front-end
planning stage. The first
three strategies are aimed at
increasing the benefit (value)
of training programs, and the
following three strategies are
aimed at decreasing costs.
Although they are presented
as interdependent, they may
be applied individually or in any
combination,
Strategy 1: Align Training with
Goals
Determining where training
budgets and resources should
be allocated is an important
decision for any training unit.
Unless the training program
is a revenue generating
business, the budget is viewed
as an expense – which is
being continually scrutinized.
Managing training budgets
and resources should be
no different than managing
any other investment,
such as major equipment
or information technology
acquisitions.
Strategy 2: Reduce Time to
Competency
Since the ultimate objective
of a training program is to
develop skills, competencies
and/or attitudes needed
to resolve a performance
deficiency and, in-turn, attain
unit and organizational goals,
closing the performance
deficiency sooner may result in
significant added benefits. For
example, sooner operational
staff acquire skills needed to
implement a new process, the
sooner the savings will kick-in.
Strategy 3:
Select the
Right Blend
of delivery
options
Certainly, most training
professionals are aware that
eLearning, for example, can
provide just-in-time access to
training material whenever
and wherever it is needed.
Moreover, it can reduce travel
costs and, in some cases, time
required to complete training.
Strategy 4: Consider Internal
Versus External Options
In certain circumstances,
external training partners
can be more effective and
economical than internal
trainers since they offer certain
expertise in specific fields that
are difficult to attain within
a training group that caters
to a wide range of needs.
By capitalizing on external
expertise, the learning curve of
the internal training group may
be reduced and costly errors
may be avoided.
Strategy 5: Duplicate Efficient
Programs and detect problems
areas
Are you comparing the
costs and benefits of various
learning initiatives and
allocating funds accordingly?
Are you forecasting budget
and resource needs for your
training program over the next
3-to-5 years, or beyond? Do you
know how your organization
measures up against other
training organizations/units? In
other words, are you making
the most of your training
budgets and resources?
By managing training
budgets and resources from
a centralized database,
data from multiple training
programs, businesses, and/
or units may be compiled,
analyzed, and compared –
to identify ways of running
training programs more
effectively and economically.
Measurement of training is
a journey, not a destination
- Training is a means to an
end. No satisfaction, learning,
or job impact measures are
meaningful if they are not
correctly applied to the right
urgent and pressing business
problems. And if we know
what these are, then we can
measure things which are
far more specific to those
initiatives, And it is a critical
one: with organizations like
yours spending 5-15% of your
total payroll on training, it is
important to know that these
budgets are being well spent.
We hope that this article and
our underlying knowledge will
help you make measurement
an easier, more business-focused,
and more valuable
process than ever before. I
look forward to hearing from
you at pavan@ittige.com with
examples of your measurement
challenges and successes.
Pavan Sriram is the Founder
and CEO of ITTIGE Learning, a
performance-based training
company that works with c-level
executives to improve learning
outcomes.
26 www.martupdate.com November 2014 November 2014 www.martupdate.com 27