The document discusses using data analytics in training and development. It defines learning analytics and differentiates it from other types of analytics like web analytics. Learning analytics should be rooted in learning sciences and evaluate programs to improve the existing system. Data analytics can provide insights into skills gaps, areas for updating, and strengths. Visualizing data allows teams to understand information and its implications. The document also discusses measuring the effectiveness of training programs through metrics like retention, sales, efficiency, customer service, and ROI.

1. By
Dr. Nilesh Thakre
Mumbai
Analytics in Training & Development
ROI

2. Contents / Objectives
Analytics in Training &
Development
ROI

3. Learning analytics is the
measurement, collection, analysis
and reporting of data about
learners and their contexts, for
purposes of understanding and
optimizing learning and the
environments in which it occurs.
Society of Learning Analytics Research

4. What are NOT learning analytics?
Classroom optimization
Staff allocation
Web analytics
These are important analytics activities, but
they are not learning analytics

5. Analytics must be
rooted in learning
sciences

6. Analytics should do more than
only evaluate what we are doing in
our existing system.

7. An LMS pulls things together by
emphasizing one space
Analytics/monitoring tools pull
things together by evaluating
relatedness outside of a central
space
Each new node amplifies value

8. Data analytics in L&D
The key to meeting any and
all moments of learning need is using
data analytics in L&D
L&D teams have a wealth of clues as to
what’s needed: The data offer insights
into what skills people lack, where they
need to update, and where they excel.

9. Data analytics in L&D
It’s difficult to pick out salient numbers
or information from all those lines and
columns of numbers. But data analysis
and data visualization—mapping the
abstract data onto charts, graphs, and
other visual forms—allows L&D teams,
managers, even learners themselves to
understand what the data are
communicating.

10.  How many learners completed a course?
 Number of hours of training employees in each
division completed
 Percentage of learners who watched a video vs.
listened to a podcast
 Employee’s or team’s sales revenue before and after
training
 Whether top-performing sales personnel are in the
group that got performance support after training?
 The group that received only training, or the control
group that received neither
Illustrative examples of Quantitative data in T & D

11. Qualitative data examples in an Training
context might be:
Percentage of learners who indicate
that the course they’ve just completed
was relevant to their job duties
Whether customer satisfaction ratings
are higher for retail employees who
have completed training vs. those who
have not
Illustrative examples of Qualitative data in T & D

12. Measuring the effectiveness of
training programme
1. Increased retention:
 Most Human Resources departments
measure the rate of retention in all or
various jobs. Many times, the front line,
high turnover jobs are the ones that receive
the most attention. If newly trained
employees feel ill-equipped for the job, they
are more likely to leave within their first 90
days. When you measure training success
this way, higher retention points to a
successful training program.

13. 2. Increased sales:
Many organizations can track efficiency
based on sales. If training is heavily geared
toward a sales or customer service force, an
effective program will eventually increase
sales numbers. You can also measure
product knowledge training as part of a sales
number – poorly educated sales people
usually do not make the sale. Sales figures
and unit sales make good metrics, but be
sure to balance any metric with other factors
that can influence sales numbers.

14. 3. Increased operational efficiency:
 In highly regulated or production-
oriented businesses, managers look for
more efficiency, which raises the bottom
line. If your training programs teach
skills, look to management’s efficiency
metrics, as a baseline, before and after
the training intervention. If you are
building a new program or product, look
at the efficiency numbers to obtain
direction on training course content.

15. 4. Customer service results:
Any organization can link training to customer
service, which can be both internal and external.
Customer service is also one of the easier place to
start: one well-written survey can identify a host
of customer related issues that can be addressed
by training programs. Remember that training
may not be the only solution to those issues. If
your organization already has a customer survey
in place, use those metrics to cross check your
programs. When your programs impact the survey
items, you can correlate an increase in customer
satisfaction back to training.

16. 5. Company-defined scorecards:
Training outsourcers tend to use client-
defined criteria to determine training
effectiveness. If your organization has a
wide variety of possible measurements,
sit down with management, and
stakeholders, to create a custom
scorecard based on expectations and the
training programs that need to be in
place.

17. 6. Cost of training:
 This is an internal training department
measurement. In high turnover
organizations, lowering cost per student
can be used as an effectiveness
measurement. Cost of training could also
relate directly back to retention – if you’re
spending less on new hire training, your
retention may be higher. Work with your
stakeholders and the HR department to
determine training costs and where you
want those numbers to be.

18. 7. Return on Investment:
 ROI has long been a “catch all” metric. In
some cases, it’s easy to define ROI, but in
more cases it’s increasingly difficult. If you
deliver soft skills training, it’s hard to put a
dollar figure on the return. There are
numerous ROI calculations available, so if
you’re thinking about using an ROI metric,
look for the formulas and plug in what you
can. If you are part of a numbers-driven
organization, you’ll be able to make friends
with the stakeholders by defining and
measuring concrete ROI.

19. 8. Revenue generation:
This metric appears most likely as a
combination of sales numbers,
operational efficiency, and customer
service. If an organization shows
increased revenue, a solid training
program can be part of that increase. If
your organization is rolling out a new
revenue generator, such as a product or
service, that is generally the best time to
use revenue generation as training metric.

20. 9. Trainers Performance:
 Trainer evaluation is an important internal
measurement. The results can come from learner
and must take into account the trainers
presentation skills, knowledge of the subject,
projection of organizational values, and adherence
to instructional guidelines. The good part about
trainer performance as a metric is that it can also
be used as an external measure. When training is
under discussion, training managers should be the
first to praise their instructors for delivering
quality instruction in every course – and instructor
evaluations provide the supporting evidence.

21. 10. Participants Satisfaction:
Your participants can measure effectiveness
quicker than anyone else, both immediately
following training and after a given time
period, such as 30 or 60 days. The
immediate results, sometimes referred to as
“smile sheets”, can give you a picture of what
happened in the classroom. The delayed
results can tell you if the material is useful
or not. Plus, end-user surveys are great tools
for proving effectiveness with management.

23. ROI
 ROI is the actual, physical calculation of
income gained from training.
 The training ROI calculations is to take the
total benefit, subtract the cost, and divide by
the total cost. The resulting number is
multiplied by one hundred to determine ROI.
 If you can do this with some certainty, try an
ROI calculation. Otherwise, try to determine
how training affected measurable factors such
as products, income, efficiency, and turnover.

25. L&D programs and employees Retention
Analytic insights can determine the
correlation of those taking training
programs to turnover, or highlight
particular training programs that lead to
the longest-tenured cohorts. With this
information, you can determine which
levers to pull to further increase employee
retention based on the programs the data
has revealed as most successful.

26. L&D and improved employee performance
Strong performers can make the
difference between hitting business
objectives and missing them. Yet, it
has been challenging for L&D
leaders to connect learning
investments with performance
outcomes. One approach has been
to run an experiment with a control
group.

27. L & D and Employee Engagement
Analytics help examine employee
engagement within locations, departments,
and teams to discover if those who are highly
engaged are the same individuals that have
recently completed a professional
development program. Likewise, learning
data can reveal which groups have the lowest
engagement, what factors may be
contributing to this low score, and whether
more training could improve the situation.

28. COMPONENTS FOR MEASURING
CORPORATE TRAINING
Measurement Strategy
- Strategy
 Compliance
- Approach
 Mandatory evaluation of every program; Must
adhere to regulators standards for evaluation;
Typically apply standardized methods.
- Reach across the Curricula
 Broad—every course is evaluated

29. Measurement Strategy
- Strategy
 Value
- Approach
 Selectively apply custom evaluation theories to
high- profile, high-impact, high-cost programs.
- Reach across the Curricula
 Narrow—a small number of courses are
evaluated
COMPONENTS FOR MEASURING
CORPORATE TRAINING

30. Measurement Strategy
- Strategy
 Mixed
- Approach
 Apply a combined compliance- and value-
based approach.
- Reach across the Curricula
 Broad to meet compliance requirements and
narrow to meet information needs
COMPONENTS FOR MEASURING
CORPORATE TRAINING

31. Recap