5 keys to improve roi on e learning

Collection
[BESTPRACTICES]
keys to improve
the ROI of your distance
learning solutions
5

2 5 keys to improve the ROI of your distance learning solutions

35 keys to improve the ROI of your distance learning solutions
Technology can make professional learning affordable, easy, and convenient. As distance learning programs
grow ever more popular, companies have discovered that technology-driven learning can be a critical tool
to communicate new information and regulations with far-flung workers, and to help businesses that are
experiencing significant growth get employees quickly on the same page.
Many executives assume that e-learning will result in tangible business gains, such as productivity
improvements, more consistent practices, cost reductions, and time savings. The problem is that these benefits
sometimes fail to materialize. To make matters worse, many companies lack a framework to truly measure the
effects of the learning they invest in.
The return on investment for distance learning efforts needn’t be a mystery. Properly measuring return on
investment (ROI) provides an answer to the question: For every dollar invested in distance learning, how many
dollars are returned after the investment is recovered?
The basic calculation of ROI is:
This well-known equation, however, does not capture all the potential benefits and nuances of e-learning
programs. To be meaningful, ROI needs to take into account other performance measures. Jack Phillips,
Chairman of the ROI Institute, has developed a 5-level methodology for measuring ROI.
Foreword
Program Benefits – Program Costs
ROI (%) = x 100
Program Costs

The Evaluation Framework
• Reaction and Planned Action: Measures participants’ reactions to and satisfaction with the learning program
and participants’ plans for action
• Learning: Measures how much participants have learned, and what skills they have gained
• Application and Implementation: Measures the extent to which participants have applied their new
knowledge and skills on the job
• Business Impact: Measures the extent to which business impact measures have changed since employees
completed learning
• Return on Investment (ROI): Compares the monetary value of the business outcomes with the costs of the
learning program
Each level of evaluation provides important data that can be used on its own to measure the efficacy of distance
learning. But when taken together, a more complete and accurate measurement of ROI emerges. This improved
picture allows executives to justify spending on technology-based learning, improve programs, eliminate
unsuccessful ones, and set smarter company priorities.
This practical guide, written by CrossKnowledge in collaboration with Jack Phillips from the ROI Institute, will
help you to have a more holistic measurement of the effectiveness of your e-learning programs, by adopting a
number of key solutions to improving ROI.
1. Create a learning management system that captures critical data and adapts to your evolving needs
2. Identify your business aims and allow them to drive learning programs
3. Include responsive and dynamic content to hold learners’ attention
4. Get the right balance for qualitative learning
5. Extend your impact with innovative learning efforts

Key
N°01
No efficient ROI
without a smart LMS
Create a learning management system that captures critical data
and adapts to your evolving needs

01
Adopting an effective Learning Management System (LMS) is essential for getting the
best ROI on distance learning. Without a versatile LMS, companies simply cannot get
the relevant information they need in order to measure the effectiveness of learning
programs.
A good LMS allows companies to track participant completion rates and user
engagement, and to include built-in surveys and feedback mechanisms to continuously
measure the program’s effectiveness.
To optimize your LMS, you should focus on the following:
Concentrate on data
The LMS is where companies collect and analyze the data that will provide them with the
building blocks for ROI. But it can be tough to know up front what data would be useful to
collect. Step back and evaluate what you would like to learn about the distance learning
programs, and use that list of needs to inform the design of your LMS.
No efficient ROI
without a smart LMS
A large telecommunication company uses
its LMS to capture a variety of data:
• course completion rates
• frequency of log-ins
• most popular courses
• time spent on content
• most popular forms of content
(video, documents, etc)
• use of social elements (message
boards, comment functions)
• reaction summaries
• planned actions
• forecasts of impact
• learning scores
• application follow-ups
• barriers to success
• enablers to success

Aim for a dedicated system
Many companies feel ill-equipped to build an LMS on their own, whether out of concern
about the cost, manpower, or technical know-how. There are plenty of off-the-shelf
learning management systems available to jumpstart e-learning efforts if your company
isn’t interested in starting from scratch. But not every size fits all. When considering
ready-made options, lean toward those that allow you to tailor systems to your specific
corporate and industry needs. Systems should also be capable of adapting over time to
match evolutions in your learning.
Design for evaluation
It’s not enough to simply capture raw data about your employees’ e-learning usage.
It’s also critical to include surveys and assessments that directly measure employees’
progress and opinions about the programs. With those evaluation measurements in
place, companies can continually assess the effectiveness of training programs, identify
weak spots, and scale up strengths. Having smart evaluation tools also allows companies
to perform small-scale tests on new training programs, gauge their usefulness, and then
expand to larger populations within the company.
No efficient ROI
without a smart LMS
01

Key
N°02
Match your e-learning
environment with
your strategic needs
Identify your business aims and allow them to drive learning
programs

To improve ROI, organizations must first define their business objectives. By isolating
the business need, and allowing those needs to drive the design of the learning program,
companies are more likely to achieve a positive ROI.
Clarify the business needs
Ask basic questions about your business aims and the need for the program such as “Is
this program required and why?” The business issue and subsequent need might be: a
decline in sales (which might suggest the need for updated lessons on new products); an
increase in compliance violations (suggesting the need for more consistent training across
the organization); or excessive turnover of talent (suggesting a lack of career development
offerings).
Clarify performance needs
When business needs are defined, performance needs can be identified that would
improve their desired business outcomes. For example, compliance discrepancies (business
need) may be increasing because employees are not following the correct procedure
(performance needs). Everything driving the e-learning environment should attempt to
bridge the gap between those business needs and the performance targets.
Clarify learning needs
After the performance needs are determined, the learning needs are identified. The
learning needs, when met, will close the performance gaps. For example, if employees are
not following a procedure, maybe it is because they do not know how. That is the e-learning
solution.
Determine program objectives
It’s critical to set clear objectives for the program in order to properly judge ROI. Some
objectives will be straightforward and explicit: Employees might need to demonstrate
proficiency with new software at the end of learning, or complete a specific compliance
certification. In those cases, specific mastery goals should be set.
Other objectives may have timeframes beyond the end of the learning program, such
as improvements in sales of new products. And others may be more focused on an
improvement in engagement, productivity, and motivation. Ideally the e-learning program
Match your e-learning environment
with your strategic needs
02

02 Match your e-learning environment
should have reaction objectives, learning objectives, application (performance) objectives,
and (business) impact objectives.
The more specific and targeted the objectives, the easier it will be to measure progress
toward their goals.
As executives attempt to match learning programs with strategic needs, they will
undoubtedly be constrained by costs. There will, after all, be a budget for an e-learning
program, and a desire to obtain the most favorable outcomes for the least investment.
Costs to consider when matching learning programs with business needs:
• Development of solutions: The cost of developing and designing the learning program,
including experts’ costs, supplies, technology, and labor.
• Acquisition of solutions: Licensed materials and technology purchased from other
sources
• Implementation and application: Salaries for coordinators and program owners, as
well as the time spent by participants in learning sessions. Also includes hardware and
software necessary for participants to engage in learning courses
• Maintenance and monitoring: Maintaining and operating learning programs in good
working order
• Evaluation and reporting: Developing an evaluation plan, collecting and analyzing data,
preparing and communicating the results

Match your e-learning environment
02
A financial services company wants to update its sales training
e-program, both to update the key competencies that are being
taught to sales associates and to reduce the amount of time
workers spend learning.
Key business need: Increase sales with updated learning while reducing time
spent in learning
Specific program objectives: The company would strive to demonstrate
an increase in sales of 15 percent within six months. To achieve this, sales
associates complete 85 percent of action items with customers using the
competencies, and 70 percent mastery in post-learning assessments.
Matching needs with objectives: In order to increase sales and instill new
sales competencies, the program was designed to improve general sales skills,
knowledge of products, customer focus skills and negotiating. The program
was also designed to allow associates with demonstrated competency in a
certain skillset to bypass training in that sector, allowing them to focus on
other areas that need improvement and reduce the overall time dedicated to
e-learning.
Costs assessed: Because sales associates already had company laptops,
technology costs were limited to the development of the software and
associated server costs. Other costs included program development expenses
(developers’ labor and testing); and participants’ time spent in learning
(salaries).
Results considered in ROI: The company was able to increase sales by 20
percent which represents $9 million in sales annually. Also, the company
reduced the number of hours new sales associates spent in training sessions,
which means they are able to start selling products earlier in their tenure.
Sales from existing clients increased as a result of the new competencies. And

02 Match your e-learning environment
y
as new accounts opened and customers were retained (based in part on the
improved competencies) new business revenues increased. Participants, with
the help of managers, allocated 10 percent of the increase (one half) to the
e-learning program.
This 50 percent represented a total sales of $4,500,000 connected to the
program. When this is converted to profit (the value added) of 20%, the
monetary benefits are $4,500,000 X 20% = $900,000. The total cost of the
program is $550,000.
So for every $1 invested in the e-learning program, there is a return of .64¢
after the costs are recovered.
This example compares the cost of e-learning to the monetary benefits driven
by the e-learning program. Sometimes it is feasible to develop the ROI by
comparing the cost of e-learning to the cost of a facilitator led version. This
comparison is only appropriate when both versions produce the same impact.
$900,000 (benefits) – $550,000 (costs)
ROI (%) = x 100 = 64%
$550,000 (costs)

06
Key
N°03
Make your content
appealing
Include responsive and dynamic content to hold
learners’ attention

03
Big investments in e-learning programs are useless if learners don’t actually use or engage
with the content. Moderator-led, in-person training is expensive, but it is often effective
because expectations and bonds are forged in the classroom. Technology-based learning, by
contrast, may not always be taken seriously, given its convenience, the fact that it is typically
done individually, and is often optional.
To maximize ROI, e-learning programs must be designed with the learner in mind.
Great care must be taken to attract and hold users’ attention. It’s a delicate line to walk:
Designers must not only make big-picture results relevant and apparent to learners
(“Why do I need to take this course?”) but also consider the minutiae that might drive
them away (such as too many clicks and screens to get to critical information and
actions). The move toward increasing gamification should also be considered carefully.
While making learning more game-like can make programs more fun, it can also create a
disconnect between the learning and a user’s job if the relevance of the game-like training
is not made explicitly clear.
In this context, the designer and developer of the e-learning programs are key to
improving ROI. In facilitator-led learning, it’s the facilitator who can make the difference.
But in distance learning, the role of designer and developer are critical if you want your
program to provide a positive pay off.
Design an interface adapted to learners’ needs and expectations
Few people want to stare at a computer screen full of nothing but text for hours at a time.
Your average e-learner is no different. Assume that the majority of your participants will
favor a graphic e-learning environment, and consider carefully the design of your content and
program flow as a result. Reduce text reading to a minimum. Make responsive design -- easy
reading and navigation with a minimum of panning and scrolling -- the first building block of
your program. And consider at the outset whether you will need the program to be available
in multiple formats, including in other languages and on alternative platforms.
Keep it logical
Having content arranged and presented logically has a huge effect on whether
information is retained by learners. If information is presented logically, participants have
a better chance of incorporating it; if it is scattershot or poorly organized, you will lose
Make your content
appealing
y

03
participants’ attention in no time. You might design an introduction, main section, and
evaluation structure, or create a series of abstract information.
Increase visual interest
Humans are visual creatures, and many of us learn concepts best when we visualize them.
In order to increase the chances that content is learned and retained, make sure the
content is appealing to the eye. Incorporate pictures and videos to attract attention and
illustrated ideas. Graphics and animations can also quickly and succinctly convey complex
information in easily digested ways. Of course, don’t include graphic elements simply for
the sake of having them. The most important thing to make sure the visuals support and
enhance the message of the content.
Foster interactions between participants
If possible, create networks, whether on social media or through the LMS, so that participants
can share information, questions, success stories, and concerns. Classroom-based learning
can create tremendous bonding, commitment, and engagement, which is often lost when
learning moves online. Facilitating interactions throughout the learning program can
improve users’ dedication and replace the role of the facilitator, creating a virtuous cycle of
engagement.
Make your content
appealing
An international engineering and construction firm is using and e-learning
program to teach time savings tips and the use of productivity tools. The
impact is actual time savings of the learners, when the time saved is used on
other productive tasks. The program was voluntary and designed to meet
learners’ needs. The program was logical, attractive, and interactive. Learners
could share their success with tips and tools and learn from each other.

Key
N°04
Instructional design: stick
to learners’ needs and
constraints
Get the right balance for qualitative learning

Improving the ROI for distance learning often means reducing logistic costs, such as those
spent on experts and facilitators. But without those experts in the classroom responding
to participants’ questions and concerns, program designers must anticipate users’ needs
and build responses to them into the learning process. Adult learners want to know why
they need to learn something. They also approach learning as problem-solving, and learn
best when they believe that the topic is of immediate value.
As a result, designers must create a system that guides learners, rationalizes their time,
and uses storytelling to make users truly feel engaged.
Make training and learning relevant to participants’ jobs
The design needs to make it clear from the outset what completely learning programs
means to users and why it is relevant to their specific roles. Make sure it is communicated
to them or built into the system why they are taking programs in the first place, what
the business need is, and how their skills and performance will be enhanced through
completion. Connecting completion of the course to the user’s success is critical for
optimal ROI.
Show that managers care
With all learning programs, no matter the format, the manager role is critical. Managers
often don’t realize they are influential in driving home the importance of learning
programs. But even small gestures and statements from managers demonstrating
support for learning can make a big difference. Similarly, saying nothing at all sends the
message that the learning isn’t of much importance. Showing support from management
can be both formal and informal. Managers might make a point of briefly mentioning
the importance of learning or asking direct reports about their progress on learning
programs.
More formally, designers might build in direct messages and statements of support from
executives and managers into the e-learning programs themselves. For instance, a video
from the CEO at a key juncture of the training pops up and voices support for the partici-
pant’s effort and reiterates how critical the program’s objectives are to the mission of the
organization. This sends a powerful message that the program is important and that the
employee’s participation and effort are valued.
Instructional design:
stick to learners’ needs
and constraints
04

04
Become storytellers
Storytelling is critical for information to be conveyed in a compelling and memorable way.
Facts and figures often don’t stick in our minds at all, but stories create “sticky” memories
by attaching emotions to things that happen, which can focus learners’ attention, provoke
curiosity, sustain engagement, and ultimately, change behavior. What does storytelling
look like in a business context, and in the context of e-e-learning? Designers should
endeavor to include case studies, videos, demos, exercises, games and examples. Of
course, technology may be constrained by available budgets, but the basic arc of stories
– question, tension, closure –can be embedded in a variety of ways throughout the
program. Story-led training creates “stickier” learning, which naturally improves ROI.
and constraints
y
In a steel manufacturing company, a message from the CEO introduces an
important e-learning safety program for employees. The executive stresses
the importance of this program, identifies which measures that should
improve, creates expectations, provides support, and offers an open door to
feedback about safety issues.

Credit unions that are experiencing rapid growth and changes in regulations
often have a difficult time keeping employees up to date on critical training. A
cooperative based in Utah, recently launched a Learning Through Storytelling
Program to help employees better understand important issues relevant to
the business.
Members of this company created several story-driven training videos to
help employees grasp new and developing business concepts. A recurring skit
helped trainees understand the difference between wills and trusts. Another
animated story explained the difference between various kinds of financial
partnerships.
The company said the training had a number of important benefits, including
appealing to Generation Y employees. The videos produced by the program
not only helped turn concepts that might be perceived as boring into fun
and interesting ideas, but also helped improve employees’ engagement and
learning retention.
In order to test the effectiveness of the storytelling videos, employees were
tested on concepts before and after watching the videos. The results were
impressive, showing a 266 percent increase in knowledge concerning the
difference between wills and trusts and a 489 percent increase in knowledge
concerning the difference between revocable and irrevocable trusts after
employees watching the videos.
04
and constraints

Key
N°05
Explore new
opportunities
to maximize ROI
Extend your impact with innovative learning programs

In our rapidly changing business environment, skills must often be continuously updated.
For companies looking for additional learning opportunities, just-in-time training and
adaptive learning are natural complements to more ‘traditional’ e-learning systems. By
building on established learning systems, the additional programs can provide more ‘bang
for the buck’ by responding to individual training needs.
Just-in-time training
Just-in-time training allows workers to access learning when and where they need it.
They can tap self-guided tutorials and other tools at any time to complete tasks and
quickly update their skills. The learning is focused on “nuggets” of information that solve
problems as they arise, rather than on updating or replacing whole suites of training at
once.
The Benefit: The programs allow employees to receive the learning they need when they
need it, rather than sit through learning that might not be relevant or convenient at the
time.
Adaptive learning:
This data-driven, personalized learning adapts the presentation of materials according to
each learner’s needs, based on that individual’s position, existing skills, prior performance,
and goals. The programs are able to tailor to the participant’s specific needs based on
accumulated data, which may include information from past courses and responses to
surveys, and then anticipate what types of content and resources learners need at a
specific juncture in order to make progress.
The Benefit: The programs can adapt automatically to different skill levels, which often
results in faster and more substantive learning as a result of the personalization.
Both tracks can result in more engaged and satisfied employees, boosting companies’
retention capabilities and ROI. Continuing education and professional development have
come to be seen as a key workplace benefit in today’s new economy. By complementing
traditional e-learning options with just-in-time training and adaptive learning programs,
companies create powerful incentives for employees to stay and work for success.
Explore new opportunities
to maximize ROI
05

05
Case Study:
Measuring ROI in an Upgrade Selling Program: A Mobile Learning Solution.
Transoft Inc.1
Brief Summary
This project involves a mobile learning application for sales associates of a large software
firm specializing in software solutions for the trucking industry. Sales associates were
provided a mobile learning solution for their iPads, designed to describe and sell an
upgrade to its most popular software product, ProfitPro. Although the release occurred
at the same time for all sales associates, not all of them logged onto the learning portal
through their iPads, which allowed an opportunity for a comparison group. This case
study highlights the key issues in calculating the impact and ROI of a mobile learning
solution on business results.
Results
The first 25 people signed up within three days of the program’s announcement. Reaction
data were collected at the end of the fifth module and are presented in Table 1. Reactions
were as expected, with the exception of intent to use, which was very high.
Table 1. Reaction Results
Rate the Following Rating
Relevant to my work 4.3
Important to my success 4.1
Intent to use 4.7
Recommend to others 4.2
Target: 4.0
(Scale: 1 = not at all; 2 = some; 3 = average; 4 = above average; 5 = very much)
Average: 4.3
1.
Elkeles,Tamar,PatriciaP.Phillips,andJackJ.Phillips.MeasuringtheSuccessofLearningThroughTechnology:AStep-by-Step
GuideforMeasuringImpactandROIonE-Learning,BlendedLearning,andMobileLearning.(Alexandra,VA:ASTDPress:2014)
p.177.
Learning measurement seemed appropriate, and quiz scores were above targets, as
shown in Table 2. Scores were slightly lower than desired for the quiz for implementation
and support. This was not a huge concern, as this information is clearly in the documents
and the product brochures.
to maximize ROI
y

05
Table 2. Learning Results
Module Quiz Categories Avg. Number of Cor-
rect Responses
1 Rationale for the upgrade 4.3
2 Key features of the upgrade 4.2
3 How upgrade will increase client profit 4.3
4 Pricing options 4.1
5 Implementation and support 3.9
Possible score: 25
Target: 20
Average: 20.8
Application data seemed to be on track as shown in Table 3. Identifying pricing options
and explaining implementation and support were off a little, but overall the objectives
were met. The time for the first call was below the objective. As expected, there were
some barriers and enablers to success, as shown in Table 4. The barriers were minimal.
However, there was a concern that 9 percent of sales associates were not encouraged
by their managers to use the program. As expected, the number one enabler was
management encouragement.
Table 3. Application Results
Extent of Use in One Month Rating
Make the first call (target: 5 days) 3.5 days
Explaining the rationale for upgrade 4.2
Identifying key features of upgrade 4.3
Describing how the upgrade increases client 4.1
Identifying pricing options 3.9
Explaining implementation and support 3.8
Target: 4.0
(Scale: 1 = not at all; 2 = some use; 3 = moderate amount of use; 4 = significant amount of use;
5 = very significant amount of use)
Average: 4.06
to maximize ROI

05
Table 4. Barriers and Enablers
Barriers % Reporting
Lack of time to be involved 12%
Lack of management encouragement 8%
Took too long to complete 8%
Technology issues 4%
Other 4%
Explaining implementation and support 3.8%
Enablers % Reporting
Management encouragement 60%
Easy to use 52%
Timely 48%
Convenient 40%
Relevant 36%
Other 16%
Table5showstheimpactdatacomparingtheexperimentalgroupof25salesassociateswiththe
controlgroupof22.Asexpected,almosteveryoneofthecontrolgroup(19of22)wasactually
sellingtheupgradethoughnotparticipatingintheprogram,butthedifferenceofthetwogroups
wasveryimpressive.Thedifferenceforthatsecondmonthisthenannualized,producingan
annualimprovementof$1,140,000.Thetimeofthefirstsalewasimpressiveforthegroup
involvedintheprogram—11dayscomparedto21daysforthecontrolgroup.
Table 5. Sales Data in Three Months After Launch
Group Avg. Sales (Month 3) Avg. Time to First Sale
Trial (Experimental) Group: 25
sales associates
$7,500 11 days
Comparison (Control) Group: 19
sales associates (out of 22)
$3,700 21 days
Difference = $3,800
Annualized = $3,800 x 12 x 25 = $1,140,000
20% profit = $228,000
As shown in Table 6, the fully loaded costs were included to make the ROI calculation
credible.
to maximize ROI
y

05
Table 6. Costs
Needs assessment (est.) $3,000
Design and development ($56,000/110 x 25) $12,725
Mobile device (prorated) $1,700
Sales associate time $2,524
Administration time (est.) $6,000
Project management (est.) $14,500
Evaluation $15,000
Total $55,449
Table 7 shows the benefit/cost ratio and the ROI calculation. As anticipated, results
exceed the ROI objective.
Table 7. BCR and ROI Calculations
In addition to the tangible sales increase converted to money, several intangibles were
connected to the program, as presented in Table 8.
Table 8. Intangible Benefits
The complete case study is published in the book Measuring the Success of Learning Through
Technology: A Step-by-Step Guide for Measuring Impact and ROI on E-Learning, Blended Learning,
and Mobile Learning.
to maximize ROI
$228,000
BCR = = 4.11
$55,449
$ 228,000 – $55,449
ROI (%) = x100 = 311%
$55,449
• Made the first sale in 11 days average
• Customer satisfaction
• Brand awareness for ProfitPro
• Job satisfaction of sales associates
• Stress reduction for sales associates
• Reputation of company

Collection
[BESTPRACTICES]
www.crossknowledge.com
ThisdocumentisapublicationoftheCrossKnowledgecompany.Allbrandandlogos,registeredorotherwise,arethepropertyoftheirrespectiveholders.
Thisisanon-contractualdocument.Design:Indexel-www.indexel.com
Collection
[BestPractices]
ways to link
distance training
to corporate needs
5
Collection
[BestPractices]
ready-to-use
training
innovations
5
Collection
[BESTPRACTICES]
keys for the successful
gamification of your
training project
10
Collection
[BestPractices]
ways to make
a success of informal
learning
7
Collection
[BESTPRACTICES]
keys for
boosting learner
engagement
10 tips for Rapid
Learning rollout
Collection
[BestPractices]
10
Collection
[BestPractices]
7keys to success
in social learning
ways to introduce
e-learning successfully
Collection
[BestPractices]
5
Collection
[BESTPRACTICES]
7golden rules for
a successful blended
learning program

5 keys to improve roi on e learning

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (20)

Similar to 5 keys to improve roi on e learning

Similar to 5 keys to improve roi on e learning (20)

Recently uploaded

Recently uploaded (20)

5 keys to improve roi on e learning