Large organizations are much less about organizing assets and the efficient production of tangible assets. It's now about bringing in talented people who can learn at a pace that gives the business competitive advantage.
3. make a
big impact
page
03
“L&D is responsible for improving our most
important asset, our people. If we can improve
L&D, we can improve the world.”
me, I said that
5. today.
Today we have a long road
ahead of us.
Data from Brandon Hall
confirms results of the LEO,
Watershed, iVentiv survey.
http://www.brandonhall.com/blogs/measurement-and-analytics-magic/
7. understanding learning
analytics: levels
page
07
measuremen
t
The simple act of tracking
things and recording
values. Can be passively
or actively collected.
evaluation
The process of trying to
make meaning from the
data we’ve measured.
Descriptive analytics.
Does the data mean
something good or bad?
advanced
evaluation
When data sets get large
enough, use advanced
evaluation techniques to
discover powerful
insights. Data mining, AI,
machine learning, etc.
predictive &
prescriptive
Make predictions and
decisions based on data
and advanced algorithms.
Recommendations
engines are the best
example in learning.
8. understanding learning
analytics:
categories
page
08
Understand more about a
specific learning activity.
Maximize effectiveness and
spot problems.
learning
experience Understand a learner or group of
learners. Ensure organizational
readiness and compliance.
learner
Understand an overall learning
program. Is this initiative helping
to meet our organization’s
business objectives?
learning program
11. example #1
analyze informal learning
page
011
What they learned:
• How do people choose to learn?
• Search:
• What are people searching for?
• What are they not finding?
• What resources are most popular?
• What resources are not being used (scrap learning)?
12. example #2
monitor cross-platform training
page
012
What they accomplished:
• Deliver compliance training across devices (e.g.,
mobile, desktop)
• Tracked without using an LMS (i.e., learners could
access from anywhere)
• Detailed understanding of how learners were using the
content
• Spotting troublesome areas and fixed problems before
too many users were affected
14. Customer Service Training
• Track activity across LMS
and support/ticketing
platform.
• Monitor customer satisfation.
• Identify how KPI trends
are impacted by other
activity.
15. xAPI &
learning analytics platform
page
015
all data
Learning is so much bigger than the
LMS. Track informal and experiential
activities.
xAPI and the LRS present a transformative
technology for learning analytics. Seamlessly
bring all your data together in one platform and
perform real-time analytics tailor made for L&D.
business
impact
Include data about behaviors and
performance in your learning analytics
real-time
Create, view, and distribute reports
seamlessly in real-time.
17. getting started
yes you can!
page
017
01
Gather your data
Just start collecting data in a common
format in a central location. Make it low
friction and ensure you have access.
04
Explore your data
Starting to understand not just what is
happening, but why. Form questions and
see if your data answers them. Often, the
best answers lead to other questions. Can
you see impacts on the larger business?
02
Get to know your data
Understand what you have. What’s out
there, what’s reliable, and what’s missing?
Do some simple evaluations and create
baselines.
05
Experiment with your data
Start a new learning program with analytics
in mind. Set up well-controlled experiments
and A/B tests to validate a hypothesis.
Create a culture of continuous
improvement.
03
Operationalize your data
Automate your way out of “Excel hell.”
Define some interesting metrics and KPIs,
and start monitoring them routinely. What
trends do you notice?
06
Show off your data
We’ve long known that learning is vital,
not we can prove it. Go forth and show the
world!
Scalable learning is the new reason for large organizations to exist
It’s no longer about assets, organization or efficiency
We’re moving from Return on Assets to Return on Talent
This is the most strategic thing or organization can be doing
Scalable learning is the strategic differentiator in the 21st century
There are so many things required to achieve scalable learning
We’re not just talking about dissemination of information we know, but also about the acquisition and discovery to new knowledge in a scalable way
How do we get there.
To achieve scale, we need measurement, we need a culture of constant improvement and we need alignment with business objectives
Because this isn’t just about scaling learning, it’s about scaling an organization’s ability to adopt to the exponential rate of change the digital economy is creating
To achieve scale, we need analytics, we need data
This is important and impactful stuff.
We live in an increasingly knowledge based economy where talent is our most precious asset
How the heck do we let the group that is responsible for improving the value of that asset, for improving the performance of our people, how the heck do we let them get by without a culture of constant improvement? Without a responsibility to use data to ensure they are impacting business processes and metrics in a positive way?
When we asked the question, the answer was consistently that it is too hard.
Well that is changing and I’m excited about the impact that can have, not just on our industry, but on the world.
If we can make corporate L&D just 10% more efficient, if we can get just 10% better at improving our workforce, is is reasonable to assume that we could make our workforce 1% more efficient at their jobs? A 1% improvement in productivity across the workforce is a really big deal.
It’s learning’s turn to take a seat at the table
To do so, we need to think strategically, we need to think in the same terms the business thinks in and align our strategy and outcomes to the strategy and outcomes of the business - often performance
We have a long way to go, but we can look to marketing for a perfect example of how to make this transition.
It was all driven by data.
Spiral or go from inner to outer level in one dimension
(Visa example)
From a client using Sharepoint
Learning experience analytics
Basic evaluation (with some text processing goodness thrown in)
Give Thursday example
(AT&T example)
Where do people bail / get stuck - is this point problematic?
Watching people cheating, etc
See actual experience of user – help debug
How long, which pages spend most time on, which questions consistently getting wrong.
Learning experience analytics
And
Learner analytics
Still just basic evaluation
Give bad button example
Learning program evaluation
Learning program evaluation – advanced analytics / prescriptive analytics
It’s all about removing friction
Removing just a little bit of friction has a big impact
Apple design methodology
Kirkpatrick model has been around since 1954, but how much of it are we using.
Remember, when we asked people why they aren’t doing more learning analytics, one of the chief responses was “it’s too hard”
That is changing, the friction is disappearing and that will have a profound impact.