Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo

Measuring the ROI of Informal
Learning
David Wentworth
Senior Learning Analyst
Brandon Hall Group
@DavidMWentworth
Alessio Artuffo
Chief Operating Officer
Docebo
@AlessioArtuffo

Learning & Development
Talent Management
Leadership Development
Talent Acquisition
Workforce Management
RESEARCH PRACTICES
© 2015 Brandon Hall Group2

Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com
Copyright © 2015 Docebo - All rights reserved. 3 www.docebo.com
Docebo, at a glance
70About
Innovation-driven, cloud-
based
learning Ecosystem for the
Extended Enterprise.
50% Product and Innovation
30% Sales and Marketing
20% Operations
100
Talents
NORTH AMERICA
Athens, GA (USA)
EUROPE
Milan (Italy)
MIDDLE EAST
Dubai (UAE)
Where
Ideal addressable market:
Organizations with 50 MIL+ in revenues
850 2
850 Partners (clients)
“We are obsessed by : Innovation and Customer Success”.

Join the Discussion
• Submit your questions
or commentsabout the
discussion to our
presenters using the
Questions tab on your
control panel.
• Presentation slides and
giveaways can be
found in the Handouts
tab on your control
panel.
© 2015 Brandon Hall Group 4
Type in your question here
HOW TO ASK QUESTIONS AND PRESENTATION RECORDING & SLIDES
Recording & Slides will also be sent out after the webinar.

Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 5
TODAY
ORGANIZATION DRIVEN
Learning Compliance
The Evolution of Learning
USER DRIVEN
Learning Experience

The State of 70/20/10

Very Little Learning is Formal
• Formal
Learning/Catalog
• Certification &
Compliance
• Courses
• Groups Associated
with Classes
• Impressions
• Expertise Location
• Idea Engine
• Real-Time
Collaboration
• Groups &
Workspaces

Poll Question
How much of the learning that occurs within
your organization would you consider informal?
• 0%
• 1%-5%
• 5%-10%
• 15%-25%
• More than 25%

What is the mix?
MIX OF FORMAL, INFORMAL AND ON THE JOB ACTIVITIES
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Formal
Informal
On the Job Activities
43%
14%
43%

The Questions We Need to Ask Ourselves
The “20” - How do we create
meaningful and impactful
informal learning?
The “70” - How do we
optimize on the job
training?
The “10” - How do we
rethink our classroom
environment?

70/20/10: A Fresh Look

Application to a Single Event
12
Pre event
Learner completes PDP,
identifies a stretch
opportunity, identifies a
learning need
Learner & Manager
discussion about
development need, current
skill level or capability
Course pre-work requires
learner to hold a formal
discussion with their
manager about the course
and current skill
level/capability
During the
event
Learner learns in a formal
setting
Learner encouraged to
reflect on application in the
workplace and think about
future development
Immediately
post event
Learner returns to role,
applies learning and
evaluates learning event in
relation to role
Learner discusses event
with manager, or others and
reflects on event and
application in the workplace
45-90 days
post event
Learner 2nd
evaluation,
application of learning in the
workplace
Makes changes to working
practices, tries new things,
challenges self to make
change
Manager evaluation of
impact on role and learner
post discussion with learner
about application in the
workplace and any further
development needs
Learning through
experience
Social learning Formal learning
© 2015 Brandon Hall Group

Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com 13
Flipping the Classroom
13©2014 Brandon Hall Group, Inc.

Application to a Program
14
Discussion
about need
and
suitability of
programme
Workshop
Apply
learning in
the
workplace
Formation of
project
groups
Project work
Coaching/
mentoring
for project
group
Application
of learning
so far in the
workplace
Workshop Project work Coaching
Application
of learning
so far in the
workplace
Workshop
Learning through
experience
Social learning Formal learning

Implementing the Framework
TOP AREAS OF FOCUS
• Solve real-world
problems
• Optimize teams
• Increase application of
training on the job
• Introduce new
approaches to work
• Access to coaches and
mentors
• Leverage e-learning

Implementing the Framework
SUCCESS ELEMENTS
• Full support of
executives
• Development of a clear
vision, mission, strategy
and governance model
• Adequate resourcing
• Solid technological
foundation

What makes the 70 and the 20 so
CRITICAL?

Embedded Learning is Essential
How often would you expect your individual contributor population, on
average, to need to connect with learning resources to effectively perform
their job?
0.8%
9.4%
29.0%
36.7%
24.1%
Less than annually
Annually
Monthly
Weekly
Daily or more often
61%
Brandon Hall Group, Social & Collaborative Learning, 2014
18© 2015 Brandon Hall Group

• Contextual
• Role
• Location
• Development path
• Small
• Short videos
• Quick updates
• Informal
• Everyday
• Ad hoc
• Mobile
• Down time
• Moment of need
• Social
• Discussions
• Recommendations
Embedded Learning is Essential

Forgetting Curve

Boost Learning

The 70:20:10 Effect
4.9%
29.4%
46.1%
19.6%
Not at all
Small extent
Moderate extent
Great extent
Improved Link to Individual
Performance
9.1%
37.4%
37.4%
16.2%
Not at all
Small extent
Moderate extent
Great extent
Improved Link to Organizational
Performance
95% 91%

High Performance Learning
• Support proactive learners -- employees who take
responsibility for their own development and seek out
learning
• Provide easy-to-use tools that present employees with
opportunities to connect to formal and informal knowledge
sources throughout the organization.
• Present each and every employee with opportunities for
development based on personal strengths, weaknesses, job
roles, or interest from sources that go beyond their
immediate leaders.
80% of High
Performing
Organizations have
these characteristics

Poll Question
How likely is your organization to adopt the
70:20:10 model?
• We already have
• No plans at all
• Would like to but, need to understand it more
• There are clear use cases in my organization
for adoption

Learning is not the outcome
PERFORMANCE IS

Vision

Adoption : Sales Enablement
• Dreams Goal WhoSectors
Create a culture of learning
and staying updated by
sharing, coaching, and
people enablement.
Transform the formal content
in supplementary content, in
support of informal and
social learning frameworks.
Sales professionals.
They are the nightmare of any
elearning training team.
They live at fast-speed and
make limited time for formal
knowledge. They are the most
natural social learners in the
corporate emisphere.
Retail,
Software,
Consulting,
High Tech,
Manufacturing
Achieve consistent and high-
quality product and services
knowledge.
Achieve high information
retention.
Decrease the investments in
one-time use learning
material.

• Adoption : Health and Safety
Transform the compliance
training programs from highly
regulated and formal only, to
engagement-heavy and
collaborative learning
opportunities.
Compliance and quality
professionals.
Energy,
Manufacturing,
Biotech,
Oil and Gas,
Healthcare
Increase the knowledge
effectiveness, quality and
retention in highly critical
H&S matters.
Allow for SOPs to be curated
by knowledge gurus as they
become obsolete and prevent
hazards via guided social
improvement.
• Dreams Goal WhoSectors

Technologies in support of the 70:20:10 model
70
20
10
INFORMAL
TRAINING
SOCIAL
TRAINING
FORMAL
TRAINING
User-generation content features,
Peer-validation and curation
Engaging the knowledge gurus,
Enable learning by collaboratuve sharing,
Integrate knowledge-rewardinggamification
Organize, distribute
track online courses and retention.

Peer Review
Video Curation
Performance support
70

20
Ask the Experts,
assign a topic
Associate topics
to experts

20
Ask questions to experts,
get answers, discuss.
Follow questions
Find “Best Answers”

Measuring:
- Coaching activity
- Reviewing activity
- Experts’
engagement
- Contributions
- Effectiveness
20
70

How do we measure the
INFORMAL?

The Trouble with Analytics
15.9%
17.9%
23.2%
25.2%
27.2%
Social learning features
Analytics features for ROI
Reporting features
Adaptability
Ease of use
Least Liked Aspects of Current LMSs
36

The Trouble with Analytics
20.1%
23.5%
29.5%
20.1%
6.7%
1 2 3 4 5
Satisfaction with LMS Analytics (1-5)
37
2.7
Average
Score

Effectiveness of Learning Measurement
14.4%
29.7%
36.9%
18.9%
Formal
50.5%
29.7%
17.1%
1.8%
Informal
43.2%
32.4%
16.2%
7.2%
Experiential
Not at all Somewhat
Moderately Highly

Kirkpatrick Measurements
Percentage of Learning Programs Measured
0% 1%-24% 25%-49% 50%-74% 75%-100%
Level 1
11.9% 6.8% 9.6% 21.5% 50.3%
Level 2
17.0% 23.9% 18.2% 27.3% 13.6%
Level 3
26.9% 31.4% 19.4% 13.7% 8.6%
Level 4
39.8% 31.8% 14.8% 8.5% 5.1%

Measurement Takeaways
• The measurement strategy should be an extension of a learning strategy
focused on performance as an outcome.
• Measuring informal learning is not that different than measuring more
traditional types of learning.
– Basics:
• Participation
• Satisfaction
• Assessments
– Performance:
• Individual performance
• Time to productivity
• Business outcomes
• There are also informal-specific measurements as well.
– High contributors
– Knowledge hubs
– Top content

Summary
• Learning is no longer the outcome
– performance is
• 70/20/10 Framework:
– Should not be applied in the traditional
formulaic approach
– Learning needs to be blended &
personalized

Questions?
David Wentworth
Brandon Hall Group
@DavidMWentworth
Alessio Artuffo
Docebo
@AlessioArtuffo

Thank You
David Wentworth
Brandon Hall Group
@DavidMWentworth
Alessio Artuffo
Docebo
@AlessioArtuffo

Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo

Similar to Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo (20)

More from DoceboElearning

More from DoceboElearning (20)

Recently uploaded

Recently uploaded (20)

Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo