Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Measuring	the	ROI	of	Informal	
Learning
David	Wentworth
Senior	Learning	Analyst
Brandon	Hall	Group
@DavidMWentworth
Alessi...
Learning	&	Development
Talent	Management
Leadership	Development
Talent	Acquisition
Workforce	Management
RESEARCH	PRACTICES...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Copyright © 2015 Docebo - All rights reserved. 3 www.docebo.co...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Join	the	Discussion
• Submit	your	questions
or	commentsabout	t...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com ©	2015	Brandon	Hall	Group 5
TODAY
ORGANIZATION	DRIVEN
Learning...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
The	State	of	70/20/10
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Very	Little	Learning	is	Formal	
• Formal	
Learning/Catalog
• C...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Poll	Question
How	much	of	the	learning	that	occurs	within	
you...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
What	is	the	mix?	
MIX	OF	FORMAL,	INFORMAL	AND	ON	THE	JOB	ACTIV...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
The	Questions	We	Need	to	Ask	Ourselves
©	2015	Brandon	Hall	Gro...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
70/20/10:	A	Fresh	Look
©	2015	Brandon	Hall	Group 11
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Application	to	a	Single	Event
12
Pre	event
Learner	completes	P...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com 13
Flipping	the	Classroom
13©2014	Brandon	Hall	Group,	Inc.
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Application	to	a	Program
14
Discussion	
about	need	
and	
suita...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Implementing	the	Framework
TOP	AREAS	OF	FOCUS	
• Solve	real-wo...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Implementing	the	Framework
SUCCESS	ELEMENTS
• Full	support	of	...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com ©	2015	Brandon	Hall	Group 17
What	makes	the	70	and	the	20	so	
...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Embedded	Learning	is	Essential
How often would you expect your...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
• Contextual
• Role
• Location
• Development	path
• Small
• Sh...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Forgetting	Curve
©	2015	Brandon	Hall	Group 20
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Boost	Learning
©	2015	Brandon	Hall	Group 21
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
The	70:20:10	Effect
4.9%
29.4%
46.1%
19.6%
Not	at	all
Small	ex...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
High	Performance	Learning
• Support	proactive	learners	-- empl...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com ©	2015	Brandon	Hall	Group 24
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Poll	Question
How	likely	is	your	organization	to	adopt	the	
70...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com ©	2015	Brandon	Hall	Group 26
Learning	is	not	the	outcome
PERFO...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Vision
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Adoption	:	Sales	Enablement
• Dreams Goal WhoSectors
Create a ...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
• Adoption :	Health and	Safety
Transform the compliance
traini...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Technologies	in	support	of	the	70:20:10	model
70
20
10
INFORMA...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Peer	Review
Video	Curation
Performance	support
70
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
20
Ask the	Experts,	
assign a	topic
Associate	topics
to	experts
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
20
Ask questions to	experts,	
get answers,		discuss.
Follow qu...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Measuring:
- Coaching activity
- Reviewing activity
- Experts’...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com ©	2015	Brandon	Hall	Group 35
How	do	we	measure	the
INFORMAL?
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
The	Trouble	with	Analytics
15.9%
17.9%
23.2%
25.2%
27.2%
Socia...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
The	Trouble	with	Analytics
20.1%
23.5%
29.5%
20.1%
6.7%
1 2 3 ...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Effectiveness	of	Learning	Measurement
14.4%
29.7%
36.9%
18.9%
...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Kirkpatrick	Measurements
©	2015	Brandon	Hall	Group 39
Percenta...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Measurement	Takeaways
• The	measurement	strategy	should	be	an	...
Follow	us	on	Twitter:	@BrandonHallGrp |	www.brandonhall.com
Summary
• Learning	is	no	longer	the	outcome	
– performance	is	...
Questions?
David	Wentworth
Senior	Learning	Analyst
Brandon	Hall	Group
@DavidMWentworth
Alessio	Artuffo
Chief	Operating	Off...
Thank	You
David	Wentworth
Senior	Learning	Analyst
Brandon	Hall	Group
@DavidMWentworth
Alessio	Artuffo
Chief	Operating	Offi...
Upcoming SlideShare
Loading in …5
×

Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo

4,273 views

Published on

People learn more through informal channels, on-the-job experiences and coaching than they do through more formal modalities. However, most companies still focus their learning efforts on the classroom.

What opportunities are you missing to help employees learn and retain new knowledge?

Published in: Business
  • Be the first to comment

Measuring the ROI of Informal Learning - Brandon Hall Group & Docebo

  1. 1. Measuring the ROI of Informal Learning David Wentworth Senior Learning Analyst Brandon Hall Group @DavidMWentworth Alessio Artuffo Chief Operating Officer Docebo @AlessioArtuffo
  2. 2. Learning & Development Talent Management Leadership Development Talent Acquisition Workforce Management RESEARCH PRACTICES © 2015 Brandon Hall Group2
  3. 3. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Copyright © 2015 Docebo - All rights reserved. 3 www.docebo.com Docebo, at a glance 70About Innovation-driven, cloud- based learning Ecosystem for the Extended Enterprise. 50% Product and Innovation 30% Sales and Marketing 20% Operations 100 Talents NORTH AMERICA Athens, GA (USA) EUROPE Milan (Italy) MIDDLE EAST Dubai (UAE) Where Ideal addressable market: Organizations with 50 MIL+ in revenues 850 2 850 Partners (clients) “We are obsessed by : Innovation and Customer Success”.
  4. 4. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Join the Discussion • Submit your questions or commentsabout the discussion to our presenters using the Questions tab on your control panel. • Presentation slides and giveaways can be found in the Handouts tab on your control panel. © 2015 Brandon Hall Group 4 Type in your question here HOW TO ASK QUESTIONS AND PRESENTATION RECORDING & SLIDES Recording & Slides will also be sent out after the webinar.
  5. 5. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 5 TODAY ORGANIZATION DRIVEN Learning Compliance The Evolution of Learning USER DRIVEN Learning Experience
  6. 6. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com The State of 70/20/10
  7. 7. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Very Little Learning is Formal • Formal Learning/Catalog • Certification & Compliance • Courses • Groups Associated with Classes • Impressions • Expertise Location • Idea Engine • Real-Time Collaboration • Groups & Workspaces © 2015 Brandon Hall Group 7
  8. 8. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Poll Question How much of the learning that occurs within your organization would you consider informal? • 0% • 1%-5% • 5%-10% • 15%-25% • More than 25% © 2015 Brandon Hall Group 8
  9. 9. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com What is the mix? MIX OF FORMAL, INFORMAL AND ON THE JOB ACTIVITIES 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Formal Informal On the Job Activities 43% 14% 43% © 2015 Brandon Hall Group 9
  10. 10. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com The Questions We Need to Ask Ourselves © 2015 Brandon Hall Group 10 The “20” - How do we create meaningful and impactful informal learning? The “70” - How do we optimize on the job training? The “10” - How do we rethink our classroom environment?
  11. 11. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com 70/20/10: A Fresh Look © 2015 Brandon Hall Group 11
  12. 12. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Application to a Single Event 12 Pre event Learner completes PDP, identifies a stretch opportunity, identifies a learning need Learner & Manager discussion about development need, current skill level or capability Course pre-work requires learner to hold a formal discussion with their manager about the course and current skill level/capability During the event Learner learns in a formal setting Learner encouraged to reflect on application in the workplace and think about future development Immediately post event Learner returns to role, applies learning and evaluates learning event in relation to role Learner discusses event with manager, or others and reflects on event and application in the workplace 45-90 days post event Learner 2nd evaluation, application of learning in the workplace Makes changes to working practices, tries new things, challenges self to make change Manager evaluation of impact on role and learner post discussion with learner about application in the workplace and any further development needs Learning through experience Social learning Formal learning © 2015 Brandon Hall Group
  13. 13. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com 13 Flipping the Classroom 13©2014 Brandon Hall Group, Inc.
  14. 14. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Application to a Program 14 Discussion about need and suitability of programme Workshop Apply learning in the workplace Formation of project groups Project work Coaching/ mentoring for project group Application of learning so far in the workplace Workshop Project work Coaching Application of learning so far in the workplace Workshop Learning through experience Social learning Formal learning © 2015 Brandon Hall Group
  15. 15. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Implementing the Framework TOP AREAS OF FOCUS • Solve real-world problems • Optimize teams • Increase application of training on the job • Introduce new approaches to work • Access to coaches and mentors • Leverage e-learning © 2015 Brandon Hall Group 15
  16. 16. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Implementing the Framework SUCCESS ELEMENTS • Full support of executives • Development of a clear vision, mission, strategy and governance model • Adequate resourcing • Solid technological foundation © 2015 Brandon Hall Group 16
  17. 17. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 17 What makes the 70 and the 20 so CRITICAL?
  18. 18. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Embedded Learning is Essential How often would you expect your individual contributor population, on average, to need to connect with learning resources to effectively perform their job? 0.8% 9.4% 29.0% 36.7% 24.1% Less than annually Annually Monthly Weekly Daily or more often 61% Brandon Hall Group, Social & Collaborative Learning, 2014 18© 2015 Brandon Hall Group
  19. 19. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com • Contextual • Role • Location • Development path • Small • Short videos • Quick updates • Informal • Everyday • Ad hoc • Mobile • Down time • Moment of need • Social • Discussions • Recommendations Embedded Learning is Essential © 2015 Brandon Hall Group
  20. 20. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Forgetting Curve © 2015 Brandon Hall Group 20
  21. 21. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Boost Learning © 2015 Brandon Hall Group 21
  22. 22. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com The 70:20:10 Effect 4.9% 29.4% 46.1% 19.6% Not at all Small extent Moderate extent Great extent Improved Link to Individual Performance 9.1% 37.4% 37.4% 16.2% Not at all Small extent Moderate extent Great extent Improved Link to Organizational Performance © 2015 Brandon Hall Group 22 95% 91%
  23. 23. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com High Performance Learning • Support proactive learners -- employees who take responsibility for their own development and seek out learning • Provide easy-to-use tools that present employees with opportunities to connect to formal and informal knowledge sources throughout the organization. • Present each and every employee with opportunities for development based on personal strengths, weaknesses, job roles, or interest from sources that go beyond their immediate leaders. 80% of High Performing Organizations have these characteristics
  24. 24. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 24
  25. 25. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Poll Question How likely is your organization to adopt the 70:20:10 model? • We already have • No plans at all • Would like to but, need to understand it more • There are clear use cases in my organization for adoption © 2015 Brandon Hall Group 25
  26. 26. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 26 Learning is not the outcome PERFORMANCE IS
  27. 27. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Vision
  28. 28. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Adoption : Sales Enablement • Dreams Goal WhoSectors Create a culture of learning and staying updated by sharing, coaching, and people enablement. Transform the formal content in supplementary content, in support of informal and social learning frameworks. Sales professionals. They are the nightmare of any elearning training team. They live at fast-speed and make limited time for formal knowledge. They are the most natural social learners in the corporate emisphere. Retail, Software, Consulting, High Tech, Manufacturing Achieve consistent and high- quality product and services knowledge. Achieve high information retention. Decrease the investments in one-time use learning material.
  29. 29. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com • Adoption : Health and Safety Transform the compliance training programs from highly regulated and formal only, to engagement-heavy and collaborative learning opportunities. Compliance and quality professionals. Energy, Manufacturing, Biotech, Oil and Gas, Healthcare Increase the knowledge effectiveness, quality and retention in highly critical H&S matters. Allow for SOPs to be curated by knowledge gurus as they become obsolete and prevent hazards via guided social improvement. • Dreams Goal WhoSectors
  30. 30. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Technologies in support of the 70:20:10 model 70 20 10 INFORMAL TRAINING SOCIAL TRAINING FORMAL TRAINING User-generation content features, Peer-validation and curation Engaging the knowledge gurus, Enable learning by collaboratuve sharing, Integrate knowledge-rewardinggamification Organize, distribute track online courses and retention.
  31. 31. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Peer Review Video Curation Performance support 70
  32. 32. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com 20 Ask the Experts, assign a topic Associate topics to experts
  33. 33. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com 20 Ask questions to experts, get answers, discuss. Follow questions Find “Best Answers”
  34. 34. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Measuring: - Coaching activity - Reviewing activity - Experts’ engagement - Contributions - Effectiveness 20 70
  35. 35. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com © 2015 Brandon Hall Group 35 How do we measure the INFORMAL?
  36. 36. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com The Trouble with Analytics 15.9% 17.9% 23.2% 25.2% 27.2% Social learning features Analytics features for ROI Reporting features Adaptability Ease of use Least Liked Aspects of Current LMSs 36
  37. 37. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com The Trouble with Analytics 20.1% 23.5% 29.5% 20.1% 6.7% 1 2 3 4 5 Satisfaction with LMS Analytics (1-5) 37 2.7 Average Score
  38. 38. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Effectiveness of Learning Measurement 14.4% 29.7% 36.9% 18.9% Formal © 2015 Brandon Hall Group 38 50.5% 29.7% 17.1% 1.8% Informal 43.2% 32.4% 16.2% 7.2% Experiential Not at all Somewhat Moderately Highly
  39. 39. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Kirkpatrick Measurements © 2015 Brandon Hall Group 39 Percentage of Learning Programs Measured 0% 1%-24% 25%-49% 50%-74% 75%-100% Level 1 11.9% 6.8% 9.6% 21.5% 50.3% Level 2 17.0% 23.9% 18.2% 27.3% 13.6% Level 3 26.9% 31.4% 19.4% 13.7% 8.6% Level 4 39.8% 31.8% 14.8% 8.5% 5.1%
  40. 40. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Measurement Takeaways • The measurement strategy should be an extension of a learning strategy focused on performance as an outcome. • Measuring informal learning is not that different than measuring more traditional types of learning. – Basics: • Participation • Satisfaction • Assessments – Performance: • Individual performance • Time to productivity • Business outcomes • There are also informal-specific measurements as well. – High contributors – Knowledge hubs – Top content © 2015 Brandon Hall Group 40
  41. 41. Follow us on Twitter: @BrandonHallGrp | www.brandonhall.com Summary • Learning is no longer the outcome – performance is • 70/20/10 Framework: – Should not be applied in the traditional formulaic approach – Learning needs to be blended & personalized © 2015 Brandon Hall Group 41
  42. 42. Questions? David Wentworth Senior Learning Analyst Brandon Hall Group @DavidMWentworth Alessio Artuffo Chief Operating Officer Docebo @AlessioArtuffo
  43. 43. Thank You David Wentworth Senior Learning Analyst Brandon Hall Group @DavidMWentworth Alessio Artuffo Chief Operating Officer Docebo @AlessioArtuffo

×