More Related Content More from Human Capital Media (20) Reporting Talent Development Metrics to Executives1. Reporting Talent Development
Metrics to Executives
You can listen to today’s webinar using your computer’s speakers
or you may dial into the teleconference.
If you would like to join the teleconference,
please dial 1.408.600.3600 and enter access code 927 649 233 #.
You will be on hold until the seminar begins.
#CLOwebinar
2. Reporting Talent Development
Metrics to Executives
Speaker: Jeffrey Berk
Chief Operating Officer
KnowledgeAdvisors Inc.
Moderator Daniel Margolis
Managing Editor
Chief Learning Officer magazine
#CLOwebinar
3. Tools You Can Use
• Q&A
– Click on the Q&A panel (?)
in the bottom right corner
– Type in your question in the
space provided
– Click on “Send.”
#CLOwebinar
4. Tools You Can Use
• Polling
– The poll will appear on the
right side of your screen
– Select the best option for
each question
– Click on “Submit”
#CLOwebinar
5. Frequently Asked Questions
Will I receive a copy of the webinar recording?
YES
Will I receive a copy of the slides?
YES
Please allow up to 2 business days to receive these materials
#CLOwebinar
9. Agenda
Review elements that comprise balanced executive
reporting metrics
Discuss core attributes on a learning leader report
Present key components of a business manager report for
learning
Offer practical yet credible examples to link learning to
business impact
February 7, 2012 © 2011, KnowledgeAdvisors 8
10. About KnowledgeAdvisors
KnowledgeAdvisors will improve learning effectiveness within your
organization.
For organizations that utilize learning
and development to drive business
outcomes
KnowledgeAdvisors offers a learning
measurement solution …
That improves the effectiveness and
business impact of learning.
February 7, 2012 © 2011, KnowledgeAdvisors 9
14. Reporting What’s Important
Outcome Metrics - Business Outcomes
Employee Contribution Margin
Productivity
Employee Retention
Employee Engagement
Profitability
Effectiveness Metrics – Program Performance
Delivery Quality
Knowledge Gain
Application to Job
Business Alignment
Value
Efficiency Metrics – Cost and Activity
# of Participants
% of Employees Trained
Total L&D Investment
L&D Investment per Participant
Courses Utilized
Classes Conducted
Instructors Utilized
Locations Used
February 7, 2012 © 2011, KnowledgeAdvisors 13
15. Strategic, Visible, Costly Programs
Leadership Q1 2010 Q2 2010 Q3 2010 Q4 2010 Q1 2011
Leadership Development
Training`
•Leader Retention Effectiveness 76% 80% 77% 75% 76%
•Employee Engagement Rating
Efficiency Rating 79% 78% 76% 78% 79%
•Employee Retention
Business 74% 74% 70% 70% 71%
•Leader Performance Ratings Outcome Rating
•Leader Fiscal Management
Onboarding Program
•Time to Performance
•Performance Rating
•Employee Engagement
•Participant Confidence
•Competency Level
Sales Training
•Growth Rate
•Profit Margin
•Win Rate
•Time to Close
•Deal Size
February 7, 2012 © 2009 KnowledgeAdvisors
14
16. Talent Development Reporting Principles
Executive initiative
championed by
KnowledgeAdvisors and
supported by leading
organizations
Standard framework to
measure learning: efficiency,
effectiveness and outcome
measures
Common terminology and
metrics
Inspired by GAAP
http://www.knowledgeadvisors.com/tdrp
© 2011 KnowledgeAdvisors Page: 15
17. Talent Development Reporting Process
Business Outcomes
– Organizational goals
– L&D’s impact on those goals
Effectiveness
– Levels 1-5, timeliness
Efficiency
– Costs
– Volume (participants, programs, classes, hours)
– Ratios
– Utilization rates
– Program and vendor management
February 7, 2012 © 2011, KnowledgeAdvisors 16
18. Talent Development Reporting Principles
Business Executive Learning Executive
Summary Report Reports
Executive
Reports L&D Summary L&D Program L&D Operational
Report Report Report
Business Learning Learning
Statements Outcomes Effectiveness Efficiency
Guiding Principles
Statement Statement Statement
Data Sets Business
Effectiveness Efficiency
Outcomes
Extract, convert and calculate
Standard Measures
Data Learning Other Sources
Sources Evaluation
Financial Data Management (e.g HRIS, ERP,
System
System CRM)
7-Feb-12 © 2011 KnowledgeAdvisors Page: 17
21. L&D Executive Report
Narrative
Text and Charts
Multiple Views
Audience
L&D Executives
L&D Management
Discuss it
90 minutes
Quarterly meeting
February 7, 2012 © 2011, KnowledgeAdvisors 20
22. Business Executive Report
Narrative
Text and Charts
Multiple Views
Audience
Business Executives
Line Management
Discuss it
90 minutes
Quarterly meeting
February 7, 2012 © 2011, KnowledgeAdvisors 21
23. Dashboard Case Study
Background: Commercial
insurance company
Business Issue: Visual, concise
place for learning metrics
Analytics Solution: Leveraged
dashboards to build custom, robust
solution
Analytics Results: Real-time
access to impact, cost, cycle-time,
satisfaction, quality and volume
dashboards.
February 7, 2012 © 2011, KnowledgeAdvisors 22
25. Quality Tab
What we learn
Ensure performance is consistent over time and compare most recent
quarter’s performance to the norm
Overall survey scores, quarterly fluctuation, scores by curriculum area
Instructor ratings, identify areas to improve
Overall instructor & course scores against benchmarks
Top/bottom rated courses based on courseware scores and associated
comments
February 7, 2012 © 2011, KnowledgeAdvisors 24
26. Volume Tab
What we learn
Total hours delivered,
number of completions
Breakout of volume by
business / function and
learning method
(eLearning, Instructor Led)
Trends in volume over time
Compare trends in volume
by usage rates / trends of
licensed course libraries,
curriculum areas (are we
buying more than we need?
Are we maximizing our
investment?)
February 7, 2012 © 2011, KnowledgeAdvisors 25
27. Six Approaches to Align Learning to Impact
High Once you determine Requires actual results
Investment Business Impact what data and advanced
analysis is appropriate, analysis techniques
select a method:
Actual
Results Causal
Typically Correlations Modeling
employs Business
surveys Impact
Advanced Templates
Business
Results
Human Evaluation
Capital
Approach
Smart
Sheets
Low Complex
Investment
February 7, 2012 © 2011, KnowledgeAdvisors 26
Simple
28. Smile Sheet to Smart Sheet
February 7, 2012 © 2011, KnowledgeAdvisors 27
29. Smart Sheet in Practice
“We need our measurement tools
Review programs and process now more than ever; if
that aren’t providing we don’t have access to this data,
how will we be able to make
impact to the intelligent decisions on what to cut?”
business - KnowledgeAdvisors Client
February 7, 2012 © 2011, KnowledgeAdvisors 28
31. Business Impact Template in Practice
Background: Large CPG based in
Canada, sales training
Business Issue: Visible sales
program, requested to link between
it and sales results
Analytics Solution: Used process
to evaluate sales change and
impact training had on sales
Analytics Results: 7% increase in
sales, 20% of the increase was due
to training, this was a $74,000
increase per sales person when the
training was $5000 per person
February 7, 2012 © 2011, KnowledgeAdvisors 30
32. Actual Results Correlations
This is a method where actual results can be input or imported at an
aggregate, periodic frequency. It is then correlated in time periods
against training evaluation data for impact indicators.
The example below is from sales data and sale training.
February 7, 2012 © 2011, KnowledgeAdvisors 31
33. Measurement Success Stories
Executive Program for Government Entity:
An executive leadership program generated more
than double the impact of typical leadership programs
equating to nearly 70% on-the-job application and a 2
to 1 benefit to cost ratio.
Strategic Sales Training
A strategic sales training program generated $74,000
in increased sales per sales person for a $5,000 per
person investment. This data was the catalyst that
convinced management to expand the program to
other sales territories.
Front-Line Staff New Hire Program
An on-boarding program for front-line staff brought
an additional 15,000 hours of increased productivity to
the business as a result of the increased speed to
competency. This analysis helped save a new hire
program that was questioned by management.
February 7, 2012 © 2011, KnowledgeAdvisors 32
34. Business Relevant Additional Examples
Simple Survey
Asked high performers to rate their high
impact factors
92% effective supply chain
89% informal learning tools
73% formal learning programs
32% technology
14% compensation
Clearly learning contributes to high
performance Source: Dr. John Sullivan, Presentation at Analytics Symposium
February 7, 2012 © 2011, KnowledgeAdvisors 33
35. Business Relevant Additional Examples
Basic Correlation
Correlate customer service scores with use of
learning tools and programs
Identified high training use CSR (customer
service reps) based on tools used, programs
completed and grouped into high and low use
Compared CSR customer satisfaction score
with learning use
Results: Top 10% customer satisfaction were
also in the high learning group, bottom 10%
were in the low learning group. Source: Dr. John Sullivan, Presentation at Analytics Symposium
February 7, 2012 © 2011, KnowledgeAdvisors 34
36. Business Relevant Additional Examples
Before vs. After
Hypothesis was that a learning support
tool could improve innovation
Before the learning tool, 15% of
employees contributed innovation ideas
After the learning tool, 24% of employees
contributed innovation ideas
Its easy to see the ‘After’ contribution
Source: Dr. John Sullivan, Presentation at Analytics Symposium
February 7, 2012 © 2011, KnowledgeAdvisors 35
37. Business Relevant Additional Examples
Control Group (side by side)
Determine if learning projects influenced
employee retention
Employees with no learning projects
averaged a turnover rate of 21%
Employees with 2 or more learning
projects averaged a turnover rate of 2%
Clearly the control group vs. learning
group shows a difference
Source: Dr. John Sullivan, Presentation at Analytics Symposium
February 7, 2012 © 2011, KnowledgeAdvisors 36
38. Business Relevant Additional Examples
Take it in, Take it out
Sales team use of social media learning
tool to see impact on sales results
Tool put In, sales went up 34%
Tool taken Out, sales went down 29%
Essentially, without the tool sales returned
to where they were originally. Clearly the
tool made a difference
Source: Dr. John Sullivan, Presentation at Analytics Symposium
February 7, 2012 © 2011, KnowledgeAdvisors 37
39. Measurement Sustainability
• Strong leadership for driving measurement
1 with governance to support it. • Roles and accountabilities for
metrics are defined with
• Culture supports and drives
2 consistent expectations on
what should be measured.
need for standards and
8 common talent development 2
processes and measurement Roles
• Strong desire to use the data for
7 3 3 • Broad base of skills in
Data
decision making Data
Utilization
Skills / analytics and
Behaviors
• Maturity in how data is reported Utilization 3 measurement and strong
1
and used to improve end to end capabilities in end-to-end
7 talent processes
Leadership
measurement methods
6 4
• Standardization for a consistent Technology Processes
and single point of analytics with 5
6 integration to and from feeder Technolog Standards,
• Established and consistent
Processes processes, technologies and
systems y Tools
4 communications for
measurement
Standards,
• Standards have been created for data collection
Tools
• Data integrity exists
5
February 7, 2012
2/7/2012 © 2010 KnowledgeAdvisors
© 2011, KnowledgeAdvisors 38
40. Events:
Analytics Symposium March 5-7 New Orleans (Executive Reporting)
http://www.knowledgeadvisors.com/events/symposium/
Learning Analytics 2 Day Workshop April 25-26 Chicago
http://www.knowledgeadvisors.com/events/workshops/learning-analytics-2-day-
workshop/
Contact Information
For further information on the information contact:
Jeffrey Berk
Chief Operating Officer
+1 312 676-4411
jberk@knowledgeadvisors.com
www.knowledgeadvisors.com
41. Join Our Next CLO Webinar
Driving Your Organization’s Culture by Building
Learning Relationships for Mutual Benefit
Tuesday, March 22, 2012
CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register at www.clomedia.com/events
Join the CLO Network: http://network.clomedia.com/
#CLOwebinar