This document discusses how to convey learning impact to executives through metrics reporting. It outlines elements of balanced executive reporting, including outcome, effectiveness, and efficiency metrics. Sample metrics include employee contribution margin, knowledge gain, and costs. The document presents examples of linking learning programs to business outcomes through methods like surveys, correlations between training and performance, and comparing trained and untrained groups. It emphasizes using data to improve processes and decision making.

1. Business Relevant Ways to Convey Learning Impact to
Executives
Jeffrey Berk
Chief Operating Officer

2. Agenda
 Review elements that comprise balanced executive
reporting metrics
Discuss core attributes on a learning leader report
Present key components of a business manager report for
learning
Offer practical yet credible examples to link learning to
business impact
May 11, 2012 © 2011, KnowledgeAdvisors 2

3. About KnowledgeAdvisors
KnowledgeAdvisors will improve learning effectiveness within your organization.
For organizations that utilize learning
and development to drive business
outcomes
KnowledgeAdvisors offers a learning
measurement solution …
That improves the effectiveness and
business impact of learning.
May 11, 2012 © 2011, KnowledgeAdvisors 3

4. About KnowledgeAdvisors
May 11, 2012 © 2011, KnowledgeAdvisors 4

5. Sample Customers
May 11, 2012 © 2011, KnowledgeAdvisors 5

6. Key Metrics
May 11, 2012 © 2011, KnowledgeAdvisors 6

7. Reporting What’s Important
Outcome Metrics - Business Outcomes
 Employee Contribution Margin
 Productivity
 Employee Retention
 Employee Engagement
 Profitability
Effectiveness Metrics – Program Performance
 Delivery Quality
 Knowledge Gain
 Application to Job
 Business Alignment
 Value
Efficiency Metrics – Cost and Activity
 # of Participants
 % of Employees Trained
 Total L&D Investment
 L&D Investment per Participant
 Courses Utilized
 Classes Conducted
 Instructors Utilized
 Locations Used
May 11, 2012 © 2011, KnowledgeAdvisors 7

8. Strategic, Visible, Costly Programs
Leadership Q1 2010 Q2 2010 Q3 2010 Q4 2010 Q1 2011
Leadership Development Training`
•Leader Retention Effectiveness 76% 80% 77% 75% 76%
•Employee Engagement Rating
Efficiency Rating 79% 78% 76% 78% 79%
•Employee Retention Business 74% 74% 70% 70% 71%
•Leader Performance Ratings Outcome Rating
•Leader Fiscal Management
Onboarding Program
•Time to Performance
•Performance Rating
•Employee Engagement
•Participant Confidence
•Competency Level
Sales Training
•Growth Rate
•Profit Margin
•Win Rate
•Time to Close
•Deal Size
May 11, 2012 © 2009 KnowledgeAdvisors
8

9. Talent Development Reporting Principles
 Executive initiative
championed by
KnowledgeAdvisors and
supported by leading
organizations
 Standard framework to
measure learning: efficiency,
effectiveness and outcome
measures
 Common terminology and
metrics
 Inspired by GAAP
http://www.knowledgeadvisors.com/tdrp
© 2011 KnowledgeAdvisors Page: 9

10. Talent Development Reporting Process
 Business Outcomes
– Organizational goals
– L&D’s impact on those goals
 Effectiveness
– Levels 1-5, timeliness
 Efficiency
– Costs
– Volume (participants, programs, classes, hours)
– Ratios
– Utilization rates
– Program and vendor management
May 11, 2012 © 2011, KnowledgeAdvisors 10

11. Talent Development Reporting Principles
Business Executive Learning Executive
Summary Report Reports
Executive
Reports L&D Summary L&D Program L&D Operational
Report Report Report
Business Learning Learning
Statements Outcomes Effectiveness Efficiency
Guiding Principles
Statement Statement Statement
Data Sets Business
Effectiveness Efficiency
Outcomes
Extract, convert and calculate
Standard Measures
Data Learning Other Sources
Sources Evaluation
Financial Data Management (e.g HRIS, ERP,
System
System CRM)
11-May-12 © 2011 KnowledgeAdvisors Page: 11

12. Dashboards, Scorecards, Statements
May 11, 2012 © 2011, KnowledgeAdvisors 12

13. L&D Summary Report
May 11, 2012 © 2011, KnowledgeAdvisors 13

14. L&D Executive Report
 Narrative
 Text and Charts
 Multiple Views
 Audience
 L&D Executives
 L&D Management
 Discuss it
 90 minutes
 Quarterly meeting
May 11, 2012 © 2011, KnowledgeAdvisors 14

15. Business Executive Report
 Narrative
 Text and Charts
 Multiple Views
 Audience
 Business Executives
 Line Management
 Discuss it
 90 minutes
 Quarterly meeting
May 11, 2012 © 2011, KnowledgeAdvisors 15

16. Dashboard Case Study
Background: Commercial
insurance company
Business Issue: Visual, concise
place for learning metrics
Analytics Solution: Leveraged
dashboards to build custom, robust
solution
Analytics Results: Real-time
access to impact, cost, cycle-time,
satisfaction, quality and volume
dashboards.
May 11, 2012 © 2011, KnowledgeAdvisors 16

17. Executive Tab
May 11, 2012 © 2011, KnowledgeAdvisors 17

18. Quality Tab
What we learn
 Ensure performance is consistent over time and compare most recent
quarter’s performance to the norm
 Overall survey scores, quarterly fluctuation, scores by curriculum area
 Instructor ratings, identify areas to improve
 Overall instructor & course scores against benchmarks
 Top/bottom rated courses based on courseware scores and associated
comments
May 11, 2012 © 2011, KnowledgeAdvisors 18

19. Six Approaches to Align Learning to Impact
High Once you determine Requires actual results
Investment Business Impact what data and advanced
analysis is appropriate, analysis techniques
select a method:
Actual
Typically
Results Causal
Correlations Modeling
employs Business
surveys Impact
Advanced Templates
Business
Results
Human Evaluation
Capital
Approach
Smart
Sheets
Low Complex
Investment
May 11, 2012 © 2011, KnowledgeAdvisors 19
Simple

20. Smile Sheet to Smart Sheet
May 11, 2012 © 2011, KnowledgeAdvisors 20

21. Smart Sheet in Practice
“We need our measurement tools and
 Review programs process now more than ever; if we
that aren’t providing don’t have access to this data, how
will we be able to make intelligent
impact to the decisions on what to cut?”
business - KnowledgeAdvisors Client
May 11, 2012 © 2011, KnowledgeAdvisors 21

22. Business Results Evaluation Approach
May 11, 2012 © 2011, KnowledgeAdvisors 22

23. Business Impact Template in Practice
Background: Large CPG based in
Canada, sales training
Business Issue: Visible sales
program, requested to link between
it and sales results
Analytics Solution: Used process
to evaluate sales change and
impact training had on sales
Analytics Results: 7% increase in
sales, 20% of the increase was due
to training, this was a $74,000
increase per sales person when the
training was $5000 per person
May 11, 2012 © 2011, KnowledgeAdvisors 23

24. Actual Results Correlations
This is a method where actual results can be input or imported at an
aggregate, periodic frequency. It is then correlated in time periods
against training evaluation data for impact indicators.
The example below is from sales data and sale training.
May 11, 2012 © 2011, KnowledgeAdvisors 24

25. Business Relevant Additional Examples
 Simple Survey
 Asked high performers to rate their high
impact factors
 92% effective supply chain
 89% informal learning tools
 73% formal learning programs
 32% technology
 14% compensation
 Clearly learning contributes to high
performance Source: Dr. John Sullivan, Presentation at Analytics Symposium
May 11, 2012 © 2011, KnowledgeAdvisors 25

26. Business Relevant Additional Examples
 Basic Correlation
 Correlate customer service scores with use of
learning tools and programs
 Identified high training use CSR (customer
service reps) based on tools used, programs
completed and grouped into high and low use
 Compared CSR customer satisfaction score with
learning use
 Results: Top 10% customer satisfaction were
also in the high learning group, bottom 10% were
in the low learning group.
Source: Dr. John Sullivan, Presentation at Analytics Symposium
May 11, 2012 © 2011, KnowledgeAdvisors 26

27. Business Relevant Additional Examples
 Control Group (side by side)
 Determine if learning projects influenced
employee retention
 Employees with no learning projects
averaged a turnover rate of 21%
 Employees with 2 or more learning
projects averaged a turnover rate of 2%
 Clearly the control group vs. learning group
shows a difference
Source: Dr. John Sullivan, Presentation at Analytics Symposium
May 11, 2012 © 2011, KnowledgeAdvisors 27

28. Measurement Sustainability
• Strong leadership for driving measurement
1 with governance to support it. • Roles and accountabilities for
metrics are defined with
• Culture supports and drives
2 consistent expectations on
what should be measured.
need for standards and common
8 talent development processes
2
and measurement Roles
• Strong desire to use the data for
7 3 3 • Broad base of skills in
Data
decision making Data
Utilization
Skills / analytics and
Behaviors
• Maturity in how data is reported and Utilization 3 measurement and strong
1
used to improve end to end talent capabilities in end-to-end
7 processes Leadership
measurement methods
6 4
• Standardization for a consistent Technology Processes
and single point of analytics with 5
6 integration to and from feeder
Technology Standards,
• Established and consistent
Processes processes, technologies and
systems Tools
4 communications for
measurement
Standards,
• Standards have been created forTools
data collection
• Data integrity exists
5
May 11, 2012
5/11/2012 © 2010 KnowledgeAdvisors
© 2011, KnowledgeAdvisors 28

29. Contact Information
For further information on the information contact:
Jeffrey Berk
Chief Operating Officer
+1 312 676-4411
jberk@knowledgeadvisors.com
www.knowledgeadvisors.com