Business Impact                  The Long-Term                  Value of Using                   Metrics That             ...
Objectives• Discuss the value proposition for  measuring learning.• Describe the long-term value of using  Metrics That Ma...
Audience Poll What is your role within your organization?           A) L&D professional           B) L&D Leader       ...
Audience Poll Are you currently a user of Metrics That  Matter®      A) Yes      B) No14-Mar-12        © 2009, Knowledg...
Agenda Archetypal L&D Organization Why measure learning effectiveness? How do KnowledgeAdvisors’ clients  measure learn...
Archetypal L&D Situation ―I can’t improve    ―Don’t tell me it will     it if I can’t    be a cold day in                 ...
We Know What Leaders Want…                                        N = 10114-Mar-12   © 2010, KnowledgeAdvisors      7
Why Measure Learning?  L&D is an integral part of the business   designed to close gaps and help the    workforce achieve ...
How Do Clients Measure Learning? Learning should be measured in a way  that aligns with business objectives. Usually thi...
Most Common Methodologies   Kirkpatrick’s 4 Levels of Evaluation   Phillip’s ROI Methodology   Bersin’s Learning Impact...
The Goal of Training Evaluation  Demonstrate cause and effect                                    Improved         Perform...
KnowledgeAdvisors: What We Do          Experts in         metrics that           matter       KnowledgeAdvisors is a human...
Integrate Measurement with ManagementMarch 14, 2012   © 2009, KnowledgeAdvisors   13
Showing Business Impact   High                                            Requires actual resultsInvestment               ...
Business Case for L&D Measurement: A Financial Link                                               Shareholder             ...
Long-term Use of Metrics That Matter®  Investigated long-term clients that have used MTM   consistently for 5+ years  Ga...
Initial Sample: Telecom CompanyWe examined the metrics for one company first to see if our  hypothesis seemed tenable. Her...
Initial Sample: Telecom CompanyEach year large numbers of evaluations were collected.
Initial Sample: Telecom CompanyWe computed the performance increase as a percentage:  (2010 value – 2006 value) / (2006 va...
Initial Sample: Telecom CompanySome data were not collected every year, but the trend upward  was evident even with only t...
Initial Sample: Telecom CompanyThe combination of all metrics provides a good summary measure.  Despite the variability ac...
Initial Sample: SummaryWhen all 5 metrics are graphed, it is clear that performance rises over time.The number of evaluati...
Case Study vs. Trend The Telecom company is only one example. Does the hypothesis hold true for other  companies or is t...
Companies with 5 Years of DataFour organizations contributed 5 years of data:  Two insurance companies, a hospital and a t...
Companies with 6 Years of DataSix organizations contributed 6 years of data:   An office solutions company, an Army Base, ...
Other companies? The analysis will continue soon with other  companies:      Small and medium businesses      Learning ...
Conclusion Measurement leads to performance improvement  for L&D departments The size of the improvement ranges from 3 –...
Upcoming SlideShare
Loading in …5
×

Business Impact: The Long-Term Value of Metrics That Matter

978 views
900 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
978
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • A CLO comes to a learning director and quotes Lord Kelvin, “I can’t improve it, if I can’t measure it.” We need a measurement strategy. (BTW, Lord Kelvin developed the principle of absolute zero.) CLO continues, “Don’t tell me it will be a cold day in Hell before we get a measurement strategy.”The director goes to his top learning manager. We need a measurement strategy. I don’t want to leave you out in the cold on this one alone, so use all the resources on your team.Learning Manager goes to her team and says, “It’s a very cold winter day outside. Unless we develop a measurement strategy, we going to be out in it looking for a new job.”
  • Discuss Pros/Cons of each
  • Business Impact: The Long-Term Value of Metrics That Matter

    1. 1. Business Impact The Long-Term Value of Using Metrics That Matter® KnowledgeAdvisors 17th March 2011
    2. 2. Objectives• Discuss the value proposition for measuring learning.• Describe the long-term value of using Metrics That Matter®14-Mar-12 © 2009, KnowledgeAdvisors 2
    3. 3. Audience Poll What is your role within your organization?  A) L&D professional  B) L&D Leader  C) HR professional  D) Business leader  E) Other14-Mar-12 © 2009, KnowledgeAdvisors 3
    4. 4. Audience Poll Are you currently a user of Metrics That Matter®  A) Yes  B) No14-Mar-12 © 2009, KnowledgeAdvisors 4
    5. 5. Agenda Archetypal L&D Organization Why measure learning effectiveness? How do KnowledgeAdvisors’ clients measure learning impact? What are the long-term benefits?14-Mar-12 © 2009, KnowledgeAdvisors 5
    6. 6. Archetypal L&D Situation ―I can’t improve ―Don’t tell me it will it if I can’t be a cold day in ―I don’t want to measure it.‖ Hell before we get leave you out in a measurement the cold on this strategy.‖ one, so leverage your whole team.‖ CLO L&D Director ―It’s a very cold winter day outside. Unless we develop a measurement strategy, we’re going to be out in it looking for new jobs.‖ L&D Manager14-Mar-12 © 2009, KnowledgeAdvisors 6
    7. 7. We Know What Leaders Want… N = 10114-Mar-12 © 2010, KnowledgeAdvisors 7
    8. 8. Why Measure Learning? L&D is an integral part of the business designed to close gaps and help the workforce achieve business goals.14-Mar-12 © 2009, KnowledgeAdvisors 8
    9. 9. How Do Clients Measure Learning? Learning should be measured in a way that aligns with business objectives. Usually this means demonstrating:  Satisfaction with learning  Knowledge and skill acquisition  Application of learning on the job  Demonstration of learning’s impact on the business (e.g., sales, quality, customer sat.)14-Mar-12 © 2009, KnowledgeAdvisors 9
    10. 10. Most Common Methodologies Kirkpatrick’s 4 Levels of Evaluation Phillip’s ROI Methodology Bersin’s Learning Impact Analysis Brinkerhoff’s Success Case Method All of these individuals are or have been on our Board of Directors and have influenced the development of Metrics That Matter®14-Mar-12 © 2009, KnowledgeAdvisors 10
    11. 11. The Goal of Training Evaluation  Demonstrate cause and effect Improved Performance Training Performance  Determine gaps or deficiencies in training that require continuous improvement March 14, 2012 11 © 2009, KnowledgeAdvisors
    12. 12. KnowledgeAdvisors: What We Do Experts in metrics that matter KnowledgeAdvisors is a human capital analytics firm. We are the world’s largest provider of learning and talent measurement solutions.March 14, 2012 © 2009, KnowledgeAdvisors 12
    13. 13. Integrate Measurement with ManagementMarch 14, 2012 © 2009, KnowledgeAdvisors 13
    14. 14. Showing Business Impact High Requires actual resultsInvestment data and advanced analysis techniques Typically Actual Causal Results Modeling employs Correlations surveys Business Impact Templates Advanced Business Human Results Capital Evaluation Approach Determine the Smart appropriate analysis, Sheets then select method LowInvestment Simple Complex
    15. 15. Business Case for L&D Measurement: A Financial Link Shareholder Returns of companies investing in L&D measurement have 17% higher returns than the S&P 500 Source: McBassi Investments, 2009 The is the cause and effect relationship we hope to demonstrate! March 14, 2012 15 © 2009, KnowledgeAdvisors
    16. 16. Long-term Use of Metrics That Matter®  Investigated long-term clients that have used MTM consistently for 5+ years  Gathered vast data sets and focused on key metrics:  Knowledge and skill acquisition (L2)  I will be able to apply training to the job (L3)  Training will improve my job performance (L4)  Training was a worthwhile investment (L5)  An average of all 4 metrics  We expected that scores would rise with time because organizations monitor performance, cull under- performing courses, and continuously improve their curricula. 14-Mar-12 © 2009, KnowledgeAdvisors 16
    17. 17. Initial Sample: Telecom CompanyWe examined the metrics for one company first to see if our hypothesis seemed tenable. Here are our results.
    18. 18. Initial Sample: Telecom CompanyEach year large numbers of evaluations were collected.
    19. 19. Initial Sample: Telecom CompanyWe computed the performance increase as a percentage: (2010 value – 2006 value) / (2006 value)
    20. 20. Initial Sample: Telecom CompanySome data were not collected every year, but the trend upward was evident even with only three years of data.
    21. 21. Initial Sample: Telecom CompanyThe combination of all metrics provides a good summary measure. Despite the variability across the four metrics, the average performance still improved 14.9%.
    22. 22. Initial Sample: SummaryWhen all 5 metrics are graphed, it is clear that performance rises over time.The number of evaluations is also overwhelmingly large lending undeniable reliability to the results. The average increase is also substantial.
    23. 23. Case Study vs. Trend The Telecom company is only one example. Does the hypothesis hold true for other companies or is the telecom example an anomaly?
    24. 24. Companies with 5 Years of DataFour organizations contributed 5 years of data: Two insurance companies, a hospital and a telecom company
    25. 25. Companies with 6 Years of DataSix organizations contributed 6 years of data: An office solutions company, an Army Base, a government defense agency, a pharmaceutical company, a US-based accounting firm, and a UK-based accounting firm
    26. 26. Other companies? The analysis will continue soon with other companies:  Small and medium businesses  Learning Providers We expect the results will be similar.14-Mar-12 © 2009, KnowledgeAdvisors 26
    27. 27. Conclusion Measurement leads to performance improvement for L&D departments The size of the improvement ranges from 3 – 5% on average but can be as large as 14.9% Revisit Lauri Bassi’s work—investments in HC lead to business improvement—outpacing the S&P 500 by 17% MTM helps make learning organizations more effective through measurement.

    ×