Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Prepare for Impact: Using Data + AI to Solve L&D's Learning Measurement Problem


Published on

Learning measurement is broken. Sure, L&D can determine whether or not people liked a training activity and test to see if knowledge or skill improved in that moment. But that's about it. And, as stakeholders challenge L&D to prove their impact on the organization, that's simply not enough. It's time to leave the traditional measurement models behind and rethink how we leverage data to enable performance. If L&D hopes to modernize their practices and explore emerging concepts, such as personalization and artificial intelligence, they must first solve the learning measurement problem.

Published in: Business
  • Be the first to comment

Prepare for Impact: Using Data + AI to Solve L&D's Learning Measurement Problem

  1. 1. Using Data + AI to Solve the Learning Measurement Problem PREPARE FOR IMPACT
  2. 2. JD Dillon Chief Learning Architect Founder + Principal
  3. 3. AI-enabled learning platform that helps frontline employees execute on what matters most to your business. Headquartered in Waterloo, ON, Canada EST. 2011 142Countries Languages 50 microlearning moments/month 40M+
  4. 4. a critical part of the workflow takes advantage of the full ecosystem applies data to guide and accelerate decision-making provides a personal experience at scale drives clear business impact fosters ongoing organizational agility modern learning mindset
  5. 5. If you cannot determine if what you are doing is working, what’s the point in doing it at all?
  6. 6. L&D has been historically unable to measure the impact of learning. Our value is in question as a result.
  7. 7. 67% of L&D leaders are now under pressure to measure learning impact LEO Learning – Measuring the Impact of Learning 2019 Brandon Hall Group – Modern Learning Measurement 2018 96% of L&D leaders say they want to measure learning impact Less than 20% of companies say they are highly effective at learning measurement
  8. 8. Traditional learning measurement just doesn’t work The Kirkpatrick Model Levels 3 + 4 are just a myth for most organizations.
  9. 9. L&D cannot move forward until we fix our measurement problem.
  10. 10. Attribution is a renewed challenge for L&D. But we’re not the first department to confront this issue. LIMITED ATTRIBUTION FULL ATTRIBUTION Support SalesMarketing pre-digital Marketing digital L&D
  11. 11. L&D must get out of its own way and recognize that impact measurement – connecting business results directly to learning solutions – is possible.
  12. 12. Impact measurement is real Productivity/ efficiency Safety Sales Customer satisfaction Quality Assurance Attribution/impact(%) Degree of impact from learning on business outcomes 45% 5% 10% 15% 20% 25% 30% 35% 40% Impact data aggregated from Axonify community based on use case.
  13. 13. To fix our measurement problem, we must first reimagine the role of learning within the modern workplace.
  14. 14. Agree on a clear, measurable business result Define the observable behavior required to achieve the result Define the knowledge required to execute the expected behavior Determine the right-fit solution DESIGN + MEASUREMENT 1 | Reconnect learning + business
  15. 15. VELOCITY VERACITY VALUEVOLUMEVARIETY Speed at which data is generated and analyzed Different types and forms of data Trustworthiness of data collected Ability to transform data into value for the business Scale of data generated and collected for analysis 2 | Redefine “learning data”
  16. 16. How are people engaging with learning opportunities? Engagement How is people’s knowledge changing over time? Learning How is learning impacting business results and delivering ROI? Outcomes How are people’s behaviors changing on the job? Behaviors How are we projected to perform in the future with our key business goals? Prediction How can we continuously adapt our support tactics to ensure optimum results? Adaptation Learning is continuous. Performance is continuous. Modern learning measurement isn’t about levels. It’s a continuous cycle.
  17. 17. Axonify collects on average 250 million data points per organization every year.
  18. 18. 3 | Increase touch points
  19. 19. Reinforcement Analytics Motivation Practice Shared experience Community of knowledge Events Online content Messaging Knowledge + skill growth Business results continuous learning experience Coaching
  20. 20. 4 | Apply data-driven technology ARTIFICIAL INTELLIGENCE MACHINE LEARNING DEEP LEARNING a program that can sense, reason, act, and adapt algorithms whose performance improve as they are exposed to more data over time multilayered neural networks learn from vast amounts of data
  21. 21. VALUE ADOPTION Personalization Content Authoring End User Administrator Chatbots Content Recommend Content Tagging NLP Search Impact Analysis System Administration Gap Identification Smart Coaching Expertise Match Translation
  22. 22. Machine Learning Process 1. DATA CLEANING Prepare data by standardizing millions of learning and business data points 2. STATISTICAL CORRELATION Establish statistically significant connections between learning and business data 3. IMPACT MEASUREMENT Learning attribution is calculated using machine learning and the expanded cleaned and correlated data 5. INSIGHTS + ACTIONS Learning activities are automatically adapted with additional recommendations made to employees and managers 4. GAP ANALYSIS At-risk business targets and employee goals are continuously flagged
  23. 23. User/Input Layer Artificial Intelligence Layer Data Layer AI-Enabled Technology (machine learning analysis + experience adaptation) User Experience (personalized learning, actionable reporting) Data (multi-dimensional user profile) demographic consumption knowledge confidence behavior results Business (KPI actuals + forecasts) Behavior (field observation) Learning (daily learning experiences) 5 | Rebuild strategy on a data foundation
  24. 24. steps you can take to 5fix the learning measurement problem 1 | Reconnect learning + business 2 | Redefine “learning data” 3 | Increase touch points 4 | Apply data-driven technology 5 | Rebuild strategy on a data foundation
  25. 25. Data + AI help us continuously identify and close individual knowledge and skill gaps – at scale.
  26. 26. We can finally solve the learning measurement problem. Fixing measurement begins with mindset, not data or technology. Learning is continuous. Therefore measurement must be continuous. AI is a foundational capability for L&D, not just a single use case tool. Once we can attribute the impact of our solutions, we can become more agile and proactive in the ways we support our employees.
  27. 27. @JD_Dillon JD Dillon
  28. 28. Using Data + AI to Solve the Learning Measurement Problem PREPARE FOR IMPACT