How to Get Started or Expand Your Learning Analytics Program
•Download as PPTX, PDF•
1 like•457 views
Watershed co-hosted a workshop with ATD Nashville and Rustici Software to help L&D pros get started with learning analytics. Facilitators Mike Rustici and John Mattox, Ph.D., explored the five steps of learning analytics and helped attendees choose the right approach for their learning analytics programs.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to How to Get Started or Expand Your Learning Analytics Program
Similar to How to Get Started or Expand Your Learning Analytics Program (20)
Recently uploaded
Recently uploaded (20)
How to Get Started or Expand Your Learning Analytics Program
- 1. Getting Started with Learning Analytics
- 2. “Scalable learning is the new reason for large organizations to exist” John Hagel Founder Deloitte Center for the Edge Innovation
- 3. Discover Discover new knowledge through study, experimentation and practice. Digest Assess the validity of knowledge and ensure it has a positive impact. Distribute Rapidly disseminate new knowledge to the entire organization. Discover Digest Distribute Learning Embedded Everywhere
- 4. ask yourself Are your learning systems going to get you there?
- 5. isn’t it because… Small moves, smartly made “In an exponential world, small moves, smartly made can set big changes in motion” – John Hagel
- 6. getting there What is the smartest move organizations can make right now to lay a foundation and set themselves on a path to becoming a scalable learning organization? What is the biggest enabler of scalable learning?
- 7. “Without data you’re just another person with an opinion” W Edward Deming Management Systems Legend
- 9. “L&D is responsible for improving our most important asset, our people. If we can improve L&D, we can improve the world” Me I said that
- 10. attitude strategy Think strategically and develop a culture of constant improvement.It’s learning’s turn
- 11. Marketing evolution Source: 5 Reasons Learning Needs to Think Like Marketing, Tribridge (2015)
- 12. marketing learning More dataMore tools Analytics
- 13. A brief history of Learning Technologies Interoperability SCORM 1.2 2001 Executive Order 13111 1999 SCORM 2004 4th Edition 2009 Project Tin Can 20102004 2006 2013 Experience API 1.0.0 2012 2014 20172000 1.0 1.1 200 4 2004 2nd ed. 2004 3rd ed. DoDI 1322.26 Enterprise- Scale Deployments 1.0.1 1.0.2 1.0.30.9 0.95 2016
- 14. xAPI & learning analytics platform all data Learning is so much bigger than the LMS. Track informal and experiential activities. xAPI and the LRS present a transformative technology for learning analytics. Seamlessly bring all your data together in one platform and perform real-time analytics tailor made for L&D. business impact Include data about behaviors and performance in your learning analytics real time Create, view and distribute reports seamlessly in real-time.
- 16. Understanding learning analytics: Complexity measurement The simple act of tracking things and recording values. Can be passively or actively collected. evaluation The process of trying to make meaning from the data measured. Descriptive analytics. Does the data mean something “good” or “bad”? advanced evaluation When data sets get large enough we can use advanced evaluation techniques to discover powerful insights. Data mining, AI, machine learning, etc. predictive and prescriptive Make predictions and decisions based on data and advanced algorithms. Recommendations engines are the best example in learning.
- 17. Learning Experience Learner Learning Program Understanding learning analytics: Categories Understand an overall learning program. Is this initiative helping to meet business objectives. Understand a learner or group of learners. Ensure organizational readiness and compliance. Understand more about a specific learning activity. Maximize effectiveness and spot problems.
- 18. The future in practice.
- 19. next generation learning ecosystem 03connected to outcomes Learning outcomes on their own are only marginally interesting. Incorporate learning into larger business objectives and bring the data together to show its effectiveness. 02 mobile, social, informal Learning happens everywhere and now we can track it. The constraints of old are gone. Embrace the types of learning that your learners want to use. Provide resources in the moment of need where the learners live. 01away from the LMS How much of what you’ve learning in your lifetime has come from an e- learning course in an LMS? LMSs aren’t going away, but it’s time for us to recognize that learning happened everywhere.
- 20. Visa University: A new ecosystem A next generation learning ecosystem brings this all together. This example from Visa shows that the LMS is just one small part of a modern learning ecosystem.
- 21. Source: Getting Buy-In From the Boss. Andy Webb, Applied
- 22. Comprehension vs. Application Source: Getting Buy-In From the Boss. Andy Webb, Applied
- 23. Source: Getting Buy-In From the Boss. Andy Webb, Applied
- 24. Code Blue When a patient's heart stops and needs to be resuscitated.
- 25. Learning Ecosystem Mock Code Blue/ In person Observation Mobile Simulation/ Defibrillator App LMS Training Lab Learning Analytics Platform (Watershed)
- 26. Does the amount of training activity over time have an impact on Code Blue KPIs? Time to defibrillation Time to chest Time to first drug Arrival times Time hands off chest Training Activities Jan 2015 Feb 2015 Mar 2015 Apr 2015 May 2015 Jun 2015 Training Activities CodeBlue KPIs Training Effectiveness
- 27. getting started The Watershed Model makes it easy to understand and get started with learning analytics. Watershed Model
- 28. Getting Started yes you can! 01 Gather your data Just start collecting data in a common format in a central location. Make it low friction and ensure you have access. 04 Explore your data Starting to understand not just “what” is happening but “why”. Form questions and see if your data can answer them. Often the best answer another question. Can you see impacts on the larger business? 02 Get to know your data Understand what you have. What is out there, what is reliable and what is missing? Do some simple evaluations and create baselines. 05 Experiment with your data Start a new learning program with analytics in mind. Set up well controlled experiments and A/B tests to validate a hypothesis. Create a culture of continuous improvement. 03 Operationalize your data Automate your way out of “Excel hell”. Define some interesting metrics and KPIs, start monitoring them routinely. What trends do you notice? 06 Show off your data We’ve long known that learning is vital, not we can probe it. Go forth and show the world!
- 29. our contacts watershed 210 Gothic Ct Franklin, TN 37067 +1-844-220-0822 http://watershedlrs.com @watershedlrs me Mike Rustici Founder and CEO mike.rustici@watershedlrs.com @mike_rustici
Editor's Notes
- Hi, I’m Mike I want to talk about big shifts and big opportunities in L&D that are happening Lots of new things are suddenly possible. Some big and lofty but also some big wins for small steps forward.
- So “what is scalable learning”? Scalable learning is not just learning new things that others know, but rather Creating and discovering new knowledge – this is more than just finding a new book to read Assessing its validity Rapidly disseminating it Ensuring it has a positive performance impact Discover -> Digest -> Distribute
- Is your LMS going to get you there? Fortunately there is a new generation of learning tools, models and ecosystems emerging Option – “is your learning org going to get you there”
- But the good news is, In an exponential world, small moves, smartly made can set big changes in motion Is this a better opening concept?
- Learning analytics is the big thing that’s now possible It’s not big and scary, many types of learning analytics that range from simple to complex
- It’s about attitude and strategy It’s learning’s turn to take a seat at the table To do so, we need to think strategically, we need to think in the same terms the business thinks in and align our strategy and outcomes to the strategy and outcomes of the business - often performance
- Let’s take a look at what happened to marketing over the past 20 years. You’ll see a lot of parallels to learning and a path forward for us. Learning depts refer to their audience as customers. And that makes so much sense.
- Marketing analogy Marketing went through this same transition over the past 20 years and we’re seeing the same thing happen in learning (summary of this article) More tools More data Missing piece is analytics
- xAPI is enabling this transition in learning, just like marketing It’s all about removing friction Removing just a little bit of friction has a big impact Apple design methodology Kirkpatrick model has been around since 1954, but how much of it are we using. Remember, when we asked people why they aren’t doing more learning analytics, one of the chief responses was “it’s too hard” That is changing, the friction is disappearing and that will have a profound impact.
- What does this look like in the real world
- There is a shift in the learning eco system New learning ecosystem (summary of this webinar) Can move away from the monolithic LMS LRS centric, mobile, social, informal, Removed constraints Connect to performance
- One of the first to deploy and publicize at enterprise scale is Visa Notice: -LMS is just one small part -Inclusion of non-learning tools Sharepoint and SurveyMonkey -Learning Experience platform as front end
- Applied is the world’s leading industrial distributor, operating at an annual revenue of nearly $3billion. Comprised of hundreds of distribution centers, every dot represents a small business with a manager and team To meet business targets, management needs to be competent on financial and sales acumen, which is why AIT invested $81 million to convert to a new ERP system to streamline reports and sales/operational accountabilities that were previously tracked manually by disparate systems. After successful SAP training and system implementation, many business KPIs dipped. SAP put a spotlight on operational metrics that were distilled into stoplight reports. But SCMs responsible for impacting results weren’t able to make improvements simply by seeing numbers on stoplight charts. Applied’s Organizational Learning (OL) department identified that SCMs could benefit from a better understanding of business insights, finance, and P&L concepts to see how their actions can directly impact financial statements
- Here’s why. Symptoms, does not point to the solution Once they got SAP, we focused on operations efficiency Revenue Gross Profit Operating Expense Billing Inventory Freight Processing Rebates What do you notice? Notice a lot of talented people at Applied If you don’t get your learning data aligned to business purpose, all your decision making is based on Intuition and hope. And, hope is not a methodology. Red = Failure ? Typical LMS style reporting – but only reflects LMS output Fluidity of Competency What does this do for leadership? Legitimized learning data Getting this done took a lot of excel work – V-Lookups, etc. What does it really tell us? Johnny did not complete some courses Background info (do not deliver this portion) A Group Should be running profitable locations. A+ group = best in class B Group Grasps financial concepts, but training and coaching needed on application of SOW. C Group Good practices, but may struggle with financial impacts of COGS and Ops expenses. Review P&L statement concepts with manager. D Group Needs the most help. Evaluate. Review webinars, coach on concepts and technique. Watch for improvement trends.
- VIDEO Not just big ROI, but little stories, with each visualization helping you to dig deeper. That’s the value of having everything in one, automated place.
- It’s important to develop skills during training before performing tasks in the real world. Real-world errors can have significant consequences. The organization needs to be confident of learner’s competency. Skills to be learnt are practical and physical.
- MedStar’s training program is made up of three main elements: The Learning Management System manages learner and organization data, and tracks a record of classroom training attended. A mobile app, called Zoll, allows clinicians to practice the steps involved in defibrillation on their own as often as they want to. In situ mock Code Blue simulations enable clinicians to practice resuscitating a medical dummy as a team.
- Wrap up and Q&A