Published on

These slides come from Ellen Wagner's presentation at ASTD TechKnowledge 2013, San Jose, CA January 30, 2013.

Analytics have ramped up everyone’s expectations of what may be possible for improving accountability, transparency, and quality in all facets of our lives, from the marketplace to the workplace and at every point in education and training value chain, including learning and development. Analytics also are sparking serious conversations about the degree to which we truly understand which variables are truly predictive of desired action states, for whom and under what conditions they predict, and the confidence with which the predictions are made.

This session will provide participants with a faced-paced overview of the analytics landscape, including an orientation to enterprise data sources, a review of analytical techniques and methods and will then demonstrate the use of a 6-phased model for applying analytics in the workplace.

Learning and development organizations simply cannot live outside today’s enterprise focus on the measurable, tangible results now driving IT, operations, finance, and other mission-critical applications. This session will help ensure that learning and development organizations have a better emerging opportunities for putting their data to work in productive ways that will lead to demonstrable impact and alignment with business goals and enterprise strategic directions.

Published in: Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Getting Started with Analyticsin the Learning Enterprise Ellen Wagner Partner and Senior Analyst January 30, 2013
  2. 2. Agenda du Jour Welcome to the world of Big Data Analytics essentials (why, where and how they matter to your learning enterprise) Reframing the opportunities Some insights for maximizing success A review of small, medium and big data in a single BIG use case Q&A 2
  3. 3. Data Are Optimizing Online Experience The digital “breadcrumbs” that online technology users leave behind about viewing, engagement and behaviors, interests and preferences provide massive amounts of information that can be mined to better optimize online experiences. Sage Road Solutions LLC 3
  4. 4. http://tinyurl.com/a3oroul 4
  5. 5. 5http://tinyurl.com/a3oroul
  6. 6. http://tinyurl.com/a3oroul 6
  7. 7. http://tinyurl.com/a3oroul 7
  8. 8. http://tinyurl.com/a3oroul 8
  9. 9. Where are we headed? Business Models Provide Guidance Courtesy Phil Ice
  10. 10. eLearning Optimization Automate learner Leverage these interactions and measurements to activities based on real- Innovate make informed time data, course goals decisions about and performance Find and apply new how to create insights to optimize optimal learning Extend learning experiences Deliver reporting Automate experiences Bring together based on a complete and dashboards to information from the view of all interactions measure learning activity Execute LMS, SIS, and internet for a Measure comprehensive view of customer interactions regardless of device Measure Optimize Courtesy Phil Ice
  11. 11. Learning Organizations and Big Data• Big data-styled analyses have ramped up everyone’s expectations for accountability, transparency and quality.• Learning and development organizations simply cannot live outside the enterprise focus on measurable, tangible results now driving IT, operations, finance and other mission critical applications. Sage Road Solutions LLC 11
  12. 12. Will Analytics REALLY OptimizeEducational Experience?RETENTION Sage Road Solutions LLC 12
  13. 13. All analyses and stakeholdersConnections are interrelated
  14. 14. Where to Begin????? Uncertainty about where to start – No established industry best practice about what to measure – No established industry best practice around methodology Corporate Culture, Learning Culture and Status Quo – Enterprise concern about what the data will show – Competing priorities and lack of incentive for collaboration between different groups Siloed data across the enterprise sure doesn’t help. Sage Road Solutions LLC 14
  15. 15. Data Driven Decision-making is aimed atmaking better decisions  Success and decision making are predicated on access to data  Understanding strengths and weaknesses is dependent on having access to all data within the enterprise  Data tells us what has happened and improves strategic planning moving forward 15
  16. 16. Modeling forSuccess 16
  17. 17. Making Data Matter Via Modeling • Model building is an iterative process • Around 70-80% efforts are spent on data exploration and understanding. 17
  18. 18. Making Data Matter Turn the Use the Gather data into information to the data information help learner
  19. 19. 6 Steps for Using Data to Improve Decision-making Collect Refine Predict Decide Implement Monitor 19
  20. 20. Where in the Worldis Your EnterpriseData?? 20
  21. 21. Data repositories in learning organizations  ERP’s  Demographics  Macro level transactions  Learning Management System  Learning transactions  Learning outcomes  Latent data  End of Course Survey  Perceptual data 21
  22. 22. The LMS Problem  LMS’s have messy data bases  The primary function was not data collection  Years of additions have created the equivalent of a bowl of “data spaghetti”  Significant abstraction work is needed to draw out anything more than cursory data 22
  23. 23. Analytical Solutions  Most LMSs now offer an “analytics package”  Apps allow you to mash data sets to obtain new insights into data.  Watch lists, progress tracking, completion, engagement all trackable with the right proxies  Web analytics tools (Google Analytics, CoreMetrics, Omniture) are the future – “the Clickstream”  Highly granular transactional data derived  Not all web analytics tools created equal 23
  24. 24. Centralizing your data assets.  Creation of a middleware database should be a priority for all institutions  SQL is a popular choice, Hadoop another for unstructured data. SPSS, SAS, others for structured data analysis  Aggregate multiple data sources  Federation  Normalization 24
  25. 25. Making Data Actionable, Making DecisionsAccountable  Data must have a “home”  Top down dissemination of analytics  Actionable reporting, visualizations, dashboards  In CONJUNCTION with other organizational initiatives 25
  26. 26. A range of approaches are required toLevels of Analysis - AGAIN satisfy stakeholder needs
  27. 27. What is enough? Just as there are different levels of analysis there are different levels of stakeholders Engaging in overkill is the worst mistake you can make
  28. 28. What About YOUR Organization?  What types of data are collected within your organization?  Are eLearning data correlated with performance objectives?  What is being done with the data that is collected?  What is missing? 28
  29. 29. Reflections There’s a difference between “Big Data” and other data. Don’t be afraid to use them all. Structured data (Machine friendly, with rows and columns) is easier to work with than unstructured data (blogs, emails, reports, course materials). The point of any and all of these data is to help YOU make better decisions. 29
  30. 30. Reflections There’s no such thing as sort of transparent. Much o the data that we currently collect really doesn’t answer many of the questions we need to answer. Even with great data, people resist new ideas if it clashes with what they already know. 30
  31. 31. Reflections It’s easier to do structured analysis on things that we can count. Latent sematic analysis can help make qualitative information more “real” Big effects can often come from small, meaningful decisions. Effective use of data for decision-making is directly related to trust, validity, reliability 31
  32. 32. THANKS for your interest Contact information: Ellen Wagner edwsonoma@gmail.com