Making Learning Analytics Matter in the Educational Enterprise Ellen Wagner Partner and Sr. Analyst , Sage Road Solutions, LLC Executive Director, WICHE Cooperative for Educational Technologies (WCET) Sage Road Solutions LLC 1
Are You in the Right Place?• You have been hearing a lot about “analytics” lately and are wondering what the buzz is all about• You are worried that “analytics” is a code word for “statistics”• You just want someone to explain what analytics are, why they matter and what you need to know
What I will be covering in today’s session• What analytics are and why they are taking the world by storm• Tips for navigating the analytics ecosystem• Why learning analytics are particularly interesting• Things to keep in mind about making learning analytics matter in your educational enterprise
Data Are Optimizing Online Experience The digital “breadcrumbs” that online technology users leave behind about viewing, engagement and behaviors, interests and preferences provide massive amounts of information that can be mined to better optimize online experiences. Sage Road Solutions LLC 5
DATA IN DAILY LIFE: LOTS OF DATA, ALL THE TIMESage Road Solutions LLC 6
Major Trends at Play• Data Warehouses and “the Cloud” make it possible to collect, manage and maintain massive numbers of records.• Sophisticated technology platforms provide computing power necessary to grind through calculations and turn the mass of numbers into meaningful patterns.• Data mining uses descriptive and inferential statistics — moving averages, correlations, regressions, graph analysis, market basket analysis, and tokenization – to look inside patterns for actionable information.• Predictive techniques, such as neural networks and decision trees, help anticipate behavior and events. Sage Road Solutions LLC 7
Gartner Pattern Based Strategy, 2010:From reacting to events that had major effects on businessstrategy to proactively seeking patterns that might indicate animpending event.The interest in Pattern-Based Strategy is likely to grow as weunderstand the technologies that are emerging to seek patterns – from both traditional (financial information, customer order data, inventory, etc.) – nontraditional sources of information (social media, news, blogs). Gartner Research, Inc. 3 August 2010 ID Number: G00205744. p.4
Emergence of Business Intelligence• Research typically reports empirical evidence to prove the tenability of ideas concepts and constructs.• Business Intelligence uses analytical techniques to mine data to make decisions and create action plans.• Techniques for analyses include many of the same tools, but the focus on structuring the research question is very different.
Putting Your Information to Work Courtesy Phil Ice, American Public University System
Learning Organizations and Data Analytics• Analytics have ramped up everyone’s expectations for accountability, transparency and quality.• Learning and development organizations simply cannot live outside the enterprise focus on measurable, tangible results driving IT, operations, finance and other mission critical applications. Sage Road Solutions LLC 11
The Case for Analytics in Learning• The learning world is starting to discover what Internet marketers have known for years.• The digital “breadcrumbs” that learners leave behind about their engagement behaviors and interests provide massive amounts of data that can be mined to improve and personalize educational experiences• This is making learning pros very, very nervous Sage Road Solutions LLC 12
Will Data REALLY Optimize Educational Experience?RETENTION Sage Road Solutions LLC 13
Where to Begin?• Uncertainty about where to start – No established industry best practice about what to measure – No established industry best practice around methodology• Organizational Culture, Learning Culture and Status Quo – Enterprise concern about what the data will show – Competing priorities and lack of incentive for collaboration between different groups• Siloed data across the enterprise sure doesn’t help.
Institutional Data Sources: One Example Courtesy Phil Ice, American Public University System, 2012
Where Learning Data Typically Live ERPs and SISs Demographics, financials, operations Macro level transactions Learning Management System (LMS) Learning transactions Learning outcomes Latent data End of Course Survey Perceptual data
“The LMS Problem” LMSs have messy data bases The primary function was not data collection per se, but learning (artifact) management and tracking Years of additions have created the equivalent of a bowl of “data spaghetti” Good analytical solutions will pay attention to how data is extracted
Learning Analytics Applications in the .edu Enterprise
Lessons from Moneyball Moneyball: The Art of Winning an Unfair Game (ISBN 0-393-05765-8) Michael Lewis, 2003 Sage Road Solutions LLC 19
SOME THINGS TO REMEMBER WHENPUTTING ANALYTICS TO WORK Sage Road Solutions LLC 20
(1) Analytics are heretoday, and they are here tostay. Get on board or getleft behind!
(2) SOMEONE on yourteam needs to knowstatistics, databasesand researchtechniques.
(3) Doing research onanalytics is fundamentallydifferent than applyinganalytics results to helplearners succeed.
(4) It’s what we do withthe analytical findingsthat really matter.
(5) We already havemore data than we canhandle. That means weneed to find better waysto handle it.
(6) Even moreinteresting datacollecting opportunitiesawait.
(7) We need to beprepared to live underthe “sword of data.”
(8) Theres no suchthing as “sort of”transparent.
(9) We have just started tounderstand the true powerthat analytics bring to thelearning enterprise.
THANKS for your interest Ellen Wagner email@example.com http://wcet.wiche.edu (9) We havent even begun to scratch the www.sageroadsolutions.com surface of the possibilities.http://twitter.com/edwsonoma +1.415.613.2690 mobile