Educational Data Mining/Learning Analytics issue brief overview
Enhancing Teaching and Learning ThroughEducational Data Mining and Learning Analytics An Issue Brief Prepared for the US Department of Education Office of Educational Technology April 10, 2012
Purpose Look at data analytics techniques in the context of a new framework for educational evidence See how commercial techniques (state-of-the-practice) might apply to education Define educational data mining and learning analytics: What questions can they answer? Categorize applications and understand benefits, and challenges
Issue Brief Questions What is educational data mining, and how is it applied? What kinds of questions can it answer, and what kinds of data are needed to answer these questions? How does learning analytics differ from data mining? Does it answer different questions and use different data? What are the broad application areas for which educational data mining and learning analytics are used? What are the benefits of educational data mining and learning analytics, and what factors have enabled these new approaches to be adopted? What are the challenges and barriers to successful application of educational data mining and learning analytics? What new practices have to be adopted in order to successfully employ educational data mining and learning analytics for improving teaching and learning?
Data Sources A review of selected publications and fugitive or gray literature (Web pages and unpublished documents) on educational data mining and learning analytics Interviews of 15 data mining/analytics experts from learning software and learning management system companies and from companies offering other kinds of Web-based services Deliberations of a technical working group of eight academic experts in data mining and learning analytics.
Inspiration for Issue Brief National Educational Technology Plan“When students are learning online, there are multiple opportunities toexploit the power of technology for formative assessment. The sametechnology that supports learning activities gathers data in the courseof learning that can be used for assessment…. An online system cancollect much more and much more detailed information about howstudents are learning than manual methods. As students work, thesystem can capture their inputs and collect evidence of their problem-solving sequences, knowledge, and strategy use, as reflected by theinformation each student selects or inputs, the number of attempts thestudent makes, the number of hints and feedback given, and the timeallocation across parts of the problem.” (U.S. Department ofEducation 2010, p. 30)
Interconnected Feedback System for Education National Educational Technology Plan“The goal of creating an interconnected feedback systemwould be to ensure that key decisions about learning areinformed by data and that data are aggregated and madeaccessible at all levels of the education system forcontinuous improvement.” (U.S. Department of Education2010, p. 35)
Research Areas Educational Data Mining: develops new techniques, tests learning theories and informs educational practice. Looks for patterns in unstructured data. Generally automates responses to learners. Learning analytics: Applies techniques and takes a “system-level” view of teaching and learning, including at the institutional level. Generally supports human decision making vs. automating responses. Visual Data Analytics: taps the ability of humans to discern patterns in visually represented complex datasets.
Application Areas User Modeling: Model a learner’s knowledge, behavior, motivation, experience, and satisfaction. User Profiling: Cluster users into similar groups. Domain Modeling: Decompose content to be learned into components and sequences. Effectiveness: Test learning principles, curricula, etc. Trend Analysis: Track changes over time. Recommendations and Improvements: Suggest resources and actions to learners; adapt system to learners.
Challenges Technical: Handling big data; interoperability of data systems; asking the right questions Institutional: Requires a culture of data-driven decision making and transparency in models that analyze data Privacy and Ethics: Maintain student and teacher privacy while allowing data aggregation to drive powerful models
Excerpt of Recommendations Educators: develop a culture of using data for making instructional decisions. Understand and communicate sources of data Researchers and Developers: Study usability and impact of “dashboards.” Understand how models can move to new contexts of use. Collaboration: Work across sectors to build capacity and knowledge. Include learning system designers (often commercial entities), learning scientists, IT departments, administrators, and educators on teams.