This document discusses learning analytics and the differences between academic analytics and learning analytics. It provides: - Definitions of academic analytics as focused on institutional decision making and management, while learning analytics focuses on supporting student learning and is aimed at learners and instructors. - An overview of how learning analytics has evolved from traditional testing and assessment to incorporate larger datasets, models, personalization techniques, and insights from digital traces like online activity logs. - Several examples of how learning analytics can provide insights at the individual student level, within groups, in the classroom, and across academic programs. - Some of the challenges in implementing learning analytics including issues around ethics, data access, and developing institutional capacity like data science

1. Learning Analytics
Stian Håklev, http://reganmian.net
Office of Online Learning Strategies
March 17, 2015
Image: Karen Lirenman

2. What is Learning Analytics?
Academic Analytics Learning Analytics
A process for providing higher
education institutions with the data
necessary to support operational and
financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement of
specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the primary
audience
* - Source: Analytics in Higher Education: Establishing a Common LanguageTeaching Analytics
Borrowed from George Siemens’ slides

3. Borrowed from George Siemens’ slides

4. Borrowed from George Siemens’ slides

5. Brand new?
old: testing, assessment, institutional research, retention
new: quantity, models, personalization, adaptivity, techniques, data
sources, processing power

6. Traces
about collecting traces that learners leave behind and using
those traces to improve learning
- online brings in many more traces
- traditional access logs
- social network analysis
- discourse/content analysis

7. Not just from LMS
Not just digital, automatic

8. Individual learners
Quantified self
Self-directed learning
Learning progression, portfolio

9. Borrowed from George Siemens’ paper

10. Group awareness
Self-monitoring, awareness
Computer-Supported Collaborative Learning

12. Classroom
Immediate feedback for instructor
Orchestration

13. From paper by Luis P. Prieto, Kshitij Sharma, Yun Wen and Pierre Dillenbourg

14. Program
Are individual courses leading to the skills we require?

16. Live analytics
vs
post-hoc analytics

17. MOOC dashboards

28. Supporting
three
MOOC
research
projects
(MRI)

29. Tools
to
collaborate
and
document

30. Connecting
database
with
other
data

31. Clicklog:
big
data,
making
it
queryable,
increasing
levels
of
abstraction

32. 20

33. From presentation at OAAI website

34. From presentation at OAAI website

35. From presentation at OAAI website

36. From presentation at OAAI website

37. From talk by Josh Baron

38. OLA Knowledge Domains
● Open Data and Models
○ Releasing data sets and models under open licenses
● Open Research
○ Publishing research in open-access journals
● Open-Source Software/Platforms
○ Open software, standards and APIs
● Open Strategy and Policy
○ Open documents on strategy and policy
● Open Learning Designs
○ Combine OER & LA to create new models of learning
From talk by Josh Baron

45. Ethics/vulnerability
• best practice with data - research ethic, operations ethics.
• custody of data. who has the right to access? custodial/access
• ethics not just for publishable research
• research councils lacking sophisticated understanding of risks
and best practices
• program evaluation/support strategy institutional vs research
ethics.
• openness to students - predictive models as black boxes.

46. Institutional capacity
• Open University
data scientists

47. Institutional capacity
• Bringing together operations and researchers
• Supporting technical skill-exchange and
documentation
• Experimenting with opening anonymized data,
competitions
• Scholarship of Teaching and Learning, CTSI,
OISE, Computer Science, IT services,
library…