The document outlines a panel session on open learning analytics, emphasizing its significance for the open education movement and the goal of fostering collaboration between networks. It discusses the need for open data, research, and models, as well as detailing the development and impact of an open-source academic early alert system aimed at identifying at-risk students. Key findings indicate that the implementation of this system positively influenced student performance and content mastery.

1. Open Learning Analytics Panel
JOSH BARON
ST IAN HÅKLEV
NORMAN BIER
HASH TAG #OPENLA

2. Panel Session Overview
Session Goal: Stimulate discussion around the
importance of open learning analytics to the future of
the larger open education movement
Longer-Term Objective: Form connections between
the OpenEd and Open Learning Analytics networks
Session Format:
 Setting the Context – Open Learning Analytics
 Examples from the Real World
 Discussion/Q&A

3. What is Learning Analytics?
Academic Analytics Learning Analytics
A process for providing higher
education institutions with the data
necessary to support operational
and financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement
of specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the
primary audience
* - Source: Analytics in Higher Education: Establishing a Common Language

4. 2014 Open Learning Analytics Summit
Society for Learning Analytics (SoLAR) began
exploring openness in learning analytics in 2011
International OLA Summit held in March 2014
Participants identified OLA “knowledge domains”
as means to organize future work

5. OLA Knowledge Domains
Open Data and Models
 Releasing data sets and models under open licenses
Open Research
 Publishing research in open-access journals
Open-Source Software/Platforms
 Open software, standards and APIs
Open Strategy and Policy
 Open documents on strategy and policy
Open Learning Designs
 Combine OER & LA to create new models of learning

6. Openness = Science
NORMAN BIER
DIRECTOR, OPEN LEARNING INI T IAT IVE
CARNEGIE MEL LON UNIVERSI TY

7. The changing value of content
Changing focus in OER community
Commoditization of content (Wiley: ‘content is
infrastructure’)
Instrumenting content is difficult and expensive
Well-instrumented content and the tools to analyze
student interactions with that content will continue to
increase in importance

8. Problems with Black-Box Systems

9. Examples
Challenges Opportunities
CC-OLI Research
MOOC Research
UC Davis
Common measures of
outcomes and
achievement
Simon DataLab
Design Analytics
Stanford Outcomes
Analytics Service
Swappable Models
Learner Centered

10. MOOC RESEARCH AND
REPRODUCIBLE SCIENCE
ST IAN HÅKLEV
INST I TUT IONAL RESEARCHER, OPEN UTORONTO
UNIVERSI TY OF TORONTO

11. Supporting three MOOC research projects
(MRI)

12. Tools to collaborate and document

13. Connecting database with other data

14. Clicklog: big data, making it queryable,
increasing levels of abstraction

15. 20

24. Open Data Models & OS
Learning Analytics Platform
JOSH BARON
SENIOR ACADEMIC TECHNOLOGY OF F ICER
MARIST COL LEGE

25. OAAI: Overview and Impact
EDUCAUSE Next
Generation Learning
Challenges (NGLC)
Funded by Bill and
Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”

26. Student Aptitude Data
(SATs, current GPA, etc.)
Student Demographic
Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Step #1: Developed
model using historical
data
Predictive
Model
Scoring
Identifies
students
“at risk” to
not
complete
course
LMS Data SIS Data
OAAI Early Alert System Overview
Intervention Deployed
“Awareness” or Online
Academic Support
Environment (OASE)
“Creating an Open Academic
Early Alert System”
Model Developed
Using Historical Data
Academic Alert
Report (AAR)

27. Research Design
Deployed OAAI system to 2200 students across
four institutions
 Two Community Colleges
 Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections
 One section was control, other 2 were treatment groups
Each instructor received an AAR three times during
the semester:
 Intervals were 25%, 50% and 75% into the semester

28. Intervention Research Findings
Final Course Grades
Analysis showed a
statistically significant
positive impact on final
course grades
 No difference between
treatment groups
Saw larger impact in
spring then fall
Similar trend amount low
income students
Mean Final Grade for "at Risk" Students
100
90
80
70
60
50
Awareness OASE Control
Final Grade (%)

29. Intervention Research Findings
Content Mastery
Student in intervention
groups were statistically
more likely to “master the
content” then those in
controls.
 Content Mastery = Grade
of C or better
Similar for low income
students.
Content Mastery for "at Risk" Students
1000
800
600
400
200
0
Yes No Yes No
Control Intervention
Frequency

30. More Research Findings…
JAYAPRAKASH, S. M. , MOODY, E. W. , LAURÍA, E. J. ,
REGAN, J. R. , & BARON, J. D. (2014) . EARLY ALERT
OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN
SOURCE ANALYTICS INITIATIVE. JOURNAL OF
LEARNING ANALYTICS, 1(1) , 6 -47.

31. Strategic Vision: Open
Learning Analytics Platform
Collection – Standards-based
data capture from any
potential source using
Experience API and/or IMS
Caliper/Senor API
Storage – Single repository
for all learning-related data
using Learning Record Store
(LRS) standard.
Analysis – Flexible
Learning Analytics
Processor (LAP) that can
handle data mining, data
processing (ETL), predictive
model scoring and reporting.
Communication –
Dashboard technology for
displaying LAP output.
Action – LAP output can be
fed into other systems to
trigger alerts, etc.

32. Access to Predictive Model and
related OS Software…
OAAI PREDICT IVE MODEL DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/ 8AWCB
APEREO LEARNING ANALYT ICS PROCESSOR DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/KWCVBQ

34. Discussion and Q&A

35. Discussion Questions
Do you feel LA will be important to OER and Open
Education in the future? How important?
Where do you see connections between the OLA
network and Open Education?
How might we best facilitate making connections
across different “networks”?
[insert more questions]

36. Additional Resources
European OLA Summit – December 1st (LACE)
 http://www.laceproject.eu/
The Asilomar Convention for Learning Research in
Higher Education
 http://asilomar-highered.info
Apereo Learning Analytics Initiative
 https://confluence.sakaiproject.org/x/rIB_BQ
Society for Learning Analytics and Research
 http://solaresearch.org

37. What is Student Success?
X
Z
Engagement
Learning Y
Credit: mike.sharkey@phoenix.com