This document describes research on using machine learning to predict student dropouts in online learning environments. It presents a new deep learning model called GRU-AE that is designed to handle the challenges of predicting dropouts in e-degree programs, which have longer student trajectories with potential gaps in activity. The GRU-AE model uses autoencoders to mitigate sparse and irregular student activity data, and gated recurrent units to learn relationships between activities over time. An experiment on real e-degree data found that GRU-AE improved prediction accuracy over other methods, especially for the longer sequences and sparser data that occur in e-degrees.

1. Drop-out
prediction in on-
line learning
environments
Paola Velardi
Sapienza Università di Roma
(joint work with Bardh Prenkaj,
Stefano Faralli, Damiano Distante)

2. Outline
Online learning environments: challenges
and opportunities
The student drop-out (SDP) problem
Machine learning solutions to SDP: models
and algorithms
GRU-AE: a deep sequential strategy for the
SDP problem
Evaluation
Conclusions

3. OLE: challenges
and opportunities
 The e-learning market volume, which stood
at $ 107 billion in 2015, is set to reach a
remarkable $ 325 billion in 2025.
 Reasons: cost, accessibility, opportunity for
working students and students living far
from large universities
 OLE have gained further importance due to
the current pandemic.
 Most institutions have adopted hybrid
learning environment, that will be the
standard de facto in the next years, opening
new opportunities and posing new
challenges
 MOOC have been so far the most popular
type of OLEs, but now full e-degrees are
gaining relevance

4. The student
drop-out problem
 Online students have a much higher chance of dropping
out than those attending conventional classrooms (up to
40%-80% more prone to abandon that in presence
students)
 Therefore, it is of paramount interest for the institutions,
students, and faculty members, to find more efficient
methodologies to reduce the dropout phenomenon
 Early adoption of mitigation strategies: Knowing
which students are likely to abandon their studies
helps distance learning institutions to develop
intervention strategies to provide individually
tailored support.
 Avoiding economic waste: Since dropouts cause
significant economic wastes, online universities
have a clear interest in investing in this type of
predictive actions. Besides, from a prestige point-
of-view, institutions who exhibit higher graduation
rates – or higher retention rates – attract a higher
number of students.

5. Machine learning
strategies for early
drop-out prediction
The problem is particularly challenging since student activities
within an online learning platform are multiple and of a variegated
nature (parallel, sequential, interactive).
While the recent push towards distance learning has aggravated
the problems of withdrawals, on the other hand, data collected by
e-learning platforms on student activities offers an
unprecedented opportunity for data analytics and machine
learning systems to advance the state of the art.
In addition to its social and economic impact, SDP represents a
new and interdisciplinary research problem, involving both social
and computer science.

6. SDP pipeline
in literature:
student
models

7. SDP pipeline
in literature:
algorithms

8. Open issues
 Finding standard benchmarks and evaluation strategies
 Deep methods should be better exploited
 More sophisticated student models to cope with e-degrees
 Temporal gaps between e-tivities do matter

9. GRU-AE: a deep
sequential
strategy for the
SDP
 A sequential deep method with following
features:
 Mitigated feature sparsity and negative
effect of inactivity periods, by employing
autoencoders as trajectory densifiers
 Exploitation of features coming from the
direct interaction of students with OLEs,
such as navigational, forum-based,
video-based and homework-based e-
tivities
 Public release of the first dataset
containing long-term e-degrees splitted
into course modules, and public release
of all benchmarks and methods
implemented for our experimental
analysis
 Performs at best in the (increasingly
relevant) context of e-degrees, that
suffer from feature sparsity and long
inactivity periods

10. Input model:
Time matrix
number of times activity ei has
been performed on day dj
k is the number of days the
student is observed (variable)

11. Gated Recurrent Unit
The task of the autoencoder is to
learn a compact and denoised
representation of the features to
mitigate the sparsity
phenomenon.
Stacked embedded and stacked raw GRUs further compress the inform
and learn (even long-distance) relations among e-tivities
The GRU-AE
architecture

12. Experiments (1)
Datasets: two MOOC (XuetangX,
KDDCup15) and one e-degree (Unitelma)
Note that Unitelma has 13 e-degrees
(>300 courses)
E-tivities are more sparse in Unitelma
Furthermore, e-tivities are more dense in
the first week(s) in MOOCs, while in e-
degrees they are more dense in the central
part

13. Experiments
(2)
 MOOC students are observed
during the entire sequence
 E-degree students are observed
during a window of (up to) 365
days during their career

14. Experiments (3)
Compared methods
 Simple ML methods: Logistic regressions,
Gaussian Naive Bayes, Decision Trees,
Support Vectors, K-neares neighbours,
Majority class, Random Forest
 Deep methods: Deep feed forwards DNN 3
and 5 (layers), LSTM, ConRec (a
combination of CNN and LSTM), and GRU-
AE
 Variable time-windows, hyperparameters and
datasets

15. AUCPR for all
methods, all
datasets, variable
k

16. Summary
average
results
 GRU-AE improves over compared
methods especially in the context
of e-degrees (long sequences,
inactivity gaps, data sparsity)
 Not significantly different from best
methods in MOOC contexts,
according to ANOVA test

17. Concluding
remarks
 State-of-the-art approaches to predicting
student withdrawal in online learning
environments (OLEs) have focused
primarily on the more common scenario of
single MOOCs.
 However, the recent pandemic has widely
expanded the use of OLE platforms,
extending it to the case of entire degree
courses and highlighting new challenges.
 We presented a deep architecture, named
GRU-AE, to mitigate the problems of feature
sparsity, long trajectories, and possibly long
temporal gaps during which students remain
inactive, that arise in the context of e-
degrees.

18. Publications
 A survey of machine learning approaches for student dropout prediction in
online courses B Prenkaj, P Velardi, G Stilo, D Distante, S Faralli, in ACM
Computing Surveys (CSUR) 53 (3), 1-34, 2020
 A reproducibility study of deep and surface machine learning methods for
human-related trajectory prediction, B Prenkaj, P Velardi, D Distante, S
Faralli, in 29th ACM Conference on Information and Knowledge
Management (CIKM), 2020
 Hidden Space Deep Sequential Risk Prediction on Student Trajectories, B
Prenkaj, P Velardi, G Stilo, D Distante, S Faralli, to appear