This document presents the results of a systematic review on utilizing learning analytics to increase student success. The review examined 41 international studies published between 2013-2017. The results provided some positive evidence that learning analytics can help improve outcomes like graduation and course completion rates when used to predict risks, target interventions, and provide personalized feedback. However, most studies were small-scale and lacked concrete evidence. There were also gaps in knowledge around topics like informal learning and a lack of geographical diversity. Future work is needed to develop standardized learning analytics systems and address ethical/privacy guidelines before broader implementation.

1. UTILIZING LEARNING ANALYTICS FOR STUDY SUCCESS: A
SYSTEMATIC REVIEW
Project Team:
Prof. Dr. Dirk Ifenthaler, Universität Mannheim (ifenthaler@uni-mannheim.de)
Dr. Dana-KrisJn Mah, Universität Mannheim (dmah@mail.uni-mannheim.de)
Dr. Jane Yin-Kim Yau, Uninversität Mannheim (jyau@uni-mannheim.de)

2. (1) Theore8cal Background
• Study dropout rate in Germany: 29 % (Heublein et al., 2017)
• PrognosJcaJon of study success or iden8fica8on of risk students
• Learning AnalyJcs (Ifenthaler, 2015)
• Use sta8cally and dynamically generated data from learners and learning
environments
• Analyse und visualisa8on in real-8me
• Goal modeling and op8miza8on of teaching-Learning processes and
learning environments
• Lack of research: SystemaJc Review with focus on prognos8s8ca8on of
study success and the effect of interven8on and preven8on measures in
the study entry phase

3. (2) Research Ques8on
(1) Is there a link between learning analy8cs and
preven8on and interven8on to increase student
outcomes from interna8onal studies?

4. (3) Methodology
BMBF-funded, 1/1/17-31/12/18
Aim: to locate empirical evidence of increased study success via LA
SystemaJc Review (Okoli & Schabram, 2010): Eight Steps
• Na8onal & interna8onal primary studies on the use of learning
analy8cs with regard to academic success

5. (4) Systema8c Review (Search)
• Conducted from 01/2017 to 12/2017
• Google Scholar, ACM Digital Library, Web of Science, Science Direct,
ERIC and DBLP
• JLA, Computers in Human Behaviour, Computers & Educa8on;
Australasian J of Educa8onal Technology, Bri8sh J of Educa8onal
Technology
• Search terms: “Learning Analy8cs” in combina8on with “study
success”, “reten8on”, “dropout preven8on”, “course comple8on”,
and “agri8on”
• Study success: “successful comple8on of a first degree in HE or
individual learning exercises or courses; Any related posi8ve learning
sa8sfac8on, improvement, experience during course comple8on”

6. (5) Systema8c Review (Inclusion)
1. Total Search Results (n = 6,220)
2. Aker removal of duplicates (n = 3,163)
3.Records screened (n = 3,163)
4. Full-text ar8cles assessed for eligibility (n = 374)
5. Studies included in systema8c review (n = 41)
• Criteria: a) HE context, b) published from 2013-17, c) English, d)
qualita8ve/quan8ta8ve findings, e) peer-reviewed
• Records excluded – n = 2789; Full-text ar8cles excluded (n = 333)

7. (6) Systema8c Review (Results: Year)
0 2 4 6 8 10 12 14 16 18
2017
2016
2015
2014
2013
Publica8on years of the key studies

8. (7) Systema8c Review (Results: Loca8on)
0 2 4 6 8 10 12 14
USA
Australia
Brazil
Ireland
UK
Taiwan
India
South Korea
Sweden
Israel
Netherlands
Pakistan
Colombia
France
Spain
Japan
South Arabia
Czech Republic
Loca8on of the key studies

9. (8) Results: Posi8ve empirical evidence
1. Four-year gradua8on rate +3.8% in 4 years aker LA was adopted (University of
Nebraska-Lincoln).
2. Course comple8on rates rose 4.2% (Columbus State University College,
Georgia).
3. Dropout rates decreased from 18% to 12% (University of New England, South
Wales).
4. Final grade was improved by 6% (Marist College).
5. Iden8fied at-risk students given interven8on resulted in 5% increase in
agendance, 12% increase in passing and 8% decrease in dropout (Strayer
University, Virginia).
6. 15% increase in student recruitment and 15% increase in reten8on (Purdue
University, Indiana).
Source: (Sclater and Mullan, 2017)

10. Categories of results DescripJon
PredicJve analyJcs using
assessment data
On average, 95% probability if a student had not submiged their
assignment -> does not complete course (Hlosta, Zdrahal, & Zendulka,
2017) .
Assessment data is used to extract learning pagerns from students to
accurately predict their learning progress.
Assessment of the first assignment accurately provides a cri8cal indicator
of the remainder of the course.
PredicJve analyJcs using
engagement indicators
This predic8ve method is useful to detect disengaging
students in MOOCs in general, using informa8on about
students behavior that is available during the course (Bote-
Lorenzo & Gomez-Sanchez, 2017) .
PredicJve analyJcs using online
plaZorm data (including self-
report data)
Can successfully iden8fy students at-risk and in need of support. Certain
formulas were used (Manai, Yamada, and Thorn, 2016):
1. If students showing higher levels of fixed mindset & to be at-risk, a
growth mindset is promoted to them by engaging them in growth
mindset ac8vi8es.
2. If students showing higher levels of belonging uncertainty, group
ac8vi8es that facilitate building a learning community for all students
in the classroom are provided.
3. If students showing low levels of math conceptual knowledge,
scaffolding for students are provided during online learning.
Personalized feedback leading to
learning gains
Numerous studies have shown that personalized learning & feedback can
lead to learning gains. LA can be used to retrieve learner informa8on in
order to facilitate personalized learning (Grawemeyer, et al., 2016).
(9) Summary of Posi8ve Results

11. (10) Results: Nega8ve/Insufficient evidence
• Most LA studies are in early stage and lack deep concrete empirical
evidence (Sclater and Mullan, 2017).
• Some successful projects – US for-profit sector with unpublished
findings (Sclater and Mullan, 2017).
• Use of LA concentrated in US, Australia and UK. Lack of geographical
spread, gaps in knowledge (informal/workplace learning, ethical
prac8ce), lack of agen8on to LA cycle, minimum evalua8on of
commercially-available tools (Ferguson & Clow, 2017).
• Concerns: temporary character of data, incompleteness of data,
ethics, data privacy (Pardo & Siemens, 2014).
• Currently, na8onal policies of LA exisit in Denmark, Netherlands,
Norway and some UK universi8es (Ferguson et al, 2016).

12. (11) Conclusion and Limita8ons
• Aim towards a standardized LA system that can be
integrated into any VLE to provide at-risk predic8on and
personalized preven8on and interven8on strategies.
• More work on ethical and privacy guidelines suppor8ng
implementa8on at HE ins8tu8ons.
• Small-scale, works-in-progress of research studies in
systema8c review.
• Future work: Policy recommenda8ons and interview
study results concerning LA readiness in universi8es.

13. (12) Future research ques8ons & methodology
(1) Which recommenda8ons for ac8on, based on the results of
the systema8c reviews, can be derived for the context of
German universi8es?
(2) How are the poten8al of the theore8cally derived preven8on
and interven8on measures to increase the academic success
at German universi8es evaluated by experts with regard to
their acceptance and implementa8on?
QualitaJve Expert Interviews
• Valida8on of the derived recommenda8ons for ac8on by means of expert interviews
(Meuser & Nagel, 2009)
Data Bank
• U8liza8on of the project results and statements on the generalizability or
transferability of the intended project results

14. Thank you for your
agen8on!!!
Any ques8ons???

15. (13) References
Bote-Lorenzo, M. and Gomez-Sanchez, E. Predic8ng the decrease of engagement indicators in a MOOC. In Interna(onal
Conference on Learning Analy(cs & Knowledge, Vancouver, Canada, 2017.
Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Migelmeier, J., Rien8es, B., Ullmann, T., Vuorikari, R. (2016). Research
Evidence on the Use of Learning Analy(cs - Implica(ons for Educa(on Policy. R. Vuorikari, J. Castaño Muñoz (Eds.). Joint Research
Centre Science for Policy Report.
Ferguson, R. & Clow, D. (2017) Where is the evidence? A call to ac8on for learning analy8cs. LAK.
Grawemeyer, B., Mavrikis, M., Holmes, W., Gu8errez-Santos, S., Wiedmann, M. and Rummel, N. Determina8on of off-task/on-task
behaviours. In Proceedings of the Sixth Interna(onal Conference on Learning Analy(cs & Knowledge, Edinburgh, UK, 2016.
Hlosta, M., Zdrahal, Z. and Zendulka, J. Ouroboros: Early iden8fica8on of at-risk students without moels based on legacy data. In
Proceedings of the Seventh Interna(onal Conference on Learning Analy(cs & Knowledge, Vancouver, Canada, 2017.
Manai, O., Yamada, H. & Thorn, C. (2016) Real-8me indicators and targeted supports: using online plaxorm data to accelerate
student learning. In Interna(onal Conference on Learning Analy(cs & Knowledge, UK, 2016.
Pardo, A. and Siemens, G. (2014) Ethical and privacy principles for learning analy8cs. Bri(sh Journal of Educa(onal Technology,
45(3), 438–450.
Sclater, N., & Mullan, J. (2017). Learning analy(cs and student success – assessing the evidence. Bristol: JISC.
review of UK and interna(onal prac(ce. Bristol: JISC.