SlideShare a Scribd company logo
Implementing Predictive Learning
Analytics on a Large Scale: The
Teacher's Perspective
Dr Christothea Herodotou
Authors: Christothea Herodotou, Bart Rienties, Avinash Boroowa, Zdenek Zdrahal,
Martin Hlosta , Galina Naydenova. Proceedings of LAK2017.
Predictive Learning Analytics (PLA)
o PLA identify students at risk + inform teachers
o Mixed-effects of providing PLA to teachers
o Difficulty in understanding and interpreting PLA data
and visualisations
o Difficulty in identifying specific interventions
o Promising outcomes (van Leeuwen et al., 2014;
McKenney and Mor, 2015)
o Identification of participation problems and intervention
o Development of curriculum materials
2
Aim of this study
Evaluate whether providing teachers
with PLA data would empower them to
identify and assist students in need for
additional support
3
OU Analyse (OUA)
•OUA: Early identification
of students at risk of
failing
•Available to tutors and
student support teams
•Aim: Improve retention of
OU students
https://analyse.kmi.open.ac.uk/
4
OUA Dashboard: Module view
5
OUA Dashboard: Student view
6
OUA Dashboard: Student view
Nearest neighbours, Predictions with real scores, Personalised recommender
7
Research Questions
1.How did 240 tutors within 10 modules made
use of OUA predictions and visualisations to
help students at risk?
2.To what extent was there a positive impact on
students' performance and retention when
using OUA predictions?
3.Which factors explain tutors' uses of OUA?
8
Methodology
1.Statistical comparisons: 240 tutors OUA
access Vs 613 tutors with no access to OUA
predictions
 formal withdrawal rates by the end of the
module,
 completion and pass rates.
2.Usage data of OUA dashboard (N=70)
3.Qualitative data: Semi-structured interviews
(N=6)
9
Results: Withdrawal rates
Formal withdrawal rates by the end of the module
2015 -OUA students 2015 - nonOUA students
count % count %
Arts 55 1728
did not reach the end of the module 15 27.27% 253 14.64%
reached 40 72.73% 1475 85.36%
Technology 239 1809
did not reach the end of the module 56 23.43% 311 17.19%
reached 183 76.57% 1498 82.81%
Law 246 1820
did not reach the end of the module 33 13.41% 359 19.73%
10
Results: Completion & pass rates
Module completion and pass rates
2015 -OUA students 2015- nonOUA students
Completion rates count % count %
Education 260 2841
did not complete 69 26.54% 931 32.77%
completed 191 73.46% 1910 67.23%
Law 245 1815
did not complete 79 32.24% 732 40.33%
completed 166 67.76% 1083 59.67%
Pass rates
Law 246 1820
did not passed 86 34.96% 797 43.79%
Passed 160 65.04% 1023 56.21%
11
Results: Usage of OUA dashboard
12
Results: Interviews (1/2)
Actual uses of OUA:
•Most useful OUA features:
o colour-coded students
o VLE activity
•Teachers define what features to use and how often (time
management)
Usefulness of OUA
•Teachers checking on students often (emails, forums)
•Teachers more proactive/enhance teaching practices
o “on top” of students
o “where I need to put my efforts"
• Predictions: complement teachers’ intuition + additional
insights
13
Results: Interviews (2/2)
Approaching students at risk
o Varied interventions - referral to student support
services, sending emails, texting, calling, doing
nothing
o Varied approaches: persistent VS less pro-active
o Personalised approach
Future intentions
o Interested in future use yet with improvements to
the system (e.g., “sensitive” to students’ activities
online)
14
Conclusions
o Blurred picture (withdrawal, completion and pass
rates)
o Variation in teachers’ degree and quality of
engagement with learning analytics.
o Lack of consensus about intervention strategies
o Predictive data - enhance and facilitate teaching
practice, especially within distance learning
contexts
o Research directions: Identify how, when, and what
interventions to trigger to support students
adequately
15
16

More Related Content

What's hot

Blackboard Learning Analytics Research Update
Blackboard Learning Analytics Research UpdateBlackboard Learning Analytics Research Update
Blackboard Learning Analytics Research UpdateJohn Whitmer, Ed.D.
 
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classes
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classesBus 231-term-paper-group-a-report-on-students-perceptions-on-online-classes
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classesMd. Asad Chowdhury Dipu
 
Using Analytics to Increase Student Success
Using Analytics to Increase Student SuccessUsing Analytics to Increase Student Success
Using Analytics to Increase Student SuccessDavid Lindrum
 
A new perspective for school evaluation and counseling
A new perspective for school evaluation and counselingA new perspective for school evaluation and counseling
A new perspective for school evaluation and counselingDr. Georgios Gaitanos
 
MAP Data report 2014
MAP Data report 2014MAP Data report 2014
MAP Data report 2014sharpscience
 
Data-Driven Development (D3) and Evaluation of Enskill English
Data-Driven Development (D3) and Evaluation of Enskill EnglishData-Driven Development (D3) and Evaluation of Enskill English
Data-Driven Development (D3) and Evaluation of Enskill EnglishLewisJohnson34
 
Using Learning Analytics to Create our 'Preferred Future'
Using Learning Analytics to Create our 'Preferred Future'Using Learning Analytics to Create our 'Preferred Future'
Using Learning Analytics to Create our 'Preferred Future'John Whitmer, Ed.D.
 
The Moodle Quiz at the Open University: how we use it & how that helps students
The Moodle Quiz at the Open University: how we use it & how that helps studentsThe Moodle Quiz at the Open University: how we use it & how that helps students
The Moodle Quiz at the Open University: how we use it & how that helps studentsTim Hunt
 
Online academic support & student retention: Early signs of a positive correl...
Online academic support & student retention: Early signs of a positive correl...Online academic support & student retention: Early signs of a positive correl...
Online academic support & student retention: Early signs of a positive correl...Studiosity.com
 
Mooc pp 4 k lppt_22nov
Mooc pp 4 k lppt_22novMooc pp 4 k lppt_22nov
Mooc pp 4 k lppt_22novKulari Lokuge
 
Linkage of funding and decentralisation tha-jumpatong
Linkage of funding and decentralisation tha-jumpatongLinkage of funding and decentralisation tha-jumpatong
Linkage of funding and decentralisation tha-jumpatongMunkh Orgil
 
Using Learning Analytics to Assess Innovation & Improve Student Achievement
Using Learning Analytics to Assess Innovation & Improve Student Achievement Using Learning Analytics to Assess Innovation & Improve Student Achievement
Using Learning Analytics to Assess Innovation & Improve Student Achievement John Whitmer, Ed.D.
 
Survey report: IT utilisation in Vietnam university
Survey report: IT utilisation in Vietnam universitySurvey report: IT utilisation in Vietnam university
Survey report: IT utilisation in Vietnam universityQ&Me Vietnam Market Research
 
Using the student voice to plan our Moodle - Mark Glynn, Eamon Costello
Using the student voice to plan our Moodle - Mark Glynn, Eamon CostelloUsing the student voice to plan our Moodle - Mark Glynn, Eamon Costello
Using the student voice to plan our Moodle - Mark Glynn, Eamon CostelloIreland & UK Moodlemoot 2012
 
The Moodle quiz at the Open University
The Moodle quiz at the Open UniversityThe Moodle quiz at the Open University
The Moodle quiz at the Open UniversityTim Hunt
 

What's hot (20)

Blackboard Learning Analytics Research Update
Blackboard Learning Analytics Research UpdateBlackboard Learning Analytics Research Update
Blackboard Learning Analytics Research Update
 
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classes
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classesBus 231-term-paper-group-a-report-on-students-perceptions-on-online-classes
Bus 231-term-paper-group-a-report-on-students-perceptions-on-online-classes
 
Using Analytics to Increase Student Success
Using Analytics to Increase Student SuccessUsing Analytics to Increase Student Success
Using Analytics to Increase Student Success
 
A new perspective for school evaluation and counseling
A new perspective for school evaluation and counselingA new perspective for school evaluation and counseling
A new perspective for school evaluation and counseling
 
MAP Data report 2014
MAP Data report 2014MAP Data report 2014
MAP Data report 2014
 
Mdd.last.monitoring
Mdd.last.monitoringMdd.last.monitoring
Mdd.last.monitoring
 
Data-Driven Development (D3) and Evaluation of Enskill English
Data-Driven Development (D3) and Evaluation of Enskill EnglishData-Driven Development (D3) and Evaluation of Enskill English
Data-Driven Development (D3) and Evaluation of Enskill English
 
Excellence rome - v03 2016
Excellence   rome - v03 2016Excellence   rome - v03 2016
Excellence rome - v03 2016
 
Edlw tel 2016_0ssiannilsson
Edlw tel 2016_0ssiannilssonEdlw tel 2016_0ssiannilsson
Edlw tel 2016_0ssiannilsson
 
Using Learning Analytics to Create our 'Preferred Future'
Using Learning Analytics to Create our 'Preferred Future'Using Learning Analytics to Create our 'Preferred Future'
Using Learning Analytics to Create our 'Preferred Future'
 
The Moodle Quiz at the Open University: how we use it & how that helps students
The Moodle Quiz at the Open University: how we use it & how that helps studentsThe Moodle Quiz at the Open University: how we use it & how that helps students
The Moodle Quiz at the Open University: how we use it & how that helps students
 
Med
MedMed
Med
 
Online academic support & student retention: Early signs of a positive correl...
Online academic support & student retention: Early signs of a positive correl...Online academic support & student retention: Early signs of a positive correl...
Online academic support & student retention: Early signs of a positive correl...
 
Mooc pp 4 k lppt_22nov
Mooc pp 4 k lppt_22novMooc pp 4 k lppt_22nov
Mooc pp 4 k lppt_22nov
 
Linkage of funding and decentralisation tha-jumpatong
Linkage of funding and decentralisation tha-jumpatongLinkage of funding and decentralisation tha-jumpatong
Linkage of funding and decentralisation tha-jumpatong
 
Using Learning Analytics to Assess Innovation & Improve Student Achievement
Using Learning Analytics to Assess Innovation & Improve Student Achievement Using Learning Analytics to Assess Innovation & Improve Student Achievement
Using Learning Analytics to Assess Innovation & Improve Student Achievement
 
[Vietnam survey] Vietnamese education for kids
[Vietnam survey] Vietnamese education for kids[Vietnam survey] Vietnamese education for kids
[Vietnam survey] Vietnamese education for kids
 
Survey report: IT utilisation in Vietnam university
Survey report: IT utilisation in Vietnam universitySurvey report: IT utilisation in Vietnam university
Survey report: IT utilisation in Vietnam university
 
Using the student voice to plan our Moodle - Mark Glynn, Eamon Costello
Using the student voice to plan our Moodle - Mark Glynn, Eamon CostelloUsing the student voice to plan our Moodle - Mark Glynn, Eamon Costello
Using the student voice to plan our Moodle - Mark Glynn, Eamon Costello
 
The Moodle quiz at the Open University
The Moodle quiz at the Open UniversityThe Moodle quiz at the Open University
The Moodle quiz at the Open University
 

Similar to Lak2017 Herodotou, Christothea

Study on students dropouts in advanced certificate in pre school education pr...
Study on students dropouts in advanced certificate in pre school education pr...Study on students dropouts in advanced certificate in pre school education pr...
Study on students dropouts in advanced certificate in pre school education pr...The Open University of Sri Lanka
 
Study on students dropout in advanced cer 11111
Study on students dropout in advanced cer 11111Study on students dropout in advanced cer 11111
Study on students dropout in advanced cer 11111Senevirathna Peellage
 
Critical issues in contemporary open education research
 Critical issues in contemporary open education research Critical issues in contemporary open education research
Critical issues in contemporary open education researchRobert Farrow
 
Teaching tools, methods and teacher student interaction
Teaching tools, methods and teacher student interactionTeaching tools, methods and teacher student interaction
Teaching tools, methods and teacher student interactionMahdi Hasan
 
Results of the EADTU Task Force on Retention and Student Services - George Ub...
Results of the EADTU Task Force on Retention and Student Services - George Ub...Results of the EADTU Task Force on Retention and Student Services - George Ub...
Results of the EADTU Task Force on Retention and Student Services - George Ub...EADTU
 
Talis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong
Talis Insight Asia-Pacific 2017: Simon Bedford, University of WollongongTalis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong
Talis Insight Asia-Pacific 2017: Simon Bedford, University of WollongongTalis
 
Suh an explanatory study of high school teachers’ integration of mobile learning
Suh an explanatory study of high school teachers’ integration of mobile learningSuh an explanatory study of high school teachers’ integration of mobile learning
Suh an explanatory study of high school teachers’ integration of mobile learningSylvia Suh
 
ABLE - NTU Danish visit February 2018
ABLE - NTU Danish visit February 2018ABLE - NTU Danish visit February 2018
ABLE - NTU Danish visit February 2018Ed Foster
 
CCC-M F2F meeting_150928
CCC-M F2F meeting_150928CCC-M F2F meeting_150928
CCC-M F2F meeting_150928McGill_RSB
 
The UCL assessment journey
The UCL assessment journeyThe UCL assessment journey
The UCL assessment journeyMarieke Guy
 
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...Fostering students’ engagement and learning through UNEDTrivial: a gamified s...
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...UNED
 
PGNA 2014: District Results
PGNA 2014: District ResultsPGNA 2014: District Results
PGNA 2014: District ResultsLaura Jacob
 
Why Student Data? (ISTE 2015)
Why Student Data? (ISTE 2015)Why Student Data? (ISTE 2015)
Why Student Data? (ISTE 2015)Julie Evans
 
OER Impact at Community Colleges
OER Impact at Community CollegesOER Impact at Community Colleges
OER Impact at Community CollegesOER Hub
 
Assessing the impact of a global health MOOC/OER
Assessing the impact of a global health MOOC/OER Assessing the impact of a global health MOOC/OER
Assessing the impact of a global health MOOC/OER Sally Parsley
 
OER Impact at Community College: eLearning 2014
OER Impact at Community College: eLearning 2014OER Impact at Community College: eLearning 2014
OER Impact at Community College: eLearning 2014Una Daly
 
Dissertation Defense presented August 8th 2015 Dr. Koffajuah Toeque Ed.D
Dissertation Defense  presented August 8th 2015 Dr. Koffajuah Toeque Ed.DDissertation Defense  presented August 8th 2015 Dr. Koffajuah Toeque Ed.D
Dissertation Defense presented August 8th 2015 Dr. Koffajuah Toeque Ed.DDr. Koffajuah Toeque Ed.D
 
Dissertation Defense Powerpoint presented Aug. 8th, 2015
Dissertation Defense Powerpoint presented Aug. 8th, 2015Dissertation Defense Powerpoint presented Aug. 8th, 2015
Dissertation Defense Powerpoint presented Aug. 8th, 2015KJ Slyusar
 
Using GradeMark to engage students in the feedback process
Using GradeMark to engage students in the feedback processUsing GradeMark to engage students in the feedback process
Using GradeMark to engage students in the feedback processSara Marsham
 

Similar to Lak2017 Herodotou, Christothea (20)

Study on students dropouts in advanced certificate in pre school education pr...
Study on students dropouts in advanced certificate in pre school education pr...Study on students dropouts in advanced certificate in pre school education pr...
Study on students dropouts in advanced certificate in pre school education pr...
 
Study on students dropout in advanced cer 11111
Study on students dropout in advanced cer 11111Study on students dropout in advanced cer 11111
Study on students dropout in advanced cer 11111
 
Critical issues in contemporary open education research
 Critical issues in contemporary open education research Critical issues in contemporary open education research
Critical issues in contemporary open education research
 
Teaching tools, methods and teacher student interaction
Teaching tools, methods and teacher student interactionTeaching tools, methods and teacher student interaction
Teaching tools, methods and teacher student interaction
 
Results of the EADTU Task Force on Retention and Student Services - George Ub...
Results of the EADTU Task Force on Retention and Student Services - George Ub...Results of the EADTU Task Force on Retention and Student Services - George Ub...
Results of the EADTU Task Force on Retention and Student Services - George Ub...
 
Talis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong
Talis Insight Asia-Pacific 2017: Simon Bedford, University of WollongongTalis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong
Talis Insight Asia-Pacific 2017: Simon Bedford, University of Wollongong
 
Suh an explanatory study of high school teachers’ integration of mobile learning
Suh an explanatory study of high school teachers’ integration of mobile learningSuh an explanatory study of high school teachers’ integration of mobile learning
Suh an explanatory study of high school teachers’ integration of mobile learning
 
ABLE - NTU Danish visit February 2018
ABLE - NTU Danish visit February 2018ABLE - NTU Danish visit February 2018
ABLE - NTU Danish visit February 2018
 
CCC-M F2F meeting_150928
CCC-M F2F meeting_150928CCC-M F2F meeting_150928
CCC-M F2F meeting_150928
 
The UCL assessment journey
The UCL assessment journeyThe UCL assessment journey
The UCL assessment journey
 
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...Fostering students’ engagement and learning through UNEDTrivial: a gamified s...
Fostering students’ engagement and learning through UNEDTrivial: a gamified s...
 
Successful Statistics Course Redesign
Successful Statistics Course RedesignSuccessful Statistics Course Redesign
Successful Statistics Course Redesign
 
PGNA 2014: District Results
PGNA 2014: District ResultsPGNA 2014: District Results
PGNA 2014: District Results
 
Why Student Data? (ISTE 2015)
Why Student Data? (ISTE 2015)Why Student Data? (ISTE 2015)
Why Student Data? (ISTE 2015)
 
OER Impact at Community Colleges
OER Impact at Community CollegesOER Impact at Community Colleges
OER Impact at Community Colleges
 
Assessing the impact of a global health MOOC/OER
Assessing the impact of a global health MOOC/OER Assessing the impact of a global health MOOC/OER
Assessing the impact of a global health MOOC/OER
 
OER Impact at Community College: eLearning 2014
OER Impact at Community College: eLearning 2014OER Impact at Community College: eLearning 2014
OER Impact at Community College: eLearning 2014
 
Dissertation Defense presented August 8th 2015 Dr. Koffajuah Toeque Ed.D
Dissertation Defense  presented August 8th 2015 Dr. Koffajuah Toeque Ed.DDissertation Defense  presented August 8th 2015 Dr. Koffajuah Toeque Ed.D
Dissertation Defense presented August 8th 2015 Dr. Koffajuah Toeque Ed.D
 
Dissertation Defense Powerpoint presented Aug. 8th, 2015
Dissertation Defense Powerpoint presented Aug. 8th, 2015Dissertation Defense Powerpoint presented Aug. 8th, 2015
Dissertation Defense Powerpoint presented Aug. 8th, 2015
 
Using GradeMark to engage students in the feedback process
Using GradeMark to engage students in the feedback processUsing GradeMark to engage students in the feedback process
Using GradeMark to engage students in the feedback process
 

More from Thea24

Learning_Science_technology_40years_openuniversity_2018
Learning_Science_technology_40years_openuniversity_2018Learning_Science_technology_40years_openuniversity_2018
Learning_Science_technology_40years_openuniversity_2018Thea24
 
Herodotou AERA symposium 2018
Herodotou AERA symposium 2018Herodotou AERA symposium 2018
Herodotou AERA symposium 2018Thea24
 
Herodotou ECSA2018 cit_inquiry
Herodotou ECSA2018 cit_inquiryHerodotou ECSA2018 cit_inquiry
Herodotou ECSA2018 cit_inquiryThea24
 
Herodotou_citizen science_20-11_17
Herodotou_citizen science_20-11_17Herodotou_citizen science_20-11_17
Herodotou_citizen science_20-11_17Thea24
 
Herodotou @nhm_london 6_6_17
Herodotou @nhm_london 6_6_17Herodotou @nhm_london 6_6_17
Herodotou @nhm_london 6_6_17Thea24
 
What characteristics of gamers’ profile should be taken into account in play...
What characteristics of gamers’ profile should be taken into account in play...What characteristics of gamers’ profile should be taken into account in play...
What characteristics of gamers’ profile should be taken into account in play...Thea24
 
Addressing the retention gap in MOOCs
Addressing the retention gap in MOOCsAddressing the retention gap in MOOCs
Addressing the retention gap in MOOCsThea24
 

More from Thea24 (7)

Learning_Science_technology_40years_openuniversity_2018
Learning_Science_technology_40years_openuniversity_2018Learning_Science_technology_40years_openuniversity_2018
Learning_Science_technology_40years_openuniversity_2018
 
Herodotou AERA symposium 2018
Herodotou AERA symposium 2018Herodotou AERA symposium 2018
Herodotou AERA symposium 2018
 
Herodotou ECSA2018 cit_inquiry
Herodotou ECSA2018 cit_inquiryHerodotou ECSA2018 cit_inquiry
Herodotou ECSA2018 cit_inquiry
 
Herodotou_citizen science_20-11_17
Herodotou_citizen science_20-11_17Herodotou_citizen science_20-11_17
Herodotou_citizen science_20-11_17
 
Herodotou @nhm_london 6_6_17
Herodotou @nhm_london 6_6_17Herodotou @nhm_london 6_6_17
Herodotou @nhm_london 6_6_17
 
What characteristics of gamers’ profile should be taken into account in play...
What characteristics of gamers’ profile should be taken into account in play...What characteristics of gamers’ profile should be taken into account in play...
What characteristics of gamers’ profile should be taken into account in play...
 
Addressing the retention gap in MOOCs
Addressing the retention gap in MOOCsAddressing the retention gap in MOOCs
Addressing the retention gap in MOOCs
 

Recently uploaded

Morse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptxMorse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptxjmorse8
 
How to Split Bills in the Odoo 17 POS Module
How to Split Bills in the Odoo 17 POS ModuleHow to Split Bills in the Odoo 17 POS Module
How to Split Bills in the Odoo 17 POS ModuleCeline George
 
MARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptxMARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptxbennyroshan06
 
How to Break the cycle of negative Thoughts
How to Break the cycle of negative ThoughtsHow to Break the cycle of negative Thoughts
How to Break the cycle of negative ThoughtsCol Mukteshwar Prasad
 
2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptxmansk2
 
Sectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdfSectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdfVivekanand Anglo Vedic Academy
 
Benefits and Challenges of Using Open Educational Resources
Benefits and Challenges of Using Open Educational ResourcesBenefits and Challenges of Using Open Educational Resources
Benefits and Challenges of Using Open Educational Resourcesdimpy50
 
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXPhrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXMIRIAMSALINAS13
 
Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345beazzy04
 
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya - UEM Kolkata Quiz Club
 
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxCapitolTechU
 
The Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational ResourcesThe Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational Resourcesaileywriter
 
How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17Celine George
 
The Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonThe Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
 
Keeping Your Information Safe with Centralized Security Services
Keeping Your Information Safe with Centralized Security ServicesKeeping Your Information Safe with Centralized Security Services
Keeping Your Information Safe with Centralized Security ServicesTechSoup
 
Industrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportIndustrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportAvinash Rai
 
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...Abhinav Gaur Kaptaan
 
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdfTelling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdfTechSoup
 

Recently uploaded (20)

Morse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptxMorse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptx
 
How to Split Bills in the Odoo 17 POS Module
How to Split Bills in the Odoo 17 POS ModuleHow to Split Bills in the Odoo 17 POS Module
How to Split Bills in the Odoo 17 POS Module
 
B.ed spl. HI pdusu exam paper-2023-24.pdf
B.ed spl. HI pdusu exam paper-2023-24.pdfB.ed spl. HI pdusu exam paper-2023-24.pdf
B.ed spl. HI pdusu exam paper-2023-24.pdf
 
MARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptxMARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptx
 
How to Break the cycle of negative Thoughts
How to Break the cycle of negative ThoughtsHow to Break the cycle of negative Thoughts
How to Break the cycle of negative Thoughts
 
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
Operations Management - Book1.p  - Dr. Abdulfatah A. SalemOperations Management - Book1.p  - Dr. Abdulfatah A. Salem
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
 
2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx
 
Sectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdfSectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdf
 
Benefits and Challenges of Using Open Educational Resources
Benefits and Challenges of Using Open Educational ResourcesBenefits and Challenges of Using Open Educational Resources
Benefits and Challenges of Using Open Educational Resources
 
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXPhrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
 
Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345
 
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
 
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
 
The Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational ResourcesThe Benefits and Challenges of Open Educational Resources
The Benefits and Challenges of Open Educational Resources
 
How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17
 
The Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonThe Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve Thomason
 
Keeping Your Information Safe with Centralized Security Services
Keeping Your Information Safe with Centralized Security ServicesKeeping Your Information Safe with Centralized Security Services
Keeping Your Information Safe with Centralized Security Services
 
Industrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportIndustrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training Report
 
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...
Research Methods in Psychology | Cambridge AS Level | Cambridge Assessment In...
 
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdfTelling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
 

Lak2017 Herodotou, Christothea

  • 1. Implementing Predictive Learning Analytics on a Large Scale: The Teacher's Perspective Dr Christothea Herodotou Authors: Christothea Herodotou, Bart Rienties, Avinash Boroowa, Zdenek Zdrahal, Martin Hlosta , Galina Naydenova. Proceedings of LAK2017.
  • 2. Predictive Learning Analytics (PLA) o PLA identify students at risk + inform teachers o Mixed-effects of providing PLA to teachers o Difficulty in understanding and interpreting PLA data and visualisations o Difficulty in identifying specific interventions o Promising outcomes (van Leeuwen et al., 2014; McKenney and Mor, 2015) o Identification of participation problems and intervention o Development of curriculum materials 2
  • 3. Aim of this study Evaluate whether providing teachers with PLA data would empower them to identify and assist students in need for additional support 3
  • 4. OU Analyse (OUA) •OUA: Early identification of students at risk of failing •Available to tutors and student support teams •Aim: Improve retention of OU students https://analyse.kmi.open.ac.uk/ 4
  • 7. OUA Dashboard: Student view Nearest neighbours, Predictions with real scores, Personalised recommender 7
  • 8. Research Questions 1.How did 240 tutors within 10 modules made use of OUA predictions and visualisations to help students at risk? 2.To what extent was there a positive impact on students' performance and retention when using OUA predictions? 3.Which factors explain tutors' uses of OUA? 8
  • 9. Methodology 1.Statistical comparisons: 240 tutors OUA access Vs 613 tutors with no access to OUA predictions  formal withdrawal rates by the end of the module,  completion and pass rates. 2.Usage data of OUA dashboard (N=70) 3.Qualitative data: Semi-structured interviews (N=6) 9
  • 10. Results: Withdrawal rates Formal withdrawal rates by the end of the module 2015 -OUA students 2015 - nonOUA students count % count % Arts 55 1728 did not reach the end of the module 15 27.27% 253 14.64% reached 40 72.73% 1475 85.36% Technology 239 1809 did not reach the end of the module 56 23.43% 311 17.19% reached 183 76.57% 1498 82.81% Law 246 1820 did not reach the end of the module 33 13.41% 359 19.73% 10
  • 11. Results: Completion & pass rates Module completion and pass rates 2015 -OUA students 2015- nonOUA students Completion rates count % count % Education 260 2841 did not complete 69 26.54% 931 32.77% completed 191 73.46% 1910 67.23% Law 245 1815 did not complete 79 32.24% 732 40.33% completed 166 67.76% 1083 59.67% Pass rates Law 246 1820 did not passed 86 34.96% 797 43.79% Passed 160 65.04% 1023 56.21% 11
  • 12. Results: Usage of OUA dashboard 12
  • 13. Results: Interviews (1/2) Actual uses of OUA: •Most useful OUA features: o colour-coded students o VLE activity •Teachers define what features to use and how often (time management) Usefulness of OUA •Teachers checking on students often (emails, forums) •Teachers more proactive/enhance teaching practices o “on top” of students o “where I need to put my efforts" • Predictions: complement teachers’ intuition + additional insights 13
  • 14. Results: Interviews (2/2) Approaching students at risk o Varied interventions - referral to student support services, sending emails, texting, calling, doing nothing o Varied approaches: persistent VS less pro-active o Personalised approach Future intentions o Interested in future use yet with improvements to the system (e.g., “sensitive” to students’ activities online) 14
  • 15. Conclusions o Blurred picture (withdrawal, completion and pass rates) o Variation in teachers’ degree and quality of engagement with learning analytics. o Lack of consensus about intervention strategies o Predictive data - enhance and facilitate teaching practice, especially within distance learning contexts o Research directions: Identify how, when, and what interventions to trigger to support students adequately 15
  • 16. 16

Editor's Notes

  1. -----even when module teams were able to unpack the underlying PLA results, most teachers found it difficult to identify specific interventions van Leeuwen et al. (2014) found that teachers who received learning analytics visualisation of collaboration activities were better able to identify participation problems, and intervened more often in “problematic” groups relative to the control group of teachers who did not receive learning analytics visualisations. In a four year study of fine-grained data collection amongst 34 teachers, McKenney and Mor (2015)indicated that by engaging with learning analytics software, teachers’ professional development was enhanced (they learn from the process) and were able to develop better curriculum materials.
  2. Model that will be rolled out across the University
  3. Yellow- current module presentation Blue – previous one Bars – average TMA scores Time machine
  4. Model that will be rolled out across the University
  5. 613 - did not have access to OUA. 17,033 students were enrolled in 10 modules - of which 4320 were supported by ALs who had access to the OUA predictions. 70 teachers received a weekly reminder (email) notifying them that the OUA predictions were available through the dashboard 170 received the OUA weekly predictions via email in excel sheets.
  6. Model that will be rolled out across the University
  7. Mixed outcomes: all 3 significant: In the case of the Technology (X² (1, N=2048) =5.59, p<.05) and Arts modules (X² (1, N=1783) = 6.65, p<.01), the OU Analyse teacher groups had higher formal withdrawal rates, while in the case of the Law module (X² (1, N=2066) =5.61, p<.05) significantly lower formal withdrawal rates were observed for the OUA group
  8. Higher completion and pass rates for OUA modules: statistically significant differences in the Education (X² (1, N=3101)= 4.23, p<.05) and the Law modules only (X² (1, N=2060)= 5.91, p<.01). The OUA teacher groups had a higher module completion rate compared to the non-OUA groups in both modules. In terms of pass rates, significant differences were observed in the Law module only with the OUA group having a higher pass rate (X² (1, N=2066) =6.9, p<.01) (see Table 4).
  9. Model that will be rolled out across the University
  10. Model that will be rolled out across the University
  11. From 10 modules 8 had no differences or negative picture for PLA – is this bc teachers did not log in systematically to the system?