Automatic classification of students in online courses using machine learning techniques

David Monllaó
David MonllaóLead Data Scientist
Automatic classification of students in online
courses using machine learning techniques
D. Monllao Olive
School of Computer Science and Software Engineering
The University of Western Australia
Crawley WA 6009, AUSTRALIA
Principal supervisor: Dr Du Huynh
Co-supervisor: Assoc/Prof Mark Reynolds
External supervisor: Dr Martin Dougiamas
Master of Philosophy - part time student
Contents
1. Problem description
1.1. Online education and Moodle
1.2. Detection of students at risk of dropping out of courses
1.3. Students engagement in online courses
2. Literature review
3. Aim
4. Progress
5. Methodology
6. Timeline
1
Online education and Moodle
● Traditional education better for social interactions
● Online education offers more flexibility but relies more in self discipline
● Moodle stats (May 2017 https://moodle.net/stats/)
○ 103 million users worldwide
○ 12 million courses
○ 215 million forum posts
○ 589 million quiz questions https://moodle.org/logo/
2
Students at risk of dropping out of courses
● Students that are not engaged in the course don’t participate
● Different stakeholders interested in reducing online courses drop out rates
○ Students
○ Teachers
○ Educational institutions
https://www.thehrdigest.com/wp-content/upload
s/2016/03/college-degree.jpg
3
Students’ engagement in online courses
● Engaged students participate in the course activities
● Engagement is not as easy to detect in online courses as in face-to-face
education
● Some examples of engagement indicators:
○ Regular accesses to the course
○ Replies to other course participants’ forum posts
○ Quick reply to teacher’s feedback
○ Percentage of accessed course resources https://userscontent2.emaze.com/images/f5e5
a8b9-e038-4620-b54f-b935902facd9/cdeb1f7
d712f03715e4b0aed325ee967.jpg 4
1. From an educational point of view
● Description of online students’ engagement indicators [1] [2]
● Factor analysis and correlations between indicators and students retention
● Limitations:
○ Not very empirically rigorous
○ Limited studied dataset, results biased to a few courses
○ Indicators correlate individually
Literature review - Learning analytics
5
[1] Katrina A. Meyer. Student engagement in online learning: What works and why. ASHE Higher Education Report, 40(6):1–114, 2014.
[2] Kate S. Hone and Ghada R. El Said. Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98:157–168, 2016.
Literature review - Educational data mining
6
[3] Carlos Marquez-Vera, Alberto Cano, Cristobal Romero, Amin Yousef Mohammad Noaman, Habib Mousa Fardoun, and Sebastian Ventura. Early
dropout prediction using data mining: a case study with high school students. Expert Systems, 33(1):107–124, 2016. EXSY-Dec-13-227.R3.
[4] J. M. Luna, C. Castro, and C. Romero. Mdm tool: A data mining framework integrated into moodle. Computer Applications in Engineering Education,
25(1):90–102, 2017
2. From a data mining point of view:
● Some recent studies using machine learning techniques like Decision
Trees, Association rules or Evolutionary algorithms or [3] [4]
● Limitations:
○ Limited studied dataset
○ Basic student engagement indicators
Aim
To find the model that better predicts students at risk of dropping out of
any ongoing Moodle course.
7
Aim - How to achieve it?
● By using multiple and different institutions’ datasets
○ To prevent the model to be overfit to courses of a particular institution or format
● By selecting a subset of the literature student engagement indicators
○ To discard indicators that don’t correlate well
● By adding course information to the training dataset
○ To make the model adaptable to all sort of courses
● By limiting the studied activity logs to the most relevant time range
○ To improve the model accuracy
8
Progress
● Student engagement indicators literature review
● Moodle analytics API developed (https://github.com/moodlehq/moodle-tool_inspire)
○ Machine learning backend plugins. Shipped with Python (Tensorflow) and PHP (php-ml)
○ Very extendable
○ Prototype: http://prototype.moodle.net/inspirephase1/
○ Experimental model included: Students at risk of dropping out
● Contributions to the most popular PHP machine learning library
○ https://github.com/php-ai/php-ml/graphs/contributors
http://php-ml.readthedoc
s.io/en/latest/assets/php
-ml-logo.png
9
Progress - Analytics API - Data flow chart
10
Progress - Prototype - Models list
11
Progress - Prototype - Predictions list
12
Progress - Prototype - Prediction details
13
Methodology - Overview
1. Training dataset preparation from raw Moodle sites data
○ One sample for each student enrolment in each course of each Moodle site
○ Features: Student engagement indicators calculations, course information and
information about the included activity logs time range
○ Label: Did the student drop out of the course?
○ Output: A .csv file
14
Methodology - Overview
2. Machine learning training and performance evaluation
a. Inputs: A .csv file
b. Cross-validation (hyper parameters tuning)
c. Prediction model performance evaluation
■ The process is repeated multiple times
d. Outputs: The average accuracy (Matthews correlation coefficient) and the standard
deviation of all performance evaluations 15
Methodology - Parameters
● Repeat the described process with different parameters:
a. Using different subsets of student engagement indicators
b. Adding more course information when required
c. Limiting the student activity logs that are used
d. Using different machine learning algorithms
■ e.g. Neural networks, Support vector machines, Random forests...
16
Timeline
17
Task / Milestone Date
Training courses and literature review September 2016 - May 2017
Thesis proposal seminar and proposal submission to the Graduate Research
School at UWA
May 2017
Learning Analytics and Educational Data Mining survey June 2017 - December 2017
Paper describing the analytics framework developed and used for this research November 2017 - July 2018
Paper detailing different combinations of parameters and results July 2018 - December 2019
Limit to nominate examiners and thesis submission August 2020 - September 2020
Questions?
Thanks for coming.
Any questions?
18
1 of 19

Recommended

A Supervised Learning Framework for Learning Management Systems by
A Supervised Learning Framework for Learning Management SystemsA Supervised Learning Framework for Learning Management Systems
A Supervised Learning Framework for Learning Management SystemsDavid Monllaó
233 views16 slides
Moodle learning analytics from different perspectives (#moothr19) by
Moodle learning analytics from different perspectives (#moothr19)Moodle learning analytics from different perspectives (#moothr19)
Moodle learning analytics from different perspectives (#moothr19)David Monllaó
1.5K views33 slides
Moodle, the de facto learning platform to facilitate research and experimenta... by
Moodle, the de facto learning platform to facilitate research and experimenta...Moodle, the de facto learning platform to facilitate research and experimenta...
Moodle, the de facto learning platform to facilitate research and experimenta...David Monllaó
787 views22 slides
Eunis 2014: Technology in Real-life Teaching of Distributed Software Development by
Eunis 2014: Technology in Real-life Teaching of Distributed Software DevelopmentEunis 2014: Technology in Real-life Teaching of Distributed Software Development
Eunis 2014: Technology in Real-life Teaching of Distributed Software DevelopmentIvana Bosnic
1.2K views28 slides
2017 UK/IE MoodleMoot: What makes a good moodle quiz? Lessons from the Open U... by
2017 UK/IE MoodleMoot: What makes a good moodle quiz? Lessons from the Open U...2017 UK/IE MoodleMoot: What makes a good moodle quiz? Lessons from the Open U...
2017 UK/IE MoodleMoot: What makes a good moodle quiz? Lessons from the Open U...Tim Hunt
1.4K views35 slides
Marta Higueras: Frameworks of language teaching competences revisited by
Marta Higueras: Frameworks of language teaching competences revisitedMarta Higueras: Frameworks of language teaching competences revisited
Marta Higueras: Frameworks of language teaching competences revisitedeaquals
421 views14 slides

More Related Content

What's hot

1. ISD Project Plan by
1. ISD Project Plan1. ISD Project Plan
1. ISD Project PlanMaruf Hamidi
13 views15 slides
6. User Interface by
6. User Interface6. User Interface
6. User InterfaceMaruf Hamidi
17 views41 slides
Section0 course introduction by
Section0 course introductionSection0 course introduction
Section0 course introductionDương Tùng
137 views5 slides
1.5 Come Together: Harnessing the Power of Peer Support Through User Groups by
1.5 Come Together: Harnessing the Power of Peer Support Through User Groups1.5 Come Together: Harnessing the Power of Peer Support Through User Groups
1.5 Come Together: Harnessing the Power of Peer Support Through User GroupsTargetX
774 views37 slides
Broadening the scope of a Maths module for student Technology teachers by
Broadening the scope of a Maths module for student Technology teachersBroadening the scope of a Maths module for student Technology teachers
Broadening the scope of a Maths module for student Technology teachersUofGlasgowLTU
408 views25 slides
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ... by
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...Blackboard APAC
202 views24 slides

What's hot(10)

Section0 course introduction by Dương Tùng
Section0 course introductionSection0 course introduction
Section0 course introduction
Dương Tùng137 views
1.5 Come Together: Harnessing the Power of Peer Support Through User Groups by TargetX
1.5 Come Together: Harnessing the Power of Peer Support Through User Groups1.5 Come Together: Harnessing the Power of Peer Support Through User Groups
1.5 Come Together: Harnessing the Power of Peer Support Through User Groups
TargetX774 views
Broadening the scope of a Maths module for student Technology teachers by UofGlasgowLTU
Broadening the scope of a Maths module for student Technology teachersBroadening the scope of a Maths module for student Technology teachers
Broadening the scope of a Maths module for student Technology teachers
UofGlasgowLTU408 views
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ... by Blackboard APAC
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...
Moodlerooms Enterprise Upgrade Process | Shirley Li (Macquarie University) & ...
Blackboard APAC202 views
TLC2016 - Inspiring a Sense of Educational Community by BlackboardEMEA
TLC2016 - Inspiring a Sense of Educational CommunityTLC2016 - Inspiring a Sense of Educational Community
TLC2016 - Inspiring a Sense of Educational Community
BlackboardEMEA169 views
Student Selection Based On Academic Achievement System using k-mean Algorithm by Nik Ridhuan
Student Selection Based On Academic Achievement System using k-mean AlgorithmStudent Selection Based On Academic Achievement System using k-mean Algorithm
Student Selection Based On Academic Achievement System using k-mean Algorithm
Nik Ridhuan50 views
GSoC Sri Lanka Meetup - Introduction to GSoC by Harshana Martin
GSoC Sri Lanka Meetup - Introduction to GSoCGSoC Sri Lanka Meetup - Introduction to GSoC
GSoC Sri Lanka Meetup - Introduction to GSoC
Harshana Martin629 views

Similar to Automatic classification of students in online courses using machine learning techniques

MOOCs & Learning Analytics by
MOOCs & Learning AnalyticsMOOCs & Learning Analytics
MOOCs & Learning AnalyticsEDSA project
492 views20 slides
Krakow presentation speak_appsmngm_final by
Krakow presentation speak_appsmngm_finalKrakow presentation speak_appsmngm_final
Krakow presentation speak_appsmngm_finalSpeakApps Project
409 views22 slides
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos by
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth CoboseMadrid network
140 views18 slides
Learning Managment System by
Learning Managment SystemLearning Managment System
Learning Managment SystemUttara University
1.5K views13 slides
SOLAR - learning analytics, the state of the art by
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the artRebecca Ferguson
1.5K views18 slides
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka... by
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...ijceronline
444 views6 slides

Similar to Automatic classification of students in online courses using machine learning techniques(20)

MOOCs & Learning Analytics by EDSA project
MOOCs & Learning AnalyticsMOOCs & Learning Analytics
MOOCs & Learning Analytics
EDSA project492 views
Krakow presentation speak_appsmngm_final by SpeakApps Project
Krakow presentation speak_appsmngm_finalKrakow presentation speak_appsmngm_final
Krakow presentation speak_appsmngm_final
SpeakApps Project409 views
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos by eMadrid network
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
eMadrid network140 views
SOLAR - learning analytics, the state of the art by Rebecca Ferguson
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the art
Rebecca Ferguson1.5K views
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka... by ijceronline
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...
ijceronline444 views
ECI519_Syllabus_Spring_2016-6 by Shaun Kellogg
ECI519_Syllabus_Spring_2016-6ECI519_Syllabus_Spring_2016-6
ECI519_Syllabus_Spring_2016-6
Shaun Kellogg274 views
Higher Education Technology Outlook in Africa by Greig Krull
Higher Education Technology Outlook in AfricaHigher Education Technology Outlook in Africa
Higher Education Technology Outlook in Africa
Greig Krull372 views
Learning analytics research informed institutional practice by Yi-Shan Tsai
Learning analytics research informed institutional practiceLearning analytics research informed institutional practice
Learning analytics research informed institutional practice
Yi-Shan Tsai89 views
Higher Education Technology Outlook in Africa by Saide OER Africa
Higher Education Technology Outlook in AfricaHigher Education Technology Outlook in Africa
Higher Education Technology Outlook in Africa
Saide OER Africa1.5K views
Learning Analytics: New thinking supporting educational research by Andrew Deacon
Learning Analytics: New thinking supporting educational researchLearning Analytics: New thinking supporting educational research
Learning Analytics: New thinking supporting educational research
Andrew Deacon1.5K views
28_09_2018 eMadrid seminar on MOOCs by Pedro J. Muñoz Merino, UC3M by eMadrid network
28_09_2018 eMadrid seminar on MOOCs by Pedro J. Muñoz Merino, UC3M28_09_2018 eMadrid seminar on MOOCs by Pedro J. Muñoz Merino, UC3M
28_09_2018 eMadrid seminar on MOOCs by Pedro J. Muñoz Merino, UC3M
eMadrid network175 views
Analytics in Action - Education by Lee Schlenker
Analytics in Action - EducationAnalytics in Action - Education
Analytics in Action - Education
Lee Schlenker119 views
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L... by eMadrid network
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
eMadrid network1.1K views
Educational Data Mining & Students Performance Prediction using SVM Techniques by IRJET Journal
Educational Data Mining & Students Performance Prediction using SVM TechniquesEducational Data Mining & Students Performance Prediction using SVM Techniques
Educational Data Mining & Students Performance Prediction using SVM Techniques
IRJET Journal81 views
V Jornadas eMadrid sobre “Educación Digital”. Pedro Muñoz Merino, Universidad... by eMadrid network
V Jornadas eMadrid sobre “Educación Digital”. Pedro Muñoz Merino, Universidad...V Jornadas eMadrid sobre “Educación Digital”. Pedro Muñoz Merino, Universidad...
V Jornadas eMadrid sobre “Educación Digital”. Pedro Muñoz Merino, Universidad...
eMadrid network740 views

More from David Monllaó

Moodle learning analytics desde diferentes perspectivas (#mootgt19) by
Moodle learning analytics desde diferentes perspectivas (#mootgt19)Moodle learning analytics desde diferentes perspectivas (#mootgt19)
Moodle learning analytics desde diferentes perspectivas (#mootgt19)David Monllaó
336 views36 slides
El equipo de integracion de Moodle HQ es tu muy mejor amigo by
El equipo de integracion de Moodle HQ es tu muy mejor amigoEl equipo de integracion de Moodle HQ es tu muy mejor amigo
El equipo de integracion de Moodle HQ es tu muy mejor amigoDavid Monllaó
208 views15 slides
Install solr and global search by
Install solr and global searchInstall solr and global search
Install solr and global searchDavid Monllaó
485 views8 slides
Add your plugin contents to global search by
Add your plugin contents to global searchAdd your plugin contents to global search
Add your plugin contents to global searchDavid Monllaó
139 views10 slides
How to improve your moodle site performance by
How to improve your moodle site performanceHow to improve your moodle site performance
How to improve your moodle site performanceDavid Monllaó
9.2K views13 slides
Testing Moodle functionality automatically by
Testing Moodle functionality automaticallyTesting Moodle functionality automatically
Testing Moodle functionality automaticallyDavid Monllaó
6.1K views13 slides

More from David Monllaó(6)

Moodle learning analytics desde diferentes perspectivas (#mootgt19) by David Monllaó
Moodle learning analytics desde diferentes perspectivas (#mootgt19)Moodle learning analytics desde diferentes perspectivas (#mootgt19)
Moodle learning analytics desde diferentes perspectivas (#mootgt19)
David Monllaó336 views
El equipo de integracion de Moodle HQ es tu muy mejor amigo by David Monllaó
El equipo de integracion de Moodle HQ es tu muy mejor amigoEl equipo de integracion de Moodle HQ es tu muy mejor amigo
El equipo de integracion de Moodle HQ es tu muy mejor amigo
David Monllaó208 views
Install solr and global search by David Monllaó
Install solr and global searchInstall solr and global search
Install solr and global search
David Monllaó485 views
Add your plugin contents to global search by David Monllaó
Add your plugin contents to global searchAdd your plugin contents to global search
Add your plugin contents to global search
David Monllaó139 views
How to improve your moodle site performance by David Monllaó
How to improve your moodle site performanceHow to improve your moodle site performance
How to improve your moodle site performance
David Monllaó9.2K views
Testing Moodle functionality automatically by David Monllaó
Testing Moodle functionality automaticallyTesting Moodle functionality automatically
Testing Moodle functionality automatically
David Monllaó6.1K views

Recently uploaded

PRIVACY AWRE PERSONAL DATA STORAGE by
PRIVACY AWRE PERSONAL DATA STORAGEPRIVACY AWRE PERSONAL DATA STORAGE
PRIVACY AWRE PERSONAL DATA STORAGEantony420421
7 views56 slides
Listed Instruments Survey 2022.pptx by
Listed Instruments Survey  2022.pptxListed Instruments Survey  2022.pptx
Listed Instruments Survey 2022.pptxsecretariat4
121 views12 slides
META.pptx by
META.pptxMETA.pptx
META.pptxvasanthan19012003
6 views10 slides
Penetration testing by Burpsuite by
Penetration testing by  BurpsuitePenetration testing by  Burpsuite
Penetration testing by BurpsuiteAyonDebnathCertified
5 views19 slides
Report on OSINT by
Report on OSINTReport on OSINT
Report on OSINTAyonDebnathCertified
6 views15 slides
DGIQ East 2023 AI Ethics SIG by
DGIQ East 2023 AI Ethics SIGDGIQ East 2023 AI Ethics SIG
DGIQ East 2023 AI Ethics SIGKaren Lopez
5 views7 slides

Recently uploaded(20)

PRIVACY AWRE PERSONAL DATA STORAGE by antony420421
PRIVACY AWRE PERSONAL DATA STORAGEPRIVACY AWRE PERSONAL DATA STORAGE
PRIVACY AWRE PERSONAL DATA STORAGE
antony4204217 views
Listed Instruments Survey 2022.pptx by secretariat4
Listed Instruments Survey  2022.pptxListed Instruments Survey  2022.pptx
Listed Instruments Survey 2022.pptx
secretariat4121 views
DGIQ East 2023 AI Ethics SIG by Karen Lopez
DGIQ East 2023 AI Ethics SIGDGIQ East 2023 AI Ethics SIG
DGIQ East 2023 AI Ethics SIG
Karen Lopez5 views
K-Drama Recommendation Using Python by FridaPutriassa
K-Drama Recommendation Using PythonK-Drama Recommendation Using Python
K-Drama Recommendation Using Python
FridaPutriassa5 views
AZConf 2023 - Considerations for LLMOps: Running LLMs in production by SARADINDU SENGUPTA
AZConf 2023 - Considerations for LLMOps: Running LLMs in productionAZConf 2023 - Considerations for LLMOps: Running LLMs in production
AZConf 2023 - Considerations for LLMOps: Running LLMs in production
Underfunded.pptx by vgarcia19
Underfunded.pptxUnderfunded.pptx
Underfunded.pptx
vgarcia1914 views
Pydata Global 2023 - How can a learnt model unlearn something by SARADINDU SENGUPTA
Pydata Global 2023 - How can a learnt model unlearn somethingPydata Global 2023 - How can a learnt model unlearn something
Pydata Global 2023 - How can a learnt model unlearn something
Data Journeys Hard Talk workshop final.pptx by info828217
Data Journeys Hard Talk workshop final.pptxData Journeys Hard Talk workshop final.pptx
Data Journeys Hard Talk workshop final.pptx
info82821711 views
PyData Global 2022 - Things I learned while running neural networks on microc... by SARADINDU SENGUPTA
PyData Global 2022 - Things I learned while running neural networks on microc...PyData Global 2022 - Things I learned while running neural networks on microc...
PyData Global 2022 - Things I learned while running neural networks on microc...
Games, Queries, and Argumentation Frameworks: Time for a Family Reunion by Bertram Ludäscher
Games, Queries, and Argumentation Frameworks: Time for a Family ReunionGames, Queries, and Argumentation Frameworks: Time for a Family Reunion
Games, Queries, and Argumentation Frameworks: Time for a Family Reunion
4_4_WP_4_06_ND_Model.pptx by d6fmc6kwd4
4_4_WP_4_06_ND_Model.pptx4_4_WP_4_06_ND_Model.pptx
4_4_WP_4_06_ND_Model.pptx
d6fmc6kwd47 views
GDG Cloud Community Day 2022 - Managing data quality in Machine Learning by SARADINDU SENGUPTA
GDG Cloud Community Day 2022 -  Managing data quality in Machine LearningGDG Cloud Community Day 2022 -  Managing data quality in Machine Learning
GDG Cloud Community Day 2022 - Managing data quality in Machine Learning
Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language... by patiladiti752
Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language...Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language...
Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language...
patiladiti7528 views
DGST Methodology Presentation.pdf by maddierlegum
DGST Methodology Presentation.pdfDGST Methodology Presentation.pdf
DGST Methodology Presentation.pdf
maddierlegum7 views
OPPOTUS - Malaysians on Malaysia 3Q2023.pdf by Oppotus
OPPOTUS - Malaysians on Malaysia 3Q2023.pdfOPPOTUS - Malaysians on Malaysia 3Q2023.pdf
OPPOTUS - Malaysians on Malaysia 3Q2023.pdf
Oppotus31 views

Automatic classification of students in online courses using machine learning techniques

  • 1. Automatic classification of students in online courses using machine learning techniques D. Monllao Olive School of Computer Science and Software Engineering The University of Western Australia Crawley WA 6009, AUSTRALIA Principal supervisor: Dr Du Huynh Co-supervisor: Assoc/Prof Mark Reynolds External supervisor: Dr Martin Dougiamas Master of Philosophy - part time student
  • 2. Contents 1. Problem description 1.1. Online education and Moodle 1.2. Detection of students at risk of dropping out of courses 1.3. Students engagement in online courses 2. Literature review 3. Aim 4. Progress 5. Methodology 6. Timeline 1
  • 3. Online education and Moodle ● Traditional education better for social interactions ● Online education offers more flexibility but relies more in self discipline ● Moodle stats (May 2017 https://moodle.net/stats/) ○ 103 million users worldwide ○ 12 million courses ○ 215 million forum posts ○ 589 million quiz questions https://moodle.org/logo/ 2
  • 4. Students at risk of dropping out of courses ● Students that are not engaged in the course don’t participate ● Different stakeholders interested in reducing online courses drop out rates ○ Students ○ Teachers ○ Educational institutions https://www.thehrdigest.com/wp-content/upload s/2016/03/college-degree.jpg 3
  • 5. Students’ engagement in online courses ● Engaged students participate in the course activities ● Engagement is not as easy to detect in online courses as in face-to-face education ● Some examples of engagement indicators: ○ Regular accesses to the course ○ Replies to other course participants’ forum posts ○ Quick reply to teacher’s feedback ○ Percentage of accessed course resources https://userscontent2.emaze.com/images/f5e5 a8b9-e038-4620-b54f-b935902facd9/cdeb1f7 d712f03715e4b0aed325ee967.jpg 4
  • 6. 1. From an educational point of view ● Description of online students’ engagement indicators [1] [2] ● Factor analysis and correlations between indicators and students retention ● Limitations: ○ Not very empirically rigorous ○ Limited studied dataset, results biased to a few courses ○ Indicators correlate individually Literature review - Learning analytics 5 [1] Katrina A. Meyer. Student engagement in online learning: What works and why. ASHE Higher Education Report, 40(6):1–114, 2014. [2] Kate S. Hone and Ghada R. El Said. Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98:157–168, 2016.
  • 7. Literature review - Educational data mining 6 [3] Carlos Marquez-Vera, Alberto Cano, Cristobal Romero, Amin Yousef Mohammad Noaman, Habib Mousa Fardoun, and Sebastian Ventura. Early dropout prediction using data mining: a case study with high school students. Expert Systems, 33(1):107–124, 2016. EXSY-Dec-13-227.R3. [4] J. M. Luna, C. Castro, and C. Romero. Mdm tool: A data mining framework integrated into moodle. Computer Applications in Engineering Education, 25(1):90–102, 2017 2. From a data mining point of view: ● Some recent studies using machine learning techniques like Decision Trees, Association rules or Evolutionary algorithms or [3] [4] ● Limitations: ○ Limited studied dataset ○ Basic student engagement indicators
  • 8. Aim To find the model that better predicts students at risk of dropping out of any ongoing Moodle course. 7
  • 9. Aim - How to achieve it? ● By using multiple and different institutions’ datasets ○ To prevent the model to be overfit to courses of a particular institution or format ● By selecting a subset of the literature student engagement indicators ○ To discard indicators that don’t correlate well ● By adding course information to the training dataset ○ To make the model adaptable to all sort of courses ● By limiting the studied activity logs to the most relevant time range ○ To improve the model accuracy 8
  • 10. Progress ● Student engagement indicators literature review ● Moodle analytics API developed (https://github.com/moodlehq/moodle-tool_inspire) ○ Machine learning backend plugins. Shipped with Python (Tensorflow) and PHP (php-ml) ○ Very extendable ○ Prototype: http://prototype.moodle.net/inspirephase1/ ○ Experimental model included: Students at risk of dropping out ● Contributions to the most popular PHP machine learning library ○ https://github.com/php-ai/php-ml/graphs/contributors http://php-ml.readthedoc s.io/en/latest/assets/php -ml-logo.png 9
  • 11. Progress - Analytics API - Data flow chart 10
  • 12. Progress - Prototype - Models list 11
  • 13. Progress - Prototype - Predictions list 12
  • 14. Progress - Prototype - Prediction details 13
  • 15. Methodology - Overview 1. Training dataset preparation from raw Moodle sites data ○ One sample for each student enrolment in each course of each Moodle site ○ Features: Student engagement indicators calculations, course information and information about the included activity logs time range ○ Label: Did the student drop out of the course? ○ Output: A .csv file 14
  • 16. Methodology - Overview 2. Machine learning training and performance evaluation a. Inputs: A .csv file b. Cross-validation (hyper parameters tuning) c. Prediction model performance evaluation ■ The process is repeated multiple times d. Outputs: The average accuracy (Matthews correlation coefficient) and the standard deviation of all performance evaluations 15
  • 17. Methodology - Parameters ● Repeat the described process with different parameters: a. Using different subsets of student engagement indicators b. Adding more course information when required c. Limiting the student activity logs that are used d. Using different machine learning algorithms ■ e.g. Neural networks, Support vector machines, Random forests... 16
  • 18. Timeline 17 Task / Milestone Date Training courses and literature review September 2016 - May 2017 Thesis proposal seminar and proposal submission to the Graduate Research School at UWA May 2017 Learning Analytics and Educational Data Mining survey June 2017 - December 2017 Paper describing the analytics framework developed and used for this research November 2017 - July 2018 Paper detailing different combinations of parameters and results July 2018 - December 2019 Limit to nominate examiners and thesis submission August 2020 - September 2020