SlideShare a Scribd company logo
Learning analytics dashboard for a Programming course
0,0 %
10,0 %
20,0 %
40,0 %
50,0 %
30,0 %
Failed
<5.5
Passed
<5.5 <9.0
Well passed
<9.0 <=10.0
Percent
Result programming N=1217
Group statistics
Cohort n= 684 Online yes/no N Mean Std. Deviation Std. Error Mean
2014 Result
0 453 4,785 3,4785 ,1634
1 231 6,833 2,6778 ,1762
•	 Many freshman computer science students failed the essential 10 week
Java programming course (42%) (figure 1).
•	 This course is supported by e-learning systems.
•	 Students who performed all their task in the e-learning system were
more successful. (table 1)
Figure 1: Result of Java programming course N=1217
Table 1: Difference in result programming Yes/no all online tasks. (t(682)= -7,84 p <.000)
Developing a dashboard application for giving feedback on tasks in e-
learning systems.
The aim is to increase the amount of students passing by giving feedback
on their online behavior.
•	 The activities of the students in e-learning environments tend to be pre-
dictable for results on the course (Hu, Lo, & Shih, 2014; Tempelaar,
Rien- ties, & Giesbers, 2014)
•	 Courses using early alert systems for at-risk students have a 6% higher
passing rate (Jayaprakash, Moody, Eitel, Regan, & Baron, 2014)
•	 User interface of dashboard is important. Graphical elements give more
user satisfaction (Ali, Asadi, Gašević, Jovanović, & Hatala, 2013)
•	 Computer-assisted instructional feedback has an effect size of 0,52 and
feedback on the task level is effective (John Hattie & Timperley, 2007)
The dashboard (figure 2) displays the progress in the e-learning systems, the expected result and com-
pared this to the performance of the total cohort (n=556).
Figure 2: Dashboard of student week 1
In Figure 3 the creation of the 8 dashboards is visualized. Every week the dashboards of the total cohort
(n=556) were generated.
•	 Moodle: online data; quiz and practical assignment results.
•	 MyProgrammingLab: Mastery score and total attempts.
•	 For the expected result, linear regression models are used, and for the risk of failure decision tree al-
gorithms (Decision Stump, Adaboost)(Hu et al., 2014) are used. These models were created through
WEKA 3.6.
Figure 3: Visualization of Dashboard generation
•	 The experiment was set up like a RCT (figure 4) (Randomized Controlled Trial) (Murnane & Willett,
2010).
•	 Opt-out: Students could chose for an opt-out. Only two students chose for an opt-out.
•	 556 students were incorporated in the experiment.
•	 The students were conditionally randomized. Every week 276 students in the treatment group re-
ceived an e-mail with a personalized link to their dashboard.
•	 The effect of the use of the dashboard is determined by the difference in the exam results of the
treatment group and the control group (figure 4).
Figure 4: Setup of the randomized controlled experiment
•	 Q1: Do students receiving a dashboard have a higher passing rate?
•	 Q2: Do students receiving a dashboard have a better result?
•	 Q3: Do students receiving a dashboard make more use of the e-learning systems?
1. Tempelaar, D. T., Rienties, B., & Giesbers, B. (2014). In search for the most informative data for feedback generation: Learning analytics in a data-rich con- text. Computers in Human Behavior. http://doi.org/10.1016/j.chb.2014.05.038
2. Hu, Y.-H., Lo, C.-L., & Shih, S.-P. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behavior, 36(0), 469–478. http://doi.org/http://dx.doi.org/10.1016/j.chb.2014.04.002
3. Jayaprakash, S. M., Moody, E. W., Eitel, J. M., Regan, J. R., & Baron, J. D. (2014). Early Alert of Academically At-Risk Students : An Open Source Analytics Ini- tiative. Journal of Learning Analytics, 1, 6–47.
4. Murnane, R. J., & Willett, J. B. (2010). Methods matter: Improving causal inference in educational and social science research. Oxford University Press.
Introduction
Objectives
Related research
Dashboard Method
Research questions
Test scores
References
Table 2: Difference in passing rate control vs. treatment group (X2 = ,319a p= ,572)
First Year
computer
science student
Objection against
experiment
Takes part of
the
experiment
Doesn’͛ t take
part of
experiment
Conditionaly
randomize
Assign to
treatment
group
Assign to
control group
No
Yes
Result
programming
Result
programming
end
Receives every
week a
dashboard
˄ Result of
intervention
0
50
100
200
150
No dashboard Dashboard
Count
Bar Chart
Passed
Failed
Passed
Participation
Improve the success rate of a Java programming by giving online feedback on the progress of e-learning tasks
Figure 5: Result exam passing rate of control and treatment group
Passed
Total
Failed Passed
Participation
No dashboard
Count 162 118 280
% within participation 57,9% 42,1% 100%
Dashboard
Count 165 109 274
% within participation 60,2% 39,8% 100%
Total
Count 327 227 554
% within participation 59,0% 41,0% 100%
Week 0 Week1
Dashboard
1
Demographic
data
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
1
week2
Dashboard
2
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
2
week3
Dashboard
3
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
3
week4
Dashboard
4
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
4
week5
Dashboard
5
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
5
week6
Dashboard
6
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
6
week7
Dashboard
7
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
6
week8
Dashboard
8
Myproglab
#Attemps
Mastery
Moodle
Quiz
result
Pracitcal
Assignme
nt
Predic
tion
Model
8
Jan Hellings. E-mail: j.f.hellings@hva.nl

More Related Content

What's hot

IRJET- Online Examination System
IRJET- Online Examination SystemIRJET- Online Examination System
IRJET- Online Examination System
IRJET Journal
 
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
TELKOMNIKA JOURNAL
 
Evaluation In Educational Technology1
Evaluation In Educational Technology1Evaluation In Educational Technology1
Evaluation In Educational Technology1
Zainab Al-shidhani
 
ICELW Conference Slides
ICELW Conference SlidesICELW Conference Slides
ICELW Conference Slides
toolboc
 
Mehrnoosh vahdat workshop-data sharing 2014
Mehrnoosh vahdat  workshop-data sharing 2014Mehrnoosh vahdat  workshop-data sharing 2014
Mehrnoosh vahdat workshop-data sharing 2014
MehrnooshV
 
NTeQ Lesson
NTeQ LessonNTeQ Lesson
NTeQ Lesson
Carmen Weaver
 
193
193193
Template abstrak & full paper seminar 50 thn
Template abstrak & full paper seminar 50 thnTemplate abstrak & full paper seminar 50 thn
Template abstrak & full paper seminar 50 thn
TeKno (Technology and Knowledge)
 
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
Janet Van Heck
 
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
Marko Teräs
 
Saving Money or Adding Value?
Saving Money or Adding Value?Saving Money or Adding Value?
Saving Money or Adding Value?
Mark Brown
 
Learning Analytics and student feedback
Learning Analytics and student feedbackLearning Analytics and student feedback
Learning Analytics and student feedback
Denise Whitelock
 
Dmw uoc may 2013
Dmw uoc may 2013Dmw uoc may 2013
Dmw uoc may 2013
cgirona
 
Problem Solution Presentation
Problem Solution PresentationProblem Solution Presentation
Problem Solution Presentation
raech101
 
Data Collection
Data CollectionData Collection
Data Collection
FDLRS
 
IRJET- Clustering Students through Data Mining and Gamified Learning
IRJET- Clustering Students through Data Mining and Gamified LearningIRJET- Clustering Students through Data Mining and Gamified Learning
IRJET- Clustering Students through Data Mining and Gamified Learning
IRJET Journal
 
Classroom Performance Systems
Classroom Performance SystemsClassroom Performance Systems
Classroom Performance Systems
guest67912
 
Research Paper on Virtual Manipulatives In the Math Classroom
Research Paper on Virtual Manipulatives In the Math ClassroomResearch Paper on Virtual Manipulatives In the Math Classroom
Research Paper on Virtual Manipulatives In the Math Classroom
smscheurer
 
Final presentation
Final presentationFinal presentation
Final presentation
Anna Harutyunyan
 
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
Biswaranjan Samal
 

What's hot (20)

IRJET- Online Examination System
IRJET- Online Examination SystemIRJET- Online Examination System
IRJET- Online Examination System
 
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
Predicting the Presence of Learning Motivation in Electronic Learning: A New ...
 
Evaluation In Educational Technology1
Evaluation In Educational Technology1Evaluation In Educational Technology1
Evaluation In Educational Technology1
 
ICELW Conference Slides
ICELW Conference SlidesICELW Conference Slides
ICELW Conference Slides
 
Mehrnoosh vahdat workshop-data sharing 2014
Mehrnoosh vahdat  workshop-data sharing 2014Mehrnoosh vahdat  workshop-data sharing 2014
Mehrnoosh vahdat workshop-data sharing 2014
 
NTeQ Lesson
NTeQ LessonNTeQ Lesson
NTeQ Lesson
 
193
193193
193
 
Template abstrak & full paper seminar 50 thn
Template abstrak & full paper seminar 50 thnTemplate abstrak & full paper seminar 50 thn
Template abstrak & full paper seminar 50 thn
 
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
Using Interactive Whiteboard Technology to Implement the Concrete-Representat...
 
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
WCOL2019: "What can learning analytics do for me?" Students' and teachers' pe...
 
Saving Money or Adding Value?
Saving Money or Adding Value?Saving Money or Adding Value?
Saving Money or Adding Value?
 
Learning Analytics and student feedback
Learning Analytics and student feedbackLearning Analytics and student feedback
Learning Analytics and student feedback
 
Dmw uoc may 2013
Dmw uoc may 2013Dmw uoc may 2013
Dmw uoc may 2013
 
Problem Solution Presentation
Problem Solution PresentationProblem Solution Presentation
Problem Solution Presentation
 
Data Collection
Data CollectionData Collection
Data Collection
 
IRJET- Clustering Students through Data Mining and Gamified Learning
IRJET- Clustering Students through Data Mining and Gamified LearningIRJET- Clustering Students through Data Mining and Gamified Learning
IRJET- Clustering Students through Data Mining and Gamified Learning
 
Classroom Performance Systems
Classroom Performance SystemsClassroom Performance Systems
Classroom Performance Systems
 
Research Paper on Virtual Manipulatives In the Math Classroom
Research Paper on Virtual Manipulatives In the Math ClassroomResearch Paper on Virtual Manipulatives In the Math Classroom
Research Paper on Virtual Manipulatives In the Math Classroom
 
Final presentation
Final presentationFinal presentation
Final presentation
 
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
Performance Analysis of Supervised Machine Learning Techniques for Sentiment ...
 

Similar to 20151112PosterJanHellingsV2_definitief

IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET Journal
 
final
finalfinal
IRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine LearningIRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET Journal
 
B05110409
B05110409B05110409
B05110409
IOSR-JEN
 
Investigating learning strategies in a dispositional learning analytics conte...
Investigating learning strategies in a dispositional learning analytics conte...Investigating learning strategies in a dispositional learning analytics conte...
Investigating learning strategies in a dispositional learning analytics conte...
Bart Rienties
 
Data Mining Techniques for School Failure and Dropout System
Data Mining Techniques for School Failure and Dropout SystemData Mining Techniques for School Failure and Dropout System
Data Mining Techniques for School Failure and Dropout System
Kumar Goud
 
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
indexPub
 
Educational Data Mining to Analyze Students Performance – Concept Plan
Educational Data Mining to Analyze Students Performance – Concept PlanEducational Data Mining to Analyze Students Performance – Concept Plan
Educational Data Mining to Analyze Students Performance – Concept Plan
IRJET Journal
 
C04622028
C04622028C04622028
C04622028
IOSR-JEN
 
Intelligent system for sTudent placement
Intelligent system for sTudent placementIntelligent system for sTudent placement
Intelligent system for sTudent placement
Femmy Johnson
 
IRJET- Using Data Mining to Predict Students Performance
IRJET-  	  Using Data Mining to Predict Students PerformanceIRJET-  	  Using Data Mining to Predict Students Performance
IRJET- Using Data Mining to Predict Students Performance
IRJET Journal
 
An Artificial Intelligence-Based Distance Education System Artimat
An Artificial Intelligence-Based Distance Education System  ArtimatAn Artificial Intelligence-Based Distance Education System  Artimat
An Artificial Intelligence-Based Distance Education System Artimat
Nicole Heredia
 
Effects of e instruction in Teaching Intro to IT
Effects of e instruction in Teaching Intro to ITEffects of e instruction in Teaching Intro to IT
Effects of e instruction in Teaching Intro to IT
Amelita Martinez
 
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
IRJET Journal
 
An Expert System For Improving Web-Based Problem-Solving Ability Of Students
An Expert System For Improving Web-Based Problem-Solving Ability Of StudentsAn Expert System For Improving Web-Based Problem-Solving Ability Of Students
An Expert System For Improving Web-Based Problem-Solving Ability Of Students
Jennifer Roman
 
Student Performance Evaluation in Education Sector Using Prediction and Clust...
Student Performance Evaluation in Education Sector Using Prediction and Clust...Student Performance Evaluation in Education Sector Using Prediction and Clust...
Student Performance Evaluation in Education Sector Using Prediction and Clust...
IJSRD
 
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
IJECEIAES
 
EDM_IJTIR_Article_201504020
EDM_IJTIR_Article_201504020EDM_IJTIR_Article_201504020
EDM_IJTIR_Article_201504020
Ritika Saxena
 
A Survey on Research work in Educational Data Mining
A Survey on Research work in Educational Data MiningA Survey on Research work in Educational Data Mining
A Survey on Research work in Educational Data Mining
iosrjce
 
G017224349
G017224349G017224349
G017224349
IOSR Journals
 

Similar to 20151112PosterJanHellingsV2_definitief (20)

IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
 
final
finalfinal
final
 
IRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine LearningIRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine Learning
 
B05110409
B05110409B05110409
B05110409
 
Investigating learning strategies in a dispositional learning analytics conte...
Investigating learning strategies in a dispositional learning analytics conte...Investigating learning strategies in a dispositional learning analytics conte...
Investigating learning strategies in a dispositional learning analytics conte...
 
Data Mining Techniques for School Failure and Dropout System
Data Mining Techniques for School Failure and Dropout SystemData Mining Techniques for School Failure and Dropout System
Data Mining Techniques for School Failure and Dropout System
 
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
ANALYSIS OF STUDENT ACADEMIC PERFORMANCE USING MACHINE LEARNING ALGORITHMS:– ...
 
Educational Data Mining to Analyze Students Performance – Concept Plan
Educational Data Mining to Analyze Students Performance – Concept PlanEducational Data Mining to Analyze Students Performance – Concept Plan
Educational Data Mining to Analyze Students Performance – Concept Plan
 
C04622028
C04622028C04622028
C04622028
 
Intelligent system for sTudent placement
Intelligent system for sTudent placementIntelligent system for sTudent placement
Intelligent system for sTudent placement
 
IRJET- Using Data Mining to Predict Students Performance
IRJET-  	  Using Data Mining to Predict Students PerformanceIRJET-  	  Using Data Mining to Predict Students Performance
IRJET- Using Data Mining to Predict Students Performance
 
An Artificial Intelligence-Based Distance Education System Artimat
An Artificial Intelligence-Based Distance Education System  ArtimatAn Artificial Intelligence-Based Distance Education System  Artimat
An Artificial Intelligence-Based Distance Education System Artimat
 
Effects of e instruction in Teaching Intro to IT
Effects of e instruction in Teaching Intro to ITEffects of e instruction in Teaching Intro to IT
Effects of e instruction in Teaching Intro to IT
 
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
M-Learners Performance Using Intelligence and Adaptive E-Learning Classify th...
 
An Expert System For Improving Web-Based Problem-Solving Ability Of Students
An Expert System For Improving Web-Based Problem-Solving Ability Of StudentsAn Expert System For Improving Web-Based Problem-Solving Ability Of Students
An Expert System For Improving Web-Based Problem-Solving Ability Of Students
 
Student Performance Evaluation in Education Sector Using Prediction and Clust...
Student Performance Evaluation in Education Sector Using Prediction and Clust...Student Performance Evaluation in Education Sector Using Prediction and Clust...
Student Performance Evaluation in Education Sector Using Prediction and Clust...
 
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
 
EDM_IJTIR_Article_201504020
EDM_IJTIR_Article_201504020EDM_IJTIR_Article_201504020
EDM_IJTIR_Article_201504020
 
A Survey on Research work in Educational Data Mining
A Survey on Research work in Educational Data MiningA Survey on Research work in Educational Data Mining
A Survey on Research work in Educational Data Mining
 
G017224349
G017224349G017224349
G017224349
 

20151112PosterJanHellingsV2_definitief

  • 1. Learning analytics dashboard for a Programming course 0,0 % 10,0 % 20,0 % 40,0 % 50,0 % 30,0 % Failed <5.5 Passed <5.5 <9.0 Well passed <9.0 <=10.0 Percent Result programming N=1217 Group statistics Cohort n= 684 Online yes/no N Mean Std. Deviation Std. Error Mean 2014 Result 0 453 4,785 3,4785 ,1634 1 231 6,833 2,6778 ,1762 • Many freshman computer science students failed the essential 10 week Java programming course (42%) (figure 1). • This course is supported by e-learning systems. • Students who performed all their task in the e-learning system were more successful. (table 1) Figure 1: Result of Java programming course N=1217 Table 1: Difference in result programming Yes/no all online tasks. (t(682)= -7,84 p <.000) Developing a dashboard application for giving feedback on tasks in e- learning systems. The aim is to increase the amount of students passing by giving feedback on their online behavior. • The activities of the students in e-learning environments tend to be pre- dictable for results on the course (Hu, Lo, & Shih, 2014; Tempelaar, Rien- ties, & Giesbers, 2014) • Courses using early alert systems for at-risk students have a 6% higher passing rate (Jayaprakash, Moody, Eitel, Regan, & Baron, 2014) • User interface of dashboard is important. Graphical elements give more user satisfaction (Ali, Asadi, Gašević, Jovanović, & Hatala, 2013) • Computer-assisted instructional feedback has an effect size of 0,52 and feedback on the task level is effective (John Hattie & Timperley, 2007) The dashboard (figure 2) displays the progress in the e-learning systems, the expected result and com- pared this to the performance of the total cohort (n=556). Figure 2: Dashboard of student week 1 In Figure 3 the creation of the 8 dashboards is visualized. Every week the dashboards of the total cohort (n=556) were generated. • Moodle: online data; quiz and practical assignment results. • MyProgrammingLab: Mastery score and total attempts. • For the expected result, linear regression models are used, and for the risk of failure decision tree al- gorithms (Decision Stump, Adaboost)(Hu et al., 2014) are used. These models were created through WEKA 3.6. Figure 3: Visualization of Dashboard generation • The experiment was set up like a RCT (figure 4) (Randomized Controlled Trial) (Murnane & Willett, 2010). • Opt-out: Students could chose for an opt-out. Only two students chose for an opt-out. • 556 students were incorporated in the experiment. • The students were conditionally randomized. Every week 276 students in the treatment group re- ceived an e-mail with a personalized link to their dashboard. • The effect of the use of the dashboard is determined by the difference in the exam results of the treatment group and the control group (figure 4). Figure 4: Setup of the randomized controlled experiment • Q1: Do students receiving a dashboard have a higher passing rate? • Q2: Do students receiving a dashboard have a better result? • Q3: Do students receiving a dashboard make more use of the e-learning systems? 1. Tempelaar, D. T., Rienties, B., & Giesbers, B. (2014). In search for the most informative data for feedback generation: Learning analytics in a data-rich con- text. Computers in Human Behavior. http://doi.org/10.1016/j.chb.2014.05.038 2. Hu, Y.-H., Lo, C.-L., & Shih, S.-P. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behavior, 36(0), 469–478. http://doi.org/http://dx.doi.org/10.1016/j.chb.2014.04.002 3. Jayaprakash, S. M., Moody, E. W., Eitel, J. M., Regan, J. R., & Baron, J. D. (2014). Early Alert of Academically At-Risk Students : An Open Source Analytics Ini- tiative. Journal of Learning Analytics, 1, 6–47. 4. Murnane, R. J., & Willett, J. B. (2010). Methods matter: Improving causal inference in educational and social science research. Oxford University Press. Introduction Objectives Related research Dashboard Method Research questions Test scores References Table 2: Difference in passing rate control vs. treatment group (X2 = ,319a p= ,572) First Year computer science student Objection against experiment Takes part of the experiment Doesn’͛ t take part of experiment Conditionaly randomize Assign to treatment group Assign to control group No Yes Result programming Result programming end Receives every week a dashboard ˄ Result of intervention 0 50 100 200 150 No dashboard Dashboard Count Bar Chart Passed Failed Passed Participation Improve the success rate of a Java programming by giving online feedback on the progress of e-learning tasks Figure 5: Result exam passing rate of control and treatment group Passed Total Failed Passed Participation No dashboard Count 162 118 280 % within participation 57,9% 42,1% 100% Dashboard Count 165 109 274 % within participation 60,2% 39,8% 100% Total Count 327 227 554 % within participation 59,0% 41,0% 100% Week 0 Week1 Dashboard 1 Demographic data Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 1 week2 Dashboard 2 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 2 week3 Dashboard 3 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 3 week4 Dashboard 4 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 4 week5 Dashboard 5 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 5 week6 Dashboard 6 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 6 week7 Dashboard 7 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 6 week8 Dashboard 8 Myproglab #Attemps Mastery Moodle Quiz result Pracitcal Assignme nt Predic tion Model 8 Jan Hellings. E-mail: j.f.hellings@hva.nl