The document analyzes data from a MOOC offered by TU/e to understand patterns in student behavior and success. It finds: 1. Successful students, especially those who earn a distinction or are in the signature track, do not necessarily study in a structured sequential way, suggesting they enter with significant prior knowledge. 2. Studying very sequentially does not guarantee success, as some failed students watched all lectures and weeks in order. 3. Signature track students may aim for a certificate while non-signature track students aim more for knowledge, as signature students watched more videos on average.

1. What is a successful
learning process in a
MOOC?
Wil van der Aalst, Joos Buijs, Hanneke
Duisterwinkel, Maikel Leemans

2. TU/e @ MOOC’s
• Partner of Coursera since 2013
• Showing the excellence of TU/e
• Emphasize niche
• International brand awareness
• Partnership with Eurotech
• Experimenting with new way’s of teaching to
improve our on-campus education
PAGE 126-5-2015

3. Projectteam MOOCs
Professor
Information
Expertise Center
Communication
Expertise Center
Video production
Educational
support
ICT Services Data expertise
PHD
Projectleader
Support
PAGE 226-5-2015

4. 3
Statistics (first run)
• 42.146 enrollments
• 821 signature tracks
• 178 different countries
• 671.927 videos watched
• 36.751 quizzes submitted
• 4,415 forum post made
Statistics
• 16.091 enrollments
• 164 signature tracks
• 153 different countries
• 204.843 videos watched
• 12.083 quizzes
submitted
• 1.031 forum post made

5. New MOOC projects:
• Eurotech MOOC
• One MOOC on Energy Efficient Building and
Communities developed by: DTU, TUM, EPFL and TU/e
• EIT ICT First Year online master Embedded Systems
• Six specializations in total
• Two specialization from TU/e
PAGE 426-5-2015

6. What is Process Mining?
PAGE 526-5-2015

7. Process Mining on MOOCs
PAGE 626-5-2015

8. Our ‘process model’: structure!
Nov 12 Nov 19 Nov 26 Dec 3 Dec 10 Dec 17 Dec 24 Dec 31 Jan 7
Week 1
Week 2
Week 3
Week 4
Week 5
Week 6
Final Quiz
Tool Quiz
Peer Ass.
Peer Review
PAGE 726-5-2015
A week:
~8 videos of ~15 min. each
+ a week quiz
For ‘distinction’

9. General Statistics
PAGE 826-5-2015
43218
29209
20868
5845 5798
1688
REGISTERED VISITED WATCHED A
LECTURE
BROWSED FORUMS SUBMITTED
EXERCISE
OBTAINED
CERTIFICATE
68% 71% 28%
29%

10. Registration behavior
PAGE 926-5-2015

11. Annotate students
• Achievement:
• Failed
• Successful
− Normal
− Distinction
• Signature track:
• Yes
• No
• Lectures watched
• Week 1 only
• Week 1&2 only
• Week 1,2 and 3
• All weeks
• Any (e.g. at least 1)
PAGE 1026-5-2015

12. Basic Group Statistics
Signature
Track?
Certificate #Students #Watched
Videos
Avg. per
student
Yes Failed 226 8,069 36
Normal 223 15,367 69
Distinction 285 21,392 75
No Failed 13,465 143,721 11
Normal 702 41,467 59
Distinction 293 19,203 66
PAGE 1126-5-2015
Heterogeneous group:
- 10,996 watched lecture from week 1
(3,451 week 1+2, 1,407 1+2+3)
- Only 269 failed students watched a video
of each week

13. Signature Track vs Non-Sign.
PAGE 1226-5-2015
Certificate Weeks
Watched
#Students Avg. videos
per student
Sign. Non-S Sign. Non-S
failed all weeks 39 269 79 69
failed any 226 13,465 36 11
successful all weeks 354 628 92 83
successful any 508 995 72 61
normal all weeks 158 427 87 82
normal any 223 702 69 59
distinction all weeks 196 201 97 86
distinction any 285 293 75 66
Signature track
always watches
more videos than
non-signature.
Failed students
watch many, but
less videos than
successful
students

14. PAGE 1326-5-2015Behavior Over Time
Late
Starters
Christmas
Delay for
‘yellow’

15. PAGE 1426-5-2015
DistinctionNormal
Signature
Non-Signature
Failed
~Previous
Slide
Seem to
give up
More intense
watching of
weeks 5 & 6
Follow
schedule

16. Do Successful Students Study Sequentially?
PAGE 1626-5-2015
Which group studies
most ‘sequentially’?
Signature yes/no?
Grading failed/normal/distinction?
Watched weeks 1-2-3/all/any?
signature grading weeks % match
no failed all weeks 0.660
no normal all weeks 0.634
no distinction all weeks 0.624
yes normal all weeks 0.593
no distinction any 0.592
no normal any 0.591
yes failed all weeks 0.573
yes normal any 0.571
no failed week 1,2,3 0.548
yes distinction all weeks 0.547
yes distinction any 0.534
no failed week 1,2 0.466

17. Behavior of Non-Sign. Failed Students
that Watched all Weeks
PAGE 1726-5-2015
Week 1
Week 6
Each lecture watched 10-
20 times by 269 students
overall

18. But how do Successful Students Watch?
PAGE 1826-5-2015
(285 Students on signature track that got distinction)
Intro &
Week 1
Week 2 & 3 Week 4, 5 & 6

19. PAGE 1926-5-2015

20. PAGE 2026-5-2015
Towards deadlines of final quiz and
peer assignment watch behavior (far)
less structured
Weeks 5 & 6 very unstructured

21. Behavior Compared
PAGE 2126-5-2015
Normal Cert. Cert. with Distinction
Signature Track Sequence but every
lecture can be skipped.
Week 6 is unordered.
(223 students)
Week 2-4 structured,
Week 1 unordered
Week 6 very unstructured
and possible repetitive
(285 students)
Non-Signature Track Sequence, even week 6.
Every lecture can be
skipped.
(702 students)
Sequence, but every
lecture can be skipped.
Week 6 is very
unstructured and repetitive
(293 students)

22. Behavior Compared
PAGE 2226-5-2015
Normal Cert. Cert. with Distinction
Signature Track Rather structured. Some
lectures in different orders.
Difficult lecture watched
multiple times.
Week 6 unordered.
(158 students)
Sequence for some weeks
without skips.
Week 1 unordered.
Difficult lectures repeated.
Weeks 4 and 5
unstructured.
(196 students)
Non-Signature Track Very sequential
(427 students)
Very sequential
(201 students)
For students that watched at least 1 Lecture of each week

23. Overall Conclusion
1. Successful students do not necessarily study
structurally
• Distinction and/or signature students have significant
preliminary knowledge?!?!
2. Structured studying does not guarantee success
3. Signature track students aim for a certificate while
non-signature track students aim for knowledge?
PAGE 2326-5-2015

24. Future Analysis
• Include more information
• Quizzes (and their results)
• Analyse timing of watching and quiz making
• Forum interaction
• Compare runs of the same MOOC
• Who re-joined?
• Is the 2nd group of students different? (gut feeling: yes!)
PAGE 2426-5-2015
http://www.edsa-project.eu/

25. PAGE 2526-5-2015

26. PAGE 2626-5-2015

27. Data Structure
PAGE 2726-5-2015