1. The document discusses an experiment using interactive programming tasks during learning videos in a MOOC course on Python basics to test their impact on learning effectiveness. 2. 577 learners participated in the course and were split into two groups - one with programming overlays and one without. Quiz results showed the group with overlays performed better, indicating the overlays could influence learning. 3. Data from the programming tasks and interactive platform provided additional insights. Task results showed some concepts were challenging and attention to accuracy is important. Statistics on learner behavior can help optimize task placement and video content.

1. SCIENCE
PASSION
TECHNOLOGY
The use of programming tasks in interactive videos to
increase learning effectiveness at MOOCs
Michael Weiß
23. November 2023
1

2. Introduction
• Motivation
§ Emergence of videos
§ Importance of interactivity in videos
§ Bachelor Thesis: Game-based learning with Alexa
Ø iMooX course with programming overlays during learning videos
2

3. Research Question
Can interactive programming tasks during learning videos support
learning efficiency and success?
• How to develop interactive programming tasks during learning videos for an
interactive video platform?
• What conclusions can be drawn from user behaviour for the optimization of the
course and interactive elements?
3

4. Content
1. Interactive video course (MOOC)
2. Programming task overlays
3. Evaluation & Outcome of course results
4

5. Content
1. Interactive video course (MOOC)
2. Programming task overlays
3. Evaluation & Outcome of course results
5

6. 1. Interactive video course (MOOC)
• iMooX Course – Python basics
§ Part 1: Programming & Basics
§ Part 2: Python Basics
§ Part 3: Input / Output of data & Control flow
§ Part 4: Lists & List functions
• Two test groups
§ Group 1: 22. March - 15. May -> watched with overlays
§ Group 2: 16. May - 14. August -> watched without overlays
• 577 learners
6

7. Course procedure
• Beginning of the course
§ Self-evaluation questionnaire
• For each part of the course
§ Self-assessment quiz before the part (5 MC-Questions)
§ Video
§ Group 1: With programming overlays
§ Group 2: Without programming overlays
§ Final quiz ( 5 MC-Questions)
• Exception: Part 1 – no programming overlays
7

8. Additional course material
• Installation instructions
• Cheat Sheet
• Additional homework
8
Figure: Cheat sheet
Figure: Installation Slides

9. Content
1. Interactive video course (MOOC)
2. Programming task overlays
3. Evaluation & Outcome of course results
9

10. 2. Programming task overalys
• Limitation with iMooX -> use LIVE
• Communication between iMooX, LIVE
and Youtube
• LIVE
§ Django – Python Webframework
§ Interaction: Programming Task
10
Figure: High-level architecture of LIVE

11. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Teacher area to display results
§ Learner area to display results
§ Download of programming tasks metadata, statistics and answers
11

12. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Teacher area to display results
§ Learner area to display results
§ Download of programming tasks metadata, statistics and answers
12

13. Plan & Create a programming task
• Teacher area
• Choose timestamp for task display
• Insert description, answer, reference output
13

14. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Teacher area to display results
§ Learner area to display results
§ Download of programming tasks metadata, statistics and answers
14

15. Display of tasks during videos
• Part 2:
§ multiplication and
variable assignment
• Part 3:
§ Simple division and
If-else statement
• Part 4:
§ List creation and
iteration
15

16. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Teacher area to display results
§ Learner area to display results
§ Download of programming tasks metadata, statistics and answers
16

17. Safe execution and evaluation of code answers
• Restricted Python Package (malicious code)
§ Define a subset of the python language
§ E.g. block import of modules, allow only certain elements
§ Execution time limit of 2 seconds
§ Execute not trusted code inside a trusted environment
17

18. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Teacher area to display results
§ Learner area to display results
§ Download of programming tasks metadata, statistics and answers
18

19. Teacher area
• Taskdisplay
• Answer list
§ Correction option
§ Teacher Note
19

20. Interaction: Programming tasks
• Functionalities
§ Planning, Creating and editing tasks
§ Display tasks during videos
§ Safe execution and evaluation of code answers
§ Answer overview & correction options for teachers
§ Answer overview for learners
§ Download of programming tasks metadata, statistics and answers
20

21. Learner area
• Taskdisplay
• Answer list
21

22. Content
1. Interactive video course (MOOC)
2. Programming task overlays
3. Evaluation & Outcome of course results
22

23. 3. Evaluation & Outcome
1. iMooX data
• Quiz results
2. LIVE data
• Programming task results
• User statistics
§ By Daniel Dohr (Visualization of students’ behavior in interactive learning videos)
23

24. 3. Evaluation & Outcome
1. iMooX data
• Quiz results
2. LIVE data
• Programming task results
• User statistics
§ By Daniel Dohr (Visualization of students’ behavior in interactive learning videos)
24

25. iMooX Data – Quiz results
25
Part 1 – Theory: No overlays
Part 2 – Python basics
Part 3 – Input / Output Control flow
Part 4 – List / List functions
G1 – watched with overlays
G2 – watched without overlays

26. Quiz results – conclusion
• Group 1 performed better than Group 2 on final quizzes
ü Programming overlays could influence learning efficiency
§ Limitation: Smaller test group 2 limits expressiveness of results
• Both groups improved from the initial assessments to final quizzes
ü Success of the course
26

27. 3. Evaluation & Outcome
1. iMooX data
§ Quiz results
2. LIVE data
§ Programming task results
§ User statistics
§ By Daniel Dohr (Visualization of students’ behavior in interactive learning videos)
27

28. 3. Evaluation & Outcome
1. iMooX data
§ Quiz results
2. LIVE data
§ Programming task results
§ User statistics
§ By Daniel Dohr (Visualization of students’ behavior in interactive learning videos)
28

29. Programming task results
• Task 1
§ mixing of addition
and multiplication
• Task 2 & Task 3
§ Inaccuracy & failure to
understand the topic
• Conclusion
• Simple mathematical operations must not be considered as pre-existing
knowledge
• importance of working accurately
29
61 %
Correct %
34 %
45 %

30. 3. Evaluation & Outcome
1. iMooX data
§ Quiz results
2. LIVE data
§ Programming task results
§ User statistics
§ By Daniel Dohr (Visualization of students’ behavior in interactive learning videos)
30

31. LIVE data
• User statistics during learning videos
§ Answer Delay
§ Attention Change
§ Drop ratio
31

32. LIVE data
• User statistics during learning videos
§ Answer Delay
§ Attention Change
§ Drop ratio
32

33. Answer Delay
• Difficulty levels of tasks
• Assumption
§ Task in Part 3 too complex
33

34. LIVE data
• User statistics during learning videos
§ Answer Delay
§ Attention Change
§ Drop ratio
34

35. Attention Change
• Attention value
• Assumptions
§ Placement of
interactions
35

36. LIVE data
• User statistics during learning videos
§ Answer Delay
§ Attention Change
§ Drop ratio
36

37. Drop ratio
• Sharp drops in
part 1 & 2
• Constant numbers in
part 3 & 4
• Assumptions
§ Lost interest
§ Get taste of course
§ Too complex
§ Improve videos
37

38. Conclusion
• Success of the course
• Programming tasks as a valuable addition
• Further research
§ Equal sized test group
§ Special characteristics of programming tasks
38

39. Thank you for your attention!
39