Presentation at ED-Media 2015 conference, Montreal, Canada

1. S C I E N C E P A S S I O N T E C H N O L O G Y
www.tugraz.at
Impacts of Interactions in
Learning-Videos:
A Subjective
and Objective Analysis
Josef Wachtler and Martin Ebner
IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

2. 2
Impacts of Interactions in Learning-Videos: A Subjective and Objective Analysis
Content
1. Introduction
2. Subjective User Feedback
3. Objective Evaluation
4. Discussion
5. Outlook & Conclusion
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

3. 3
Impacts of Interactions in Learning-Videos: A Subjective and Objective Analysis
Graz University of Technology
IT Services – Unit Social Learning
Europe, Austria, Graz
http://www.tugraz.at
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

4. 4
Introduction
Motivation
selective attention is the most crucial resource for
human learning
supporting and managing this attention enhances
both, behavioral and neuronal performance
attention is heavily influenced by interaction and
communication in all forms and directions
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

5. 5
Introduction
Why Interactions?
missing interactions are leading to passive learners
and to a much more surficial work
avoid that learners become tired or annoyed
increase the attention and the contribution
feedback for teachers:
Is it possible for the learners to follow the content?
Is the speed appropriate?
...
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

6. 6
Introduction
Web-Application named LIVE
on-demand video or live-broadcasting
attention analysis
different methods of interaction:
automatically asked questions and captchas
asking questions to the lecturer
asking text-based questions to the attendees
multiple-choice questions at pre-defined positions
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

7. 7
Introduction
Interactions during a Video
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

8. 8
Introduction
Research Question
What is the ideal distribution of the interactions over a
learning video to reach acceptable results to the
multiple-choice questions?
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

9. 9
Subjective User Feedback
Setting
used at several lectures at Graz University of
Technology
ranging from a large freshman course in computer
science to a course in the last semester of the
bachelor program in computer science and a course
in adult education regarding cleanroom technology
students were asked to report their experiences with
the interactions
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

10. 10
Subjective User Feedback
Reported Feedback
questions with a content, which is related to the topic
of the video are highly appreciated
general questions are disturbing because they are
seeming to be useless
a maximum of ten interactions per hour is acceptable
interactions should be spread evenly across the video
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

11. 11
Objective Evaluation
Setting
used at the lecture “Logic and Computability”
beginning with the third video the number of students
who watched most of the video (more than 75%) is
quite high
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

12. 12
Objective Evaluation
Setting
the evaluation of the distribution of the interactions
and their impacts to the multiple-choice questions
shows a very similar outcome at each video
results are shown only for the third video which holds
as a representative for the remaining five ones
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

13. 13
Objective Evaluation
MC-Questions answered
starts decreasing at the fifth question
not every student watched the full video and due to
that they never reached the later questions
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

14. 14
Objective Evaluation
MC-Questions correct/false
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

15. 15
Objective Evaluation
Issues
three issues could be seen if we consider the
distribution of the questions as the influencing factor
of the success rate
Lazy Start: the number of correct answers to the
first question is not very high
Correct after Question Pause: correct answers
are numerous at the third question despite the
fact that the timespan since the last question is
quite high
Tight-Placed Errors: if questions are placed very
tight the number of correct answers is decreasing
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

16. 16
Objective Evaluation
Occurrence of the Random Interactions
help to explain the issues
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

17. 17
Discussion
Lazy Start
it was not necessary to answer a content related
question until the occurrence of this question
the last random interaction could be happened long
ago
contradiction: previous research studies always
indicate that the attention is decreasing along the
duration of the video
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

18. 18
Discussion
Correct after Question Pause
the timespan since the last random question is
smaller
randomly occurring general questions are probably
leading to a better performance at the multiple-choice
questions by helping to overcome longer periods of
no interactivity
contradiction to the subjective feedback
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

19. 19
Discussion
Tight-Placed Errors
a placement of the questions with minor space is
followed by a low performance at the questions
confirmed by the subjective feedback
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

20. 20
Discussion
How to Place Interactions in Learning Videos
max. number of interactions per hour is ten
content related questions are important for the
satisfaction of the students
general questions seem to be useful to support the
attention between content related questions (one per
green slots)
the space between content related questions should
not be too small
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

21. 21
Outlook & Conclusion
Outlook
issues are based on observations gathered at six
videos
it is required to fully prove the accuracy of these
observations mathematically
videos with different distributions of interactions will
be generated
study will start in August 2015
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

22. 22
Outlook & Conclusion
Conclusion
analysis of the impacts of different forms of
interactions in learning videos
three issues:
Lazy Start
Correct after Question Pause
Tight-Placed Errors
some recommendations of how to place
multiple-choice questions in videos based on
observed trends
further research is planned to prove the issues and
trends
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015

23. 23
Thank you ...
... for your attention!
Questions?
Josef Wachtler, josef.wachtler@tugraz.at
Martin Ebner, martin.ebner@tugraz.at
Graz University of Technology
IT Services – Unit Social Learning
M¨unzgrabenstraße 35A, A-8010 Graz
http://elearningblog.tugraz.at
Josef Wachtler and Martin Ebner , IT Services - Unit Social Learning - TU Graz
EDMEDIA 2015