Reading subtitles across devices : A study into the differences in reading patterns of people watching subtitled videos on smartphone, tablet and computer screen

1. Reading subtitles across devices:
A study into the differences in reading patterns of people watching
subtitled videos on smartphone, tablet and computer screen
Agnieszka Szarkowska1, Monika Laskowska1, Andreu Oliver3, Olga Pilipczuk1
1 University of Warsaw
3 Universitat Autònoma de Barcelona
ICEAL
Warsaw, September 2015

2. About the study
 Reading across devices
– part of the HBBTV4ALL project
(Universitat Autònoma de Barcelona)
 Reading subtitles on three devices:
– Monitor (22-inch LCD, 1920x1200)
– iPad Air
– iPhone (4S)
http://www.hbb4all.eu/

3. Previous eyetracking studies on mobile devices
(Al-Showarah et al. 2014)
 Influence of age on use of mobile devices
– the elderly have difficulties with processing
information and browsing smartphone interfaces
across all screen sizes
 Influence of screen size of mobile devices
– the smaller the screen – the worse the performance,
irrespectively of age

4. Study design
 3x2 design
– 3 devices:
• iPhone
• iPad
• monitor
– 2 languages:
• English
• Norwegian
 Subtitle watching
experience
– Comprehension
– Reading patterns
– Preferences

5. Study material
 English film
– “Joining the dots”,
dir. Pablo Romero Fresco (2012)
– 3 x 3 min. video
 Norwegian film
– “Headhunters”,
dir. Morten Tyldum (2011)
– 3 x 3 min. videos
 Polish interlingual subtitles
displayed at 15 cps

6. Eyetracking
 SMI Red (250 Hz)
– Monitor
 Tobii X2 (30Hz)
– iPhone
– iPad
 Eyetracking measures
– Number of fixations
– Fixation duration
 Fixation threshold: 80 ms

7. Procedure
 Informed consent
 Videos in counterbalanced order
– Calibration before each clip
– 10 multiple choice comprehension
questions after each clip
• 5 on subtitle content (text only)
• 5 on visual aspects
 Demographic
and preference survey
 Total duration: ca. 45 minutes

8. Participants
 33 people aged 20-35
– Mean age: 24 (SD=3,4)
– 8 men, 25 women
 Declared proficiency on 1-10 scale
(1 – no knowledge, 10 – proficiency)
– English: 8.79 (SD=1.19)
– Norwegian: 1.36 (SD=0.55)

9. 55%
6%
39%
How often do you watch foreign
films with subtitles?
always
sometimes
rarely
Participants’ background
12%
21%
67%
PhD
BA
MA
Education

10. Comprehension
results

11. Comprehension – textual
4,14
4,32 4,24
1
2
3
4
5
Subtitle comprehension
No.ofcorrectanswers
phone
tablet
monitor
two-way ANOVA
(variables: device & language)
device: p=0.4471

12. Comprehension – visual
3,55
3,74 3,74
0
1
2
3
4
5
Visual comprehension
No.ofcorrectanswers
phone
tablet
monitor
two-way ANOVA
(Variables: device & language)
Device: p=0.4917

13. Eyetracking results

14. Areas of interest
on each subtitle

15. Fixation count per device
No. of fixations
per subtitle
SD
iPhone 5.28 1.95
iPad 6.13 2.26
monitor 5.65 1.94
two-way ANOVA
(variables: device & language)
Device: p=.0647
Bonferroni correction
The only significant difference:
between iPhone and iPad (p=.0592)

16. Mean fixation duration per device
Fixation duration SD
iPhone 250 ms 43.14
iPad 183 ms 20.73
monitor 172 ms 44.59
two-way ANOVA
(Variables: device & language)
Device: p=.000
Bonferroni correction:
iPhone vs. iPad & iPhone vs. monitor (p=.000)
iPad and monitor (p=.1173)

17. Mean fixation duration
by device and language
177
188
250
167
178
240
0 50 100 150 200 250 300
monitor
iPad
iPhone
milliseconds
Norwegian
English
two-way ANOVA
(Variables: device & language)
Device: p=.0256

18. Preferences

19. How often do you watch subtitled videos
on these devices?
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
always
sometimes
rarely
never
monitor
tablet
phone

20. The comfort of watching subtitled videos
by device
0 20 40 60 80 100
1
2
3
4
5
Percentage of answers
1-verybad,5-verygood
monitor
tablet
phone

21. Conclusions
 iPhone – the worst device to watch subtitled videos
– Lowest comprehension results
– Longest mean fixation duration
– Fewest fixations – less reading
– Nobody’s preferred device
 Habits inform preferences – other participants?
 Longer mean fixation duration in English
than in Norwegian clips
– Parallel processing of visual and audio content
– Higher cognitive effort, but more rewarding
(higher comprehension scores in English)
 Testing across eyetrackers…

22. So what…?
HBBTV (hybrid broadcast broadband)
= digital broadcasting + Internet
+ mobile devices
Smartphones
– secondary devices only with subtitles?

23. Contact
a.szarkowska@uw.edu.pl
www.avt.ils.uw.edu.pl
www.facebook.com/AVTLab