This document summarizes a presentation on how subtitles are read and processed. It discusses key differences between reading subtitles and printed text, including the fleeting and dynamic nature of subtitles. It also outlines major eye movements in reading, factors that influence word fixation and skipping, and research measuring these behaviors for subtitles of varying speeds and text segmentation. The document then discusses how deaf, hard-of-hearing and hearing viewers may process subtitles differently and the role of subtitler expertise, particularly in live subtitling using respeaking.
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
What do we do at AVT Lab
1. AGNIESZKA
SZARKOWSKA
Audiovisual Translation Lab (AVT Lab)
Institute of Applied Linguistics
University of Warsaw
a.szarkowska@uw.edu.pl
@ASzarkowska
https://avt.ils.uw.edu.pl/
https://www.facebook.com/AVTLab/
7. HOW IS READING SUBTITLES DIFFERENT
FROM READING PRINTED TEXT?
▪ ‘Fleeting text on a dynamic background’
▪ The viewer has no control over
the presentation speed of subtitled text
▪ Competing sources of information
▫ Visual
▫ Auditory
▫ Verbal
▫ Non-verbal
8. MAJOR TYPES OF EYE MOVEMENTS
▪ Saccades
▫ Rapid eye movements from 10 to 100 ms
▫ Forward and backward (regressions)
▪ Fixations
▫ Moments when our eyes are relatively still
▫ 200-300 ms (when reading English printed text)
26. HOW DOES SUBTITLE SPEED
AFFECT THE READING PROCESS?
▪ Fast subtitles
→ lower comprehension
→ less enjoyment
→ higher cognitive load
→ higher mean fixation duration
▪ Slow subtitles
→ more regressions and re-reading
27. ▪ Subtitle speed
▫ Fast (20 cps)
▫ Medium (16 cps)
▫ Slow (12 cps)
▪ 77 participants
▫ 30 English
▫ 21 Polish
▫ 26 Spanish
28. WHAT DID WE MEASURE?
▪ Comprehension
▪ Self-reported cognitive load
▫ Difficulty
▫ Effort
▫ Frustration
▪ Scene and subtitle
recognition
▪ Eye tracking
▫ Absolute reading time
▫ Proportional reading time
▫ Fixation count
▫ Revisits
32. Fast subtitles
▪ No negative effect on:
▫ comprehension
▫ scene recognition
▫ subtitle recognition
▫ reading experience
Slow subtitles
▪ Higher frustration
▪ Lower enjoyment
▪ More re-reading
▪ Higher mean fixation duration
WHAT DID WE FIND?
33.
34.
35. TEXT SEGMENTATION IN SUBTITLING
Syntactically
segmented
Non-syntactically
segmented
36.
37. ““Line breaks have their value,
yet when you are reading fast,
most of the time
it becomes less relevant.”
43. VIEWERS WITH HEARING LOSS
▪ Types of users
▫ The Deaf vs. the deaf
▫ The hard of hearing
▪ Degrees of hearing loss
▫ Mild, moderate, severe, profound
▪ Onset of hearing loss
▫ Pre-, peri- and post-lingual
Deafness
and poor
reading skills
often coincide
44. “
Our objective is to serve
as great a number
of the audience as possible
by seeking a more common
denominator: a lower reading
level and reading speed.
Downey, G. J. (2008). Closed captioning: subtitling, stenography,
and the digital convergence of text with television.
Baltimore: Johns Hopkins University Press.
Philip Collyer from WGBH,
The first director
of the Caption Centre
45. EDITING DOWN SUBTITLES
▪ Vocabulary simplification
▫ Modernising archaic words
▫ Removing idioms, word play, etc.
▪ Syntax simplification
▫ Complex to simple sentences
▫ Passive to active voice
▫ Subject-verb-object order
46.
47. “We thought we were doing
the right thing.
Jensema, C., & Burch, R. (1999). Caption Speed and Viewer
Comprehension of Television Programs
64. INTRA- AND INTERLINGUAL RESPEAKING
▪ Respeaking in education
https://www.youtube.com/watch?v=yTc7XEF0r58&t=2s
▪ Interlingual respeaking at conferences
https://www.youtube.com/watch?v=rGL0kJKfYPo
▪ Live events - The Witcher
https://www.youtube.com/watch?v=Oo0QZSd1yIY
88. SUBTLEX
▪ A corpus of subtitles used to estimate word frequencies
▪ "Subtitle-based word frequencies
are the best estimate of reading behaviour"
▪ SUBTLEX-US
▪ SUBTLEX-PL
▪ http://crr.ugent.be/programs-data/subtitle-frequencies
89.
90. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency
norms and the introduction of a new and improved word frequency measure for American English. Behavior Research
Methods, 41(4), 977-990.
91. WHY SUBTLEX MAY BE PROBLEMATIC?
▪ Text condensation in subtitling
▫ Removing elements of spoken language, repetitions,
false starts, vocatives
▫ Simplifying grammatical structures
▫ Using shorter synonyms for brevity
▪ Low-frequency words - underrepresented
▪ Written-to-be-spoken-as-if-not-written
92. AMATEUR SUBTITLES
▪ Using fansubs!
http://opensubtitles.org
▪ Relying on data generated
by unqualified teenagers in their free time
▪ Copyrights – relying on illegal material
93. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency
norms and the introduction of a new and improved word frequency measure for American English. Behavior Research
Methods, 41(4), 977-990.