This a case-based qualitative research presentation in which how the data have been collected, analyzed, and reported in papers have been discussed. The research is entirely focused on investigating an Accessibility related research premise.

1. | 1
Akhter Al Amin
Golisano College of Computing and Information Sciences
Rochester Institute of Technology
Perspective of Deaf or Hard of Hearing
Users for TV Caption Appearance
ISTE-798: Final Presentation

2. | 2
Introduction & Motivation
▪ Captioning ensure Deaf and Hard of Hearing(DHH) viewers’
access to auditory information while watching TV programs.
▪ Prior research demonstrated DHH viewers’ dissatisfaction with
current caption quality.[1]
▪ Accurate transcription quality may not be enough.[2]
▪ It is important to
• understand the DHH users’ preference for a wide range of visual
properties of captions.
• explore the confounding factors influence DHH viewers’ preference.
• provide empirical evidence through qualitative inquiry.

3. | 3
Epistemology and Qualitative Framing
▪ We aim to use Constructivist Epistemology to unveil
participants’ insightful experience, concerns, and
recommendations.[3]
▪ This qualitative inquiry is informed by Grounded Theory
approach using Phenomenology. [3]

4. | 4
Problem Statement
The effect of these visual properties of captions on Deaf
and Hard of Hearing (DHH) users' TV-watching
experience needs to be explored to construct a
research-based guidelines and to inform the design of
state-of-the-art caption evaluation metrics.

5. | 5
▪ What caption appearance or style are preferred by viewers
who are Deaf and Hard of Hearing (DHH)?
▪ Are there any key factors that influence DHH users’
prioritization of specific caption appearance type such as
caption font, background, speaker indicator type?
Research Question

6. | 6
Methods
▪ We have recruited participants those who are Deaf and Hard
of Hearing and regularly watch captioned TV programming.
▪ A total of 5 participants were recruited for these structured
interview sessions.
▪ These interviews were 30 minutes long.
▪ The participants were asked
• to share their prior experiences with different types of captions.
• to share which caption styles matter to them and why.
▪ The session was conducted in story-telling mode.

7. | 7
Methods
▪ We divided our sessions in 5 visual attributes of captions:
• Background of caption text
• Font size of caption, e.g. large, medium, and small
• Caption font color and typeface e.g. arial, calibri
• Movement types of caption, e.g. scrolling or pop-up
• Placement of caption, e.g. static central location, dynamic location
▪ These sections have outlined our discussions across the
sessions in a formative manner.

8. | 8
Data
▪ The interview session was conducted both in ASL and
English.
▪ We recorded the interviews to generate accurate interview
transcripts later.
▪ We verified the transcription against the recording before
delving into in-depth analysis.
▪ Demographics:
• Three females, one males and one non-binary
• Aged 23 to 26 (median = 25)
• Four participants described as D/deaf and one as hard-of-hearing

9. | 9
Analysis
▪ Open-coding followed by thematic analysis.
Preliminary Themes Selected Codes
Caption blocking onscreen graphical information region
is a concern.
● Large font may block graphical information e.g.,
speakers’ lips, sports score, speakers’ name.
● Black background reduces background visibility.
● Circumstantial factor
Genre specific caption style is preferred ● Genre information carrying Typeface can be
beneficial.
● This is crucial in a social gathering setup.
Caption movement preference regulated by several
factors
● Genre-specific Scrolling and Pop-up preference
● Number of speakers on the screen at a time can
be an issue
● Importance of Movement
Caption placement should be static, but... ● Dynamic caption is not entirely bad, but uncertain
appearance.
● Interesting to see who is talking.

10. | 10
Selected Memos
Memo #1: Large font can overlap onscreen material which seems disturbing to participants. It is necessary to remember this factor while
choosing font size during broadcast. While some comments give an impression that bigger fonts increase readability, fast reader may not
fond of bigger font due to presence of less number of text onscreen at a time.
Memo #2: Another source of caption blocking onscreen content is caption background. Participants feel that if caption background possesses
black or ash color which blocks the background entirely, impede viewers ability to watch what is going on behind.
Memo #4: Some participants have experience of watching such genre-specific caption typeface which is pretty interesting. I think they like
that. There can be relationship between DHH viewers’ perception of the genre of the TV program and the typeface variation. It is interesting
to see that those who did not experience such typeface, are interested to see how does that feel which seems promising.
Memo #8: There are several relationships I observed in participants’ responses which motivates the viewers to choose one caption
movement over another. Scrolling type is more of a real time type expressed by participants, which means that scrolling might be better for
live captioning. But there is a niche relationship between the genre of the TV program and type pf caption movement. This also extends to
number of onscreen speakers. In case of lesser number or single speaker onscreen might require pop-up, on the other hand in case multiple
participants it is required to have scrolling type. We think there might be a three way relationship We can derive from this.
Memo #10: This was pretty predictable that participants do not want caption to be move around, since they need to look for caption which is
distracting is most of the time. But I think this could be a premise of investigation that how number of onscreen speakers might influence
viewers choice of caption placement. I think there can be a relationship between number of onscreen speakers and viewers priority of
caption placement.

11. | 11
▪ Themes that emerged from the analysis are four-fold:
• Caption blocking onscreen graphical information is a concern.
I think background is important, specially what is going on behind the scene is vital. I think this one I will tell others that you may
miss if the background covers some parts of the background. On top of that, I think keeping color simple matters, you know like
distracting stems from color.-P1
• Genre-specific caption style might disseminate vital information.
I have never changed the font for captions, but it would be cool for font color to match the genre of the show, e.g. Harry Potter
(wizard style), or scary movie (spooky font). I saw few of them in different movies and channels. I find them interesting they carry
some sort of information you know what I mean - P4
• Caption movement preference may be regularized by genre of the
program or number of onscreen speakers.
If one person is talking, I prefer pop-up. Then, I can read it really quick, and have time to glance at their face, in-between reading
text. Otherwise, if a lot of people, I prefer word-by-word scrolling style. -P1
• Relative concerns regarding dynamic caption placement.
I prefer central location so I dont need to look for caption. But there should indication who is speaking. If the people who are
talking are introduced themselves by name I would like to move caption out of the way. -P2
Findings

12. | 12
Limitations
▪ Naturalistic inquiry might unveil interesting insights.
▪ Our study focused on entire users’ preferences.
▪ Demographics diversity required.
▪ Investigated limited number of caption attributes.

13. | 13
Thanks for your attention.
Question?

14. | 14
References
1. Larwan Berke, Khaled Albusays, Matthew Seita, and Matt Huenerfauth. Preferred appearance of captions generated by automatic speech
recognitionfor deaf and hard-of-hearing viewers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing
Systems, CHI EA ’19,page 1–6, New York, NY, USA, 2019. Association for Computing Machinery.
2. Ofcom. 2019. Measuring live subtitling quality, UK. Retrieved from:
https://www.ofcom.org.uk/__data/assets/pdf_file/0019/45136/sampling-report.pdf. Last accessed 19 Dec, 2020.
3. Patton, M. Q., Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, Calif: Sage Publications.