Slides presented at the 2017 #a11yTO Camp. It has been said that captioning videos is both an art and a science. This presentation provides a basic explanation of how captioning is supported by HTML5 and CSS (the "science") and covers a range of tips and ideas for making captions work for d/Deaf, deafened, and hard of hearing consumers (the "art").

1. THE ART AND SCIENCE OF
CAPTIONING
David Fourney, PhD.
2017 #a11yTO Camp - Accessibility Toronto Camp

2. DISCLAIMER
• The opinions expressed are entirely my own.

3. AGENDA
• What are captions?
• How to add captions to a video (the “Science” of captions).
• Tips, tricks, and ideas (the “Art” of captions).

4. WCAG WANTS CAPTIONS
• 1.2.2 Captions (Prerecorded): Captions are provided for all prerecorded
audio content in synchronized media, except when the media is a media
alternative for text and is clearly labeled as such. (Level A)
• 1.2.4 Captions (Live): Captions are provided for all live audio content in
synchronized media. (Level AA)
• Sign Languages?

5. WHAT ARE CAPTIONS?
• synchronized visual and/or text alternative for both speech and non-speech
audio information needed to understand the media content
• Note 1: Captions are similar to dialogue-only subtitles except captions convey not only
the content of spoken dialogue, but also equivalents for non-dialogue audio information
needed to understand the program content, including sound effects, music, laughter,
speaker identification and location.
• Note 2: Closed Captions are equivalents that can be turned on and off with some players.
• Note 3: Open Captions are any captions that cannot be turned off. For example, if the
captions are visual equivalent images of text embedded in video.
• Note 4: Captions should not obscure or obstruct relevant information in the video.
• Note 5: In some countries, captions are called subtitles.
• Note 6: Audio descriptions can be, but do not need to be, captioned since they are
descriptions of information that is already presented visually.
WCAG 2.0 / ISO/IEC 40500:2012

6. THE WRITING ON THE WALL

7. CAPTIONING IS NOT SUBTITLING!
• Subtitling assumes:
• Hearing audience
• Translation to another language
• Captioning rarely provides translation

8. CAPTIONS CONVEY
1. Spoken dialogue
2. Non-speech audio information needed to understand the program
content (sound effects, music, laughter, speaker identification and
speaker location)

9. A LOT HAS CHANGED SINCE THE 1970’S

10. BUT NOT CAPTIONS!
• Digital captions first demo’ed in 1971
• Current standard Line 21 TV captioning was established in 1976
• Monospaced white text on black background
• Maximum 32 characters per line
• Internet captioning uses the same visual design!
• Suffers from the same information architecture!
• Has the same usability problems!

11. Wikipedia

12. THE SCIENCE
How captions get into your video

13. MAKING CAPTIONS
• W3C developed Web Video Text Tracks Format (WebVTT) for captioning
• Supports HTML5, CSS, Unicode (UTF-8)

14. WEBVTT FORMAT
WEBVTT
00:00:19.000 --> 00:00:20.040
Hi, Joe!
00:00:20.060 --> 00:00:23.060
I was walking to the store.

15. WEBVTT STYLING
• Handled in the main HTML / CSS
<style type="text/css">
::cue(v[voice=”Bill"]) {
color: yellow;
background-color: black;
font-size: 150%;
} </style>

16. TOGETHER, WE GET …
WEBVTT
00:00:19.000 --> 00:00:20.040 position:5% align:start
<v Bill>Hi, Joe! </c>
00:00:20.060 --> 00:00:23.060 align:end
<v Joe>I was walking to the store.</c>

17. ADDING CAPTIONS
• HTML5 supports “text tracks”
• Can be any added text: captions, translations, etc.
<video>
<source src=”myvideo.mp4" type="video/mp4">
<track src=”myvideo.vtt”
kind="captions” type="text/vtt" > </track>
</video>

18. ADDING MORE CAPTIONS
<video width="320" height="240" controls>
<source src=”myvideo.mp4" type="video/mp4">
<source src=”myvideo.ogg" type="video/ogg">
<track
src="subtitles_en.vtt" kind=”captions”
srclang="en" label="English">
<track
src="subtitles_no.vtt" kind="subtitles”
srclang=”fr" label=”French">
</video>

19. YOUTUBE
• YouTube will generate your transcript for you
• This is only the dialogue!
• Only 1 of WCAG’s 2 parts of captions
• Other important information (non-speech audio information) is missing.
• Please review the automatic captions for accuracy!

21. THE ART
How captions should behave

22. CAPTIONING STANDARD
• ISO/IEC 20071-23:2018
• "Information technology — User interface component accessibility — Part 23:
Visual presentation of audio information (including captions and subtitles)”
• Provides over 70 requirements and recommendations on presenting speech
and non-speech information
• Identifies a broad range of visual alternatives including captions, graphics,
avatars, etc.

23. PLACEMENT
• Don’t obstruct important video content
• Covering faces, other text, soccer ball, etc.
• Place captions according to who is speaking to help identify speaker

24. moving across the field now. Looks like
she’s lined up for the shot, and the

25. DESCRIPTIVE VS. INFORMATIVE
• Sound effects, music, etc. as well as
actor’s emotions and tone provide
information that captions should
convey
• Example: Captioning traffic noise
• Man on the street interview?
• Maybe if interviewee startled by horn
• Car chase scene?
• Part of the scene experience

26. SOUND EFFECTS
• Assume your audience cannot hear your audio!
• Use commonly understood descriptions, onomatopoeia (“barking”, “woof”)
• Tell me where is the sound from (“dog barking”)
• Tell me why the sound is there (“dog barking at intruder”)
• Is it a one-time sound (“bark”) or sustained (“barking continues”, “barking
stops”)
• Is there no sound (“silence”)
• Don’t rely on hearing experiences. Assume never heard a particular
sound!
• “quietly slowly played guitar” VS “soft guitar” (what makes it soft??)

27. MUSIC
• Show the presence of music ( ♪ )
• What is this music?
• Who is performing?
• What’s it called?
• Why is the music there?
• What is the purpose of the music?
• What emotion should I now experience?
• Has the tone of the scene changed?

28. OTHER ISSUES
• Spelling and homonyms
• “Tax hains” (tax havens)
• “Road scholar” (Rhodes Scholar)
• WCAG requirements (e.g., colour contrast with background)

29. THANK YOU!