A talk I gave at Web Directions South 2009 about the current possibilities of the HTML5 video element, and its shortfalls in accessibility. Also briefly mentions audio and media fragment URIs. Takes a broad sweep at accessibility and usability features of HTML5 media elements.

1. Taking HTML5
<video>
a step further
Silvia Pfeiffer
Mozilla / W3C / Xiph /
Annodex / Vquence

2. Storyboard
» Capabilities of <video>
» Aspects of video usability
and accessibility
» Designing solutions

3. Capabilities of
<video>

4. <video> is simple, sexy
<video
src="elephants_dream.ogv"
poster="elephants_dream.png"
controls>
</video>

5. Compare to Flash
<OBJECT
classid="clsid:D27CDB6E-AE6D-11cf-96B8-
444553540000"
codebase="http://download.macromedia.com/p
ub/shockwave/cabs/flash/swflash.cab#version=6,
0,40,0" WIDTH="550" HEIGHT="400"
id="myMovieName">
<PARAM NAME=movie VALUE="myFlashMovie.swf">
<EMBED href="myFlashMovie.swf" WIDTH="550"
HEIGHT="400" NAME="myMovieName" ALIGN=""
TYPE="application/x-shockwave-flash"
PLUGINSPAGE="http://www.macromedia.com/go/
getflashplayer"></EMBED>
</OBJECT>

6. Video and CSS
http://www.zachstronaut.com/lab/isocube.html
.left {
-webkit-transform: rotate(15deg)
skew(15deg, 15deg);
-moz-transform: rotate(15deg)
skew(15deg, 15deg);
}

7. Video, Canvas, CSS + js
https://developer.mozilla.org/samples
/video/chroma-key/index.xhtml
#c2 {
background-image: url(foo.png);
background-repeat: no-repeat;
}

8. Video and Face Detection
http://hacks.mozilla.org/2009/06/conn
ecting-html5-video/comment-page-
1/
1:16m : twitter and video
2:00m : face recognition
Full-screen

9. Dynamic Content Injection
http://people.mozilla.com/~prouget/demo
s/DynamicContentInjection/play.xhtml
<canvas id="mirrorVideo"/>
<svg id="playButton"
xmlns="http://www.w3.org/2000/svg" width="100"
height="100">
<path d="M 40 70 L 70 50 L 40 30"/> <circle
cx="50" cy="50" r="40"
onclick="processor.playVideo()"/>
</svg>

10. Video in SVG
http://www.double.co.nz/video_test/video.
svg
<foreignObject>
<div xmlns="http://www.w3.org/1999/xhtml">
<video src="myvideo.ogg"/>
</div>
</foreignObject>

11. Video and Physics
http://people.mozilla.com/~blizzard/l
aunch/
Javascript and video for the win!

12. Aspects of
Video Usability

13. Keyboard Control
» @controls
 space bar
 →/←
 CTRL + → / ←
 HOME + → / ←
 ↑ / ↓ on volume button
» Exposure to screen readers

14. Deep Access to Video
http://www.annodex.net/~silvia/itext/
mediafrag.html#t=30
Media Fragment URIs:
• Time offset browser bar
• Time offset <video> @src

15. Aspects of
Video Accessability

16. Screen Reader ARIA
» @aria-label
<video class="v" src="elephant.ogv"
poster="elephant.png”
controls
aria-label="Elephants Dream video"
title="Elephants Dream video">
</video>

17. Sign Language
http://www.ehow.com/video_4403510_sign
-language-movies.html
Signing by humans
http://www.vcom3d.com/vault_files/forest_asl/
Or by avatars in the future

18. Audio Descriptions
http://www.youtube.com/watch?gl=
AU&hl=en-GB&v=i2VXp0s0BLw
Spoken by a human

19. What can be done
with text

20. Captions / Subtitles / Audio
Descriptions
Intro video
http://localhost/~silvia/mozilla/itext/w
ebjam.html

21. Other time-aligned text
» Lyrics
» Karaoke
» Ticker Text
» Annotations/active regions
» Transcripts
» Linguistic markup
» Timed Metadata

22. Advantages
» Search engines
» Cross-site use
» Linking between videos
» Advertising -> Monetisation

23. A structured
approach:
out-of-band text

24. SRT example
1
00:00:15,000 --> 00:00:17,951
At the left we can see...
2
00:00:18,166 --> 00:00:20,083
At the right we can see the...
3
00:00:20,119 --> 00:00:21,962
...the head-snarlers

25. LRC example
[ti:Let's Twist Again]
[ar:Chubby Checker oppure Beatles, The]
[au:Written by Kal Mann / Dave Appell, 1961]
[al:Hits Of The 60's - Vol. 2 – Oldies]
[00:12.00]Lyrics beginning ...
[00:15.30]Some more lyrics ...

26. Text Abstraction
» Start time
» End time
» Text
» Onenter event
» Onleave event
» [Styling]
» [Hyperlink]
https://wiki.mozilla.org/Accessibility/HTML5_
captions_v2

27. Associate with <video>
<video src="elephants_dream.ogv controls>
<itextlist category=”SUB">
<itext src=”subtitle_en.srt" lang="en"/>
<itext src=”subtitle_de.srt" lang="de"/>
<itext src=”subtitle_fr.srt" lang="fr"/>
<itext src=”subtitle_jp.srt" lang="jp"/>
</itextlist>
</video>

28. Capabilities itextlist
» API:
 @category @onenter
 @active @onleave
 @name
» Default display in menu in
audio/video controls
» Default styling for categories

29. Capabilities itext
» API:
 @src @charset
 @lang @delay
 @type @stretch
» JavaScript API:
 currentText
 allText
 langName
» Default choice by browser language

30. Actual Examples
http://www.annodex.net/~silvia/itext/
Subtitles
Captions
Chapter Markers
Textual Audio Descriptions
Lyrics
Styling

31. A structured
approach:
in-line text

32. Multitrack video
» Video:
 Sign language
» Audio:
 Dubbed tracks
 Audio descriptions
» Text:
 Captions / subtitles / etc

33. Aim
» Identical Interface
» JavaScript
» CSS
» Media-format

34. State of
Affairs

35. Implementations
» JavaScript prototype
 Firefox 3.5 / Chromium / Safari 4.03 /
Opera 9.62
» Firefox
 Keyboard control 
 Deep linking ()
 Out-of-band patch
 In-line patch

36. Standardisation
» WHATWG discussions
» W3C discussions
 HTML
 WAI-XTECH
» In-principle Google, Opera
» 1st November, Stanford Uni,
W3C Workshop on Video A11y

37. Silvia Pfeiffer
silvia.pfeiffer1@gmail.com
http://blog.gingertech.net
Twiter: @silviapfeiffer