Taking HTML5 video a step further

28,023 views

Published on

A talk I gave at Web Directions South 2009 about the current possibilities of the HTML5 video element, and its shortfalls in accessibility. Also briefly mentions audio and media fragment URIs. Takes a broad sweep at accessibility and usability features of HTML5 media elements.

Published in: Technology
  • Be the first to comment

Taking HTML5 video a step further

  1. 1. Taking HTML5 <video> a step furtherSilvia PfeifferMozilla / W3C / Xiph /Annodex / Vquence
  2. 2. Storyboard» Capabilities of <video>» Aspects of video usability and accessibility» Designing solutions
  3. 3. Capabilities of <video>
  4. 4. <video> is simple, sexy<video src="elephants_dream.ogv" poster="elephants_dream.png" controls></video>
  5. 5. Compare to Flash<OBJECT classid="clsid:D27CDB6E-AE6D-11cf-96B8- 444553540000" codebase="http://download.macromedia.com/p ub/shockwave/cabs/flash/swflash.cab#version=6, 0,40,0" WIDTH="550" HEIGHT="400" id="myMovieName"><PARAM NAME=movie VALUE="myFlashMovie.swf"><EMBED href="myFlashMovie.swf" WIDTH="550" HEIGHT="400" NAME="myMovieName" ALIGN="" TYPE="application/x-shockwave-flash" PLUGINSPAGE="http://www.macromedia.com/go/ getflashplayer"></EMBED></OBJECT>
  6. 6. Video and CSShttp://www.zachstronaut.com/lab/isocube.html.left { -webkit-transform: rotate(15deg) skew(15deg, 15deg); -moz-transform: rotate(15deg) skew(15deg, 15deg);}
  7. 7. Video, Canvas, CSS + jshttps://developer.mozilla.org/samples /video/chroma-key/index.xhtml#c2 { background-image: url(foo.png); background-repeat: no-repeat;}
  8. 8. Video and Face Detectionhttp://hacks.mozilla.org/2009/06/conn ecting-html5-video/comment-page- 1/1:16m : twitter and video2:00m : face recognitionFull-screen
  9. 9. Dynamic Content Injectionhttp://people.mozilla.com/~prouget/demo s/DynamicContentInjection/play.xhtml<canvas id="mirrorVideo"/><svg id="playButton" xmlns="http://www.w3.org/2000/svg" width="100" height="100"> <path d="M 40 70 L 70 50 L 40 30"/> <circle cx="50" cy="50" r="40" onclick="processor.playVideo()"/></svg>
  10. 10. Video in SVGhttp://www.double.co.nz/video_test/video. svg<foreignObject> <div xmlns="http://www.w3.org/1999/xhtml"> <video src="myvideo.ogg"/> </div></foreignObject>
  11. 11. Video and Physicshttp://people.mozilla.com/~blizzard/l aunch/Javascript and video for the win!
  12. 12. Aspects ofVideo Usability
  13. 13. Keyboard Control» @controls  space bar  →/←  CTRL + → / ←  HOME + → / ←  ↑ / ↓ on volume button» Exposure to screen readers
  14. 14. Deep Access to Videohttp://www.annodex.net/~silvia/itext/ mediafrag.html#t=30Media Fragment URIs:• Time offset browser bar• Time offset <video> @src
  15. 15. Aspects ofVideo Accessability
  16. 16. Screen Reader ARIA» @aria-label<video class="v" src="elephant.ogv" poster="elephant.png” controls aria-label="Elephants Dream video" title="Elephants Dream video"></video>
  17. 17. Sign Languagehttp://www.ehow.com/video_4403510_sign -language-movies.htmlSigning by humanshttp://www.vcom3d.com/vault_files/forest_asl/Or by avatars in the future
  18. 18. Audio Descriptionshttp://www.youtube.com/watch?gl= AU&hl=en-GB&v=i2VXp0s0BLwSpoken by a human
  19. 19. What can be done with text
  20. 20. Captions / Subtitles / Audio DescriptionsIntro videohttp://localhost/~silvia/mozilla/itext/w ebjam.html
  21. 21. Other time-aligned text» Lyrics» Karaoke» Ticker Text» Annotations/active regions» Transcripts» Linguistic markup» Timed Metadata
  22. 22. Advantages» Search engines» Cross-site use» Linking between videos» Advertising -> Monetisation
  23. 23. A structured approach:out-of-band text
  24. 24. SRT example100:00:15,000 --> 00:00:17,951At the left we can see...200:00:18,166 --> 00:00:20,083At the right we can see the...300:00:20,119 --> 00:00:21,962...the head-snarlers
  25. 25. LRC example[ti:Lets Twist Again][ar:Chubby Checker oppure Beatles, The][au:Written by Kal Mann / Dave Appell, 1961][al:Hits Of The 60s - Vol. 2 – Oldies][00:12.00]Lyrics beginning ...[00:15.30]Some more lyrics ...
  26. 26. Text Abstraction» Start time» End time» Text» Onenter event» Onleave event» [Styling]» [Hyperlink]https://wiki.mozilla.org/Accessibility/HTML5_ captions_v2
  27. 27. Associate with <video><video src="elephants_dream.ogv controls> <itextlist category=”SUB"> <itext src=”subtitle_en.srt" lang="en"/> <itext src=”subtitle_de.srt" lang="de"/> <itext src=”subtitle_fr.srt" lang="fr"/> <itext src=”subtitle_jp.srt" lang="jp"/> </itextlist></video>
  28. 28. Capabilities itextlist» API:  @category @onenter  @active @onleave  @name» Default display in menu in audio/video controls» Default styling for categories
  29. 29. Capabilities itext» API:  @src @charset  @lang @delay  @type @stretch» JavaScript API:  currentText  allText  langName» Default choice by browser language
  30. 30. Actual Exampleshttp://www.annodex.net/~silvia/itext/SubtitlesCaptionsChapter MarkersTextual Audio DescriptionsLyricsStyling
  31. 31. A structuredapproach: in-line text
  32. 32. Multitrack video» Video:  Sign language» Audio:  Dubbed tracks  Audio descriptions» Text:  Captions / subtitles / etc
  33. 33. Aim» Identical Interface» JavaScript» CSS» Media-format
  34. 34. State of Affairs
  35. 35. Implementations» JavaScript prototype  Firefox 3.5 / Chromium / Safari 4.03 / Opera 9.62» Firefox  Keyboard control   Deep linking ()  Out-of-band patch  In-line patch
  36. 36. Standardisation» WHATWG discussions» W3C discussions  HTML  WAI-XTECH» In-principle Google, Opera» 1st November, Stanford Uni, W3C Workshop on Video A11y
  37. 37. Silvia Pfeiffersilvia.pfeiffer1@gmail.comhttp://blog.gingertech.netTwiter: @silviapfeiffer

×