HTML5 multimediaWhere we are, where were goingBruce Lawson
Anne van Kesteren annevk at opera.comWed Feb 28 05:47:56 PST 2007Hi,Opera has some internal expiremental builds with an im...
<object width="425" height="344">  <param name="movie"value="http://www.example.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param>  <...
<video src=pudding.webm  controls  autoplay  poster=poster.jpg  width=320 height=240>    <a href=video.webm>Download movie...
<audio src=bieber.ogg  controls  autoplay>     <a href=bieber.ogg>Download horrid pap</a></audio>
<video src=pudding.webm loop>
<audio   src=bieber.ogg   preload><audio   src=bieber.ogg   preload=auto><audio   src=bieber.ogg   preload=none><audio   s...
video as native object...why is it important?●    keyboard accessibility built-in●    “play nice” with rest of the page●  ...
"In addition to giving video an HTMLelement, we must also agree on a baselinevideo format that will be universallysupporte...
Ogg Theora“free and open”, no licensing/royaltiesnot many tools for it, not web optimised
MP4 / H.264ubiquitous, patent encumbered, licensing/royalties,               hardware accelerated
WebM"open and royalty-free", web optimised,   hardware acceleration on its way
video formats               webM            Ogg/ Theora    mp4/ h264Opera            yes               yesChrome          ...
The politics of codecs
audio formats          mp3       Ogg/ Vobis   wavOpera                  yes       yesChrome     yes         yesFirefox    ...
Giving everybody video<video controls autoplay poster=… width=… height=…>  <source src=pudding.mp4 type=video/mp4>  <sourc...
<video controls poster="…" width="…" height="…">   <source src="movie.mp4" type="video/mp4" />   <source src="movie.webm" ...
Help●    archive.org converts and hosts creative-commons    licensed media●   vid.ly has a free conversion/ hosting servic...
powerful (simple) API
www.w3.org/TR/html5/video.html#media-elements
controlling <video> with JavaScriptvar v = document.getElementByTagName(video)[0];v.play();v.pause();v.volume = … ;v.curre...
events fired by <video>var v = document.getElementById(player);v.addEventListener(loadeddata, function() { … }, true)v.add...
Race condition:html5demos.com/video
video media queries<video controls><source src=hi-res.ogv media="(min-device-width:800px)"><source src=lo-res.ogv></video>...
Full-screen video●  Currently, WebkitEnterFullscreen();●  May 11, WebKit announced its implementing Gecko API  https://wik...
<video> accessibility
WebM release does not support subtitlesWHATWG / W3C RFC will release guidance on subtitles and otheroverlays in HTML5 <vid...
<track> element<video controls poster=… width=… height=…>    <source src=movie.webm type=video/webm>    <track src=subtitl...
webVTTWEBVTT FILE101:23:45,678 --> 01:23:46,789Hello world!201:23:48,910 --> 01:23:49,101HelloWorld!www.iandevlin.com/blog...
WebVTT formatting            Supports positioning of text               Supports <b> and <i>           Colouring individua...
<track> polyfillhttp://www.delphiki.com/html5/playr/
Synchronising media elementsEach media element can have a MediaController. A MediaController is an objectthat coordinates ...
How mature is all this?                               Looping latency               people.opera.com/patrickl/experiments/...
“...extending the language to better support Webapplications … This puts HTML in direct competitionwith other technologies...
DRM on HTML5 without changes                  Henri Sivonenlists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-             ...
getUserMedia API(previously known as "HTML5 <device>")
<video autoplay></video><script>var video = document.querySelector(video);If (navigator.getUserMedia){navigator.getUserMed...
my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
Thanks Mike Taylor @miketaylr for JS help, and Patrick Lauke@patrick_h_lauke for the Hixie voice. (What does that "H"     ...
Upcoming SlideShare
Loading in...5
×

HTML5 Multimedia: where we are, where we're going

7,404

Published on

A much-​​​​hyped feature of HTML5 is native multimedia. In this session we’ll look at embedding <audio> and <video> into your pages, and how to make it work cross-​​​​browser and degrade gracefully in older browsers. Sound too good to be true? It’s not!

We’ll look at the pros and the cons of HTML5 multimedia and see how to write simple controls with JavaScript. Most excitingly, we’ll also look at how HTML5 builds in support for subtitles and captions for multimedia accessibility. And you might pick up a Turkish dancing tip on the way.

---

Edited version of my Web Directions London talk on 26 May 2011. Slides that don't make sense out of context are removed.

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,404
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
62
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

HTML5 Multimedia: where we are, where we're going

  1. 1. HTML5 multimediaWhere we are, where were goingBruce Lawson
  2. 2. Anne van Kesteren annevk at opera.comWed Feb 28 05:47:56 PST 2007Hi,Opera has some internal expiremental builds with an implementation of a <video> element. The elementexposes a simple API (for the moment) much like the Audio() object: play() pause() Stop()The idea is that it works like <object> except that it has special <video> semantics much like <img> hasimage semantics. In markup you could prolly use it as follows: <figure> <video src=news-snippet.ogg> ... </video> <legend>HTML5 in BBC News</legend> </figure>I attached a proposal for the element and as you can see there are still some open issues. The element andits API are of course open for debate. Were not enforcing this upon the world ;-)Cheers, http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2007-February/009702.html
  3. 3. <object width="425" height="344"> <param name="movie"value="http://www.example.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param> <param name="allowFullScreen"value="true"></param> <param name="allowscriptaccess"value="always"></param> <embedsrc="http://www.example.com/v/9sEI1AUFJKw&hl=en&fs=1&" type="application/x-shockwave-flash"allowscriptaccess="always" allowfullscreen="true"width="425" height="344"></embed></object>
  4. 4. <video src=pudding.webm controls autoplay poster=poster.jpg width=320 height=240> <a href=video.webm>Download movie</a></video>
  5. 5. <audio src=bieber.ogg controls autoplay> <a href=bieber.ogg>Download horrid pap</a></audio>
  6. 6. <video src=pudding.webm loop>
  7. 7. <audio src=bieber.ogg preload><audio src=bieber.ogg preload=auto><audio src=bieber.ogg preload=none><audio src=bieber.ogg preload=metadata>
  8. 8. video as native object...why is it important?● keyboard accessibility built-in● “play nice” with rest of the page● Simple API for controls
  9. 9. "In addition to giving video an HTMLelement, we must also agree on a baselinevideo format that will be universallysupported, just like the GIF, JPEG and PNGimage format are universally supported. Itsimportant that the video format we choosecan be supported by a wide range of devicesand that its royalty-free (RF). RF is a well-established principle for W3C standards." http://people.opera.com/howcome/2007/video/
  10. 10. Ogg Theora“free and open”, no licensing/royaltiesnot many tools for it, not web optimised
  11. 11. MP4 / H.264ubiquitous, patent encumbered, licensing/royalties, hardware accelerated
  12. 12. WebM"open and royalty-free", web optimised, hardware acceleration on its way
  13. 13. video formats webM Ogg/ Theora mp4/ h264Opera yes yesChrome yes yes Nope (Chrome.soon)Firefox Yes (FF4) yesSafari yesIE9 Yes (if installed) yes
  14. 14. The politics of codecs
  15. 15. audio formats mp3 Ogg/ Vobis wavOpera yes yesChrome yes yesFirefox yes yesSafari yes yesIE9 yes yes
  16. 16. Giving everybody video<video controls autoplay poster=… width=… height=…> <source src=pudding.mp4 type=video/mp4> <source src=pudding.webm type=video/webm> <!-- fallback content --></video>
  17. 17. <video controls poster="…" width="…" height="…"> <source src="movie.mp4" type="video/mp4" /> <source src="movie.webm" type="video/webm" /> <source src="movie.ogv" type="video/ogg" /> <object width="…" height="…" type="application/x-shockwave-flash" data="player.swf"> <param name="movie" value="player.swf" /> <param name="flashvars" value=" … file=movie.mp4" /> <!-- fallback content --> </object></video> still include fallback for old browsers http://camendesign.com/code/video_for_everybody
  18. 18. Help● archive.org converts and hosts creative-commons licensed media● vid.ly has a free conversion/ hosting service (max 1GB source file) see vid.ly/5u4h3e● Miro video converter is a drag and drop GUI skin on FFMPEG
  19. 19. powerful (simple) API
  20. 20. www.w3.org/TR/html5/video.html#media-elements
  21. 21. controlling <video> with JavaScriptvar v = document.getElementByTagName(video)[0];v.play();v.pause();v.volume = … ;v.currentTime = … ;…
  22. 22. events fired by <video>var v = document.getElementById(player);v.addEventListener(loadeddata, function() { … }, true)v.addEventListener(play, function() { … }, true)v.addEventListener(pause, function() { … }, true)v.addEventListener(timeupdate, function() { … }, true)v.addEventListener(ended, function() { … }, true)…
  23. 23. Race condition:html5demos.com/video
  24. 24. video media queries<video controls><source src=hi-res.ogv media="(min-device-width:800px)"><source src=lo-res.ogv></video>http://dev.w3.org/html5/spec/video.html#the-source-element
  25. 25. Full-screen video● Currently, WebkitEnterFullscreen();● May 11, WebKit announced its implementing Gecko API https://wiki.mozilla.org/Gecko:FullScreenAPI● Opera likes this approach, too
  26. 26. <video> accessibility
  27. 27. WebM release does not support subtitlesWHATWG / W3C RFC will release guidance on subtitles and otheroverlays in HTML5 <video> in the near future. WebM intends to followthat guidance. http://code.google.com/p/webm/issues/detail?id=11 Egg image Kacper "Kangel" Aniołek http://commons.wikimedia.org/wiki/File:Egg.jpg
  28. 28. <track> element<video controls poster=… width=… height=…> <source src=movie.webm type=video/webm> <track src=subtitles-en.vtt kind=subtitles srclang=en> <track src=subtitles-de.vtt kind=subtitles srclang=de> <!-- fallback content --></video>http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element
  29. 29. webVTTWEBVTT FILE101:23:45,678 --> 01:23:46,789Hello world!201:23:48,910 --> 01:23:49,101HelloWorld!www.iandevlin.com/blog/2011/05/html5/webvtt-and-video-subtitlesblog.gingertech.net/2011/03/29/webvtt-explained/
  30. 30. WebVTT formatting Supports positioning of text Supports <b> and <i> Colouring individual speakers Support vertical text Supports RTL Supports ruby annotationshttp://www.whatwg.org/specs/web-apps/current-work/webvtt.html
  31. 31. <track> polyfillhttp://www.delphiki.com/html5/playr/
  32. 32. Synchronising media elementsEach media element can have a MediaController. A MediaController is an objectthat coordinates the playback of multiple media elements, for instance so that asign-language interpreter track can be overlaid on a video track, with the twobeing kept in sync....Media elements with a MediaController are said to be slaved to their controller. TheMediaController modifies the playback rate and the playback volume of each of themedia elements slaved to it, and ensures that when any of its slaved mediaelements unexpectedly stall, the others are stopped at the same time.When a media element is slaved to a MediaController, its playback rate is fixed tothat of the other tracks in the same MediaController, and any looping is disabled. http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#synchronising- multiple-media-elements http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API
  33. 33. How mature is all this? Looping latency people.opera.com/patrickl/experiments/audio/hacky-looper/ HTML5 video issues on the iPad and how to solve themblog.millermedeiros.com/2011/03/html5-video-issues-on-the-ipad-and-how-to-solve-them/ Unsolved HTML5 video issues on iOS http://blog.millermedeiros.com/2011/04/unsolved-html5-video-issues-on-ios/ Audio Sprites (and fixes for iOS) remysharp.com/2010/12/23/audio-sprites/
  34. 34. “...extending the language to better support Webapplications … This puts HTML in direct competitionwith other technologies intended for applications deployedover the Web, in particular Flash and Silverlight.”Ian Hickson, Editor of HTML5http://lists.w3.org/Archives/Public/public-html/2009Jan/0215.html
  35. 35. DRM on HTML5 without changes Henri Sivonenlists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010- July/027051.html
  36. 36. getUserMedia API(previously known as "HTML5 <device>")
  37. 37. <video autoplay></video><script>var video = document.querySelector(video);If (navigator.getUserMedia){navigator.getUserMedia(video,successCallback,errorCallback);function successCallback(stream) { video.src = stream; }...</script>
  38. 38. my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
  39. 39. Thanks Mike Taylor @miketaylr for JS help, and Patrick Lauke@patrick_h_lauke for the Hixie voice. (What does that "H" stand for?) Ian Devlin @iandevlin and Silvia Pfeiffer @silviapfeiffer for webVTT advice Philip Jägenstedt @foolip and Simon Pieters @zcorpan forimplementing/ QA-ing multimedia in Opera and explaining it to me slowly and patiently
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×