HTML5 multimedia - where we are, where we're going


Published on

Bruce Lawson's talk at Frontrow Conference,

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

HTML5 multimedia - where we are, where we're going

  1. 1. HTML5 multimediaWhere we are, where were goingBruce Lawson
  2. 2. Anne van Kesteren annevk at opera.comWed Feb 28 05:47:56 PST 2007Hi,Opera has some internal expiremental builds with an implementation of a <video> element. The elementexposes a simple API (for the moment) much like the Audio() object: play() pause() Stop()The idea is that it works like <object> except that it has special <video> semantics much like <img> hasimage semantics. In markup you could prolly use it as follows: <figure> <video src=news-snippet.ogg> ... </video> <legend>HTML5 in BBC News</legend> </figure>I attached a proposal for the element and as you can see there are still some open issues. The element andits API are of course open for debate. Were not enforcing this upon the world ;-)Cheers,
  3. 3. <object width="425" height="344"> <param name="movie"value=""></param> <param name="allowFullScreen"value="true"></param> <param name="allowscriptaccess"value="always"></param> <embedsrc="" type="application/x-shockwave-flash"allowscriptaccess="always" allowfullscreen="true"width="425" height="344"></embed></object>
  4. 4. <video src=pudding.webm controls autoplay poster=poster.jpg width=320 height=240> <a href=video.webm>Download movie</a></video>
  5. 5. <audio src=bieber.ogg controls autoplay> <a href=bieber.ogg>Download horrid pap</a></audio>
  6. 6. <video src=pudding.webm loop>
  7. 7. <audio src=bieber.ogg preload><audio src=bieber.ogg preload=auto><audio src=bieber.ogg preload=none><audio src=bieber.ogg preload=metadata>
  8. 8. powerful (simple) API
  9. 9.
  10. 10. controlling <video> with JavaScriptvar v = document.getElementByTagName(video)[0];;v.pause();v.volume = … ;v.currentTime = … ;…
  11. 11. events fired by <video>var v = document.getElementById(player);v.addEventListener(loadeddata, function() { … }, true)v.addEventListener(play, function() { … }, true)v.addEventListener(pause, function() { … }, true)v.addEventListener(timeupdate, function() { … }, true)v.addEventListener(ended, function() { … }, true)…
  12. 12. video as native object...why is it important?● keyboard accessibility built-in● “play nice” with rest of the page● Simple API for controls
  13. 13. "In addition to giving video an HTMLelement, we must also agree on a baselinevideo format that will be universallysupported, just like the GIF, JPEG and PNGimage format are universally supported. Itsimportant that the video format we choosecan be supported by a wide range of devicesand that its royalty-free (RF). RF is a well-established principle for W3C standards."
  14. 14. Ogg Theora“free and open”, no licensing/royaltiesnot many tools for it, not web optimised
  15. 15. MP4 / H.264ubiquitous, patent encumbered, licensing/royalties, hardware accelerated
  16. 16. WebM"open and royalty-free", web optimised, hardware acceleration on its way
  17. 17. video formats webM Ogg/ Theora mp4/ h264Opera yes yesChrome yes yes Nope (Chrome.soon)Firefox Yes (FF4) yesSafari yesIE9 Yes (if installed) yes
  18. 18. The politics of codecs
  19. 19. audio formats mp3 Ogg/ Vobis wavOpera yes yesChrome yes yes yesFirefox yes yesSafari yes yesIE9 yes
  20. 20. Giving everybody video<video controls autoplay poster=… width=… height=…> <source src=pudding.mp4 type=video/mp4> <source src=pudding.webm type=video/webm> <source src=pudding.ogv type=video/ogg> <!-- fallback content --></video>
  21. 21. <video controls poster="…" width="…" height="…"> <source src="movie.mp4" type="video/mp4" /> <source src="movie.webm" type="video/webm" /> <source src="movie.ogv" type="video/ogg" /> <object width="…" height="…" type="application/x-shockwave-flash" data="player.swf"> <param name="movie" value="player.swf" /> <param name="flashvars" value=" … file=movie.mp4" /> <!-- fallback content --> </object></video> still include fallback for old browsers
  22. 22. Help● converts and hosts creative-commons licensed media● has a free conversion/ hosting service (max 1GB source file) see● Miro video converter is a drag and drop GUI skin on FFMPEG
  23. 23. video media queries<video controls><source src=hi-res.ogv media="(min-device-width:800px)"><source src=lo-res.ogv></video>
  24. 24. Full-screen video● Currently, WebkitEnterFullscreen();● May 11, WebKit announced its implementing Gecko API● Opera likes this approach, too
  25. 25. <video> accessibility
  26. 26. WebM release does not support subtitlesWHATWG / W3C RFC will release guidance on subtitles and otheroverlays in HTML5 <video> in the near future. WebM intends to followthat guidance. Egg image Kacper "Kangel" Aniołek
  27. 27. <track> element<video controls poster=… width=… height=…> <source src=movie.webm type=video/webm> <track src=subtitles-en.vtt kind=subtitles srclang=en> <track src=subtitles-de.vtt kind=subtitles srclang=de> <!-- fallback content --></video>
  28. 28. WebVTT or TTML ?
  29. 29. webVTTWEBVTT FILE101:23:45,678 --> 01:23:46,789Hello world!201:23:48,910 --> 01:23:49,101Mój poduszkowiec jest pełen węgorzy
  30. 30. WebVTT formatting Supports positioning of text Supports <b> and <i> Colouring individual speakers Support vertical text Supports RTL Supports ruby annotationsValidator:
  31. 31. subtitling polyfills
  32. 32. playr
  33. 33. LeanBack Player
  34. 34. MediaElement.js
  35. 35. Synchronising media elements
  36. 36. Synchronising media elementsEach media element can have a MediaController. A MediaController is an objectthat coordinates the playback of multiple media elements, for instance so that asign-language interpreter track can be overlaid on a video track, with the twobeing kept in sync....Media elements with a MediaController are said to be slaved to their controller. TheMediaController modifies the playback rate and the playback volume of each of themedia elements slaved to it, and ensures that when any of its slaved mediaelements unexpectedly stall, the others are stopped at the same time.When a media element is slaved to a MediaController, its playback rate is fixed tothat of the other tracks in the same MediaController, and any looping is disabled. multiple-media-elements
  37. 37. How mature is all this? Looping latency HTML5 video issues on the iPad and how to solve Unsolved HTML5 video issues on iOS Audio Sprites (and fixes for iOS)
  38. 38. “...extending the language to better support Webapplications … This puts HTML in direct competitionwith other technologies intended for applications deployedover the Web, in particular Flash and Silverlight.”Ian Hickson, Editor of HTML5
  39. 39. Image © "I will draw anything you like to raise funds for the Sick Kids"
  40. 40. DISCLAIMER:This is my personal perspective and does not reflect theopinion of Opera.Or my wife, children or hamster. Or anyone else who mightbe embarrassed by association with me (which is mostlyeveryone, to be honest).As this is a conference, in the land of Żubrówka, Ill probablyget drunk and try to chat you up, so lets make thisdisclaimer run until Saturday, OK?
  41. 41. DRM on HTML5 without changes Henri July/027051.html
  42. 42. getUserMedia API(previously known as "HTML5 <device>")
  43. 43. <video autoplay></video><script>var video = document.querySelector(video);If (navigator.getUserMedia){navigator.getUserMedia(video,successCallback,errorCallback);function successCallback(stream) { video.src = stream; }</script>Tutorial:
  44. 44.
  45. 45. WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities viasimple Javascript APIs. The WebRTC components have been optimized to best serve this purpose. The WebRTC initiative is a project supported by Google, Mozilla and Opera. and-peer-to-peer-communication.html
  46. 46. "We expect to see WebRTC support in Firefox, Opera, and Chrome soon!"
  47. 47. Case study
  48. 48. <div><button>Q</button><button>W</button>...</div><div><button>A</button><button>S</button>...</div>
  49. 49. <video loop><source src=hixie.mp4 type=video/mp4><source src=hixie.webm type=video/webm></video>
  50. 50. //grab all the buttons buttons = document.querySelectorAll(button), //loop through all the <button> elements, and createcorresponding //audio elements buildAudio = (function(){ for (var i = 0, l = buttons.length; i < l; i++) { var key = buttons[i].textContent.toLowerCase(); //using the fancy new Audio() constructor here document.body.appendChild(new Audio(key + .ogg)); }; })()
  51. 51. document.addEventListener(ended, function(e){ if ( == "AUDIO") { // pause the video since the audio //has stopped vid.pause(); }}, true)
  52. 52. <button data-text="<q> element">Q</button><button data-text="Web Workers">W</button>
  53. 53. text = document.getElementById(thinger) …text.textContent = el.dataset ? el.dataset.text : el.getAttribute(data-text);
  54. 54. #thinger {position:absolute; left:40px;top:200px; text-align: center; font-family: hixie, cursive, monospace; font-size:3em; color:red; transform: matrix(0.98,-0.17,-0.17,0.98,0,0);} AEZ Segar font, made into fontface kit with
  55. 55. Thanks Mike Taylor @miketaylr for JS help, and Patrick Lauke@patrick_h_lauke for the Hixie voice. (What does that "H" stand for?) to me slowly and patiently
  56. 56. All rights reserved cornerThanks Takara Tomy for permission to use Gigapudding video. Buy the pudding.Thanks Jack Henderson for permission to his scary DRM picture. Buy his book. HTML5 nazi-killing Bruce on unicorn picture by Marina Lawson. Buy her Dads book. All others by me or (I believe) public domain.
  57. 57.