1. HTML5 multimedia
BROWSER-NATIVE VIDEO AND AUDIO
Patrick H. Lauke / JSDay / Verona / 17 Maggio 2012

2. <video>

3. Adobe Flash currently most common
video/audio delivery mechanism

4. <object width="425" height="344">
<param name="movie"
value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en
&fs=1&"></param>
<param name="allowFullScreen"
value="true"></param>
<param name="allowscriptaccess"
value="always"></param>
<embed
src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&f
s=1&" type="application/x-shockwave-flash"
allowscriptaccess="always" allowfullscreen="true"
width="425" height="344"></embed>
</object>

5. plug-in is a black box

6. <video src="video.webm"></video>
originally proposed by Opera in 2007
dev.opera.com/articles/view/labs-a-call-for-video-on-the-web

7. <video src="video.webm"
controls
autoplay
muted
loop
preload="none|metadata|auto"
poster="poster.jpg"
width="320" height="240">
<a href="video.webm">Download movie</a>
</video>

8. video as native object

10. people.opera.com/patrickl/experiments/webm/hover+transition

11. powerful (simple) API

12. <video> and JavaScript
var v = document.getElementById('player');
v.play()
v.pause()
v.volume = …
v.currentTime = …
v.addEventListener('loadeddata', function() { … }, true)
v.addEventListener('play', function() { … }, true)
v.addEventListener('pause', function() { … }, true)
v.addEventListener('timeupdate', function() { … }, true)
v.addEventListener('ended', function() { … }, true)

13. www.w3.org/2010/05/video/mediaevents.html

14. people.opera.com/patrickl/experiments/webm/basic-controls

15. HTML5 does not specify
how browser controls should look

17. people.opera.com/patrickl/experiments/webm/fancy-controls

18. people.opera.com/patrickl/experiments/webm/fancy-swap

19. video formats
the big debate?

20. HTML5 does not specify
video container/codec
(same as with images in HTML 4.01)

21. MP4/H.264
or
Ogg Theora
or
WebM/VP8

22. MP4 / H.264
ubiquitous, licensing/royalties

23. Ogg Theora
free and open, no licensing fees
not many tools for it, not web optimised

24. WebM / VP8
open and royalty-free, web-optimised
support by hardware and software vendors

25. http://tools.google.com/dlpage/webmmf

26. Browsers vs codecs
Chome Firefox IE Opera Safari
Ogg Theora ✔ ✔ ✔
WebM / VP8 ✔ ✔ ? ✔
MP4 / H.264 ✔ ✔ ✔
mobile and devices slightly better...

27. providing multiple sources
<video controls width="…" height="…">
<source src="movie.mp4" type="video/mp4">
<source src="movie.webm" type="video/webm">
<source src="movie.ogv" type="video/ogg">
<!-- fallback content -->
</video>

28. specifying codecs
<video controls width="…" height="…">
<source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'>
<source … type='video/webm; codecs="vorbis,vp8"'>
<source … type='video/ogg; codecs="theora,vorbis"'>
<!-- fallback content -->
</video>

29. understanding fallback
<video controls width="…" height="…">
<source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'>
<!-- fallback content →
<p>Shown to browsers that don't support video element,
and NOT just browsers that don't support MP4/H.264</p>
</video>

30. people.opera.com/patrickl/experiments/webm/wrongsource

31. feature-detection for codecs
v.canPlayType('video/webm;codecs="vp8,vorbis"')
"probably" | "maybe" | "" < WAT?

33. video and Responsive Web Design
<source … media="(min-width:1000px)">
<source … media="(max-width:1000px)">

34. people.opera.com/patrickl/experiments/webm/mediaquery

35. people.opera.com/patrickl/experiments/webm/mediaquery

36. flash fallback for older browsers
http://camendesign.com/code/video_for_everybody

37. <video controls width="…" height="…">
<source src="movie.mp4" type="video/mp4" />
<source src="movie.webm" type="video/webm" />
<source src="movie.ogv" type="video/ogg" />
<object width="…" height="…" type="application/x-
shockwave-flash" data="player.swf">
<param name="movie" value="player.swf" />
<param name="flashvars" value=" … file=movie.mp4" />
<!-- fallback content -->
</object>
</video>

38. <audio>

39. audio...exactly the same as video
<audio src=”music.mp3” controls autoplay … ></audio>
<audio controls autoplay>
<source src="music.mp3" type="audio/mpeg" />
<source src="music.oga" type="audio/ogg" />
<!-- fallback content -->
</audio>
formats: MP3 vs Ogg Vorbis (vs WAV)

40. people.opera.com/patrickl/experiments/audio/windchime

41. no need for <audio> or <video> in your DOM
var s = document.createElement('audio');
s.src = 'aaargh.oga'; s.play();

42. audio “sound sprites” to counter
latency and overhead
people.opera.com/~emoller/audio-demo

43. <canvas>

44. canvas drawing ready-made images
ctx = canvas.getContext("2d");
var logo = new Image();
logo.src = 'logo.png';
ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2);
or call in an existing image already on the page

45. canvas access to image data array
ctx = canvas.getContext("2d");
canvasData = ctx.getImageData(x,y,w,h);
[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]

46. canvas also works with video
ctx = canvas.getContext("2d");
v = document.getElementById('player');
ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2);
grab currently displayed frame (update as appropriate)

47. html5doctor.com/video-canvas-magic

48. media.chikuyonok.ru/ambilight

49. is it all safe to use, right now?

50. www.youtube.com/html5

51. don't do browser sniffing
http://www.flickr.com/photos/timdorr/2096272747/

53. feature-detection
progressive enhancement, graceful degradation
diveintohtml5.info/everything.html

54. feature-detection for audio/video
if (!!document.createElement('video').canPlayType;) { … }
if (!!document.createElement('audio').canPlayType;) { … }

55. sublimevideo.net

56. www.jplayer.org

57. www.textfiles.com/underconstruction

58. dev.w3.org/2011/webrtc/editor/webrtc.html

59. dev.w3.org/2011/webrtc/editor/getusermedia.html

60. camera access
// Opera
navigator.getUserMedia({video:true},success,error);
[…]
video.src = stream;
// WebKit
navigator.webkitGetUserMedia("video",success,error);
[…]
video.src = window.webkitURL.createObjectURL(stream);
gUM Shield: http://html5doctor.com/getusermedia/

61. www.opera.com/browser/next

63. neave.com/webcam/html5

64. shinydemos.com/qr-code

66. www.opera.com/developer
patrick.lauke@opera.com