audio, video and canvas in HTML5
NATIVE MULTIMEDIA SUPPORT




Patrick H. Lauke / standards>next / Manchester / 29 Septemb...
HTML5
<!DOCTYPE html>
“...extending the language to better
support Web applications [...] This puts
HTML in direct competition with other
techno...
making your site Fonzie compliant...
<video>
Adobe Flash currently most common
    video delivery mechanism
<object width="425" height="344">
  <param name="movie"
value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en
&fs=1&"></param>...
<video src="video.webm"
  controls
  Autoplay
  Loop
  preload="none"
  poster="poster.jpg"
  width="320" height="240">
  ...
video as native object
● “plays nice” with rest of the page
●
  keyboard accessibility built-in
video formats
   the big debate
MP4 / H.264




ubiquitous, patent encumbered, licensing/royalties
Ogg Theora




“free and open”, no licensing/royalties
not many tools for it, not web optimised
www.webmproject.org
WebM




open and royalty-free, web optimised
www.youtube.com/html5
providing multiple sources
<video controls autoplay poster="…" width="…" height="…">
   <source src="movie.webm" type="vid...
still include fallback for old browsers
     http://camendesign.com/code/video_for_everybody
<video controls autoplay poster="…" width="…" height="…">
   <source src="movie.webm" type="video/webm" />
   <source src=...
powerful (simple) API
    to script your own controls
icant.co.uk/easy-youtube
www.w3.org/TR/html5/video.html#media-elements
controlling a <video> element
var v = document.getElementById('player');

v.play();
v.pause();
v.volume = … ;
v.currentTim...
events fired by <video> element
var v = document.getElementById('player');

v.addEventListener('loadeddata', function() { ...
<audio>
audio exactly the same as video
<audio src=”music.mp3” controls autoplay></audio>

[...]

<audio controls autoplay>
   <so...
<canvas>
canvas = “scriptable images”

<canvas width="…" height="…"></canvas>
canvas has standard API methods for drawing
ctx = canvas.getContext("2d");
ctx.fillRect(x, y, width, height);
ctx.beginPat...
canvas drawing ready-made images
ctx = canvas.getContext("2d");

var logo = new Image();
logo.src = 'logo.png';

ctx.drawI...
canvas access to image data array
ctx = canvas.getContext("2d");
canvasData = ctx.getImageData(x,y,w,h);

[R,G,B,A,R,G,B,A...
canvas exporting image data as data:uri
ctx = canvas.getContext("2d");
…
canvas.toDataURL("image/png");
// returns "data:i...
canvas also works with video
ctx = canvas.getContext("2d");
v = document.getElementById('player');

ctx.drawImage(v,x1,y1,...
canvas accessibility concerns
video, audio and canvas on any device
           without plugins
         (Java / Flash / Silverlight not ubiquitous)
HTML5 (and friends) has
lots more APIs for developers
       (for powerful client-side apps)
isgeolocationpartofhtml5.com
geolocation
navigator.geolocation.getCurrentPosition(success, error);
navigator.geolocation.watchCurrentPosition(success, ...
offline detection...
window.addEventListener('online', function(){ … }, true);
window.addEventListener('offline', function...
is it all safe to use, right now?
don't do browser sniffing



      http://www.flickr.com/photos/timdorr/2096272747/
feature-detection
progressive enhancement, graceful degradation – or use shims
                   http://diveintohtml5.org...
feature-detection for audio/video
if (!!document.createElement('video').canPlayType;) { … }

if (!!document.createElement(...
feature-detection for audio/video codecs
var v = document.createElement('video');
if (!!(v.canPlayType)&&
     ((v.canPlay...
ready-made HTML5 audio/video players
           (for the lazy)
sublimevideo.net
videojs.com
www.happyworm.com/jquery/jplayer
HTML5 as Flashkiller?
not a question of HTML5 replacing Flash...
giving developers a choice!
www.opera.com/developer
people.opera.com/patrickl/presentations/standards-next_29.09.2010/standards-next_29.09.2010.pdf
  ...
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010
Upcoming SlideShare
Loading in …5
×

audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010

4,539 views
4,355 views

Published on

Part II of the standards-next.org workshop on HTML5 with Bruce Lawson, concentrating on audio, video and canvas (plus hints of additional HTML5 API niceness)

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,539
On SlideShare
0
From Embeds
0
Number of Embeds
86
Actions
Shares
0
Downloads
95
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010

  1. 1. audio, video and canvas in HTML5 NATIVE MULTIMEDIA SUPPORT Patrick H. Lauke / standards>next / Manchester / 29 September 2010
  2. 2. HTML5 <!DOCTYPE html>
  3. 3. “...extending the language to better support Web applications [...] This puts HTML in direct competition with other technologies[...] , in particular Flash and Silverlight.” Ian Hickson, Editor of HTML5 http://lists.w3.org/Archives/Public/public-html/2009Jan/0215.html
  4. 4. making your site Fonzie compliant...
  5. 5. <video>
  6. 6. Adobe Flash currently most common video delivery mechanism
  7. 7. <object width="425" height="344"> <param name="movie" value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en &fs=1&"></param> <param name="allowFullScreen" value="true"></param> <param name="allowscriptaccess" value="always"></param> <embed src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&f s=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed> </object>
  8. 8. <video src="video.webm" controls Autoplay Loop preload="none" poster="poster.jpg" width="320" height="240"> <a href="video.webm">Download movie</a> </video>
  9. 9. video as native object ● “plays nice” with rest of the page ● keyboard accessibility built-in
  10. 10. video formats the big debate
  11. 11. MP4 / H.264 ubiquitous, patent encumbered, licensing/royalties
  12. 12. Ogg Theora “free and open”, no licensing/royalties not many tools for it, not web optimised
  13. 13. www.webmproject.org
  14. 14. WebM open and royalty-free, web optimised
  15. 15. www.youtube.com/html5
  16. 16. providing multiple sources <video controls autoplay poster="…" width="…" height="…"> <source src="movie.webm" type="video/webm" /> <source src="movie.ogv" type="video/ogg" /> <source src="movie.mp4" type="video/mp4" /> <!-- fallback content --> </video>
  17. 17. still include fallback for old browsers http://camendesign.com/code/video_for_everybody
  18. 18. <video controls autoplay poster="…" width="…" height="…"> <source src="movie.webm" type="video/webm" /> <source src="movie.ogv" type="video/ogg" /> <source src="movie.mp4" type="video/mp4" /> <object width="…" height="…" type="application/x- shockwave-flash" data="player.swf"> <param name="movie" value="player.swf" /> <param name="flashvars" value=" … file=movie.mp4" /> <!-- fallback content --> </object> </video>
  19. 19. powerful (simple) API to script your own controls
  20. 20. icant.co.uk/easy-youtube
  21. 21. www.w3.org/TR/html5/video.html#media-elements
  22. 22. controlling a <video> element var v = document.getElementById('player'); v.play(); v.pause(); v.volume = … ; v.currentTime = … ; …
  23. 23. events fired by <video> element var v = document.getElementById('player'); v.addEventListener('loadeddata', function() { … }, true) v.addEventListener('play', function() { … }, true) v.addEventListener('pause', function() { … }, true) v.addEventListener('timeupdate', function() { … }, true) v.addEventListener('ended', function() { … }, true) …
  24. 24. <audio>
  25. 25. audio exactly the same as video <audio src=”music.mp3” controls autoplay></audio> [...] <audio controls autoplay> <source src="music.mp3" type="audio/mpeg" /> <source src="music.oga" type="audio/ogg" /> <!-- fallback content --> </audio> same format debacle: MP3 vs Ogg Vorbis (vs WAV)
  26. 26. <canvas>
  27. 27. canvas = “scriptable images” <canvas width="…" height="…"></canvas>
  28. 28. canvas has standard API methods for drawing ctx = canvas.getContext("2d"); ctx.fillRect(x, y, width, height); ctx.beginPath(); ctx.moveTo(x, y); ctx.lineTo(x, y); ctx.bezierCurveTo(x1, y1, x2, y2, c1, c2);
  29. 29. canvas drawing ready-made images ctx = canvas.getContext("2d"); var logo = new Image(); logo.src = 'logo.png'; ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2); or call in an existing image already on the page
  30. 30. canvas access to image data array ctx = canvas.getContext("2d"); canvasData = ctx.getImageData(x,y,w,h); [R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]
  31. 31. canvas exporting image data as data:uri ctx = canvas.getContext("2d"); … canvas.toDataURL("image/png"); // returns "data:image/png;base64,…" canvas.toDataURL("image/jpg"); // returns "data:image/jpeg;base64,…" Note: do it on the canvas, not the context!
  32. 32. canvas also works with video ctx = canvas.getContext("2d"); v = document.getElementById('player'); ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2); grab currently displayed frame (update as appropriate)
  33. 33. canvas accessibility concerns
  34. 34. video, audio and canvas on any device without plugins (Java / Flash / Silverlight not ubiquitous)
  35. 35. HTML5 (and friends) has lots more APIs for developers (for powerful client-side apps)
  36. 36. isgeolocationpartofhtml5.com
  37. 37. geolocation navigator.geolocation.getCurrentPosition(success, error); navigator.geolocation.watchCurrentPosition(success, error); function success(position) { /* where's Wally? */ var lat = position.coords.latitude; var long = position.coords.longitude; ... }
  38. 38. offline detection... window.addEventListener('online', function(){ … }, true); window.addEventListener('offline', function(){ … }, true); and application cache <html manifest=”blah.manifest”> CACHE MANIFEST # send this with correct text/cache-manifest MIME images/sprites.png scripts/common.js scripts/jquery.js styles/global.css data.xml and many more... (even beyond HTML5)
  39. 39. is it all safe to use, right now?
  40. 40. don't do browser sniffing http://www.flickr.com/photos/timdorr/2096272747/
  41. 41. feature-detection progressive enhancement, graceful degradation – or use shims http://diveintohtml5.org/everything.html
  42. 42. feature-detection for audio/video if (!!document.createElement('video').canPlayType;) { … } if (!!document.createElement('audio').canPlayType;) { … }
  43. 43. feature-detection for audio/video codecs var v = document.createElement('video'); if (!!(v.canPlayType)&& ((v.canPlayType('video/webm;codecs="vp8,vorbis"') == 'probably') || (v.canPlayType('video/webm;codecs="vp8, vorbis"') == 'maybe'))) { … }
  44. 44. ready-made HTML5 audio/video players (for the lazy)
  45. 45. sublimevideo.net
  46. 46. videojs.com
  47. 47. www.happyworm.com/jquery/jplayer
  48. 48. HTML5 as Flashkiller?
  49. 49. not a question of HTML5 replacing Flash...
  50. 50. giving developers a choice!
  51. 51. www.opera.com/developer people.opera.com/patrickl/presentations/standards-next_29.09.2010/standards-next_29.09.2010.pdf patrick.lauke@opera.com

×