• Save
"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller
Upcoming SlideShare
Loading in...5
×
 

"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller

on

  • 2,575 views

A brief history of audio on the web, its present shape as of mid-2011, where things are going, and some examples of how you might use audio today on the web.

A brief history of audio on the web, its present shape as of mid-2011, where things are going, and some examples of how you might use audio today on the web.

Statistics

Views

Total Views
2,575
Views on SlideShare
2,571
Embed Views
4

Actions

Likes
2
Downloads
0
Comments
1

2 Embeds 4

http://www.hot.ee 2
http://www.appletruth.com 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller "Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller Presentation Transcript

  • “ Probably, Maybe, No”: The State Of HTML5 Audio
  • Hi.
    • Scott Schiller (@schill)
    • HTML Technician (front-end engineering)
    • Photography / DJing / carnivorous plant enthusiast
    • Y! Photos (2005-2007,) Flickr (present)
  • The Point / Things I’m Going To Talk Aboot
  • The Point / Things I’m Going To Talk Aboot
    • Where we:
    • ( were / are / going ) with web audio
    • How we got this far
    The Point / Things I’m Going To Talk Aboot
    • “ Show me something shiny ”
    The Point / Things I’m Going To Talk Aboot
    • Inform + inspire + excite -> go and make shiny things
    The Point / Things I’m Going To Talk Aboot
  • A (largely) silent web
    • Sound is one of the senses.
    • Yet, it seems to be largely missing from the web.
    • WTF?
    • It’s complicated.
    • [ anti-audio rant here ]
  • A Brief History of Web Audio
    • <bgsound=&quot;xfiles.mid&quot; controls=&quot;console&quot; loop=&quot;5&quot;>
    • <!-- inline player UI, often QuickTime -->
    A Brief History of Web Audio
    • <!-- autostart === auto-annoy, hidden evil -->
    • <embed src=&quot;hamsterdance.wav&quot; autostart=true loop=true volume=100 hidden=true>
    • <noembed>
    • <bgsound src=&quot;hamsterdance.wav&quot;>
    • </noembed>
    A Brief History of Web Audio
    • <!-- Adobe Flash embed code, 2011 -->
    • <OBJECT classid=&quot;clsid:D27CDB6E-AE6D-11cf-96B8-444553540000&quot; codebase=&quot;http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0&quot; WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; id=&quot;myMovieName&quot;>
    • <PARAM NAME=movie VALUE=&quot;flashSoundPlayer.swf&quot;>
    • <PARAM NAME=quality VALUE=high>
    • <PARAM NAME=bgcolor VALUE=#FFFFFF>
    • <EMBED href=&quot;flashSoundPlayer.swf&quot; quality=high bgcolor=#FFFFFF WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; NAME=&quot;myMovieName&quot; ALIGN=&quot;&quot; TYPE=&quot;application/x-shockwave-flash&quot; PLUGINSPAGE=&quot;http://www.macromedia.com/go/getflashplayer&quot;></EMBED>
    • </OBJECT>
    A Brief History of Web Audio
  • The (Mis-)Truth Is Out There.
    • The scary part...
    • These are from search results. In 2011.
  • <BGSOUND> + <EMBED> Limitations
    • Slow. Buggy. Inconsistent.
    <BGSOUND> + <EMBED> Limitations
    • <embed> loads a plugin/helper binary
    • (QT, RealPlayer, WMP)
    <BGSOUND> + <EMBED> Limitations
    • Parameters differ. loop=true vs. loop=5
    <BGSOUND> + <EMBED> Limitations
    • <embed> sometimes scriptable via JS. Little/no API docs per plugin. (+buggy.) *
    <BGSOUND> + <EMBED> Limitations * Excluding Flash 8+, documented and scriptable via ExternalInterface.
  • Which brings us to…
  •  
  • Ah, the future. So shiny.
    • <audio src=&quot;yayhtml5.mp3&quot; controls>
    • new Audio('yayhtml5.mp3');
  • So simple, too!
  •  
  • Browser format support is varied.
  • (Shock, horror.)
  • So it’s not that simple.
    • <audio src=&quot;html5.mp3&quot; controls>
    • <!– for non-HTML5 browsers… -->
    • Download <a href=&quot;html5.mp3&quot;>html5.mp3</a>.
    • </audio>
  • Furthermore…
  • No single format is supported in all major browsers.* * Not even .WAV, as of March 2011. Boourns.
  • So it’s not really that simple.
  • At least two formats are required.
    • <audio controls>
    • <source src=&quot;html5.mp3&quot; />
    • <source src=&quot;html5.ogg&quot; />
    • Download <a href=&quot;html5.mp3&quot;>html5.mp3</a> or <a href=&quot;html5.ogg&quot;>html5.ogg</a>
    • </audio>
  • AND…
  • Your client wants it to work in IE 6.
  • (As with any new technology, there are always a few “buts”…)
  • If only <audio> were as simple as <img> .
  • This is unlikely, given technical, philosophical and legal complexities.
  • If only <audio> were as simple as <img> . The A/V Codec Fight
  • Format Wars
    • Mozilla + Google (OGG/WEB-M) vs. Apple + Microsoft (AAC/MP4/H.264)
    Format Wars
    • Web-M = “free”*, H.264 = IP/$$$
    Format Wars * Mumble mumble something about MPEG-LA threatening Web-M, allegedly violating IP, etc.
    • Developer headaches, 2+ encodings, confusion
    Format Wars
  • And then there’s mobile.
  • “ Special snowflake” devices
  • Only one sound at a time
  • *Under most circumstances. iOS 4.2 locks this down pretty well. No auto-play allowed*
  • Pop-up blocker-style security model for Audio.play()
  • o nfinish()-> play() allowed, otherwise, fail
  • (As with any new technology, there are always a few “buts”…)
  • So it’s really not that simple.
  • Other Annoyances
  • Occasional Safari/OS X audio load failure.
  • &quot;Safari&quot; (Windows) = No HTML5 A/V support.
  • Download &quot;Safari + QuickTime&quot; = HTML5 A/V.
  • How is HTML5 Audio Implemented?
    • Multiple vendors targeting W3 spec
    How is HTML5 Audio Implemented?
    • &quot;Plug-in&quot; architecture (eg. Safari -> QT Plugin)
    How is HTML5 Audio Implemented?
    • Per spec, user should be able to install plugins
    How is HTML5 Audio Implemented?
    • XiphQT -> Safari (and iTunes) do OGG
    How is HTML5 Audio Implemented?
    • Chrome, Firefox /Mozilla include F/OSS-type libraries for playback
    How is HTML5 Audio Implemented?
    • IE 9: WMP /DirectAudio / ??
    How is HTML5 Audio Implemented?
  • Potential vs. Reality
    • HTML5, like AJAX, is not “magical.” *
    Potential vs. Reality * Unless you’re in marketing, and then maybe it is.
    • The potential is there.
    Potential vs. Reality
    • We'll get there, eventually.
    Potential vs. Reality
    • For now, bugs and quirks will likely:
    Potential vs. Reality
    • For now, bugs and quirks will likely:
    • * Enhance job security
    Potential vs. Reality
    • For now, bugs and quirks will likely:
    • * Enhance job security
    • * Enhance grey hair
    Potential vs. Reality
  • (As with any new technology, there are always a few “buts”…) [ enhance ]
  • Can I Use HTML5 Audio Today?
  • (Might as well) Go for it.
  •  
  • Use responsibly, tread carefully.
  • HTML5 Audio via JavaScript
    • Audio().canPlayType() =
    • 'probably','maybe', 'no' *
    • The most un-computer-like response I’ve ever seen. Reflects the reality of fragmented format support.
    HTML5 Audio via JavaScript * Spec changed 'no' to '', to be a non-truthy return value.
    • Audio().canPlayType('audio/mp3'); Audio().canPlayType('audio/mpeg; codecs=&quot;mp3&quot;'); More detail should = “probably”
    HTML5 Audio via JavaScript
    • Audio can use HTTP partials, range requests. Arbitrary seek is a benefit.
    HTML5 Audio via JavaScript
    • A taste of events: canplay, load, ended, error, progress, timeupdate, waiting
    HTML5 Audio via JavaScript
  • And How About Flash?
    • Single-vendor/platform consistency advantage
    And How About Flash?
    • Overall, best non-HTML5 fallback for MP3/MP4
    And How About Flash?
  • Is Flash Dead Yet?
  • No.
  • http://isflashdeadyet.com/
  • HTML5 Isn’t Going To Kill Flash?
  • No.
  • Flash remains a handy fallback for non-HTML5 UAs and is the de-facto method for “secure” RTMP streaming, etc.
  • So HTML5 And Flash Can Co-Exist…
  • CAN I USE BOTH AT THE SAME TIME TO PLAY AUDIO ON BOTH IE 6 AND THE iPHONE, DEPENDING ON WHICH TECHNOLOGY IS AVAILABLE?
  • Exactly. :)
  • This is why SoundManager 2 exists.
    • JavaScript API for sound
    SoundManager 2
    • HTML5 Audio() and/or Flash as needed (abstracted) ~10 KB (gzip)
    SoundManager 2
    • soundManager.onready(function() { soundManager.play( 'mySound', '/path/to/an.mp3’ );
    • }); JS talks to Flash (ExternalInterface) or HTML5 Audio(), depending on support. Transparent to developer.
    SoundManager 2
    • [ demos ]
    SoundManager 2
    • Why use an abstraction?
    SoundManager 2
    • Why use an abstraction? Support for both HTML4 + HTML5
    SoundManager 2
    • Why use an abstraction? Support for both HTML4 + HTML5 Normalize HTML5 differences
    SoundManager 2
  • SO ULTIMATELY TO PLAY MP3/MP4 EVERYWHERE, YOU MUST HAVE FLASH?
  • Correct. It’s either that, or split formats.
  • THAT IS LAME!
  • Unfortunately, yes. LAME is also a great free MP3 encoder. ;)
  • Bonus Flash Bits
    • Flash audio API: Well-designed, overall
    Bonus Flash Bits
    • Revamped with Flash 8, extended in 9, moreso in 10
    Bonus Flash Bits
    • Some conveniences/extras beyond HTML5 audio
    Bonus Flash Bits
    • Audio metadata access (waveform/spectrum) = nifty visualizations http://flic.kr/p/9btbyE
    Bonus Flash Bits
  • A Glimpse Into The Future
    • HTML5 audio: A good start
    A Glimpse Into The Future
    • Meets basics for sound apps (near-Flash parity)
    A Glimpse Into The Future
    • Pending: Bug finding, fixes, improvements and eventual stability
    A Glimpse Into The Future
  • What’s Next?
  • Mozilla Audio Data API / Webkit “Web Audio” API
    • Access/manipulate raw audio bytes during playback
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
    • Generate sound entirely from JavaScript
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
    • Mozilla released their API with Firefox 4 http://videos.mozilla.org/serv/blizzard/audio-slideshow/ https://wiki.mozilla.org/Audio_Data_API
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
    • Chrome 10+ beta with &quot;Web Audio&quot; enabled in about:flags http://chromium.googlecode.com/svn/trunk/samples/audio/o3d-webgl-samples/pool.html http://chromium.googlecode.com/svn/trunk/samples/audio/simple.html
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
  • A Glimpse Into The Future: Mozilla/Webkit Audio APIs
    • [ demo? ]
  • Show me the shiny.
  • In late 2010, Technics announced that they were ceasing production of the DJ industry hallmark, the SL-1200 turntable.
  • 1972 - 2010
  • As a DJ / Web Developer, It seemed appropriate to take some sort of action.
  • So I started looking at CSS, which does boxes quite well.
  •  
  • CSS3’s border-radius finally lets us draw circles.
  •  
  • We can also apply background:url() and overflow:hidden …
  •  
  • Mozilla and Webkit give us -moz-transform and –webkit-transform:rotate()
  •  
  •  
  • <input type=“range” … />
  • <input type=“crossfader”> ? ;)
  • Prototype Goals
    • Recreate core turntable UI + behaviours in a browser.
    Prototype Goals
    • HTML5 audio should cover most sound requirements. Load, pause/play, seek etc.
    Prototype Goals
    • Hardware acceleration should keep it fast, too.
    Prototype Goals
    • Initially considered canvas, went with HTML + CSS. Layout advantage, fast render still possible.
    Prototype Findings
    • HTML5 audio works, despite browser quirks (and support issues.) The fix: Flash fallback.
    Prototype Findings
    • Heavily dependent on acceleration. transform:rotate3d() etc. Smooth in Webkit on OS X, some audio lag on Windows. Might be OK on IE 9. Acceleration is still “new.”
    Prototype Findings
    • [ demo ]
    Turntable Prototype
    • How about pitch bending, scratch, EQ effects etc.? (This is where Flash 10 / Audio Data APIs come in.)
    Turntable Prototype
    • [ demo ]
    Turntable Prototype
    • [ demo ]
    Turntable Prototype Layout/box model mode (RGBA outline and background colors)
    • [ demo ]
    Turntable Prototype For fun, an alternate skin (background-color, etc.)
  • The (Near) End
  • In Closing
    • HTML5: Tons of potential.
    • Bugs to be found + fixed.
    • X-platform/device support.
    • Audio Data APIs = shiny.
  • In Closing
    • Use abstractions as needed to get similar experiences in more places.
    • My hope is that SoundManager 2 becomes redundant over time.
  • “ You see, HTML5 audio is like a shark…”
  •  
  • In Closing Creative Commons-Licensed Photos &quot;Butts&quot; by arbyreed - http://flic.kr/p/jvCCh &quot;Untitled&quot; (facepalm) by Zack Sheppard - http://flic.kr/p/7gwXNt &quot;20100216_0082_1600x1067&quot; (hockey photo) by Les_Stockton - http://flic.kr/p/7DL6NU &quot;Snow Phone&quot; by WKeown - http://flic.kr/p/7Ef7Aa &quot;Safari&quot; by aditza121 - http://flic.kr/p/k7kkq &quot;Holly (and shadow) mid jump&quot; by Ali Smiles :) - http://flic.kr/p/5ZKWKY &quot;I can't believe this illustration had no caption&quot; by Ross Harmes - http://flic.kr/p/9jmjyt &quot;Water Slide&quot; by Ross Harmes - http://flic.kr/p/9jJRB7 fin.