Your SlideShare is downloading. ×
  • Like
  • Save
"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller

  • 2,323 views
Published

A brief history of audio on the web, its present shape as of mid-2011, where things are going, and some examples of how you might use audio today on the web.

A brief history of audio on the web, its present shape as of mid-2011, where things are going, and some examples of how you might use audio today on the web.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,323
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
1
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. “ Probably, Maybe, No”: The State Of HTML5 Audio
  • 2. Hi.
    • Scott Schiller (@schill)
    • HTML Technician (front-end engineering)
    • Photography / DJing / carnivorous plant enthusiast
    • Y! Photos (2005-2007,) Flickr (present)
  • 3. The Point / Things I’m Going To Talk Aboot
  • 4. The Point / Things I’m Going To Talk Aboot
    • Where we:
    • ( were / are / going ) with web audio
  • 5.
    • How we got this far
    The Point / Things I’m Going To Talk Aboot
  • 6.
    • “ Show me something shiny ”
    The Point / Things I’m Going To Talk Aboot
  • 7.
    • Inform + inspire + excite -> go and make shiny things
    The Point / Things I’m Going To Talk Aboot
  • 8. A (largely) silent web
  • 9.
    • Sound is one of the senses.
  • 10.
    • Yet, it seems to be largely missing from the web.
  • 11.
    • WTF?
  • 12.
    • It’s complicated.
  • 13.
    • [ anti-audio rant here ]
  • 14. A Brief History of Web Audio
  • 15.
    • <bgsound=&quot;xfiles.mid&quot; controls=&quot;console&quot; loop=&quot;5&quot;>
    • <!-- inline player UI, often QuickTime -->
    A Brief History of Web Audio
  • 16.
    • <!-- autostart === auto-annoy, hidden evil -->
    • <embed src=&quot;hamsterdance.wav&quot; autostart=true loop=true volume=100 hidden=true>
    • <noembed>
    • <bgsound src=&quot;hamsterdance.wav&quot;>
    • </noembed>
    A Brief History of Web Audio
  • 17.
    • <!-- Adobe Flash embed code, 2011 -->
    • <OBJECT classid=&quot;clsid:D27CDB6E-AE6D-11cf-96B8-444553540000&quot; codebase=&quot;http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0&quot; WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; id=&quot;myMovieName&quot;>
    • <PARAM NAME=movie VALUE=&quot;flashSoundPlayer.swf&quot;>
    • <PARAM NAME=quality VALUE=high>
    • <PARAM NAME=bgcolor VALUE=#FFFFFF>
    • <EMBED href=&quot;flashSoundPlayer.swf&quot; quality=high bgcolor=#FFFFFF WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; NAME=&quot;myMovieName&quot; ALIGN=&quot;&quot; TYPE=&quot;application/x-shockwave-flash&quot; PLUGINSPAGE=&quot;http://www.macromedia.com/go/getflashplayer&quot;></EMBED>
    • </OBJECT>
    A Brief History of Web Audio
  • 18. The (Mis-)Truth Is Out There.
  • 19.
    • The scary part...
  • 20.
    • These are from search results. In 2011.
  • 21. <BGSOUND> + <EMBED> Limitations
  • 22.
    • Slow. Buggy. Inconsistent.
    <BGSOUND> + <EMBED> Limitations
  • 23.
    • <embed> loads a plugin/helper binary
    • (QT, RealPlayer, WMP)
    <BGSOUND> + <EMBED> Limitations
  • 24.
    • Parameters differ. loop=true vs. loop=5
    <BGSOUND> + <EMBED> Limitations
  • 25.
    • <embed> sometimes scriptable via JS. Little/no API docs per plugin. (+buggy.) *
    <BGSOUND> + <EMBED> Limitations * Excluding Flash 8+, documented and scriptable via ExternalInterface.
  • 26. Which brings us to…
  • 27.  
  • 28. Ah, the future. So shiny.
  • 29.
    • <audio src=&quot;yayhtml5.mp3&quot; controls>
    • new Audio('yayhtml5.mp3');
  • 30. So simple, too!
  • 31.  
  • 32. Browser format support is varied.
  • 33. (Shock, horror.)
  • 34. So it’s not that simple.
  • 35.
    • <audio src=&quot;html5.mp3&quot; controls>
    • <!– for non-HTML5 browsers… -->
    • Download <a href=&quot;html5.mp3&quot;>html5.mp3</a>.
    • </audio>
  • 36. Furthermore…
  • 37. No single format is supported in all major browsers.* * Not even .WAV, as of March 2011. Boourns.
  • 38. So it’s not really that simple.
  • 39. At least two formats are required.
  • 40.
    • <audio controls>
    • <source src=&quot;html5.mp3&quot; />
    • <source src=&quot;html5.ogg&quot; />
    • Download <a href=&quot;html5.mp3&quot;>html5.mp3</a> or <a href=&quot;html5.ogg&quot;>html5.ogg</a>
    • </audio>
  • 41. AND…
  • 42. Your client wants it to work in IE 6.
  • 43. (As with any new technology, there are always a few “buts”…)
  • 44. If only <audio> were as simple as <img> .
  • 45. This is unlikely, given technical, philosophical and legal complexities.
  • 46. If only <audio> were as simple as <img> . The A/V Codec Fight
  • 47. Format Wars
  • 48.
    • Mozilla + Google (OGG/WEB-M) vs. Apple + Microsoft (AAC/MP4/H.264)
    Format Wars
  • 49.
    • Web-M = “free”*, H.264 = IP/$$$
    Format Wars * Mumble mumble something about MPEG-LA threatening Web-M, allegedly violating IP, etc.
  • 50.
    • Developer headaches, 2+ encodings, confusion
    Format Wars
  • 51. And then there’s mobile.
  • 52. “ Special snowflake” devices
  • 53. Only one sound at a time
  • 54. *Under most circumstances. iOS 4.2 locks this down pretty well. No auto-play allowed*
  • 55. Pop-up blocker-style security model for Audio.play()
  • 56. o nfinish()-> play() allowed, otherwise, fail
  • 57. (As with any new technology, there are always a few “buts”…)
  • 58. So it’s really not that simple.
  • 59. Other Annoyances
  • 60. Occasional Safari/OS X audio load failure.
  • 61. &quot;Safari&quot; (Windows) = No HTML5 A/V support.
  • 62. Download &quot;Safari + QuickTime&quot; = HTML5 A/V.
  • 63. How is HTML5 Audio Implemented?
  • 64.
    • Multiple vendors targeting W3 spec
    How is HTML5 Audio Implemented?
  • 65.
    • &quot;Plug-in&quot; architecture (eg. Safari -> QT Plugin)
    How is HTML5 Audio Implemented?
  • 66.
    • Per spec, user should be able to install plugins
    How is HTML5 Audio Implemented?
  • 67.
    • XiphQT -> Safari (and iTunes) do OGG
    How is HTML5 Audio Implemented?
  • 68.
    • Chrome, Firefox /Mozilla include F/OSS-type libraries for playback
    How is HTML5 Audio Implemented?
  • 69.
    • IE 9: WMP /DirectAudio / ??
    How is HTML5 Audio Implemented?
  • 70. Potential vs. Reality
  • 71.
    • HTML5, like AJAX, is not “magical.” *
    Potential vs. Reality * Unless you’re in marketing, and then maybe it is.
  • 72.
    • The potential is there.
    Potential vs. Reality
  • 73.
    • We'll get there, eventually.
    Potential vs. Reality
  • 74.
    • For now, bugs and quirks will likely:
    Potential vs. Reality
  • 75.
    • For now, bugs and quirks will likely:
    • * Enhance job security
    Potential vs. Reality
  • 76.
    • For now, bugs and quirks will likely:
    • * Enhance job security
    • * Enhance grey hair
    Potential vs. Reality
  • 77. (As with any new technology, there are always a few “buts”…) [ enhance ]
  • 78. Can I Use HTML5 Audio Today?
  • 79. (Might as well) Go for it.
  • 80.  
  • 81. Use responsibly, tread carefully.
  • 82. HTML5 Audio via JavaScript
  • 83.
    • Audio().canPlayType() =
    • 'probably','maybe', 'no' *
    • The most un-computer-like response I’ve ever seen. Reflects the reality of fragmented format support.
    HTML5 Audio via JavaScript * Spec changed 'no' to '', to be a non-truthy return value.
  • 84.
    • Audio().canPlayType('audio/mp3'); Audio().canPlayType('audio/mpeg; codecs=&quot;mp3&quot;'); More detail should = “probably”
    HTML5 Audio via JavaScript
  • 85.
    • Audio can use HTTP partials, range requests. Arbitrary seek is a benefit.
    HTML5 Audio via JavaScript
  • 86.
    • A taste of events: canplay, load, ended, error, progress, timeupdate, waiting
    HTML5 Audio via JavaScript
  • 87. And How About Flash?
  • 88.
    • Single-vendor/platform consistency advantage
    And How About Flash?
  • 89.
    • Overall, best non-HTML5 fallback for MP3/MP4
    And How About Flash?
  • 90. Is Flash Dead Yet?
  • 91. No.
  • 92. http://isflashdeadyet.com/
  • 93. HTML5 Isn’t Going To Kill Flash?
  • 94. No.
  • 95. Flash remains a handy fallback for non-HTML5 UAs and is the de-facto method for “secure” RTMP streaming, etc.
  • 96. So HTML5 And Flash Can Co-Exist…
  • 97. CAN I USE BOTH AT THE SAME TIME TO PLAY AUDIO ON BOTH IE 6 AND THE iPHONE, DEPENDING ON WHICH TECHNOLOGY IS AVAILABLE?
  • 98. Exactly. :)
  • 99. This is why SoundManager 2 exists.
  • 100.
    • JavaScript API for sound
    SoundManager 2
  • 101.
    • HTML5 Audio() and/or Flash as needed (abstracted) ~10 KB (gzip)
    SoundManager 2
  • 102.
    • soundManager.onready(function() { soundManager.play( 'mySound', '/path/to/an.mp3’ );
    • }); JS talks to Flash (ExternalInterface) or HTML5 Audio(), depending on support. Transparent to developer.
    SoundManager 2
  • 103.
    • [ demos ]
    SoundManager 2
  • 104.
    • Why use an abstraction?
    SoundManager 2
  • 105.
    • Why use an abstraction? Support for both HTML4 + HTML5
    SoundManager 2
  • 106.
    • Why use an abstraction? Support for both HTML4 + HTML5 Normalize HTML5 differences
    SoundManager 2
  • 107. SO ULTIMATELY TO PLAY MP3/MP4 EVERYWHERE, YOU MUST HAVE FLASH?
  • 108. Correct. It’s either that, or split formats.
  • 109. THAT IS LAME!
  • 110. Unfortunately, yes. LAME is also a great free MP3 encoder. ;)
  • 111. Bonus Flash Bits
  • 112.
    • Flash audio API: Well-designed, overall
    Bonus Flash Bits
  • 113.
    • Revamped with Flash 8, extended in 9, moreso in 10
    Bonus Flash Bits
  • 114.
    • Some conveniences/extras beyond HTML5 audio
    Bonus Flash Bits
  • 115.
    • Audio metadata access (waveform/spectrum) = nifty visualizations http://flic.kr/p/9btbyE
    Bonus Flash Bits
  • 116. A Glimpse Into The Future
  • 117.
    • HTML5 audio: A good start
    A Glimpse Into The Future
  • 118.
    • Meets basics for sound apps (near-Flash parity)
    A Glimpse Into The Future
  • 119.
    • Pending: Bug finding, fixes, improvements and eventual stability
    A Glimpse Into The Future
  • 120. What’s Next?
  • 121. Mozilla Audio Data API / Webkit “Web Audio” API
  • 122.
    • Access/manipulate raw audio bytes during playback
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
  • 123.
    • Generate sound entirely from JavaScript
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
  • 124.
    • Mozilla released their API with Firefox 4 http://videos.mozilla.org/serv/blizzard/audio-slideshow/ https://wiki.mozilla.org/Audio_Data_API
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
  • 125.
    • Chrome 10+ beta with &quot;Web Audio&quot; enabled in about:flags http://chromium.googlecode.com/svn/trunk/samples/audio/o3d-webgl-samples/pool.html http://chromium.googlecode.com/svn/trunk/samples/audio/simple.html
    A Glimpse Into The Future: Mozilla/Webkit Audio APIs
  • 126. A Glimpse Into The Future: Mozilla/Webkit Audio APIs
    • [ demo? ]
  • 127. Show me the shiny.
  • 128. In late 2010, Technics announced that they were ceasing production of the DJ industry hallmark, the SL-1200 turntable.
  • 129. 1972 - 2010
  • 130. As a DJ / Web Developer, It seemed appropriate to take some sort of action.
  • 131. So I started looking at CSS, which does boxes quite well.
  • 132.  
  • 133. CSS3’s border-radius finally lets us draw circles.
  • 134.  
  • 135. We can also apply background:url() and overflow:hidden …
  • 136.  
  • 137. Mozilla and Webkit give us -moz-transform and –webkit-transform:rotate()
  • 138.  
  • 139.  
  • 140. <input type=“range” … />
  • 141. <input type=“crossfader”> ? ;)
  • 142. Prototype Goals
  • 143.
    • Recreate core turntable UI + behaviours in a browser.
    Prototype Goals
  • 144.
    • HTML5 audio should cover most sound requirements. Load, pause/play, seek etc.
    Prototype Goals
  • 145.
    • Hardware acceleration should keep it fast, too.
    Prototype Goals
  • 146.
    • Initially considered canvas, went with HTML + CSS. Layout advantage, fast render still possible.
    Prototype Findings
  • 147.
    • HTML5 audio works, despite browser quirks (and support issues.) The fix: Flash fallback.
    Prototype Findings
  • 148.
    • Heavily dependent on acceleration. transform:rotate3d() etc. Smooth in Webkit on OS X, some audio lag on Windows. Might be OK on IE 9. Acceleration is still “new.”
    Prototype Findings
  • 149.
    • [ demo ]
    Turntable Prototype
  • 150.
    • How about pitch bending, scratch, EQ effects etc.? (This is where Flash 10 / Audio Data APIs come in.)
    Turntable Prototype
  • 151.
    • [ demo ]
    Turntable Prototype
  • 152.
    • [ demo ]
    Turntable Prototype Layout/box model mode (RGBA outline and background colors)
  • 153.
    • [ demo ]
    Turntable Prototype For fun, an alternate skin (background-color, etc.)
  • 154. The (Near) End
  • 155. In Closing
    • HTML5: Tons of potential.
    • Bugs to be found + fixed.
    • X-platform/device support.
    • Audio Data APIs = shiny.
  • 156. In Closing
    • Use abstractions as needed to get similar experiences in more places.
    • My hope is that SoundManager 2 becomes redundant over time.
  • 157. “ You see, HTML5 audio is like a shark…”
  • 158.  
  • 159. In Closing Creative Commons-Licensed Photos &quot;Butts&quot; by arbyreed - http://flic.kr/p/jvCCh &quot;Untitled&quot; (facepalm) by Zack Sheppard - http://flic.kr/p/7gwXNt &quot;20100216_0082_1600x1067&quot; (hockey photo) by Les_Stockton - http://flic.kr/p/7DL6NU &quot;Snow Phone&quot; by WKeown - http://flic.kr/p/7Ef7Aa &quot;Safari&quot; by aditza121 - http://flic.kr/p/k7kkq &quot;Holly (and shadow) mid jump&quot; by Ali Smiles :) - http://flic.kr/p/5ZKWKY &quot;I can't believe this illustration had no caption&quot; by Ross Harmes - http://flic.kr/p/9jmjyt &quot;Water Slide&quot; by Ross Harmes - http://flic.kr/p/9jJRB7 fin.