The document summarizes the state of HTML5 audio, including its history, current limitations, and future potential. It discusses how browser support for audio formats is fragmented, requiring multiple formats for broad compatibility. It also notes that Flash still provides the best cross-browser support and is needed as a fallback for non-HTML5 browsers. The document demonstrates an HTML5 audio player prototype built with CSS but dependent on Flash for full functionality across browsers.

1. “ Probably, Maybe, No”: The State Of HTML5 Audio

2. Hi. Scott Schiller (@schill) HTML Technician (front-end engineering) Photography / DJing / carnivorous plant enthusiast Y! Photos (2005-2007,) Flickr (present)

3. The Point / Things I’m Going To Talk Aboot

4. The Point / Things I’m Going To Talk Aboot Where we: ( were / are / going ) with web audio

5. How we got this far The Point / Things I’m Going To Talk Aboot

6. “ Show me something shiny ” The Point / Things I’m Going To Talk Aboot

7. Inform + inspire + excite -> go and make shiny things The Point / Things I’m Going To Talk Aboot

8. A (largely) silent web

9. Sound is one of the senses.

10. Yet, it seems to be largely missing from the web.

11. WTF?

12. It’s complicated.

13. [ anti-audio rant here ]

14. A Brief History of Web Audio

15. <bgsound=&quot;xfiles.mid&quot; controls=&quot;console&quot; loop=&quot;5&quot;> <!-- inline player UI, often QuickTime --> A Brief History of Web Audio

16. <!-- autostart === auto-annoy, hidden evil --> <embed src=&quot;hamsterdance.wav&quot; autostart=true loop=true volume=100 hidden=true> <noembed> <bgsound src=&quot;hamsterdance.wav&quot;> </noembed> A Brief History of Web Audio

17. <!-- Adobe Flash embed code, 2011 --> <OBJECT classid=&quot;clsid:D27CDB6E-AE6D-11cf-96B8-444553540000&quot; codebase=&quot;http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0&quot; WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; id=&quot;myMovieName&quot;> <PARAM NAME=movie VALUE=&quot;flashSoundPlayer.swf&quot;> <PARAM NAME=quality VALUE=high> <PARAM NAME=bgcolor VALUE=#FFFFFF> <EMBED href=&quot;flashSoundPlayer.swf&quot; quality=high bgcolor=#FFFFFF WIDTH=&quot;550&quot; HEIGHT=&quot;400&quot; NAME=&quot;myMovieName&quot; ALIGN=&quot;&quot; TYPE=&quot;application/x-shockwave-flash&quot; PLUGINSPAGE=&quot;http://www.macromedia.com/go/getflashplayer&quot;></EMBED> </OBJECT> A Brief History of Web Audio

18. The (Mis-)Truth Is Out There.

19. The scary part...

20. These are from search results. In 2011.

21. <BGSOUND> + <EMBED> Limitations

22. Slow. Buggy. Inconsistent. <BGSOUND> + <EMBED> Limitations

23. <embed> loads a plugin/helper binary (QT, RealPlayer, WMP) <BGSOUND> + <EMBED> Limitations

24. Parameters differ. loop=true vs. loop=5 <BGSOUND> + <EMBED> Limitations

25. <embed> sometimes scriptable via JS. Little/no API docs per plugin. (+buggy.) * <BGSOUND> + <EMBED> Limitations * Excluding Flash 8+, documented and scriptable via ExternalInterface.

26. Which brings us to…

28. Ah, the future. So shiny.

29. <audio src=&quot;yayhtml5.mp3&quot; controls> new Audio('yayhtml5.mp3');

30. So simple, too!

32. Browser format support is varied.

33. (Shock, horror.)

34. So it’s not that simple.

35. <audio src=&quot;html5.mp3&quot; controls> <!– for non-HTML5 browsers… --> Download <a href=&quot;html5.mp3&quot;>html5.mp3</a>. </audio>

36. Furthermore…

37. No single format is supported in all major browsers.* * Not even .WAV, as of March 2011. Boourns.

38. So it’s not really that simple.

39. At least two formats are required.

40. <audio controls> <source src=&quot;html5.mp3&quot; /> <source src=&quot;html5.ogg&quot; /> Download <a href=&quot;html5.mp3&quot;>html5.mp3</a> or <a href=&quot;html5.ogg&quot;>html5.ogg</a> </audio>

41. AND…

42. Your client wants it to work in IE 6.

43. (As with any new technology, there are always a few “buts”…)

44. If only <audio> were as simple as <img> .

45. This is unlikely, given technical, philosophical and legal complexities.

46. If only <audio> were as simple as <img> . The A/V Codec Fight

47. Format Wars

48. Mozilla + Google (OGG/WEB-M) vs. Apple + Microsoft (AAC/MP4/H.264) Format Wars

49. Web-M = “free”*, H.264 = IP/$$$ Format Wars * Mumble mumble something about MPEG-LA threatening Web-M, allegedly violating IP, etc.

50. Developer headaches, 2+ encodings, confusion Format Wars

51. And then there’s mobile.

52. “ Special snowflake” devices

53. Only one sound at a time

54. *Under most circumstances. iOS 4.2 locks this down pretty well. No auto-play allowed*

55. Pop-up blocker-style security model for Audio.play()

56. o nfinish()-> play() allowed, otherwise, fail

57. (As with any new technology, there are always a few “buts”…)

58. So it’s really not that simple.

59. Other Annoyances

60. Occasional Safari/OS X audio load failure.

61. &quot;Safari&quot; (Windows) = No HTML5 A/V support.

62. Download &quot;Safari + QuickTime&quot; = HTML5 A/V.

63. How is HTML5 Audio Implemented?

64. Multiple vendors targeting W3 spec How is HTML5 Audio Implemented?

65. &quot;Plug-in&quot; architecture (eg. Safari -> QT Plugin) How is HTML5 Audio Implemented?

66. Per spec, user should be able to install plugins How is HTML5 Audio Implemented?

67. XiphQT -> Safari (and iTunes) do OGG How is HTML5 Audio Implemented?

68. Chrome, Firefox /Mozilla include F/OSS-type libraries for playback How is HTML5 Audio Implemented?

69. IE 9: WMP /DirectAudio / ?? How is HTML5 Audio Implemented?

70. Potential vs. Reality

71. HTML5, like AJAX, is not “magical.” * Potential vs. Reality * Unless you’re in marketing, and then maybe it is.

72. The potential is there. Potential vs. Reality

73. We'll get there, eventually. Potential vs. Reality

74. For now, bugs and quirks will likely: Potential vs. Reality

75. For now, bugs and quirks will likely: * Enhance job security Potential vs. Reality

76. For now, bugs and quirks will likely: * Enhance job security * Enhance grey hair Potential vs. Reality

77. (As with any new technology, there are always a few “buts”…) [ enhance ]

78. Can I Use HTML5 Audio Today?

79. (Might as well) Go for it.

81. Use responsibly, tread carefully.

82. HTML5 Audio via JavaScript

83. Audio().canPlayType() = 'probably','maybe', 'no' * The most un-computer-like response I’ve ever seen. Reflects the reality of fragmented format support. HTML5 Audio via JavaScript * Spec changed 'no' to '', to be a non-truthy return value.

84. Audio().canPlayType('audio/mp3'); Audio().canPlayType('audio/mpeg; codecs=&quot;mp3&quot;'); More detail should = “probably” HTML5 Audio via JavaScript

85. Audio can use HTTP partials, range requests. Arbitrary seek is a benefit. HTML5 Audio via JavaScript

86. A taste of events: canplay, load, ended, error, progress, timeupdate, waiting HTML5 Audio via JavaScript

87. And How About Flash?

88. Single-vendor/platform consistency advantage And How About Flash?

89. Overall, best non-HTML5 fallback for MP3/MP4 And How About Flash?

90. Is Flash Dead Yet?

91. No.

92. http://isflashdeadyet.com/

93. HTML5 Isn’t Going To Kill Flash?

94. No.

95. Flash remains a handy fallback for non-HTML5 UAs and is the de-facto method for “secure” RTMP streaming, etc.

96. So HTML5 And Flash Can Co-Exist…

97. CAN I USE BOTH AT THE SAME TIME TO PLAY AUDIO ON BOTH IE 6 AND THE iPHONE, DEPENDING ON WHICH TECHNOLOGY IS AVAILABLE?

98. Exactly. :)

99. This is why SoundManager 2 exists.

100. JavaScript API for sound SoundManager 2

101. HTML5 Audio() and/or Flash as needed (abstracted) ~10 KB (gzip) SoundManager 2

102. soundManager.onready(function() { soundManager.play( 'mySound', '/path/to/an.mp3’ ); }); JS talks to Flash (ExternalInterface) or HTML5 Audio(), depending on support. Transparent to developer. SoundManager 2

103. [ demos ] SoundManager 2

104. Why use an abstraction? SoundManager 2

105. Why use an abstraction? Support for both HTML4 + HTML5 SoundManager 2

106. Why use an abstraction? Support for both HTML4 + HTML5 Normalize HTML5 differences SoundManager 2

107. SO ULTIMATELY TO PLAY MP3/MP4 EVERYWHERE, YOU MUST HAVE FLASH?

108. Correct. It’s either that, or split formats.

109. THAT IS LAME!

110. Unfortunately, yes. LAME is also a great free MP3 encoder. ;)

111. Bonus Flash Bits

112. Flash audio API: Well-designed, overall Bonus Flash Bits

113. Revamped with Flash 8, extended in 9, moreso in 10 Bonus Flash Bits

114. Some conveniences/extras beyond HTML5 audio Bonus Flash Bits

115. Audio metadata access (waveform/spectrum) = nifty visualizations http://flic.kr/p/9btbyE Bonus Flash Bits

116. A Glimpse Into The Future

117. HTML5 audio: A good start A Glimpse Into The Future

118. Meets basics for sound apps (near-Flash parity) A Glimpse Into The Future

119. Pending: Bug finding, fixes, improvements and eventual stability A Glimpse Into The Future

120. What’s Next?

121. Mozilla Audio Data API / Webkit “Web Audio” API

122. Access/manipulate raw audio bytes during playback A Glimpse Into The Future: Mozilla/Webkit Audio APIs

123. Generate sound entirely from JavaScript A Glimpse Into The Future: Mozilla/Webkit Audio APIs

124. Mozilla released their API with Firefox 4 http://videos.mozilla.org/serv/blizzard/audio-slideshow/ https://wiki.mozilla.org/Audio_Data_API A Glimpse Into The Future: Mozilla/Webkit Audio APIs

125. Chrome 10+ beta with &quot;Web Audio&quot; enabled in about:flags http://chromium.googlecode.com/svn/trunk/samples/audio/o3d-webgl-samples/pool.html http://chromium.googlecode.com/svn/trunk/samples/audio/simple.html A Glimpse Into The Future: Mozilla/Webkit Audio APIs

126. A Glimpse Into The Future: Mozilla/Webkit Audio APIs [ demo? ]

127. Show me the shiny.

128. In late 2010, Technics announced that they were ceasing production of the DJ industry hallmark, the SL-1200 turntable.

129. 1972 - 2010

130. As a DJ / Web Developer, It seemed appropriate to take some sort of action.

131. So I started looking at CSS, which does boxes quite well.

133. CSS3’s border-radius finally lets us draw circles.

135. We can also apply background:url() and overflow:hidden …

137. Mozilla and Webkit give us -moz-transform and –webkit-transform:rotate()

140. <input type=“range” … />

141. <input type=“crossfader”> ? ;)

142. Prototype Goals

143. Recreate core turntable UI + behaviours in a browser. Prototype Goals

144. HTML5 audio should cover most sound requirements. Load, pause/play, seek etc. Prototype Goals

145. Hardware acceleration should keep it fast, too. Prototype Goals

146. Initially considered canvas, went with HTML + CSS. Layout advantage, fast render still possible. Prototype Findings

147. HTML5 audio works, despite browser quirks (and support issues.) The fix: Flash fallback. Prototype Findings

148. Heavily dependent on acceleration. transform:rotate3d() etc. Smooth in Webkit on OS X, some audio lag on Windows. Might be OK on IE 9. Acceleration is still “new.” Prototype Findings

149. [ demo ] Turntable Prototype

150. How about pitch bending, scratch, EQ effects etc.? (This is where Flash 10 / Audio Data APIs come in.) Turntable Prototype

151. [ demo ] Turntable Prototype

152. [ demo ] Turntable Prototype Layout/box model mode (RGBA outline and background colors)

153. [ demo ] Turntable Prototype For fun, an alternate skin (background-color, etc.)

154. The (Near) End

155. In Closing HTML5: Tons of potential. Bugs to be found + fixed. X-platform/device support. Audio Data APIs = shiny.

156. In Closing Use abstractions as needed to get similar experiences in more places. My hope is that SoundManager 2 becomes redundant over time.

157. “ You see, HTML5 audio is like a shark…”

159. In Closing Creative Commons-Licensed Photos &quot;Butts&quot; by arbyreed - http://flic.kr/p/jvCCh &quot;Untitled&quot; (facepalm) by Zack Sheppard - http://flic.kr/p/7gwXNt &quot;20100216_0082_1600x1067&quot; (hockey photo) by Les_Stockton - http://flic.kr/p/7DL6NU &quot;Snow Phone&quot; by WKeown - http://flic.kr/p/7Ef7Aa &quot;Safari&quot; by aditza121 - http://flic.kr/p/k7kkq &quot;Holly (and shadow) mid jump&quot; by Ali Smiles :) - http://flic.kr/p/5ZKWKY &quot;I can't believe this illustration had no caption&quot; by Ross Harmes - http://flic.kr/p/9jmjyt &quot;Water Slide&quot; by Ross Harmes - http://flic.kr/p/9jJRB7 fin.