• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Effective HTML5 game audio
 

Effective HTML5 game audio

on

  • 1,517 views

This presentation presents the strengths and challenges of using HTML5 audio. It then presents a working game audio system architecture which leverages HTML5 audio. Finally, it looks at the future of ...

This presentation presents the strengths and challenges of using HTML5 audio. It then presents a working game audio system architecture which leverages HTML5 audio. Finally, it looks at the future of web audio.

Statistics

Views

Total Views
1,517
Views on SlideShare
1,517
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Split into 2 sectionsTime for questions between the 2 sectionsAlso have demos
  • Note emphasis on stream – has a noticeable impact when implementing an HTML5 audio system. We’ll talk about the impact in a short whileWill talk a little bit more about the Web Audio API if there’s timeW3C Working Audio Group is responsible for HTML5 Audio and Web Audio API
  • Firefox loop bug issue:https://bugzilla.mozilla.org/show_bug.cgi?id=449157
  • https://developer.mozilla.org/en-US/docs/Media_formats_supported_by_the_audio_and_video_elements#Browser_compatibility
  • On timely activation,Specs does not state that playback should begin immediately on play activation
  • On server functionality,HTML5 audio is a stream element, therefore it relies on server streaming support i.e. make partial requests
  • Demo:Demo Firefox: perfect working exampleDemo IE 9: delayed response + fixSafari delayed response + performance issueChrome: Server no partial content bug
  • Sound Man
  • Why multiple sound channels?Allows elements which logically belong together to be manipulated as one (e.g. mute speech only)
  • Why per channel volume control?Tuning:Per channel basis is great for tuning sound mixesWhy per active sound volume control?Some games need to manipulate sound volumes post sound launch e.g. ship thruster sound volumeSound performance impact?- Talk more about this when we talk about optimizations
  • This is the proposed game audio stack
  • Blue are public classesGreen are private classesThe audio class are enhanced HTML5 audio element instances
  • Blue are public classesGreen are private classesThe audio class are enhanced HTML5 audio element instances
  • Reason behind Hyper Grav audio decomposition:In-game SFX behavior is wholly different from FE SFX behaviorWanted to make sure all in-game objects were recycled when returning to the front endIn-game audio interface allowed this by maintaining all game-specific object references required for audio playbackWished I had decomposed AudioMan’s interface further
  • Reason behind Hyper Grav audio decomposition:In-game SFX behavior is wholly different from FE SFX behaviorWanted to make sure all in-game objects were recycled when returning to the front endIn-game audio interface allowed this by maintaining all game-specific object references required for audio playbackWished I had decomposed AudioMan’s interface further

Effective HTML5 game audio Effective HTML5 game audio Presentation Transcript

  • EFFECTIVE HTML5 GAME AUDIOChris Khoo (chris.khoo@wappworks.com)
  • TOPICS1. HTML5 Audio  Definition  The good, the bad, and the ugly2. HTML5 audio game layer architecture
  • INTRODUCTION TO HTML5AUDIO
  • HTML5 AUDIO HTML5 audio is part of the HTML5 specifications managed by the World Wide Web Consortium (W3C) The <audio> element  Represents a sound or audio stream  Not designed for real-time audio mixing W3C is working on a Web Audio API as an alternative audio solution  Advanced sound manipulation functionality  Not purely stream related
  • THE GOOD HTML5 audio API support is standard across all modern browsers  EXCEPTION: Firefox versions prior to Jan 1, 2012 do not support the loop flag It is possible to implement a game sound engine using HTML5 audio
  • THE BAD Limited audio control  Coarse control of audio playback position and volume  No audio panning support  No sound filters Non-contiguous looping  Expect a slight pause whenever a sound sample loops
  • THE BAD Non-standard audio formats across browsers  No one audio format works for all browsers  A list of supported audio formats on a per browser basis is available on Mozilla‟s website  To ensure the highest browser compatibility, be sure to support multiple audio formats (ogg, mp3, wav)
  • THE BAD Spotty HTML5 audio support on default mobile device web browsers  Android  Android 2.x: Audio does not play  Android 3.0+: Only one audio instance allowed to produce sound at a time  iOS  Audio must be triggered through user interaction  Apple is actively working to prevent workarounds  Support in third party mobile web browsers is much better  Firefox mobile implements HTML5 audio perfectly
  • THE UGLY No guarantee of timely activation  Some browsers have a noticeable delay between calling the play() function and sound playback starting  Safari - always true  IE 9 - occurs when modifying an audio element‟s src parameter prior to play back Beware of audio stream stalls  HTML5 audio streaming can sometimes stall without reason  It seems to happen more frequently with WebKit browsers  A “stalled” event is fired whenever this occurs  Call the load() function to manually restart streaming
  • THE UGLY Server functionality can impact HTML5 audio behavior  On WebKit browsers (Chrome and Safari), audio/video seek functionality is quietly disabled if the audio web server does not support partial requests  Disabling seek causes  play() call completes without error but playback does not occur  „ended‟ event does not fire on playback complete  „Sound file format corrupted‟ exception is thrown when trying to replay sounds  Generally happens the first time an audio is referenced  Doesn‟t happen on subsequent accesses if the audio is cached Other known quirks  Firefox 3.6: short sounds don‟t play  No longer a problem since Mozilla is phasing out this version  Safari: Terrible audio playback performance
  • DEMO TIME
  • SHOULD WE USE HTML5 AUDIO?Pros Browser audio support is steadily improving It works across all platforms including mobile web  It‟s possible to work around most of the issues through careful sound design and good engineering but audio experience, in general, is compromised Popular HTML5 game acceleration platforms support audio through the <audio> element  CocoonJs and AppMobi take this approachCons Limited audio manipulation functionality Inherently more complicated to implement Audio experience still varies from browser to browser
  • QUESTIONS?
  • BUILDING AN HTML5 GAMEAUDIO SYSTEM
  • SYSTEM REQUIREMENTS Graceful degradation  What if there‟s no HTML5 audio support? Sound loading/caching Sound playback  Sound looping support Mute/Unmute Volume control  Globally  Per active sound Multiple sound channels  Per channel volume control  Stop all active sounds in a channel
  • SYSTEM REQUIREMENTS Polyphony limits  Limit maximum number of simultaneous sounds per channel  Why?  Performance reasons: sound playback has a performance impact  Less is more: too many sounds playing simultaneously can be chaotic  Option to drop the oldest sound or ignore the latest sound request when polyphony limit reached
  • GAME AUDIO STACK
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEMAudioSys Game-specific audio interface Isolates game audio implementation from the rest of the game systems Helps in implementing graceful degradation  Game must continue to function when audio functionality is not available Audio samples are loaded and managed by an AudioCache instance Music and SFX audio are played back using separate AudioChannel instances
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEMAudioCache Handles audio asset loading  selects the best audio format Maps sound IDs to actual audio samples
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEMAudioChannel Handles sound playback Supports looping Provides channel playback controls  Stop  Volume control  Mute Monitors and enforces polyphony limits Plays sounds from an AudioCache and returns an AudioChannel::Handle for managing an actively playing sound Reusable component
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEMAudioChannel::Handle Façade class for managing an AudioChannel‟s active audio instance Provides playback functionality  Stop  Set volume Provide functionality to check playback status
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEMAudioFactory Instantiates augmented HTML5 audio instances Why?  Adds a custom loop API  Addresses missing loop parameter support in earlier Firefox versions  Replaces the existing volume control with a two tier volume control system Augmentation is applied on an audio instance instead of through class extension  It is not recommended to extend an HTML5 audio class directly because HTML5 audio class name is not standard
  • GAME AUDIO SYSTEM
  • GAME AUDIO SYSTEM
  • DEMO TIME
  • AUDIO SYSTEM IMPROVEMENTS Decompose AudioSys into multiple classes based on use case  Improves readability and maintainability  Hyper Grav‟s audio system decomposition  AudioMan: Game audio system  AudioListener: In-game specific audio interface Modify AudioCache to manage sound pools on a per sound asset basis  AudioChannel requests an audio instance from AudioCache and it returns an instance from the sound asset‟s instance pool  Fixes sound playback delay in IE 9
  • AUDIO SYSTEM IMPROVEMENTS Support asset filtering in AudioCache  Current implementation selects format based on the order of source URLs and the browser‟s audio format support  Add filtering to control audio format selection on a per platform basis  Why?  On some platforms, audio format selection has to be made for performance reasons e.g. audio compression has a significant performance impact on mobile
  • ALTERNATIVE WEB AUDIO APIS W3C Web Audio API  Official Web Audio API  Designed to handle more complex audio applications (e.g. games)  API is still in development  Currently, only Chrome implements a version of the Web Audio API Mozilla Audio API extension  Extends existing HTML5 audio and video elements to support more complex audio application  Mozilla has officially declared the API deprecated
  • THE END Questions?
  • REFERENCES W3C Audio Working Group http://www.w3.org/2011/audio/ Latest W3C HTML5 audio specification http://www.w3.org/TR/html5/the-audio-element.html Mozilla‟s audio data browser compatibility list https://developer.mozilla.org/en- US/docs/Media_formats_supported_by_the_audio_and_ video_elements#Browser_compatibility Proposed W3C Web Audio API http://www.w3.org/TR/webaudio/ Mozilla Audio API extension https://developer.mozilla.org/en- US/docs/Introducing_the_Audio_API_Extension