Nothing Odd about Audio

4,005 views

Published on

Presentation about importance of audio in games, and some research directions.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,005
On SlideShare
0
From Embeds
0
Number of Embeds
1,206
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Nothing Odd about Audio

    1. 1.  PRESENTATION FOR Silicon Knights Karen Collins, University of Waterloo www.GamesSound.com There's Nothing Odd About Audio
    2. 2. FOCUS <ul><li>Who cares about audio? </li></ul><ul><li>Research directions in game audio </li></ul><ul><li>The future of game audio </li></ul>
    3. 3. Who Cares about Game Audio? <ul><li>“ Enhance your gaming world with a custom soundtrack on your Xbox 360™” </li></ul><ul><li>Call of Duty® 2: “Can anything beat storming enemy positions to the sounds of the Bee Gees ‘Stayin’ Alive’?” </li></ul>
    4. 4. Who Cares about Game Audio?
    5. 5. 2007 Tarriff 22A <ul><li>ESA: music “is never the main feature of any communication that might occur on a video game publisher’s site.” </li></ul>Who Cares about Game Audio?
    6. 6. <ul><li>83% of Adult Gamers listed sound as one of the most important video game console elements </li></ul><ul><li>47% of Game Console owners (18-25) hook up their console to a home theatre system </li></ul><ul><li>48% of Hardcore Gamers said surround sound is a purchase driver for next gen consoles. [According to the ESA] </li></ul>Who Cares about Game Audio?
    7. 7. Who Cares about Game Audio?
    8. 8. What difference does sound make?
    9. 9. What difference does sound make?
    10. 10. Recent research... (Jørgensen) “ ... I thought it’s duller to play without sound. But I didn’t ... think it affected the game so much.” “ it’s like you’re reminded that this actually is a computer game. So when there’s no sound, it’s just like two animated figures standing there, shooting at each other.” “ It becomes like, you become more systematic, they become almost like numbers. Like, that unit fell, that unit fell, and now I lost one and stuff. There’s almost no emotion behind the fact that they’re falling” “ The first thing I notice is that my time of reaction has increased by ten...” “ I didn’t always notice things happening” “ What difference does sound make to games?
    11. 11. Music driving gameplay elements. New Super Mario Bros (Nintendo DS 2006)‏ (Koji Kondo)‏ What difference does sound make?
    12. 12. What difference does sound make? <ul><li>God of War: excellent mixing, good use of music and sound effects </li></ul><ul><li>Interesting dialogue and vocal samples </li></ul><ul><li>-won over a dozen &quot;Game of the Year Awards&quot;. </li></ul><ul><li>-named best PlayStation 2 Action game of all time by. </li></ul><ul><li>Academy of Interactive Arts & Sciences 9th Annual D.I.C.E. (Design, Innovate, Create, Entertain) Awards: Outstanding Achievement: Original Music Composition and Sound Design </li></ul><ul><li>--coincidence? </li></ul>
    13. 13. Current Academic Research, and the Future of Game Audio
    14. 14. A very brief intro to semiotics, and how it can be useful for games…. Semiotics of Sound
    15. 15. Semiotics and Game Audio <ul><li>Sound/music as a symbolic language </li></ul><ul><li>What (and how) does music/sound communicate? </li></ul><ul><li>Semiotics helps us to understand what messages are being communicated to us. </li></ul>An Example (“stand-alone” music, not associated with visual image)
    16. 16. An Example: User Testing <ul><li>Futuristic </li></ul><ul><li>Night time </li></ul><ul><li>Dark </li></ul><ul><li>Chase </li></ul><ul><li>Bad guy/monster/insectoid </li></ul>
    17. 17. A Very Quick Semiotic Analysis
    18. 18. A VERY QUICK SEMIOTIC ANALYSIS <ul><li>Where have we heard similar sounding motifs before? </li></ul><ul><li>It’s militant, so we could check military movies </li></ul><ul><li>It’s about the future, so we could check futuristic movies... </li></ul>
    19. 19. So... What is this saying? =
    20. 20. Our Future= ??
    21. 21. What Does This Have to do with Games? <ul><li>Specific combinations of sounds result in specific meanings </li></ul><ul><li>How can we study and break these meanings down into “rules”/grammar to generate algorithms for generative audio or user- generated playlists? </li></ul><ul><ul><li>What combinations are effective? </li></ul></ul><ul><ul><li>What variations/substitutions can be made with and without changing meanings? </li></ul></ul>
    22. 22. Algorithmically Generative Audio <ul><li>Problems: </li></ul><ul><li>How do we create emotionally effective algorithmic adaptive audio? </li></ul><ul><li>What aspects of audio carry meaning ? </li></ul><ul><ul><li>How do these work individually and together? </li></ul></ul><ul><ul><li>What universals (within the Western world) are there that can be codified? </li></ul></ul><ul><li>How generalized/simplified can/do the rules / grammar need to be? </li></ul>
    23. 23. Parameter Based Music: <ul><li>Number/action of non-playing characters </li></ul><ul><li>Number/action of playing characters </li></ul><ul><li>Actions </li></ul><ul><li>Locations (place, time of day, etc.)‏ </li></ul><ul><li>Scripted or unscripted events </li></ul><ul><li>Player health or enemy health </li></ul><ul><li>Difficulty </li></ul><ul><li>Timing </li></ul><ul><li>Player properties (skills, endurance)‏ </li></ul><ul><li>Bonus objects </li></ul><ul><li>Movement (speed, direction, rhythm)‏ </li></ul><ul><li>“ Camera” angle </li></ul>
    24. 24. Generative audio in spore
    25. 25. Generative audio: Ballblazer (LucasArts 1985)
    26. 26. GENERATIVE SOUNDTRACKS: SONY’S CINESCORE
    27. 27. Generative/Procedural Aspects in Games .kkrieger: An FPS in 97 kb: procedurally generated in real time.
    28. 28. Input Sample Synthesized Result &quot;Texture Synthesis from Multiple Sources&quot;, by Li-Yi Wei. In SIGGRAPH 2003 Applications and Sketches. Generative Game Elements
    29. 29. Making a Sound Granular Parker and Behm 2007 ”Generating Audio Textures by Example”, Journal of Game Development, 2007
    30. 30. Granular Synthesis Examples <ul><li>Crowd </li></ul><ul><li>Tennis </li></ul><ul><li>Speech </li></ul>Crowd and speech examples borrowed from Leonard Paul at Vancouver Film School
    31. 31. Granular: Remaining Open Questions <ul><li>What elements in a sound effect can be varied while still maintaining the “ meaning ” of the sound? </li></ul><ul><li>How can we create AI systems that are aware of these potential meanings, and make real-time adjustments to sounds in a game? </li></ul><ul><li>How to develop an “ audio physics engine ”: e.g. footsteps change based on how much player is carrying, etc. </li></ul>
    32. 32. Methods for better algorithms <ul><li>Neural networks on tagged MIDI files </li></ul><ul><li>tagging games </li></ul>
    33. 33. Distributed Classification Examples
    34. 34. What Does the User Get? <ul><li>Contribution to knowledge </li></ul><ul><li>Feeling of being part of community </li></ul><ul><li>fun! </li></ul><ul><li>See Luis von Ahn, “Games with a Purpose” IEEE Computer Magazine or </li></ul><ul><li>“ Why do tagging systems work?” Conference on Human Factors in Computing Systems CHI 06 </li></ul>
    35. 35. ESP Game Player 1 Player 2 GUESSING: KID GUESSING: BOY GUESSING: CAR GUESSING: HAT GUESSING: CAR SUCCESS! Consensus on: CAR Input:
    36. 36. Games for Audio Tagging: Interactively Building an Online Database <ul><li>Three games under development: </li></ul><ul><ul><li>Game like ESP game but for audio </li></ul></ul><ul><ul><li>Audio-visual game in which users select image to audio </li></ul></ul><ul><ul><li>Audio-visual based game where users select appropriate audio content for visual image </li></ul></ul>
    37. 37. Adapting the Algorithms For MIR <ul><li>MIR = Music Information Retrieval </li></ul><ul><li>Retrieval based on bpm, harmonic content, melodic intervals, timbre, etc. </li></ul><ul><li>How can we use MIR techniques to make better game audio? </li></ul><ul><ul><li>User-generated playlists + new algorithms = </li></ul></ul><ul><ul><li>appropriate and new user-generated audio content </li></ul></ul>
    38. 38. Unified Architecture Routing, allocation and scheduling (includes system clocks) Input: Game Data Parameters Detection (Beat tracking, phrase matching, pitch matching, harmony and key matching). Prediction (Neural nets, fuzzy logic.) Wave banks Audio Data (MIDI) Algorithmic composition/modelling Samplers, synths, tone generators Intelligent mixing engine AI Audio Engine

    ×