Problems and Solutions in Game Audio

[object Object],[object Object],[object Object]

Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object]

A Brief Historical Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],2.1 8-BIT

Pong (Atari 1972) (Al Alcorn) 2.1 8-BIT

Space Invaders (Midway 1978) ,[object Object],2.1 8-BIT

Rally X (Namco/Midway 1980) ,[object Object],2.1 8-BIT

Making a “beep” in assembly ( Time & memory constraints ) ,[object Object],From Using Assembly Language by Allen L Wyatt 2.1 8-BIT

Technological Constraints ,[object Object],2.1 8-BIT

Atari VCS (2600) ,[object Object],2.1 8-BIT

Captain Comic (Color Dreams 1988) 2.1 8-BIT

Working With Constraints: Nintendo NES ,[object Object],[object Object],2.1 8-BIT

Ballblazer (LucasArts 1984) (Peter Langston) ,[object Object],[object Object],[object Object],[object Object],[object Object],2.8-BIT

Working With Constraints: Commodore 64 Track 1 (e.g. “Level_One”) Instrument 1 (envelope, waveform, effect filters, etc.) Instrument 2 (envelope, waveform, effect filters, etc.) Rob Hubbard’s Module Format 2.1 8-BIT Pattern 1 (sequence of notes) Pattern 2 (sequence of notes) Pattern 3 (sequence of notes)

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],;C900 pat38: .byte $86, $0D, $00 ; set instrument .byte $23, $D5 .byte $9B, $05, $23 ; play note .byte $86, $0E, $00 ; set instrument .byte $9B, $05, $5A ; play note .byte $D6 .byte $9B, $05, $5A ; play note .byte $86, $0D, $00 ; set instrument .byte $9B, $05, $23 .byte $D5 .byte $9B, $05, $23 ; play note .byte $86, $0E, $00 ; set instrument .byte $9B, $05, $5A ; play note .byte $D6 .byte $9B, $05, $5A ; play note .byte $86, $0F, $00 ; set instrument .byte $00, $D0 .byte $9B, $03 ; restore state .byte $00 .byte $00, $00 2.1 8-BIT

Shadow of the Beast 2 (Psygnosis 1989) (David Whittaker) 2.2 16-BIT MOD/Tracker on Amiga

Combining modules (in MIDI) with control statements MIDI and the Creation of iMUSE 2.2 16-BIT Land, Michael Z. and Peter N. McConnell. Method and Apparatus for Dynamically Composing Music and Sound Effect Using a Computer Entertainment System . US Patent No. 5,315,057. 24 May, 1994.

Super Mario World (Nintendo 1991) (Koji Kondo) 2.2 16-BIT Musical layering technique Mario jumps on Yoshi & gets extra layer of music (SNES).

Legend of Zelda: Ocarina of Time (Nintendo 1999) (Koji Kondo) (N64) ,[object Object],2.3 64-BIT

The Sims (Maxis 2000) ,[object Object],2.4 128-BIT

Music driving gameplay elements. New Super Mario Bros (Nintendo DS 2006) (Koji Kondo) 2.5 Mobile

State-of-the-Art Today ,[object Object],[object Object],[object Object],[object Object],[object Object],2.6 TODAY

[object Object],[object Object]

Fundamental Problems ,[object Object],[object Object],[object Object],[object Object],[object Object]

3.1 Mixing Who needs mixing? Chicken Shift (Bally 1984)

Problem: Mixing: Unpredictability, Variability 3.1 Mixing

Problem: Mixing: current state of dynamic range … in a popular film … in a popular game Graphics adapted from those supplied by Rob Bridgett of Swordfish Studios . 3.1 Mixing

Solutions: Real-time Weighted Mixing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],3.1 Mixing

Solution: Location-Based Run-Time Mixing ,[object Object],[object Object],[object Object],[object Object],3.1 Mixing

3.2 Variability ,[object Object],[object Object],[object Object]

Solution: Granular Synthesis 3.2 Variability

“ Granular” Synthesis ,[object Object],[object Object],[object Object],3.2 Variability

Granular synthesis: Graphic Equivalent 3.2 Variability Input Sample Synthesized Result "Texture Synthesis from Multiple Sources", by Li-Yi Wei. In SIGGRAPH 2003 Applications and Sketches.

Making a Sound Granular 3.2 Variability Parker and Behm 2007 ”Generating Audio Textures by Example”, Journal of Game Development, 2007

Granular Synthesis Examples ,[object Object],[object Object],[object Object],3.2 Variability Crowd and speech examples borrowed from Leonard Paul at Vancouver Film School

Granular: Remaining Open Questions ,[object Object],[object Object],[object Object],3.2 Variability

3.3 Adaptability Problem : Games are non-linear, unpredictable and very long! A to B: 16 units A to all: 376 units 30 rooms: 11280 units 10 levels: 100K+ units Transitional Units

Solution: Game Audio Algorithms ,[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

Variable Musical Parameters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

#8 Variable (Open) Form ,[object Object],[object Object],[object Object],3.3 Adaptability

#8 Example: Variable (Open) Form 3.3 Adaptability http://www.home.cs.utwente.nl/~zsofi/mozart/ Variations: 11 14 x 2 2 = 1 518 999 334 332 964

#8 Variable Form: Non-linear Sequencing ,[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

#9 Parameter Based Music: Parameters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],The transition matrix approach and the creation of transitional units 3.3 Adaptability

#9 Example: Parameter-Based Music ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

#9 Example: No One Lives Forever ,[object Object],[object Object],[object Object],3.3 Adaptability

#10. Algorithmic Variations (ongoing research focus) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

Semiotics ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

Semiotics of sound: Why is it important? An example… 3.3 Adaptability

Defining a Sound Semiotics Grammar ,[object Object],[object Object],3.3 Adaptability

Distributed Classification Examples 3.3 Adaptability

What Does the User Get? ,[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

ESP Game 3.3 Adaptability Player 1 Player 2 GUESSING: KID GUESSING: BOY GUESSING: CAR GUESSING: HAT GUESSING: CAR SUCCESS! Consensus on: CAR Input:

Games for Audio Tagging: Interactively Building an Online Database ,[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

Adapting the Algorithms For MIR ,[object Object],[object Object],[object Object],[object Object],[object Object],3.3 Adaptability

Unified Architecture Routing, allocation and scheduling (includes system clocks) Input: Game Data Parameters Detection (Beat tracking, phrase matching, pitch matching, harmony and key matching). Prediction (Neural nets, fuzzy logic.) Wave banks Audio Data (MIDI) Algorithmic composition/modelling Samplers, synths, tone generators Intelligent mixing engine AI Audio Engine

Why CS needs Arts (and vice versa)

Thank-you to … Further information: [email_address] www.GamesSound.com (my web site) www.algorithmic.net www.granularsynthesis.com www.audiokinetic.com www.iasig.org

Problems and Solutions in Game Audio

Recommended

Recommended

More Related Content

Similar to Problems and Solutions in Game Audio

Similar to Problems and Solutions in Game Audio (20)

Recently uploaded

Recently uploaded (20)

Problems and Solutions in Game Audio

Editor's Notes