Video Game Music Overview

Game audio Karen Collins

OUTLINE
Defining Games Audio
History
Functions of Games Audio
Approaches to composition
diegesis and dynamic audio

Who cares about games?
Majority of people in western world play games (75% of heads of households (in the United States) play computer or video games)‏
average age for a game player is thirty
nearly half of game players are female (55% male, 43% female).
“ Women over the age of 18 represent a greater portion of the gameplaying population (28%) than boys from ages 6 to 17 (21%).

Who cares about game audio?
83% of Adult Gamers listed sound as one of the most important video game console elements
47% of Game Console owners (18-25 yr) hook up their game console to a home theatre system
48% of Hardcore Gamers said surround sound is a purchase driver for next generation consoles

What is game audio?
Dialogue/speech sounds
Sound effects
Music/score
Ambience
Interface sounds (menu, etc.)‏

What’s wrong?

How games are not movies: The problems of games audio
Linearity Vs. Non-Linearity/ unpredictability, mixing
Interactivity with player
Temporality (length)‏
Budgets
Delivery methods/technology

What is Interactive Audio?
sound events occur in direct reaction to a player’s movements. The player triggers the cue , and can repeatedly activate it, such as by making a character jump up and down.

Interactive Audio: Footsteps Enemy music cue

What is adaptive audio?
“ adaptive” audio is generally referred to as sound that reacts to transformations in the gameplay environment—such as a change from day to night set by a game’s timer mechanism . Adaptive audio is not directly triggered by a player

Adaptive Audio Change in music day to night Wolf, crow

What is Dynamic Audio
Dynamic audio is a broad term to refer to both interactive and adaptive audio

A brief history of game audio
(in 4 minutes!)‏

History

Functions of Games Sound
Commercial functions
Structural reinforcement
Immersion/suspension of disbelief
Semiotics functions
Narrative functions
Emotional functions
Aesthetic functions
Kinetic functions

1. Commercial functions
Sell games
Sell artists
Sell soundtracks
Sell movies
Sell other games
Branding: familiarization

Sales, marketing and branding

2. Structural reinforcement
Mask structure and editing
Enhance/reinforce structure of game
Create structure
Situate player in game matrix
Indicate change in narrative
Enhance continuity

Structural reinforcement

3. Immersion/suspension of disbelief
Add realism:
Emphasize real sounds
Represent location/spatialization
Create illusion
Cover external noise

Immersion/suspension of disbelief

4. Semiotic functions
Sonic symbols, leitmotifs: identify objects and decrease learning curve
Preparatory function
Acousmatic sound: Focus attention, identify goals
Convey meaning, emotion
Comment
Add slant or bias

Semiotic Preparatory Function

A test of the semiotics of space/place in Zelda
Ranch (farm)‏
Windmill
Fire temple
Evil Castle underground
Valley of gypsy thieves
Flying fairy
Dawn
Temple of time
Ice cavern
A B C D E F G H I

5. Narrative functions
Advancement of plots
Locate player in storyline
Anchor player in terms of game matrix/level, etc.
anticipate/foreshadow
Dialogue can be a major event in plot
Access character’s thoughts/reveal character details
Reveal goals

Narrative: correct response

6. Emotional functions
Emotional and physiological: manipulate player
Disturb or enhance gameplay: battle scenes
Mood induction
Create intimacy (immersive)‏

Emotional

7. Aesthetic functions
Improve our appreciation of the game
Make the game better/more enjoyable
Make it “cool”
Indicate genre/style
Inter-textual referencing

8. Kinetic functions
Aerobic exercise: fitness
Music as motivating factor for movement
Edutainment for toddlers, etc.

Kinetic Functions
Dance Dance Revolution, etc.

Problems (and some solutions) of Interactive Audio in Games

Mixing: Problems: -too many sounds -no “post production” mixing -unpredictable timings : results in “muddy”, clash of sounds

Mixing Solutions!
Reduce amount of sounds
Control frequencies of sounds
Assign priorities in code
E.g. dialogue = priority “1”
Ambience = priority “4”
Gunshot = priority “2”
Music = priority “3”
Footsteps = priority “5”

Problem: technological constraints
Trying to write songs in a few kB for constrained technology such as mobile phones

Solutions: technological constraints:
Playing a single sample at different pitches to produce variation.
The "pitch it up, play it down" technique for saving space.

Sampling to save size
This game contains only a single "trumpet fall" sound effect
modify the playback sample rate so that each time you pick up a piece of cheese, the sound is played at a different pitch
randomly vary the pitch during gameplay, so there's no repeating pattern to it
The sound tends to mask the transition from one mix to another, helping to create a more seamless audio experience.
same kind of pitch-shifting effect applied to car horn beeps: (fig 1)‏

Sampling to save space
Another way to save space is using the "pitch it up, play it down" technique.
take your original, high-resolution sound effect and transpose it up an octave, halving the length.
Then convert it to a low-resolution compressed format, like this: (cat growl, octave up, car start, octave up. Then in the game, play it down an octave: (octave down).
Although the game sound might be a little crunchy, you've just cut the size of your file in half without losing too much audio fidelity. Obviously, the higher you pitch the sound, the "crunchier" the playback will become, but the technique can be used for custom instruments as well as sound effects, and is particularly effective on the tiny speakers in mobile devices.

Problem: transitions from cue to cue
No “post” = unpredictable timings
Abrupt jumps are jarring

(1) Solution: transitions: Cross fade

Problem: Multi-player interactivity
Unscripted events!
parameter-based music is difficult:
Music cannot be tied to specific events or locations
Music cannot be tied to specific parameters

Solutions: multi-player interactivity
Work with multi-players!
Asheron’s Call II, Lord of the Rings: players can bring instruments and jam out with each other

Multi-player music

Problem: listener fatigue
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.
Loops are boring.

Solutions: Listener fatigue
Learn how to use silence!
“ boredom switch”: timer for set levels to stop
VARIABILITY!

Dynamic Composition: Variability
variable tempo
variable pitch
variable volume
variable DSP
variable melodies (algorithmic generation)‏
Variable harmony (chord arrangements)‏
Variable harmony (keys)‏
variable mixing
Variable sequencing
variable structure

Variable tempo

Variable pitch

Variable Pitch

Variable volume

Variable DSP
Change DSP effects to change music:
E.g. reverb to make it softer, dreamier
Phasing to create a “dazed” effect
Overdrive to make it more aggressive, etc.

Variable melodies
Ballblazer (1984): the music plays forever, without repeating itself but without straying too far from the original theme. The bassline doesn't vary at all, only the lead line in the higher register is algorithmically varied.

Variable harmony
Asheron’s Call 2: changes in mode indicate changes in gameplay:
A player standing alone in an area would hear the background score in Lydian, but as creatures fill the area, the music transitioned to Phrygian. If monsters outnumber player, music transition to diminished mode and quickened drum groove

Variable mixing
Adjust mix by
Bringing instruments up in mix that were quiet or silent before
Drop elements from mix

Cheese Racer
Muting and unmuting multiple tracks in a MIDI file to produce various mixes.
2 drum loop samples,
three tones: bass, melody and chords
In the code for the first level, ten different combinations of tracks: percussion + bass, percussion + bass + melody, bass + melody + chords, etc.

(cont..)‏
Notice how the mix changes every time the mouse gets a piece of cheese. (Fig 2 : exit) (Fig 3)‏
Music loop 40 seconds. 10 different combinations: therefore, each level has more than 6 minutes of different music mixes. “Because the mix changes depending on gameplay, the music will never play exactly the same way twice, thereby increasing variation and decreasing "ear fatigue."”
In fact, there are two more sets of tracks playing two more styles of music, using the same tempo and percussion tracks as the first level.
Entire game contains almost 20 minutes of various music mixes, using only 68K of compressed sample and MIDI data.

Dynamic Music- example
Layer of instrumentation added when Mario rides Yoshi in Super Mario World.

Russian Squares
Using layers, each cue increment represents a puzzle row being cleared by the player.

Variable sequencing
Alter sequences of music
Zelda Ocarina: Hyrule field

Variable structure
Create “transition matrix”: chart out all possible directions for each sequence, and create transitions for marker points/jumps

Transition Matrix: No One Lives Forever
Music and Arranging by Guy Whitmore and Tobin Buttram
This music clip moves through all six of the music states for the Ambush theme, demonstrating transitions called from a transition matrix. Time Music State 0:00 6 (combat 2) 1:22 transition 1:30 2 (sub-ambient) 2:14 transition 2:16 4 (suspense) 3:10 transition 3:12 3 (ambient) 4:08 transition 4:12 5 (combat 1) 5:33 transition 1 (silence)‏

No One Lives Forever
EarthOrbit :
The Ambush theme starts in music state 5 (combat 1), transitions to music state 2 (ambient), then transitions to music state 6 (combat 2) with motifs.

What else can be done?
Dynamic music offers all kinds of interesting possibilities to have music as a driving force in a game!

Super Mario DS
New Super Mario Bros DS
Enemies jump and fly in time to music

Electroplankton

Vib Ribbon

Levels of dynamic sound
non-dynamic, non-diegetic sound
Adaptive non-diegetic sound
Interactive non-diegetic sound
Non-dynamic diegetic sound
Adaptive diegetic sound
Interactive diegetic sound
Kinetic diegetic and non-diegetic
(reminder: Non-Diegetic: Outside the character’s space. Diegetic: “real sounds”, inside character’s space)‏

Non-dynamic, non-diegetic sound
non-dynamic, linear music or sound found most frequently in the cinematic sequences (movie sequences in which the player’s functions are stopped while a short clip to advance the plot is played). In other words, the player has no control over the possibility of interrupting the music (short of resetting/turning the game off).

Non-dynamic, non-diegetic sound

Sound events occurring in reaction to gameplay, but which are unaffected by the player’s direct movements
The player cannot re-trigger these events.

Sound events occurring in the character’s space
The player can re-trigger these events.

In non-dynamic diegetic audio , the sound event occurs in the character’s space, but the character does not directly interact with it.

Adaptive diegetic sound
Diegetic sounds adapt to gameplay environment (e.g. night to day)‏

Adaptive Diegetic Sounds

Interactive diegetic sounds
occur in the character’s space and with which the player’s character can interact. The player instigates the audio cue, but does not necessarily affect the sound of the event once the cue is triggered.

Interactive Diegetic Sounds

Kinetic interactive Sound
Player interacts with the sound by mimicking the sound/image kinetically
Guitar Hero
Donkey Konga

http://www.gamessound.com	901
http://gamessound.com	65
http://www.slideshare.net	8

Video Game Music Overview

by Karen Collins on Apr 26, 2008

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 974

Statistics

Video Game Music Overview Presentation Transcript