Games & Music
MAC281
@rob_jewitt
robert.jewitt@sunderland.ac.uk 1
Overview
1. In praise of aural
2. Technological developments
3. Taxonomy of game music archetypes
– Adaptive music
– Rhythm-based
– Drama
2
Key reading
3
Visual culture
4
Aural culture?
5
6
7
sensory experience of the environment is a
multimedia one
-TrevorWishart, 1986: 49
8
9
Recorded sound waves are perceived in the exact same
way as non-recorded sound waves
10
Technological developments
11
Technological developments
When CD- and later DVD-based games first allowed
soundtracks to include popular tracks precisely as they would
sound on a CD, game designers leapt at the opportunity to
include a song or two (though seldom much more, due to space
restrictions).
-- Gibbons, 2011
12
13
Format Size Platform
ROM cartridge 16 Mb Sega Mega Drive, SNES
3.5” diskette 1.44 MB PC,Amiga, Atari ST
CD 650 MB PC, Sega Mega CD, Sony PS1, Atari
Jaguar
DVD 4.7 GB (single layer) PC, Sony PS2, Xbox, Xbox 360,Wii
Bluray 25 GB (single layer) Sony PS3
Xenon 2: Megablast
14
Bitmap Brothers
1989
BombThe Bass
1988
Call of Duty: Black Ops
15
Activision,Treyarch, 2010
Call of Duty: Black Ops
16
Activision,Treyarch, 2010
Non-diegetic use ofThe Rolling Stones – ‘Sympathy for the Devil’
Reference to Apocalypse Now’s use of ‘(I Can’t Get No) Satisfaction’
Spec Ops:The Line
17
Spec Ops:The Line
18
Diegetic vs Mimetic
19
For the benefit of the audienceIn the narrative world of the characters
Taxonomy of game music
archetypes
20
Adaptive music
Aka ‘dynamic’ music
Will adapt to the drama, often increasing in
tempo, pitch or volume depending on the action.
21
Licensed linearity
“there is limited adaptability inherent in most
popular music, whereas games require songs that
may need to adapt to gameplay states or player
interaction. Licensed songs are (for the most part)
designed as linear music, and therefore the
placement of this music in a game is generally
limited...as is the genre of the game where such
music may be appropriate....”
--Karen Collins, 2008: 119
22
Non-linear game scoring
23
Paul Haslinger, score composer
The inadequacy of looping
Not only have you eliminated the emotional
effectiveness of the music by generalizing it and not
applying it to a context, but by looping it over and
over, you’ve completely detached the player from even
registering it altogether
-- Scott Morton, 2005
24
The inadequacy of looping
What’s worse, it usually becomes annoying after a
time. Now we've moved down from "why should we
even have music playing here" to "why shouldn't we
turn off the music altogether and listen to MP3s?"
-- Scott Morton, 2005
25
popular music in games became less a
function of the narrative and more an
element of the ludic game mechanics
26
There’s always exceptions…
Final Fantasy X-2
Square Enix, 2003
27
Rhythm-based music games
28
20072005
[They] created a model in which music is not only removed from
the narrative, it is in effect a replacement for it - the music is the
game, and no (or almost no) overarching narrative exists outside
of it
-- Gibbons, 2011
29
30
DJ Hero
31
Rez
32
Child of Eden
33
Audiosurf
34
The weird and the wonderful
35
The weird and the wonderful
36
It fills the gaps…
37
“The new MTV”? (Tessler, 2008)
Narrative-based games
38
GrandTheft Auto III
39
GrandTheft Auto:Vice City
40
41
Fallout 3
Radio stations…
This clever strategy relieves game designers from the
onus of relating the music to the onscreen action. A
radio, after all, can hardly be expected to change its
tune to fit individual circumstances …
-- Gibbons, 2011
42
Radio stations…
A significant downside, however, is that although
players can customize the soundscape of their own
experiences, the songs cannot function in a narrative
way.
-- Gibbons, 2011
43
Bioshock
Employs licensed music into the narrative
Events trigger key uses of music
44
Blues,Twentieth-century blues, they're gettin' me
down.
Blues, escape those weary twentieth-century
blues.
Why, if there's a god in the sky, why shouldn't he
grin
High above this dreary twentieth-century din?

In
this strange illusion, chaos and confusion,
People seem
to lose their way.
What is there to strive for, love or keep
alive for?
Say hey hey, call it a day.

Blues, nothing to
win or to lose, it's getting me down.
Blues, escape those
weary twentieth-century blues.

Why is it that civilized
humanity can make this world so wrong?
In this hurly-
burly of insanity, our dreams cannot last long.
We've
reached a deadline, a press headline, every
sorrow;
Blues value is news value tomorrow.
45
Noël Coward
"Twentieth-Century Blues”
(1932)
The lyrics comment directly upon the
failure of Rapture?
It had to be you…
46
How much is that doggie in the
window?
47
La mer … Beyond the sea…
48
• Acousmatic sound
• Active onscreen sound; often in a different room;
beckons the attention
• Michel Chion (1994, p85)
• Immersion?
49
“La Mer (Beyond the Sea)”
- Django Reinhardt &
StéphaneGrappelli
- Bobby Darin (1959)
Original ‘La Mer’ by
CharlesTrénet (1946)
Quite different lyrics
Somewhere beyond the sea
Somewhere waiting
for me
My lover stands on golden sands
And
watches the ships that go sailin'

Somewhere
beyond the sea
She's there watching for me
If I
could fly like birds on high
Then straight to her
arms 
I'd go sailin'

It's far beyond the stars
It's
near beyond the moon
I know beyond a
doubt
My heart will lead me there soon

We'll
meet beyond the shore
We'll kiss just as
before
Happy we'll be beyond the sea
And
never again I'll go sailin'

I know beyond a
doubt
My heart will lead me there soon
We'll
meet (I know we'll meet) beyond the shore
We'll
kiss just as before
Happy we'll be beyond the
sea
And never again I'll go sailin’
50
Original “La Mer” by
CharlesTrénet (1946)
Quite different lyrics
51
La mer
Qu'on voit danser le long des
golfes clairs
A des reflets d'argent
La mer
Des reflets changeants
Sous la pluie
La mer
Au ciel d'été confond
Ses blancs moutons
Avec les anges si purs
La mer bergère d'azur Infinie
Voyez
Près des étangs
Ces
grands roseaux
mouillés
Voyez
Ces oiseaux
blancs
Et ces maisons rouillées
La mer
Les a bercés
Le long des golfes clairs
Et d'une chanson d'amour
La mer
A bercé mon cœur pour la vie
The sea
Seen dancing along the clear
gulfs
to gleams of silver.
The sea
Of changing reflections
Under the rain.
The sea
Confuses the summer sky's
sheep
With angels so pure,
The sea
Shepherd of blue infinity.
Look!
Next to the ponds
Those tall wet reeds.
Look!
Those white birds
And those rustyhouses
The Sea
Has cradled them
Along the clear gulfs.
The Sea
Has cradled my heart for life.
Summary
Music can play an integral and varied part of the
gaming experience
Imperceptible design decisions employed to add to the
overall ambience
Improved immersion?
52
Work cited & further reading
• KarenCollins (ed.) (2008) Game Sound: An Introduction to the History,Theory, and Practice ofVideo Game Music and Sound
Design. Cambridge, MA and London: MIT Press
• WilliamGibbons (2011), ‘WrapYourTroubles in Dreams: Popular Music, Narrative, and Dystopia in Bioshock, in Games
Studies:The international journal of computer game research,Vol 11, Iss 3. http://gamestudies.org/1103/articles/gibbons
• ClaudiaGorbman (1987). Unheard Melodies: Narrative Film Music. Bloomington: Indiana University Press.
• Scott Morton (2005) ‘Enhancing the Impact of Music in Drama-OrientedGames’, Gamasutra
http://www.gamasutra.com/view/feature/2189/enhancing_the_impact_of_music_in_.php
• Rod Munday (2007). “Music inVideoGames.” In Jamie Sexton (Ed.), Music, Sound and Multimedia: From the Live to the
Virtual (pp. 51-67). Edinburgh: EdinburghUniversity Press.
• HollyTessler (2008) ‘The new MTV?: ElectronicArts and “playing” music’ in Karen Collins (ed.) From Pac-Man to Pop Music:
Interactive Audio in Games and New Media.Aldershot:Ashgat,e
• ZachWalen (2007) “Film Music vs. Game Music:TheCase of Silent Hill.” In Jamie Sexton (Ed.), Music, Sound and Multimedia:
From the Live to theVirtual (pp. 68-81). Edinburgh: EdinburghUniversity Press.
• ZachWhalen (2004). “PlayAlong:An Approach toVideogame Music.” Games Studies:The international journal of computer
game research,Vol 4 No 1 http://www.gamestudies.org/0401/whalen/.
• TrevorWishart (1986) ‘Sound Symbols and Landscapes’ in Simon Emmerson (ed.) The Language Electroacoustic
Music, London: Macmillan
• Resource mapping articles on game music: http://www.ludomusicology.org/bibliography/ 53
Images
• #4 Sam Bald (2008) ‘Eyeball’
• #5 MarkWhale (2010) ‘ Apple iPhone White’
• #8 tantrum_dan (2008) ‘Machesney Alarm Clock’
• #11 robjewitt (2011) ‘Aural sex’
• #13 robjewitt (2011) ‘Street Fighter 2 Sega Mega Drive’
• #19 oc fernando (2011) ‘Pac Man’
• #44 Kaleanderson (2011) ‘Creative Commons’
54
Diegetic vs Mimetic
• In the narrative world of
the characters
• For the benefit of the
audience
55
Uncharted 3
56
Alice: Madness Returns
57
Dead Island
58
Taking a cinematic approach
59
Getting it wrong
• “There are entire movies contained inside today's games …
Unfortunately, video game developers and the players
themselves don't often see this connection. Corners are
cut, sacrifices made, flat-out wrong practices are repeated
time and again, and the gaming media looks upon it and
proclaims it good”
– Andrew High (composer), 2012, Gamasutra
60
Getting it wrong
61
Getting it right
62
Saints Row:TheThird
63

Video Games and Music

  • 1.
  • 2.
    Overview 1. In praiseof aural 2. Technological developments 3. Taxonomy of game music archetypes – Adaptive music – Rhythm-based – Drama 2
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
    sensory experience ofthe environment is a multimedia one -TrevorWishart, 1986: 49 8
  • 9.
  • 10.
    Recorded sound wavesare perceived in the exact same way as non-recorded sound waves 10
  • 11.
  • 12.
    Technological developments When CD-and later DVD-based games first allowed soundtracks to include popular tracks precisely as they would sound on a CD, game designers leapt at the opportunity to include a song or two (though seldom much more, due to space restrictions). -- Gibbons, 2011 12
  • 13.
    13 Format Size Platform ROMcartridge 16 Mb Sega Mega Drive, SNES 3.5” diskette 1.44 MB PC,Amiga, Atari ST CD 650 MB PC, Sega Mega CD, Sony PS1, Atari Jaguar DVD 4.7 GB (single layer) PC, Sony PS2, Xbox, Xbox 360,Wii Bluray 25 GB (single layer) Sony PS3
  • 14.
    Xenon 2: Megablast 14 BitmapBrothers 1989 BombThe Bass 1988
  • 15.
    Call of Duty:Black Ops 15 Activision,Treyarch, 2010
  • 16.
    Call of Duty:Black Ops 16 Activision,Treyarch, 2010 Non-diegetic use ofThe Rolling Stones – ‘Sympathy for the Devil’ Reference to Apocalypse Now’s use of ‘(I Can’t Get No) Satisfaction’
  • 17.
  • 18.
  • 19.
    Diegetic vs Mimetic 19 Forthe benefit of the audienceIn the narrative world of the characters
  • 20.
    Taxonomy of gamemusic archetypes 20
  • 21.
    Adaptive music Aka ‘dynamic’music Will adapt to the drama, often increasing in tempo, pitch or volume depending on the action. 21
  • 22.
    Licensed linearity “there islimited adaptability inherent in most popular music, whereas games require songs that may need to adapt to gameplay states or player interaction. Licensed songs are (for the most part) designed as linear music, and therefore the placement of this music in a game is generally limited...as is the genre of the game where such music may be appropriate....” --Karen Collins, 2008: 119 22
  • 23.
    Non-linear game scoring 23 PaulHaslinger, score composer
  • 24.
    The inadequacy oflooping Not only have you eliminated the emotional effectiveness of the music by generalizing it and not applying it to a context, but by looping it over and over, you’ve completely detached the player from even registering it altogether -- Scott Morton, 2005 24
  • 25.
    The inadequacy oflooping What’s worse, it usually becomes annoying after a time. Now we've moved down from "why should we even have music playing here" to "why shouldn't we turn off the music altogether and listen to MP3s?" -- Scott Morton, 2005 25
  • 26.
    popular music ingames became less a function of the narrative and more an element of the ludic game mechanics 26
  • 27.
    There’s always exceptions… FinalFantasy X-2 Square Enix, 2003 27
  • 28.
  • 29.
    [They] created amodel in which music is not only removed from the narrative, it is in effect a replacement for it - the music is the game, and no (or almost no) overarching narrative exists outside of it -- Gibbons, 2011 29
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
    The weird andthe wonderful 35
  • 36.
    The weird andthe wonderful 36
  • 37.
    It fills thegaps… 37 “The new MTV”? (Tessler, 2008)
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
    Radio stations… This cleverstrategy relieves game designers from the onus of relating the music to the onscreen action. A radio, after all, can hardly be expected to change its tune to fit individual circumstances … -- Gibbons, 2011 42
  • 43.
    Radio stations… A significantdownside, however, is that although players can customize the soundscape of their own experiences, the songs cannot function in a narrative way. -- Gibbons, 2011 43
  • 44.
    Bioshock Employs licensed musicinto the narrative Events trigger key uses of music 44
  • 45.
    Blues,Twentieth-century blues, they'regettin' me down.
Blues, escape those weary twentieth-century blues.
Why, if there's a god in the sky, why shouldn't he grin
High above this dreary twentieth-century din?

In this strange illusion, chaos and confusion,
People seem to lose their way.
What is there to strive for, love or keep alive for?
Say hey hey, call it a day.

Blues, nothing to win or to lose, it's getting me down.
Blues, escape those weary twentieth-century blues.

Why is it that civilized humanity can make this world so wrong?
In this hurly- burly of insanity, our dreams cannot last long.
We've reached a deadline, a press headline, every sorrow;
Blues value is news value tomorrow. 45 Noël Coward "Twentieth-Century Blues” (1932) The lyrics comment directly upon the failure of Rapture?
  • 46.
    It had tobe you… 46
  • 47.
    How much isthat doggie in the window? 47
  • 48.
    La mer …Beyond the sea… 48
  • 49.
    • Acousmatic sound •Active onscreen sound; often in a different room; beckons the attention • Michel Chion (1994, p85) • Immersion? 49
  • 50.
    “La Mer (Beyondthe Sea)” - Django Reinhardt & StéphaneGrappelli - Bobby Darin (1959) Original ‘La Mer’ by CharlesTrénet (1946) Quite different lyrics Somewhere beyond the sea
Somewhere waiting for me
My lover stands on golden sands
And watches the ships that go sailin'

Somewhere beyond the sea
She's there watching for me
If I could fly like birds on high
Then straight to her arms 
I'd go sailin'

It's far beyond the stars
It's near beyond the moon
I know beyond a doubt
My heart will lead me there soon

We'll meet beyond the shore
We'll kiss just as before
Happy we'll be beyond the sea
And never again I'll go sailin'

I know beyond a doubt
My heart will lead me there soon
We'll meet (I know we'll meet) beyond the shore
We'll kiss just as before
Happy we'll be beyond the sea
And never again I'll go sailin’ 50
  • 51.
    Original “La Mer”by CharlesTrénet (1946) Quite different lyrics 51 La mer Qu'on voit danser le long des golfes clairs A des reflets d'argent La mer Des reflets changeants Sous la pluie La mer Au ciel d'été confond Ses blancs moutons Avec les anges si purs La mer bergère d'azur Infinie Voyez
Près des étangs
Ces grands roseaux mouillés
Voyez
Ces oiseaux blancs
Et ces maisons rouillées La mer Les a bercés Le long des golfes clairs Et d'une chanson d'amour La mer A bercé mon cœur pour la vie The sea Seen dancing along the clear gulfs to gleams of silver. The sea Of changing reflections Under the rain. The sea Confuses the summer sky's sheep With angels so pure, The sea Shepherd of blue infinity. Look! Next to the ponds Those tall wet reeds. Look! Those white birds And those rustyhouses The Sea Has cradled them Along the clear gulfs. The Sea Has cradled my heart for life.
  • 52.
    Summary Music can playan integral and varied part of the gaming experience Imperceptible design decisions employed to add to the overall ambience Improved immersion? 52
  • 53.
    Work cited &further reading • KarenCollins (ed.) (2008) Game Sound: An Introduction to the History,Theory, and Practice ofVideo Game Music and Sound Design. Cambridge, MA and London: MIT Press • WilliamGibbons (2011), ‘WrapYourTroubles in Dreams: Popular Music, Narrative, and Dystopia in Bioshock, in Games Studies:The international journal of computer game research,Vol 11, Iss 3. http://gamestudies.org/1103/articles/gibbons • ClaudiaGorbman (1987). Unheard Melodies: Narrative Film Music. Bloomington: Indiana University Press. • Scott Morton (2005) ‘Enhancing the Impact of Music in Drama-OrientedGames’, Gamasutra http://www.gamasutra.com/view/feature/2189/enhancing_the_impact_of_music_in_.php • Rod Munday (2007). “Music inVideoGames.” In Jamie Sexton (Ed.), Music, Sound and Multimedia: From the Live to the Virtual (pp. 51-67). Edinburgh: EdinburghUniversity Press. • HollyTessler (2008) ‘The new MTV?: ElectronicArts and “playing” music’ in Karen Collins (ed.) From Pac-Man to Pop Music: Interactive Audio in Games and New Media.Aldershot:Ashgat,e • ZachWalen (2007) “Film Music vs. Game Music:TheCase of Silent Hill.” In Jamie Sexton (Ed.), Music, Sound and Multimedia: From the Live to theVirtual (pp. 68-81). Edinburgh: EdinburghUniversity Press. • ZachWhalen (2004). “PlayAlong:An Approach toVideogame Music.” Games Studies:The international journal of computer game research,Vol 4 No 1 http://www.gamestudies.org/0401/whalen/. • TrevorWishart (1986) ‘Sound Symbols and Landscapes’ in Simon Emmerson (ed.) The Language Electroacoustic Music, London: Macmillan • Resource mapping articles on game music: http://www.ludomusicology.org/bibliography/ 53
  • 54.
    Images • #4 SamBald (2008) ‘Eyeball’ • #5 MarkWhale (2010) ‘ Apple iPhone White’ • #8 tantrum_dan (2008) ‘Machesney Alarm Clock’ • #11 robjewitt (2011) ‘Aural sex’ • #13 robjewitt (2011) ‘Street Fighter 2 Sega Mega Drive’ • #19 oc fernando (2011) ‘Pac Man’ • #44 Kaleanderson (2011) ‘Creative Commons’ 54
  • 55.
    Diegetic vs Mimetic •In the narrative world of the characters • For the benefit of the audience 55
  • 56.
  • 57.
  • 58.
  • 59.
    Taking a cinematicapproach 59
  • 60.
    Getting it wrong •“There are entire movies contained inside today's games … Unfortunately, video game developers and the players themselves don't often see this connection. Corners are cut, sacrifices made, flat-out wrong practices are repeated time and again, and the gaming media looks upon it and proclaims it good” – Andrew High (composer), 2012, Gamasutra 60
  • 61.
  • 62.
  • 63.

Editor's Notes

  • #19 Nowhere to run … Nowhere to hide
  • #57 Starting at about 40 seconds into the launch trailer, notice the up-down-up-down-up-down as the speech pops in and out. It's distracting and annoying.Creating audio space is like arranging a bunch of 3D bubbles. Whatever is in the center on the X- and Y-axes and at the front of the Z (depth) axis is going to grab the most attention, and that should always be the vocals. Ducking is a cheap way of pushing the music back on the Z-axis, but that constant shifting is noticeable; the much preferable method is to make space on the front plane of X and Y around the dialogue. It is quite possible to write around dialogue, as thousands upon thousands of hours of Opera and film scoring will attest. Why shouldn't video games do this, as well?
  • #58 A beat, in scoring terms, refers to a particular visual point of action that should be accented. This can be a hard cut in the footage, a punch, just about anything. This can be accented in the music in many ways, typically depending on the requirements of the visuals. Here's a quick example that runs the gamut:The low percussion (likely an udu) as the dragonfly lands on her nose is a beat. The sitar note as it flies away is another. The harp gliss for the reveal of Wonderland is another. The cymbal roll for discovering the caterpillar is yet another; the harsh low brass almost immediately after as his expression sours another still. 25 seconds in and we've hit five beats already. This is a fairly common pace for higher-energy sections and trailers. Notice that each of these has a different effect, but all come together to add interest and impact to what's happening on screen.
  • #59 It is possible to create music for a beat-heavy visual without using beats, but then it's up to the foley and sound designers to pick up your slack. See here for a quintessential example:The opposite effect, hard transitions and visual beats without any aural punch at all, feels so unnatural that I can't even find good examples of it, because no one does it.
  • #63 Lack of music can also build tension, especially when it is released properly. Here's a great example from the first Gears of War (NSFW language)There is no music at all until a full minute in, and when it comes in it's a barely-perceptible string pad to simply build the tension a little more. And then, 20 seconds later, release. The score continues its minimalism after that, but it's a good example of how very few lines can be used to good effect.