Dissertation - Keeran Johnson

“Sonic Engagement: How does
Interactive and Adaptive Audio
Contribute to an Immersive Gameplay
Experience?”
University of St Mark & St John
Keeran Johnson
Honours Project submitted in part-fulfillment of the requirements for
BA Hons Live Music Production (Top up)
2016

i
Table of Contents
Table of Contents.........................................................................................................................i
List of Figures..............................................................................................................................ii
StatementofOriginality ............................................................................................................. iii
Abstract...................................................................................................................................... iv
Chapter 1 - Introduction.............................................................................................................1
1.1-Field Of Study....................................................................................................................1
1.2-Game Changers.................................................................................................................3
Chapter 2 - Understanding the Important..................................................................................5
2.1-Categorization of Game Audio..........................................................................................5
2.2-The Concept of Immersion ...............................................................................................8
Chapter 3 - Putting It All Together............................................................................................11
3.1-Game Music....................................................................................................................11
3.2-Interactive/Adaptive Sounds ..........................................................................................17
Chapter 4 - Case Study..............................................................................................................23
Chapter 5 - Conclusion..............................................................................................................29
Bibliography..............................................................................................................................32
Appendix A Participant 1 (The Last Of Us)................................................................................36
Appendix B Participant 2 (The Last Of Us)................................................................................36
Appendix C Participant 3 (The Last Of Us)................................................................................36
Appendix D Participant 4 (Star Wars Battlefront).....................................................................37
Appendix E Participant 5 (Star Wars Battlefront).....................................................................37

ii
List of Figures
Figure 1(IEZA: A framework for game audio) .............................................................................7
Figure 2 Santiago, (no date)......................................................................................................11
Figure 3 (Reserved, 2015).........................................................................................................12
Figure 4 (GameRevolution GR, 2013) .......................................................................................14
Figure 5 (ICXM, no date)...........................................................................................................16
Figure 6 (Supersizevideos, 2010)..............................................................................................19

iii
Statement of Originality
I confirm that I have fully acknowledged all sources of information and help received and
that where such acknowledgement is not made the work is my own.
Signed: ……………………….
Dated: ……………………….

iv
Abstract
This research has brought together the concept of immersion with interactive and adaptive
audio, giving valid examples and reasons as to how and why they contribute to the each
other, and the gaming industry. Through relevant research and examples with supporting
CD artifacts, this text will then aim to reinforce the importance of interactive and adaptive
audio and its important contribution to the development of an immersive gaming experience
according to past and present standards. Underpinning this will be a relevant case study
carried by the author, in order to support or challenge any findings.

1
Chapter 1 - Introduction
1.1-Field Of Study
This honours project will explore the framework of adaptive and interactive audio within the
gaming world. With contribution deriving from various external sources, both primary and
secondary, this text will build a critical discussion with intent to both inform and answer
important questions revolving around this area of study. My reason for choosing this topic is
in preparation for a master study in the same field, and then progressing into the industry.
Michael Greenhalgh writing for ‘SoundOnSound’ says that ‘Video games use sound in a far
more integral way than any other medium’. This paper seeks to study this further and look in
depth at the contribution of interactive and adaptive audio. ‘The complexity of sound design
within games is also developing exponentially.’ (Greenhalgh, 2015).
It is important to address, where relevant, the history of game development. The
technological advances that have been made in the gaming industry have made an
astounding difference to the impact it has on its consumers, giving consumers a more
extreme level of immersion. 'Gamespot' article on a 'History of Video Game Music' talks
about sound and music being a 'Forgotten Element' (McDonald, G. and Staff, G. 2004). This
text will discuss how sound is used in context with its game in order to fully immerse and
draw in its consumers. Technology has changed exponentially over the last 20 years, such as
processing speed and data storage space. These developments have given game developers
more room to play with particular aspects of games. In this case, sound. These developments
will show how games could’ve been back then, and how this has changed our games of the
present day.
Gamespot states that 'sound elements have a more subtle effect than do splashy visuals or
hyperspeedy gameplay. In fact, oftentimes the mark of superior sound design is that you

2
don't consciously notice it at all.' This often involves using sounds to address certain symbolic
meanings, involving sounds that are contextualised and linked to certain triggers or events
within a game. E.g. tense music means danger. Sounds are used on various levels throughout
a game for different purposes in order to achieve different goals, whether it be a single jump
scare or a consistent flow of ambience throughout the game in its entirety. Game developers
are often compared to the industry of TV and Film. There are solid similarities when
comparing these industries, providing a strong idea of the inspiration that various key
concepts have.
“The video game market is huge, and the ability to tell stories, and tell different
kinds of stories in the gaming space is quickly evolving and changing for the
better.” (Lee, n.d.)
Concepts, ideas and methods on how audio is used on various media formats have
transformed a great deal since the start of the gaming industry. This project looks to address
the gaming industry and its intricate use of audio for interactive and adaptive purposes.
“Although there has been significant academic research into related areas of
multimedia and audio in terms of technology, communication, and development,
work into the sonic aspects of audio-visual media has neglected games. Similarly,
studies and theories of video games have, for the most part, disregarded the
audio. While there have been a scattering of articles published sporadically in the
last few years, video games audio remains largely unexplored.” (Collins, 2010)
In 2008, Collins also said, “I would argue at this stage, games are so new to academic study
that we are not yet able to develop truly useful theories without basic, substantial empirical
research into their practice, production and consumption.” With games having now taken an
8 year leap, this text aims to contribute to the industry by giving a critical discussion within
this field of study, highlighting the importance of sound designers, sound programmers and
their work, and subsequently re-apply Collins’ statement and its relevance in the present day.

3
1.2-Game Changers
After being limited for a long period of time through data storage and memory, particularly
in the earlier years of PlayStation 1 and Nintendo 64, technology has advanced to a point
where this is less of an issue, but not nonexistent. Since the days of PlayStation 2 and Xbox,
work efforts into game audio have multiplied exponentially, providing consumers with ever
improving audio stimulation. Becoming fully immersed into a game, and fully experiencing its
beauty is always a game developer’s top priority. Graphically, games achieve a high standard
on a regular basis. This is a big factor in an immersive experience, and this standard is
continuously being raised year by year.
“ What I thought was fifty years away, was only ten years away. And what I
thought was ten years away… it was already here. I just wasn’t aware of it yet. ”
(Manuel, R. (n.d.)
The first video games offered basic synthesized sounds. From the days of old, and early Atari
machines, video game audio ‘blipped and farted along’ ‘using complex processes to make
primitive sounds’, more commonly known today as 8-Bit. Al Alcorn (who worked on Atari
machines) was quoted in an interview. (Madden, 2014)
“How do you do that with digital circuits? Ones and zeroes? I had no idea, so I
went in there that afternoon and in less than an hour poked around and found
different tones that already existed in the sync generator, and gated them out
and it took half a chip to do that. And I said ‘there’s the sound – if you don’t like it
you do it!’” (Alcorn, 2008)
Today, games such as ‘Fallout 4’ (released November 2015) boast over 111,000 lines of
recorded dialogue. (Martin, 2015) This impressive development means a larger workforce,
more technology and more concentration on captivating player through the use of game
audio. Game platforms are now easily capable of producing a high quality auditory

4
experience, however, visuals and graphics still predominantly take a higher position in the
hierarchy; leaving the audio team to make the best of what is left.
Processing power and memory is still something of a limitation for game developers and
consoles. This is not the case for the long running PC (Personal Computer). PC is often
referred to as ‘PC Master Race’ with the website ‘Reddit’ having a full forum page using this
name, along with the slogan ‘MAY OUR FRAME RATES BE HIGH AND OUR TEMPERATURES
LOW’ (125 et al., 2016). PC’s run technical specifications that far exceed any other console
due to its larger size, larger parts and long running development history, with manufacturers
now making increasingly powerful chips and processers at even smaller sizes. It is not until
the recent 2014 release of the Xbox One and PlayStation 4 that this technology is becoming
the core for all things gaming, giving games console a chance to catch up.
Adaptive and interactive audio has been around for a long time, for example classic arcade
games from 1970’s and 1980’s such as space invaders, when the player shoots, the game
makes that classic synthesized ‘pew’ sound. This level of interactivity was on a very basic
scale. With limitations being slowly lifted, the level of audio creation is becoming increasingly
in depth.
The next chapter will go on to examine the categorization of game audio using examples to
highlight its relevance to the gaming industry. Further chapters will then go on to examine
the use of interactive and adaptive games audio in relation to an immersive gameplay
experience.

5
Chapter 2 - Understanding the Important
2.1-Categorization of Game Audio
Game audio has been categorized by few scholars over the last two decades such as Alex
Stockburger, Karen Collins and Zach Whalen. In order to understand and pull apart an
‘Immersive Gameplay Experience’, it is imperative to understand how a games auditory
experience is categorized. One of the earliest written works in this field comes from Alex
Stockburger in ‘Space Time Play’ (2003). He divides the concept of interactive game audio
into ‘sound objects’ (Grimshaw, 2011 P. 82). These are then broken down into ’score sound
objects, zone sound objects, interface sound objects, speech sound objects and a range of
different effect sound objects’ (Grimshaw, 2011 P. 82). This means, according to
Stockburgers’ work, that each sound has a direct link to an object within the game. An
example would be footsteps, directly linked to the playable character, or Non-playable
character. Being 2003, Stockburger does not discuss the interactivity or adaptive capabilities
of diegetic and non-diegetic sound; instead he covers them as ‘matter-of-fact’ basic concepts
(Grimshaw, 2011 P. 82). In comparison, Zach Whalen (2004), in his study on ‘Video Game
Music’ covers diegetic and non-diegetic sound in a different manner. Dissimilar to what the
title suggests, Whalen states that the ‘broad strokes’ of his argument ’apply generally to
sound as well’. Whalen addresses music/sound as a ‘problem’ that is ‘further complicated’ by
the categorization of diegetic and non-diegetic audio. This use of the word ‘problem’ is also
reflected in other authors’ works, such as Karen Collins in ‘Game Sound’ (2008). He goes on
to argue that much of the audio a player hears, is not represented by any visual ‘object’ on
screen (Non-diegetic), drawing the conclusion that the term ‘musical sound’ would
potentially be the better ‘label’ (Whalen, 2004), thus leading to an alternative
categorization.

6
Karen Collins writing in ‘Game Sound: An Introduction to the History, Theory and Practice of
Video Game Music and Sound Design’ covers interactive and adaptive sound according to
‘functionality’. Collins separates interactive sound as ‘sound events that react to the players
direct input’ (Collins, 2008, pp. 4 – 4). An example of this would be the sound of Gordan
Freeman’s (the main character you play in Half-Life) gun as the player chooses to reload. This
categorization also links back to Stockburger (2003) showing that ‘each sound has a direct
link to an object within the game’. Collins then continues to classify adaptive audio as sound
that ‘reacts to the game-states’ and ‘various in-game parameters’, for example, a low health
heart beat sound. This example is used in various titles such as Grand Theft Auto, Half-Life,
Bloodborne and many more. It is then evident further into Collins text (pp.125) that game
sound can be categorized further.
With the distinction between interactive and adaptive already apparent, Collins states that
diegetic and non-diegetic sound is only a broad categorization, and that it can be split further
into ‘nondynamic and dynamic sound’, and from there looping back around in order to
discover its relation to ’diegesis’ and the player. (Collins, 2008, p. 125) This demonstrates that
many sounds within a game are not absolute. These sounds give blurred lines and alter their
categorization mid-game, making them ‘fluid’ rather than ‘fixed’. A good example of this
concept would be the gun fire in Call Of Duty Black Ops III. When the player chooses to fire
his/her gun, you get specific sound for that specific gun (interactive). However when the
player gets incapacitated due to low health or sufficient damage, the gun fire sound changes
to become dull and distant, lacking in a lot of high frequency detail (adaptive). This
emphasizes Collins’ point of many sounds being ‘fluid’ and not necessarily ‘fixed’.

7
Between 2003 and 2008, Richard Van Tol and Sander Huiberts developed the IEZA
Framework. This framework is a model for categorizing, contextualizing and analyzing audio
for video games. This framework is separated into four categories, ‘Zone, Effect, Affect and
Interface’. These are then further classified as either ‘Diegetic, Non-Diegetic, Activity or
Setting’ (See Figure 1). Effect relates to audio that is directly linked to ‘specific sound sources’
that exist within the diegetic measure of the video game. This could the simple sounds of the
main playable character, such as footsteps. In a racing game, an example would be the sound
of the car itself. The Zone categorization applies to diegetic audio that originates from the
game environment itself. This could be the sound of the crowd in a sports game.
Alternatively, this could simply be the basic ambience or natural sound of the gaming world.
Both of the categorizations link directly to Collins’ work, in that they would both contain
interactive and adaptive audio. Furthermore, sounds that then lie in either section could be
considered ‘fluid’ and therefore not absolute. The same principle also applies to both
‘interface’ and ‘affect’ (IEZA: A framework for game audio, no date). These both stay within
the Non-Diegetic side of the framework. Interface refers to sounds that do not exist within
the gaming world, such as ‘Heads Up Display (HUD)’ audio. Affect then applies the non-
Figure 1(IEZA: A framework for game audio)

8
diegetic environment, meaning music that conveys the tonality and genre of the gaming
world, such as ‘orchestral music in an adventure game and horror sound effects in a survival
horror game’. These categories would also be considered ‘fluid’, relating to Collins’ work, in
that ‘interface’ audio could also be considered ‘affect’ audio, providing the ‘HUD’ sounds
convey the non-diegetic setting of the game. Collins concept of interactive and adaptive
audio could be applied to all sections within this framework. Does this mean that the
majority of game sound carries ambiguity? Collins’ acknowledges this through the
categorization of ‘interactive and adaptive’ which are then either ’fluid or fixed’.
It is evident from this how complex sound categorization can become, especially with game
production becoming increasingly detailed and in-depth. All scholars here show similar theo-
ries in the distinction of game audio, which has improved over time, evident from Collins’
work. Collins categorization of game audio will be employed throughout this paper in order
to provide concise points with supporting evidence.
2.2-The Concept of Immersion
In 2003, Oliver Grau classified immersion as the ‘key to any understanding of the
development of the media’ (Grau, 2003). But what are the signs that a player is immersed
within a game? The concept of immersion, or presence as sometimes known, is accompanied
by a strong following. But what is immersion? It is clear that there is no objective answer;
however, studies and texts that cover immersion to appear to show similarities that allow a
strong definition to be almost absolute.
‘Immersion is a state in which players start to imagine or visualize their next
move in their mind’s eye along with the response of the game environment to it.
This is totally different from merely thinking about the current state of the game
or consciously planning their next move, and closely resembles the state of mind
during a guided meditation’. (Kaveh, 2010)

9
Immersion is often considered to be a measurable state when playing video games. This is
determined by certain factors that may occur during an immersive experience. This could po-
tentially be something as simple as loosing track of time, becoming withdrawn form the
world around you oblivious to people and events around you. Physical reactions also show
clear signs of immersion, a faster heart rate during a fast paced, tense action sequence, em-
pathy with characters etc. (Stuart, 2015) Ian Mahar distinguishes immersion into two sorts,
‘impersonal’ and ‘personal’. This separation is the difference between ‘identifying’ with the
character you play (impersonal), meaning that particular characters behavior and appear-
ance is set in stone and you are simply taking him/her through their story. Mahar uses Mario
as an example here. The alternative means you are ‘literally putting yourself into the game’.
This means that the games fictional context and narrative have a direct effect on your ‘emo-
tional and behavioral responses’. (Mahar, 2010)
Writer Andrew Glassner in ‘Interactive Storytelling’ gives a different, more in depth outlook
on immersion. Glassner states that there are ’several levels of engagement immersion, which
are characterized by the depth of the bond between the audience member and the
character’, which he describes as measurable on a ‘finer scale’. The most basic form of
immersion is ‘curiosity’. Feeling the impulse to play a game or continue playing due to either
a ‘casual desire’ or a ‘burning curiosity’. The next step is ‘sympathy’, which occurs when
players will start to see events that unfold through the protagonist’s eyes. Glassner’s point
about ‘identification’ is contradictory to that of Mahar. Glassner states that ‘identification is
the first of the strong forms of connection to the character’. This shows that rather than being
‘impersonal’ as Mahar states, it is deeper and gives the player a connection with the
character, meaning they can see ’elements’ of themselves in the character or vice versa.
‘Empathy’ is then used as the penultimate depth of immersion. This state gives the player a
tight ‘emotional bonding’ to the protagonist, to the point where players become ‘engrossed’.

10
The final state is the extreme; this state of immersion means players can potentially ‘lose the
boundary’ between themselves and the protagonist, essentially leaving the real world.
(Glassner, 2004)
It is evident that certain keywords have a direct link to the term, giving a more concise
definition. The use of words such as emotion, imagination, subconscious and reality all have
a clear connection to the concept of immersion. Grau states that ‘It is characterized by
diminishing critical distance to what is shown and increasing emotional involvement in what
is happening’ (Grau, 2003). It is therefore clear to draw the conclusion that immersion,
although subjective, is when a player becomes emotionally attached to the game, its
narrative and its protagonist. This attachment triggers identification and empathy to events
that unfold, allowing players to relate to the characters circumstances, causing them to
make choices and have emotional responses as if players were actually in the game,
essentially drawing them in and getting them lost in the game.

11
Chapter 3 - Putting It All Together
3.1-Game Music
Music is regarded very highly in the audiovisual spectrum of a video game. It is important to
highlight that the music of a video game can be just as interactive and adaptive as any other
sound, therefore contributing to a games immersive experience. Collins gives a good example
of this. The image provided shows the training process in Kokiri Forest (See figure 2).
‘For instance, in Zelda’s Kokiri Forest, during the first portion of the game, we are
continuously in daytime mode as we get trained in gameplay, and the Kokiri
theme that plays throughout does not change except at those points where a
player enters a building or encounters an enemy. While interactive, it is not
adaptive at this point. After we complete our first major task and arrive at the
next portion of the game (there are no distinct “levels” in this game), we then
experience the passing of time, and can return to the forest. Now, if we return at
night, the music has faded out to silence. At dawn, it will return to the main
theme: the theme has become adaptive. In other words, a cue which is
interactive or adaptive at one point in the game does not necessarily remain so
throughout.’ (Collins, 2008, p. 125)
Figure 2 Santiago, (no date)

12
An alternative example of this interactive and adaptive music would be EA Sports’ Need For
Speed Most Wanted (2005). This racing game consists of many high speed races and police
chases, where the player has to earn his/her way up the street racing ranks. Throughout the
majority of its gameplay, the music comes from a track list that plays itself on shuffle. This
then changes as soon as the player (interactive) gets themselves spotted by a police vehicle,
triggering the start of a police chase along with its fast-paced, action packed and incredibly
tense Police Chase Theme (See Artifact 1for example). The image shown in figure 3 is an
example of a police chase. This theme then becomes adaptive according to different
gameplay states throughout the chase. When the player ends up out of view from any
impending police vehicle, the game states enters a cooldown mode, showing the player that
providing they stay out of sight, they will be successful in evading capture. During this state,
the theme begins to slow down, cleverly building tension, causing the player to become
uneasy in their desperate attempt to flee (See Artifact 1for this example). The immersive
nature of this theme is its ability to build tension and reinforce an action packed police
chase. This result means players then become empathic and engrossed in the games context,
which is Glassner’s penultimate depth of immersion.
Figure 3 (Reserved, 2015)

13
Collins then goes to give a second example of interactive and adaptive audio. This example
shows that interactive and adaptive audio can give game music a different level of diegesis,
causing them to become fluid or fixed.
Similarly, in Asheron's Call 2: The Fallen Kings (Turbine Software 2003), the non-
diegetic music that plays in the background of scenes becomes diegetic when
players decide to have their character play an instrument or sing along with the
music. Not only has the music changed from non-dynamic to interactive, but it
has also gone from non-diegetic to diegetic. As such, then, although I have
distinguished levels of sound here, they must be viewed as fluid, rather than
fixed, for many types of audio cues.’ (Collins, 2008, p. 125)
The games used for this example were made over a decade ago. Asheron’s Call 2: The Fallen
Kings was brought to us in 2003. The Legend of Zelda: Ocarina of Time was a further five
years prior pushing into the two decade boundary. The limitations of games and their respec-
tive consoles would have been a much larger factor, especially in 1998 with minimal memory
and lack of processing power. It is therefore clear that the process of immersing players
through interactive and adaptive audio has remained the same, even in older generations of
development.
Game music takes many different forms and can play a variety of roles alongside other vari-
ous sounds. The non-diegetic sounds (for the most part), e.g. music, prove to be just as vital
to the audiovisual performance as the vast array of diegetic sounds used. In some cases, the
music or ‘theme’ for a video game often becomes iconic and something that numerous peo-
ple can identify and relate to. The Super Mario Bros. is regarded as ‘one of the most iconic
themes of all time’, coming from a time when games had limited storage. This meant Koji
Kondo (the composer) had to score a theme that was not only repetitive and limited in order
to stick to storage specifications, but that encouraged people to continue playing and enjoy
the game, triggering an immersive response. ‘Many people happily associate’ the Super Mar-
io Theme with ‘childhood innocence and fun’. (Staff, 2013)

14
Music has been found to be a very fulfilling emotional stimulus. A part of the brain known as
the Hippocampus has a sufficient connection to a human’s emotional response. This means it
responds to the emotional tonality and mood of the music, whether it is fear, tension, love or
pleasure. Other parts of the brain known as the nucleus accumbens and the amygdala also
link to the emotional music response. These parts of the brain are triggered when music is
deemed pleasurable. This is then associated with ‘primary’ and ‘secondary rewards’ such as
‘food, sex, money and power’ (Vicky, 2014). It is therefore evident that when interactive and
adaptive music triggers an emotional response within a video game, players are getting
drawn in and immersed within its gameplay. Game soundtracks are created and scored to
make players feel a particular way. ‘In an audiovisual world, music is by far the best way to
establish mood and tone. It plays on our senses, creeps into our minds’ (Staff, 2013). A good
example is the cut scene called Joel’s Choice from The Last of Us. This scene involves Joel
changing his mind and deciding to take Ellie (the 14 year old girl he has been protecting) and
look after her himself, rather than passing her on to someone else (see Figure 4). Players are
given this scene straight off the back of Joel and Ellie’s father and daughter argument in the
Figure 4 (GameRevolution GR, 2013)

15
previous part of the game, so emotional responses and the level of immersion is running
deep at this point. The adaptive music that plays throughout this scene is titled ‘The Choice’
played by an orchestra. The composition (by Gustavo Santaolalla) is slow and low key. While
Joel is choosing to take Ellie himself, the music plays throughout (See Artifact 2 for example).
This then dramatically increases the emotional content that is being displayed on screen,
thus further immersing the player (GameRevolution GR, 2013). It is important to note here
that cut scenes, although still a key ingredient for both narrative and context, adopt the ma-
jority of their cinematic focus from films, particularly in their use of diegetic and non-diegetic
sounds. This allows game developers to draw in consumers the same way a film would draw
in its audience, through narrative, context, mis-en-scene, cinematography and more appro-
priately, sound. (Whalen, 2004)
Whalen shows that game audio, specifically music, plays an important role in the narrative of
a video game. This form of ‘adaptive’ audio contributes to making a more immersive
‘Gameplay Experience’. The soundtrack for The Evil Within is one of horror, terror and total
fear. Throughout the game, the protagonist is constantly caught up in terrifying combat
sequences whilst being teleported to different locations within the insane mind of a brutal
murderer. This combination means that Sebastian (the main protagonist) is constantly facing
off against monsters that only the sick and twisted could think of. The result is an intelligently
scored soundtrack that was clearly made to scare and terrify. Chapter 9 sees Sebastian
having to run and hide form Ruvic (the evil murderer whose mind the player is trapped in).
Randomly throughout this chapter, Ruvic will appear from the shadows and begin to hunt
down the player (see Figure 5). This change in game state is emphasized by the evidently
scary, adaptive soundtrack titled Lurking in the Dark (Epic Video Game Music, 2014). This
heavily reverberated track uses ambient sounds such as high pitch screeching noises,
ambient fuzzing and crackling, accompanied by an extremely loud intermittent pulsing slap

16
that emanates every time Ruvic teleports towards the player to hunt him/her down (See
Artifact 3 for example). Providing the player then escapes this gameplay phase, the music
dies away into another track, however, the gameplay at this point is still very much
dangerous. The player still has enemies to combat, and there is a high chance that Ruvic will
appear again, thus triggering the same sequence.
This means that the more subtle sound track that plays as a constant throughout this chapter
still reinforces the game state, which is one of danger and fear. This example shows that any
game environment could be a ‘geographical embodiment’ (Whalen, 2004) of the games
musical narrative and context. By achieving this result, game developers can the trigger
players emotions, in this case fear and danger, allowing for a much more immersive
gameplay experience. Whalen also gives an alternative example of this concept in Silent Hill
(1999). He states that the musical content never presents the players with a ‘safe state’ and
instead ficuses on aspects of tension and ‘danger’ therefore compelling players to move
‘through the games spaces’. (Whalen, 2004) Whalen states:
Figure 5 (ICXM, no date)

17
‘The mood of the game is crucial to the horrific "feel," but it is also provides
motivation by compelling continual progress through the game. The town of
Silent Hill is never a safe place: players maintain the game's contiguity by trying
to escape Silent Hill, a geographical embodiment of the musical danger state’.
(Whalen, 2004)
This illustrates that music is just as important to the narrative and context of a gameplay
sequence as anything else, meaning that game developers can use these adaptive
soundtracks in order to trigger particular emotions with players, therefore creating a much
more immersive gameplay experience.
‘In general survival horror games rely on conventions of horror film sound to
effectively create the mood of horror required for the game (echoing effects,
screeching violins, dissonant bursts of symphonic noise at "startle" moments,
etc.) (Whalen, 2004)
To conclude here, it is evident that the connection consumers have with music in general is
utilized within the process of game production in order to trigger emotional responses.
Game developers have been doing this for over a decade, so it is clear that these immersion
techniques are not new to the world of gaming. Becoming immersed in these soundtracks
can also give them an iconic status, as discussed with Super Mario connection, and the Police
Chase Theme in Need For Speed. It is already clear how player’s emotions can be triggered
through music, and how this can immerse them into the gaming world, but how much
further can it go with the rest of game audio? Is there a state of absolute immersion? The
next chapter will discuss interactive and adaptive audio in regards to the sounds themselves.
3.2-Interactive/Adaptive Sounds
“Sound and music make up more than half of communicating a story, greater
even than what you’re seeing...” Steven Spielberg (2015, 2015)
Diegetic and Non-diegetic sound effects make up a huge amount of today’s games, with
video games often reaching 40 – 50 Gigabytes as a result. Current video games on the market

18
look to build games form the ground up, this means ‘sound designers imagine, record,
create, and edit sound effects’ from scratch (Isaza, 2010). A video game that could exist
without its fundamental diegesis is not a topic of consideration for the industry. Sound
design within the gaming industry is now such a strong factor, that it has its own career path.
Following Collins’ categorization of interactive and adaptive audio, sound effects always
have a connection to one, the other, or even both (fluid or fixed). The purpose of these
sounds is to feedback to the players, further immersing them into the virtual reality of the
game. Miguel Isaza states that sound is ‘the only real sense fully experienced by the player’
and that the sounds players hear, play a specific role for immersion. The primary function of
game sound effects is to provide the player with feedback as to what is happening within the
gaming world. This can be done both interactively and adaptively.
Game sounds follow tonality and set the mood. If a game is light hearted, the sounds that
accompany the game will emphasize this. If a game is designed to be scary, then its sounds
are dark, ambient and focused. In the game Unreal (1998 – See Figure 6) you take on the
role of Prisoner 849 who escapes from a crash landed spaceship on planet Na Pali. In the
opening gameplay, there is no music, only the sound of the protagonist’s footsteps. As you
step out of the prison cell, the player is swamped by an unnerving distant growling (an alien
of some kind), an over-bearing security voice over announcing your escape, along with the
deep rumble of earthquakes and explosions. To maximize this sensory overload, the sounds
of distant screams also take place, creating an extremely unsettling environment (See
Artifact 4 for example). It is clear here that sounds are used on various levels to make
players feels certain emotions, such as fear or love, and trigger certain cognitive responses,
such as fight or flight, thus further immersing the player within the game.

19
‘Information’ is another key function of these interactive and adaptive sounds. This means
that the diegetic or non-diegetic audio can supply the player with detailed information
within their virtual reality. Going back to the example of Unreal, after moving through a few
rooms on your escape from the ship, you come across a closed sliding blast door, with a
slight gap at the bottom. Upon approach to this door you hear loud screams and the distinct
horrifying sound of someone being attacked. Although the player knows where this is
happening, the sound remains completely non-diegetic. Eventually the attack stops and the
door opens shortly after, leaving you with just your imagination to figure out what happened
(See Artifact 4 for this example). Alternatively, first person shooter game Left4Dead uses
the very distinct sound of an echoing cry to inform you of a nearby Witch, a dangerous
enemy that can chase you down rapidly and deal a lot of damage. This sound is unique and
purely contextual to a Witch encounter, triggering a players fight or flight response (See
Artifact 5 for example). This immersive value is also shown in similar fashion in Naughty
Figure 6 (Supersizevideos, 2010)

20
Dog’s The Last Of Us. This post-apocalyptic game sees the player encounter a variety of
‘infected’ human creatures throughout their play through, one of which is The Clicker. A
blind infected that uses echo-location to find its next target. This distinct clicking/screaming
combination, informs the player of imminent danger, much like the Witch in Left 4 Dead (The
music and sound design for The Last Of Us, 2013). This concept then reinforces the previous
point made, that all of these sounds have a specific purpose and provide the player with a
specific meaning in order to trigger a certain response.
Another function of these sounds is to provide a sense of reward or failure. Dave Russell, in
an article on ‘diegesis theory’, uses the example of Tetris:
‘The sounds and flashes that go with completing a row in Tetris make the act
much more pleasurable. It is a short-term award that supports the long-term
award of not dying and beating a high-score.’ (Russell, 2012)
In Bthesda’s roll-playing game Skyrim, upon leveling up your character you are rewarded with
a medieval like chant accompanied by the sound of a drawing sword. This is the same
concept, only in a larger video game. In the same game, if the player dies, the sound is
ambient and deep. Through immersion, players learn what these sounds mean. It then
becomes part of their gaming journey to either avoid or trigger these sounds. (Russell, 2012)
As collaboration, a game’ interactive and adaptive sound adds realism, making on screen
objects and events come to life. Through this realism, levels of immersion become stronger,
making for a better gaming experience. Ben Minto, who works for the DICE! Sound
development team, shows just how much detail is included into video games of the present
day, speaking about Star Wars Battlefront:
‘What we were really after were recordings of the “air” or the “feel” of the pacific
coastal redwoods, which is the location where the forest moon of Endor was
filmed. If you can get away from today’s noise pollution, you can experience this

21
very delicate and quiet sound that is something that we strive to add in as a layer
when designing ambiences. It really helps to put the player in that specific
environment and sell the sensation of being surrounding by those giant trees.’
(Ltd, 2016)
Star Wars Battlefront is the video game production of the original Star Wars trilogy. The
players can take part in a series of epic battles that take place on famous Star Wars planets,
choosing to side with either the Rebel Alliance or the Imperials. In addition to this, players
can then earn themselves a chance to play as one of the films heroes, such as Han Solo or
Luke Skywalker. The sound development team spent a lot of man hours with Lucasfilm in
order to make their sounds as original as possible, such as the ambient sounds of each
planet, the blaster sound and droid sounds. This presents the idea that consumers are better
immersed when they have a product to relate to, in this case, the Star Wars films. If the
blaster sound heard within the game takes a consumer back into the film, they can then
identify and relate to that sound, thus become immersed within the product. Interestingly,
the context of these sounds does not exist in the real world; however, players still want
them to sound ‘realistic’. This then emphasizes the concept of immersion through sound,
because even though these sounds are part of a developed fantasy, consumers still demand
highly realistic standards in order to enjoy the game and become immersed. (Ltd, 2016)
Having this level of sound capability is something that players can experience more and
more of in the modern years of the gaming industry. Looking at games before 2005, the year
of next generation consoles with the release of the Xbox 360 and a year later the PlayStation
3, games with an in depth level of sound detail still existed despite technological limitations
such as processing power, memory and disc format capabilities. In Lord of the Rings: The
Third Age, where the player takes on the role of Berethor in a journey through Middle Earth
that shadows the films trilogy, it is evident that the sound concepts are the same that
players experience today. When walking through the wilderness to get to Moria, the sound

22
of wildlife, blowing trees and ambience remains a constant. Does this mean that perhaps
that the graphical limitations of older games due to lack of technology, allowed for more
space than developers realized to incorporate good sound? This is not a concept that would
have been considered in previous years because each year we see better video graphics,
meaning in 2004, relatively speaking, The Third Age looked just as good because consumers
and developers had not made anything better. Upon release of this game, IGN posted a
review comparing its performance across its three contending platforms.
‘The Xbox's in-game Dolby Digital really doesn't offer up a noticeably more
immersive sound field, but the quality of effects, voice, and music is better.
GameCube and PlayStation are very similar in both the area of surround and
quality. ‘ (Davis, 2004)
This shows that cross platform differences can also have an impact on the quality of these
interactive and adaptive sounds, therefore providing the argument that in this particular
case, the Xbox provides consumers with a more immersive gameplay experience.
To conclude, it is clear that interactive and adaptive sounds are all symbolic. Every sound
provides the player with a meaning. The level of realism that can be created through using
detailed interactive and adaptive audio is clear, which that deepen a players immersive
experience. In addition to realism, information, tonality, mood and context are the other
uses of these interactive and adaptive sounds. These sounds then allow the player to apply
the same immersive connection as game music, identification, empathy, sympathy and an
emotional connection, all contributing to the immersive level of the video game. The next
section will move on to looking at game sound as a whole product through a comparison in
case studies, which will then put into practice the theories and concepts referenced and
supported by this paper.

23
Chapter 4 - Case Study
The purpose of this case study is to demonstrate this area of critical discussion. This has been
achieved by using five participants and setting them up to play a small section of a video
game. The first gameplay section used for this case study is Chapter 1/Prologue ‘Hometown’
from the Naughty Dog title ‘The Last Of Us’. The gameplay section sees the player take
control of both Sarah and Joel, a single father and his daughter, in a tense distressing attempt
to find safety at the outbreak of extreme sickness that brings about the end of the modern
world (see Artifact 6 for this example). The second game used is the 2015 DICE game ‘Star
Wars Battlefront’. In this game, the player assumes the role of any soldier or Hero they
choose in order to battle the opposing faction within the Star Wars universe, the two
factions being the Imperials and the Rebel Alliance (See Artifact 7 for this example). As shown
earlier in the text, the level of sound detail put into this game is vast, therefore providing a
lot of interactive and adaptive audio for the players to immerse themselves in. Participants
for this game played a Survival game mode for 10-15 minutes. After the participant
completed their section of gameplay for either game, they then proceeded to replay it with
the sound completely removed. Upon completion, participants then gave a response in
relation to the study.
This case study was carried out in the same environment for each participant. This
environment consisted of a comfortable living space, much like a household living room, this
is to ensure participants feel relaxed, allowing them to focus on the gameplay. The games
stated, were setup on a current generation PlayStation 4 along with a 48inch Samsung full
high definition wide screen TV. For experiments on cross platform differences in sound, Xbox
One and PC could also be setup for a direct comparison. As this experiment focuses on the
interactive and adaptive sound, it was essential to have a sound setup of sufficient quality. To

24
achieve this, a Marantz PM6005 stereo amplifier was used alongside a pair of Mordaunt
Short Aviano 6 floor standing full range speakers. This setup allows for high quality sound
delivery, ruling out any lack of immersion caused by poor sound quality. Participants were
only required to play their respective game for 10 – 15 minutes, this is so that concentration
is higher, with focus being maintained on only a small section of the game, allowing for a
more detailed analysis to be drawn by each participant. Five participants were used rather
than just one with the objective of gathering more responses, in order to analyze and
breakdown the participant responses with more detail. In doing this, a better conclusion can
be drawn, and further comparison can be made. Full quotes from each participant can be
found in the appendix.
Upon completion of this case study, it is apparent that all of the participants had similar
responses, with slightly different focal points. Participant 1 (Appendix A) states:
‘In comparison the second play through with the sound off was miles worse than
the first one. It just wasn’t the same.’
In addition to this, all participants shared a similar use in language, using words such as
‘emotion’, ‘detached’, ‘connection’ and ‘immersion’. When participants use these words it is in
negative conjunction with the sound being turned off. Participants 4 (Star Wars Battlefront)
states that ‘With no audio I felt extremely detached from the game.’ which is a similar
response to that of Participant 2 (Appendix B) (The Last Of Us) who explained that ‘when you
play it with the sound off you get nothing, you become completely detached with the game
and you are just watching a moving picture and interacting with next to no immersion,.’
Participants 1, 3 and 5 (Refer to Appendix A, C and E) also gave similar responses, testifying
that when the sound is off ‘there is no immersion’ and that ‘as soon as the sounds were
turned off, it was like the game lost all of its life.

25
With all participants experiencing the same loss of ‘connection’ and ‘immerison’, it is then
evident from this case study that interactive and adaptive sound is a primary contributing
factor to an immersive gameplay experience.
A big contributing factor that has arisen through this study is the impact of sound as a means
to provide information and other immersive details. Participant 1 (Appendix A) states:
‘Playing the game with sound gives you so much more information, such as the
dialogue from characters, explosions, noises and such. It made such a difference
when that information wasn’t being fed to me. I didn’t know where to go, what
to do and became all but clueless as to what was actually happening. The only
reason I knew what to do was because I had already played it through once with
the sound on.’
This demonstrates not only the immersive value of this sounds, but also, how much a player
can identify and relate to through gameplay. If a player cannot experience the ‘explosions’,
‘dialogue’ or other informative noises, then how is it possible for the player to identify and
relate to those sounds, apart from potential visuals? Furthermore, these visual are only a
contributing factor on screen (diegetic) meaning when sound is turned off, any non-diegetic
information is then non-existent, which is a huge factor in gameplay. Additionally, the level
of information that these sounds provide, also has a big impact on playability. It is inherent
through playing these games that sounds often provide with information on how to
progress, especially if it is in the form of dialogue. This therefore supports section 3.2 of this
paper, and how sounds provide information. Participant 4 (Appendix D) also supports this
conclusion by stating that:
‘The game audio provides a tremendously immersive experience without being
obvious. Engagement with my allies is necessary to aid survival, and their dialog
brings useful information and warnings of events. Environment noise provides
clues on direction, enemy attack, and a feeling of being close to the action, with
enemy weapon fire surrounding me, alerting me to danger and heightening my
level of cautiousness. Additionally, the soundtrack audio adds great excitement
and also provides clues by the use of tone and tempo.

26
This mention of tone and tempo also supports section 3.1 of this paper, on the discussion of
game music that provides context, and can convey particular emotions.
All of these responses also show a distinct connection to Mahar and Glassner classifications
on game immersion in section 2.2. The consistent use of words such as ‘connection,
emotion, drawn in, detached’ all have a direct link to both Mahar’ two stages of immersion
(impersonal and personal) along with Glassner’ ‘finer scale’ measurement of ‘curiosity,
identification etc.’. This therefore shows that this case study has provided a direct link
between the concept of immersion and the interactive and adaptive audio within a game,
thus proving that audio is a fundamental creating an immersive gameplay experience.
To draw a comparison to this, an alternative case study is to be used. Terry Garrett is a blind
student from the University of Colorado (Diver, 2016). As seen in artifact 8, Terry provides a
small example on how he plays through The Legend of Zelda: Ocarina of Time. Using a
precise stereo environment, Terry is able to localize sounds within the game, and navigate
his way through particular sections. The majority of this though is achieved through the
sounds themselves, what they mean and the information they provide the player (Diver,
2016).
‘Terry had previously finished the 2D PlayStation game Oddworld: Abe's Oddysee
using just audio cues in order to progress, and the best part of five years ago he
set himself the challenge of taking that approach to "seeing" the virtual world in
front of him into a three-dimensional space. His chosen game was Ocarina of
Time, and here he is in May 2011, beginning his quest in the Deku Tree, taking
out spider enemies by listening for their movements.’ (Diver, 2016)
This case study reinforces the importance of interactive and adaptive sound and how they
provide players with information. A simple knocking noise when you walk into a wall, tells
you that you cannot move any further. Having the localization of enemy sounds across a ste-
reo field, allows players to determine the location of that enemy. All of these sound details

27
provide players with information to decode and relate to, in particular, Terry, because it
means that he can then navigate his way through the game, just using these sounds.
Comparing this to the previous case study it is clear to see similarities. Both case studies
highlight the importance of the information that interactive and adaptive sounds provide.
This is of much more importance for the second case study, because as shown, the sounds
are imperative for progressing through the game. The first case study also demonstrates
this. Participant 1 (Appendix A) states:
‘I didn’t know where to go, what to do and became all but clueless as to what
was actually happening. The only reason I knew what to do was because I had
already played it through once with the sound on.’
In addition, Participant 3 (Refer to Appendix C) states that ‘Every sound is put there for a rea-
son, to tell you something’. This validates that sound is also important for those playing the
game with visuals as well as sound. All of the participants in the first case study stated that
they had no ‘immersion’ or ‘enjoyment’ when playing the game with the sound off. This high-
lights that players are unable to identify or relate to the game in the same way with sound
turned off. In particular, as stated by Participant 2 (Appendix B):
‘I found the biggest difference at the end. The level of emotion you get filled with
when Sarah dies is quite overwhelming and surprising when it comes from a
game. But then when you play it with the sound off you get nothing, you become
completely detached with the game and you are just watching a moving picture
and interacting with next to no immersion,. It just goes to show how important it
is really.’
To conclude here, it is profoundly evident that the interactive and adaptive sounds are all
too important to a games immersive experience. Without these sounds, players have no
means of identification or relation, unless the visuals mange to display otherwise. In addi-
tion, sounds also provide huge in roads for the portrayal of emotion. The lack of these
sounds gives players minimal in the way of telling you where to go, what to do, or even what

28
is happening. So it is therefore questionable that these interactive and adaptive sounds ex-
pand even beyond creating an immersive game. Do these sounds make a game playable?

29
Chapter 5 - Conclusion
It is evident that the game sound industry is rapidly changing and growing. This rapid change
has given game developers the required technological advancements, such as data storage
and processing power, in order to provide gamers with a deeper level of immersion than ever
before through their production of game sound. This is achieved through sound designers
now making their game sound from scratch, recording original material, then mixing and
layering those sounds for use within a particular game for a specific purpose. Through the
work of Collins, Grimshaw and other referenced scholars throughout this paper, research
within game sound industry has also been fortified; giving developers and consumers a much
stronger groundwork to build on. This includes incorporation of Diegetic and Non-diegetic,
along with Collins’ categorization of game audio, interactive and adaptive. This growing
industry means sound development teams are also increasing; making bigger games with
more technology requires a bigger workforce.
A game developer’s goal is to maximize immersion levels within its consumers, a state in
which players become emotionally attached to the game, experiencing the various levels of
immersion throughout. Although this is also done through gameplay and graphics, it is
largely done through sound, as sound is often the only thing that consumers can completely
relate to. This concept is achieved through interactive and adaptive game music , with the
imminent danger of Ruvic from The Evil Within, the start of a tense police chase in Need For
Speed Most Wanted and the emotional captivity of Joel and Ellie’s relationship in The Last Of
Us. Furthermore, the interactive, adaptive, diegetic and non-diegetic sounds themselves
also contribute to an immersive experience. Examples of this are given such as, Prisoner 849
escaping in Unreal along with the detailed blaster and light sabre sounds from Star Wars
Battlefront.

30
The case study given then offers direct links to the text and provides tangible responses that
support this paper, such as the use of sound for information, and the link between audio and
game immersion. The comparison to the alternative case study then reverses sensory
information, giving a different perspective on the use of these sounds. Having this
comparison allows for further context to be drawn and a more detailed conclusion to be
made.
Drawing a conclusion to this research, it can be clearly argued that interactive and adaptive
audio is imperative in providing consumers with an immersive gameplay experience. Game
developers must continue to adopt the discussed theories in order to maintain the
production of highly immersive games. The interactive and adaptive sounds provided within
a game give players different levels of immersion through identification, empathy, sympathy
and emotional attachment. This connection is even stronger when it comes to music. Music
already provides consumers with strong emotional connections; therefore using it to
maximize a game’s immersive potential is something that game developers achieve on a
regular basis. These music soundtracks can also develop iconic status, therefore raising a
games profile, drawing in more consumers and driving up sales. This can potentially put a
game in the timeless category. The interactive and adaptive sound effects also carry out this
same duty, adding context, detail and providing information for the players to attach
themselves to. Having this attachment then emphasizes that interactive and adaptive audio
had a considerable contribution to the immersive value of a video game.
Technology progression, especially within the gaming industry, has been exponential since
early titles such as The Legend of Zelda. Acknowledging this progression raises intriguing
questions about the future progression of interactive and adaptive audio and its contribution
to immersion for video games. How much more will data storage increase? How much faster

31
will processing speeds become? How interactive and adaptive can these sounds become?
Can they trigger news states of immersion? All of these questions have profound implications
that can influence the future of immersion, game audio and the gaming industry.

32
Bibliography
125, drummer, jojimboy, 8, braininajar, noM, P. 3, 45, D., 1, lord_mbarrow, YOLO, noscope
420, 1983, D., 2, gamedude, 67, M., 474000, ak, 123, crest and 19, pedro (2016) PC master
race - spearheading the Second Golden Age of PC gaming • /r/pcmasterrace. Available at:
https://www.reddit.com/r/pcmasterrace/ (Accessed: 5 February 2016).
2015, G.E. (2015) The importance of audio in gaming: Investing in next generation sound.
Available at: http://gdcvault.com/play/667/The-Importance-of-Audio-In (Accessed: 23 March
2016).
Alcorn, A. (2008) Al Alcorn interview - IGN - page 2. Available at:
http://uk.ign.com/articles/2008/03/11/al-alcorn-interview?page=2 (Accessed: 5 February
2016).
Collins, K. (2008) Game sound: An introduction to the history, theory, and practice of video
game music and sound design. Cambridge, MA: MIT Press.
Collins, K. (2010) ‘An introduction to the participatory and non-linear aspects of video games
audio’, .
Davis, Z. (2004) Lord of the rings: The Third age head-to-head - IGN - page 4. Available at:
http://uk.ign.com/articles/2004/11/11/lord-of-the-rings-the-third-age-head-to-head?page=4
(Accessed: 24 March 2016).
Day, P. (2012) Top tips for starting your own business. Available at:
http://www.bbc.co.uk/news/business-16595152 (Accessed: 13 January 2016).
Diver, M. (2016) A blind man has finally completed ‘ocarina of time’ after Five years | VICE |
United Kingdom. Available at: https://www.vice.com/en_uk/read/a-blind-man-has-finally-
completed-ocarina-of-time-after-five-years-235 (Accessed: 12 April 2016).
Epic Video Game Music (2014) The evil within full OST (by Masafumi Takada). Available at:
https://www.youtube.com/watch?v=fUA43PHWSmg (Accessed: 2 March 2016).
Find (2016) Available at: https://www.psychologytoday.com/basics/neuroscience (Accessed:
28 January 2016).
Frisina, M. (2016) ‘Foundations of video game sound: Gameplay through the context of it’s
audio framework’, .
GameRevolution GR (2013) The last of us - Joel’s choice. Available at:
https://www.youtube.com/watch?v=AyiwPPxGhGI (Accessed: 2 March 2016).
GameRevolution GR (2013) The last of us - Joel’s choice. Available at:
https://www.youtube.com/watch?v=AyiwPPxGhGI (Accessed: 11 April 2016).

33
GameSoundCon 2010: ‘Introduction to game audio’ (no date) Available at:
http://soundworkscollection.com/videos/gamesoundcon (Accessed: 15 January 2016).
Glassner, A. (2004) Interactive storytelling: Techniques for 21st century fiction. A K Peters/CRC
Press.
Grau, O. (2003) Virtual art: From illusion to immersion. Cambridge, MA: MIT Press.
Greenhalgh, M. (2015) Sounding off: Derivative movie sound design?. Available at:
http://www.soundonsound.com/sos/dec15/articles/sounding-off-1215.htm (Accessed: 15
January 2016).
Grimshaw, M. (ed.) (2011) Game sound technology and player interaction: Concepts and
developments. United States: Information Science Reference.
History - digital sound and music in computer games (no date) Available at:
http://3daudio.info/gamesound/history.html (Accessed: 15 January 2016).
Huiberts, S. (2010) Captivating sound - the role of audio for immersion in computer games.
Available at: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf
(Accessed: 15 January 2016).
ICXM (no date) REVIEW: The evil within. Available at: http://www.icxm.net/x/review-the-evil-
within.html (Accessed: 11 April 2016).
IEZA: A framework for game audio (no date) Available at:
http://www.gamasutra.com/view/feature/131915/ieza_a_framework_for_game_audio.php?
page=3 (Accessed: 15 January 2016).
Isaza, M. (2010) Aaron Marks special: Function of game sound effects. Available at:
http://designingsound.org/2010/10/aaron-marks-special-function-of-game-sound-effects/
Kaveh, B. (2010) A fresh look at the concept of immersion. Available at:
http://www.gamedesignideas.com/video-games/a-fresh-look-at-the-concept-of-
immersion.html (Accessed: 24 February 2016).
Lee, J. (1964) Jim Lee quotes at BrainyQuote.com. Available at:
http://www.brainyquote.com/quotes/quotes/j/jimlee606952.html?src=t_gaming (Accessed:
4 February 2016).
Ltd, L. (2016) Creating the sounds of Star WarsTM
BattlefrontTM
. Available at:
http://starwars.ea.com/starwars/battlefront/news/creating-the-sounds-of-star-wars-
battlefront (Accessed: 24 March 2016).
Ltd, L. (2016) Sounds of Star Wars battlefront: Part III. Available at:

34
http://starwars.ea.com/starwars/battlefront/news/sounds-of-star-wars-battlefront-part-3
(Accessed: 11 April 2016).
Madden, B. (2014) Level up: The evolution of video game audio. Available at:
http://www.sonicscoop.com/2014/08/14/level-up-the-evolution-of-video-game-audio/
(Accessed: 5 February 2016).
Mahar, I. (2010) The Two ways you become immersed in video games. Available at:
http://kotaku.com/5949897/the-two-ways-you-become-immersed-in-video-games
Mann, C. and Twiss, G. (1910) Physics. .
Manuel, R. (1973) Rob Manuel quotes at BrainyQuote.com. Available at:
http://www.brainyquote.com/quotes/quotes/r/robmanuel661227.html?src=t_gaming
Martin, M. (2015) Fallout 4 has more lines of Dialog than fallout 3 and Skyrim combined.
Available at: http://uk.ign.com/articles/2015/09/04/fallout-4-has-more-lines-of-dialog-than-
fallout-3-and-skyrim-combined (Accessed: 2 February 2016).
McDonald, G. and Staff, G. (2004) A history of video game music. Available at:
http://www.gamespot.com/articles/a-history-of-video-game-music/1100-6092391/
(Accessed: 15 January 2016).
Prochazka, A. (1999) What do reflex and voluntary mean? Modern views on an ancient
debate. Available at: http://www.biusante.parisdescartes.fr/chn/docpdf/clarac2.pdf
Russell, D. (2012) Video game audio: Diegesis theory. Available at:
http://devmag.org.za/2012/04/19/video-game-audio-diegesis-theory-2/ (Accessed: 24
March 2016).
Santiago, D. (no date) Ocarina of time Walkthrough – inside the great Deku tree. Available at:
http://www.zeldadungeon.net/ocarina-of-time-walkthrough/inside-the-great-deku-tree/
Santiago, D. (no date) Ocarina of time Walkthrough – inside the great Deku tree. Available at:
http://www.zeldadungeon.net/ocarina-of-time-walkthrough/inside-the-great-deku-tree/
(Accessed: 11 April 2016).
Staff, A. (2013) The importance of music in video games. Available at:
http://attackofthefanboy.com/community/importance-music-video-games/ (Accessed: 23
February 2016).

35
Stuart, K. (2015) What do we mean when we call a game ‘immersive’? Available at:
http://www.theguardian.com/technology/gamesblog/2010/aug/10/games-science-of-
immersion (Accessed: 24 February 2016).
Subscribe (2013) Available at: http://www.epicsound.com/2013/06/the-music-and-sound-
design-for-the-last-of-us/ (Accessed: 15 January 2016).
The music and sound design for The Last Of Us (2013) Available at:
http://www.epicsound.com/2013/06/the-music-and-sound-design-for-the-last-of-us/
Vicky (2014) Music, emotion and the brain 12. Available at:
http://musicpsychology.co.uk/music-emotion-and-the-brain/ (Accessed: 2 March 2016).
Whalen, Z. (2004) Play along - an approach to Videogame music. Available at:
http://www.gamestudies.org/0401/whalen/ (Accessed: 5 February 2016).
reserved, A. rights (2015) Download game need for speed most wanted black edition.
Available at: http://speed-new.com/need-for-speed-most-wanted-black-edition-free-
download-pc (Accessed: 11 April 2016).
supersizevideos (2010) Unreal 1 Gameplay level 1. Available at:
https://www.youtube.com/watch?v=Op-tys_98Ek (Accessed: 11 April 2016).

36
Appendix A Participant 1 (The Last Of Us)
‘In comparison the second play through with the sound off was miles worse than the
first one. It just wasn’t the same. It was almost as if when the sound was turned off, it also
turned off all possible connection with the game. I think it was really quite clever actually, to
use this particular chapter, because when you play it the first time around the connection
between you and the characters develops so quickly and then at the end it becomes so
emotional. But when the sound is gone that connection just goes as well. Even though you
already know what is said and how it all sounds. There is still no connection. There is no
immersion.
‘Playing the game with sound gives you so much more information, such as the
dialogue from characters, explosions, noises and such. It made such a difference when that
information wasn’t being fed to me. I didn’t know where to go, what to do and became all
but clueless as to what was actually happening. The only reason I knew what to do was
because I had already played it through once with the sound on.’
Appendix B Participant 2 (The Last Of Us)
‘It’s not something I am new to. I have played quite a few games with the sound off
and it does make a huge difference, especially in this case. You get so much tension building
throughout the gameplay and most of that is done through the sounds that you hear. When
you hear the screams, the explosions, the gunshots, and the sound of Sarah’s frightened
voice and the tense music, all of those build tension and make gameplay so immersive.
‘I found the biggest difference at the end. The level of emotion you get filled with
when Sarah dies is quite overwhelming and surprising when it comes from a game. But then
when you play it with the sound off you get nothing, you become completely detached with
the game and you are just watching a moving picture and interacting with next to no
immersion,. It just goes to show how important it is really.’
Appendix C Participant 3 (The Last Of Us)
‘It’s very strange to compare the two. It’s hard to explain. When the sound is off,
you’re still playing the game, but it just feels like you’re not playing the game. Sounds provide
meanings in a game. Every sound is put there for a reason, to tell you something. Only when
those sounds are taken away, there is literally no point because you can’t enjoy it. You don’t
really think about it properly until you do something like this. I believe the sounds make the
game. If the sounds are wrong then you can’t draw the player in. It just doesn’t work’
‘I think if you had done the experiment the other way around, so you got me to play
the game without sound first, then it would have had a bigger impact. Only because I had
already played it first, and then I knew what to do the second time around. The amount of
information given through sound is incredible.’

37
Appendix D Participant 4 (Star Wars Battlefront)
‘The game audio provides a tremendously immersive experience without being
obvious. Engagement with my allies is necessary to aid survival, and their dialog brings useful
information and warnings of events. Environment noise provides clues on direction, enemy
attack, and a feeling of being close to the action, with enemy weapon fire surrounding me,
alerting me to danger and heightening my level of cautiousness. Additionally, the soundtrack
audio adds great excitement and also provides clues by the use of tone and tempo.
‘From the start, I was drawn deeply into the game and the impending onslaught. I felt
connected with the game and unaware of the fact that it was a game; more of a battle I was
living. Interestingly, I was also somewhat unaware of my real-word environment; such was
my concentration on the game and the feeling of immersion.
‘This is not something I was really aware of until the audio was muted. With no audio
I felt extremely detached from the game. It became, quite simply, an interactive picture that
made me feel as if I was merely controlling a character in a game, rather than being the
character. Excitement was little to none, as was the level of engagement with other charac-
ters or connection with the environment. My character felt alone in the game, and I felt in-
creasingly bored of the silence as time passed.’
Appendix E Participant 5 (Star Wars Battlefront)
‘The first thing that struck me was how realistic the sounds actually are, and how
many sounds there are. When you are in a battle and you can hear countless shots firing
around, along with distant explosions and light sabers, I became so immersed in its realism
and authenticity. The sounds drew me in. When playing the sounds made such a huge differ-
ence. I knew where enemies were, I knew where I was going, and I knew how much progress I
was making. There was almost a feeling to what I was doing within the game and I became
very immersed, through these sounds.’
‘As soon as the sounds were turned off, it was like the game lost all of its life. Playing
a game with just the picture and you controlling a character provided me with no enjoyment
whatsoever. I couldn’t relate to those amazing sounds anymore. Those blaster shots and light
saber sounds didn’t provide me with any more thrills and it became impossible for me to be-
come drawn into the game. Not something I will do again in the foreseeable future.’

Dissertation - Keeran Johnson

More Related Content

Similar to Dissertation - Keeran Johnson

Dissertation - Keeran Johnson