1. On the Explicit and Implicit
Effects of
In-Game Advertising
A master's thesis by Joël Bosch
2. On the Explicit and Implicit Effects of In-
Game Advertising
Joël Bosch
s0823287
mail@joelbosch.nl
January 2014
Master's Thesis
Communication and Information Sciences
Radboud University Nijmegen
Tutor: Prof. dr. W. Spooren
Second assessor: I. Stassen, MA
3. 1
Summary
In this thesis, a study is reported on the effects of in-game advertising. The study
combines qualitative and quantitative methods. Interviews with in-game advertising
professionals have been conducted to help understand the current thoughts of professionals
about the effectiveness of in-game advertising and how it is currently used within the
industry. After that an experiment was conducted to investigate the effectiveness of in-game
billboards using a paper questionnaire and an Implicit Association Test. The professionals
were pessimistic about the current state of in-game advertising, optimistic about its
possibilities and potential, but noted that game developers were hesitant to conduct research
themselves out of fear for unbeneficial results. The experimental results provide evidence for
the positive effects of in-game brands on the explicit and implicit attitudes and possible
moderating effects of cognitive capacity. Additionally, it was found that recognition may play
an important role in the effects of in-game advertising on the explicit attitude.
6. 4
Chapter 1 - Introduction
§ 1.1 - In-game advertising and the gaming industry
In-game advertising is a form of advertising that holds a lot of promise for advertisers,
game developers and even gamers. It is hard to place a number on how many people play
games exactly because of how many different gaming platforms there are and because not all
companies involved share sales figures. It is apparent, however, that games enjoy a large
audience, most of which is very interesting for advertisers. A few examples of the popularity
of some of the more successful games will illustrate the impressive size of the gaming
industry:
The game World of Warcraft (Blizzard Entertainment, 2004) currently has over 8 million
active subscribers (Nunneley, 2013; Tassi, 2013), all paying a fairly substantial fee of €11 to
€13 per month. The fairly recent game Call of Duty: Black Ops 2 (Activision Blizzard, 2012)
hit one billion dollars in sales in just fifteen days (Thier, 2012). And one of the currently most
played pc games is League of Legends (Riot Games, 2009). The game is played by 32 million
players each month and they play it for more than one billion hours every month (Merrill,
2012).
Besides these high-budget games, low-budget games are sold more and more with the
increased popularity of social media and smartphones. The popular game Candy Crush Saga
(King, 2012), available both on Facebook and as an application on phones, has over 6.6
million daily active users with an estimated $632.867 daily revenue (MacIsaac, 2013). Other
big players on the market are Rovio and Zynga. Rovio has had its games downloaded more
than 1 billion times (Rovio, 2013) thanks to the success of Angry Birds (Rovio, 2009) and in
2012 Zynga had a total revenue of close to $1.3 billion thanks to popular games such as
Farmville (Zynga, 2009; Zynga, 2013).
The gaming audience is more diverse than is usually assumed. The stereotypical image of
a gamer is a young male. According to research by Newzoo (2011) amongst 20.000 people
from ten different countries, 47 % of all gamers are female, although with an average of 3.6
hours per week they spend less time on gaming than males (5.3 hours per week on average).
According to research by the Entertainment Software Association (ESA; 2012), 47 % of the
gamers in the USA is female as well (p. 3). While games are mostly played by younger
people, older people also play games frequently. According to the ESA the average age of
gamers in the USA is thirty and only 32 % of the gamers is younger than eighteen (p. 2).
7. 5
Besides the potential to reach a very large audience, another important advantage of in-
game advertising for advertisers is that this type of advertising cannot be easily avoided.
Because advertisements are mainly seen as a nuisance by many consumers, they may try to
avoid advertising (Stühmeister & Wenzel, 2010, p. 2) and they have various ways of doing so.
They can avoid television commercials by changing the channel, diverting their attention to
other things, by leaving the room entirely, muting or turning off the television while the
commercials are playing and with the aid of a digital video recorder they can even fast-
forward through the commercials to the next program (Wilbur, 2008, p. 143). On the internet
it is not so different. Of all the different forms of advertising, unsolicited e-mails and pop-up
ads are found to be the most intrusive and annoying (Kim & Pasadeos, 2006), causing
consumers to ignore the advertisements (called 'banner blindness', see Cho & Cheon, 2004, p.
89) and even use software like spam-filters and ad-blockers (e.g., adblockplus.org) to prevent
themselves from being exposed to the advertisements entirely. In-game advertising does not
suffer nearly as much from these issues. Even though in-game advertising can be ignored,
there is hardly any software and no hardware available that will remove it completely and
because gamers are busy working on their game objectives, they cannot easily walk away
from the advertising.
Another key aspect of in-game advertising is that most gamers do not want to avoid
exposure to it. Several studies have indicated that gamers do not mind in-game advertising or
even appreciate it when used congruently and in an appropriate gaming genre, so that it may
enhance the realism of the game (Lewis & Porter, 2010; Nelson, 2002; Nelson, Keum &
Yaros, 2004). This is both of benefit to consumers, who can enjoy games with in-game
advertising more, and to game developers, who do not receive as much criticism for including
in-game advertisements.
Game developers can sell in-game advertising space to advertisers to help finance the
investments that are involved with the development of the games, which can be up to €40
million for a top-quality game (Yildiz, personal communication, March 29th, 2011). Except
for the income some developers manage to get through pre-order sales, in-game advertising is
the only income games can generate before they are released. Another way in which game
developers can benefit from in-game advertising is through joint promotion. An example is
the extra downloadable content (DLC) for the game Your Shape Fitness Evolved (Ubisoft,
2010) by Nivea. This DLC involved some extra workouts for the game provided by Nivea and
had plenty of in-game advertising for Nivea. The benefit for Ubisoft was that this DLC would
be also be included in a Nivea advertising campaign, creating much more exposure for the
8. 6
game than Ubisoft would have been able to afford by themselves (Yildiz, personal
communication, March 29th, 2011).
In-game advertising sees great use, achieving an extra yearly revenue in the gaming
industry of between one and two billion dollars (Batchelor, 2011; MarketingCharts, 2007;
Yildiz, 2007), although they are not sure about the actual effectiveness of the advertisements
for the advertised brands (Bosch, 2013, pp. 6-7). In order to paint a more accurate picture of
how in-game advertising is currently valued by developers and advertisers, interviews with
professionals have been conducted.
§ 1.2 - Shortcomings of in-game advertising research
The main issues that are addressed in this thesis deal with the actual effectiveness of in-
game advertising. There has been plenty of research that has looked into this problem, but it
has not yet yielded a complete image.
The majority of the early research on in-game advertising was focused on how well
gamers could consciously recognize and recall the advertisements (Chaney, Lin & Chaney,
2004; Grigorovici & Constantin, 2004; Herrewijn & Poels, 2011; Janssen & Helmich, 2011;
Lee & Faber, 2007; Lemon, 2006; Leng, Quah & Zainuddin, 2010; Lewis & Porter, 2010;
Nelson, 2002; Nelson, Yaros & Keum, 2006; Schneider & Cornwell, 2005; Winkler &
Buckner, 2003; Yang, Roskos-Ewoldsen, Dinu & Arpan, 2006). The results of these studies
mostly point at mediocre recall and recognition, suggesting that in-game advertising is not
very efficient. Psychological research in other areas, however, has unveiled the possibility of
information affecting people's memory and behaviour without the necessity for recalling or
recognizing the information (Graf & Schacter, 1985; Schacter, 1987; Shapiro, MacInnis &
Heckler, 1997; see also: Milner, Corkin & Teuber, 1968). Recent research seems to indicate
that these subconscious effects are also present in in-game advertising (Yang et al., 2006;
Glass, 2007; Bosch, 2013). This might be one of the most important aspects of the
effectiveness of in-game advertising and warrants further investigation.
For the purpose of achieving a better understanding of the effects of in-game advertising,
this thesis investigates the effects on both the conscious, explicit attitude and the
subconscious, implicit attitude (Greenwald & Banaji, 1995). For the experiment, participants
played a game in which in-game billboards were manipulated to be one of two distinct brands.
After the gaming session, their explicit and implicit attitudes were measured with the help of
survey questions and an Implicit Association Test (IAT; Greenwald, McGhee & Schwartz,
1998), respectively.
9. 7
Chapter 2 - Literature review
§ 2.1 - Subconscious influences
In the 1950s, when the western world had started to recover from the Second World War,
the consumer society was flourishing in the United States. At first factories were producing
their products en masse and the only thing that could hinder their profit was their inability to
produce more. But as technological advances enabled factories to produce more and more, the
American consumers started to be satisfied with what they had. The economy would surely
collapse if consumers would stop buying new products because their needs were already
satisfied. This is when the demand for advertising experts grew, for they were the ones to
create demand for their products and keep the consumers consuming (Packard, 1957/1960).
With the need for advertising, the need for advertising research grew as well. It became
apparent to the researchers that it was hard to predict consumer behaviour. When a brewery
that produced two different kinds of beer commissioned a study, the interviewers asked
customers what their favourite kind of beer was: the regular beer or the export-quality. 75 %
of the customers answered that they preferred the export-quality beer, but in reality the
brewery sold 9 times more regular beer than export-quality (Packard, 1957/1960, p. 21). This
led the researchers to believe there was more than meets the eye and they started investigating
subconscious processes.
In the beginning researchers tried to find consumers' true motives by using motivational
research and psycho-analytical methods such as the Rorschach inkblot test, the Thematic
Apperception Test and the Szondi test (Packard, 1957/1960, pp. 42-44). All of these tests
seemed to indicate that people could have hidden motives differing from their conscious
motives and that these hidden motives could influence consumer behaviour without them
being aware of it. And not only did the researchers find that it was possible for consumers'
decisions to be influenced by hidden motives, they later also discovered that it was possible
for outside sources to influence these hidden motives without consumers being aware of it
(e.g., Shapiro et al., 1997; Tulving & Schacter, 1990; Zajonc, 2001). Experiments with
patients suffering from amnesia have indicated that people are capable of processing
information without being fully aware of it (Graf & Schacter, 1985; Schacter, 1987; Tulving,
Schacter & Stark, 1982; see also: Milner et al., 1968). Graf and Schacter (1985) call this
‘implicit memory’; they called the conscious processing of information ‘explicit memory’.
10. 8
§ 2.2 - Explicit and implicit attitudes
The distinction between explicit and implicit memory can also be made for attitudes
(Greenwald & Banaji, 1995; Wilson, Lindsey & Schooler, 2000). People can have two
attitudes about the same object that can differ from each other. Which attitude has the most
influence on behaviour is determined by the amount of available cognitive capacity and the
motivation to reason (Friese, Hofmann & Wänke, 2008; Wilson et al., 2000). When sufficient
cognitive capacity is available and the motivation to reason is high, one will consciously
consider one’s actions and behaviour will most likely be guided by the conscious, explicit
attitude. When an individual is stressed for time or otherwise not motivated or able to
consciously consider his actions, he will act spontaneously and his actions will unknowingly
be guided by his implicit attitude (Fazio & Olson, 2003, pp. 304-305; Rydell & McConnell,
2006). Not only do they guide behaviour in different situations, explicit and implicit attitude
are also formed in different ways. Whereas explicit attitudes changed quickly and were
affected by deliberate processing goals, implicit attitudes changed slowly, were unaffected by
processing goals and more influenced by associative information (Rydell & McConnell, 2006,
p. 1006).
Two studies have looked at the potential effects of in-game advertising on the implicit
attitude. They both seem to indicate that in-game advertising can affect the implicit attitude.
Glass (2007) found that in-game product placements can positively affect the participants’
implicit attitude for the advertised brands. Bosch (2013) found that in-game billboards can
positively affect the participants’ implicit attitude for the advertised brands when they were
not under a high cognitive load. Results from studies that have looked into the effects on the
explicit attitude (Grigorovici & Constantin, 2004; Herrewijn & Poels, 2011; Sharma, Mizerski
& Lee, 2007) suggest that in-game billboards and product placements also have a positive
effect on the corresponding explicit attitude.
Distinguishing between explicit and implicit attitudes and including both in the same
analysis can help yield a more complete image about the effects of in-game advertising.
Hence the following hypotheses will be tested:
Hypothesis 1: The presence of in-game billboards has a positive effect on the explicit attitude
towards the advertised brand.
Hypothesis 2: The presence of in-game billboards has a positive effect on the implicit attitude
towards the advertised brand.
11. 9
§ 2.3 - The moderating effect of difficulty level
Previous in-game advertising research has indicated that there may be an important
moderator involved in the effects of in-game advertising on brand attitude. Herrewijn and
Poels (2011) investigated the moderating effect of difficulty level on the effect of in-game
advertising on the explicit brand attitude. They found that the positive effect of in-game
advertising on the explicit attitude was significantly stronger in the lowest difficulty level than
in the highest. Their explanation of this moderation was based on associations. They argued
that this could be explained by participants enjoying the game more at the lower difficulty
level and thus forming more positive associations with the advertised brands than participants
who enjoyed the game less because they were more frustrated by playing at a higher difficulty
level. In this thesis, an alternative explanation is proposed.
This alternative explanation is provided by the Limited Capacity Model (LCM; Lang,
2000) and is based on cognitive processing capacity. According to the LCM, a human being’s
information processing capacity is limited. Before information can be processed any further, it
will first have to reach our senses. Our sensory store may well be able to store a virtually
unlimited amount of information (Lang, 2000, p.48), but is available for only a short amount
of time. This information will have to be stored in the long term memory in order to be
remembered or recognized. However, people cannot house all of the information in their
sensory store in their long term memory, so specific information will have to be selected for
further processing. Both controlled and automatic processes can influence this selection
(Donohew, Lorch & Palmgreen, 1998, p. 454; Lang, 2000, p. 48). Controlled processes work
towards the goals of individuals. They can, for instance, decide to pay extra attention to
people in white shirts, which could mean they will no longer have sufficient cognitive
capacity available to notice what else might happen (Simons & Chabris, 1999). Automatic
processes are mostly activated by the stimulus because it is relevant to the goals and needs of
the individual or represents change or an unexpected occurrence in the environment (Lang,
2000, p. 49). These controlled and automatic processes guide the allocation of cognitive
processing capacity. When playing a game, cognitive capacity will be allocated to the
processing of the information that is needed to play the game through controlled processes.
Through automatic processes, other stimuli in the digital environment may attract the
allocation of the remaining available cognitive processing capacity. In the study by Herrewijn
and Poels (2011), it is possible that there was a moderating effect of difficulty level because
participants in higher difficulty levels would have to process more information through in
order to play the game than in lower difficulty levels. This could lead participants in the
12. 10
higher difficulty levels to be so occupied with playing the game that they had insufficient
cognitive resources left to process advertising to a degree where they would be influenced as
much as the participants in the lower difficulty levels, who had more cognitive resources left
to process other stimuli in the digital environment, like advertisements.
This cognitive explanation would expect the same results as found in the article by
Herrewijn and Poels (2011). Results from another study concerning the effects of in-game
advertising on the explicit attitude seem to support the proposed cognitive explanation. This
study by Grigorovici and Constantin (2004) included secondary tasks. It was found that for at
least one out of three brands, the secondary tasks moderated the effects of the in-game
advertising, indicating that participants under a high cognitive load were less positively
affected by the in-game advertising. The cognitive explanation seems to fit the results of both
of these studies. To test this, the following hypothesis is stated:
Hypothesis 3: When the difficulty level of a game is higher, the positive effect of in-game
billboards on the explicit attitude is weaker.
A similar hypothesis was stated for the moderating effect of the difficulty level of the
game on the relation between the in-game brand and the implicit attitude, even though Bosch
(2013, p. 32) argued that an increased cognitive load would not necessarily hinder the effect
of in-game advertising on the implicit attitude, because it is known that attention for the
advertisement is no prerequisite for implicit memory (Shapiro & Krishnan, 2001) or the
influence of the advertisement on the consideration of advertised brands (Shapiro et al.,
1997). Surprisingly, the results from his study showed that the positive effect of in-game
advertising on the implicit attitude could only be found in the low difficulty condition, which
does suggest a moderating effect of difficulty level. However, only two difficulty levels were
used, which does not indicate whether the moderating effect might be linear or shaped like an
inverted U. Adding a medium difficulty level will give us more information about the
moderating effect of cognitive load. Hence the last hypothesis is as follows:
Hypothesis 4: When the difficulty level of a game is higher, the positive effect of in-game
billboards on the implicit attitude is weaker.
However, testing these hypotheses will not give an estimation of whether the associative
explanation by Herrewijn and Poels (2011) is superior to the proposed cognitive explanation
13. 11
of the effect. If the moderating effect of difficulty level is best explained by the associative
explanation because difficulty level of the game correlates with the frustration level of the
player which affects the associations with the advertised brand, then a more direct
measurement of frustration during the play session should be able to give better a prediction
of the player’s explicit or implicit attitude than the difficulty level. Whether this holds true
will be examined in § 5.2.
Chapter 3 - Method
§ 3.1 - Interviews
To paint a proper picture of how in-game advertising is currently valued by developers
and advertisers, interviews have been conducted with professionals. These professionals were
contacted using a snowball sample, starting with professionals who had previously been
interviewed by the researcher. For the purpose of this research, the professional opinion of the
interviewed on a number of subthemes were interesting and for this reason half-structured
interviews (Baudoin, 2010) were used, using a topic list in accordance with the Grounded
Theory approach (Hijmans & Wester, 2006, p. 508) to be able to expand upon the existing
understanding of the in-game advertising business as is described in the first chapter. The
topic list initially featured a set of topics regarding present professional opinions about in-
game advertising and expected future directions. Once the first interviews had been conducted
and relevant new topics had been brought up or previous topics had reached the point of
saturation, the topic list was altered and new interviews were conducted until a relatively
cohesive picture was discovered. The interviews were all conducted in Dutch, hence the
interviewed will not be quoted, only paraphrased.
The three in-game advertising professionals who were interviewed for this thesis were
Hufen (personal communication, October 9th, 2013), owner of BrandNewGame and author of
a book about in-game advertising, Yildiz (personal communication, October 10th, 2013),
strategic sales manager at Ubisoft and Te Brake (personal communication, October 21th,
2013), owner of iQU. Hufen has worked for a game developer (Atari Benelux), a clothing
brand that has actively advertised in games (Diesel) and an advertising agency (Crossmarks).
After that he founded BrandNewGame, a consultancy and concepting company. Yildiz has
been successful as a strategic sales manager at game developer Ubisoft, setting up numerous
partnerships that involved in-game advertising. Te Brake has been a successful investor,
14. 12
specializing in games and gaming companies. These three professionals were able to share the
viewpoints of advertisers, game developers and investors in gaming companies.
§ 3.2 - Experimental design and sample
A two by three between-subjects experiment was conducted with male students at the
Radboud University campus in Nijmegen in the Netherlands. The experiment was a variation
of the experiment done by Bosch (2013). It differed from the original experiment in that an
extra difficulty level was added during the gaming session and a different measure was used
for the explicit attitude. The participants started the experiment with a gaming session, using a
game in which billboard advertisements had been manipulated. Participants then filled in a
paper questionnaire (Appendix III) to measure their explicit attitude and additional variables
and demographics. After completing the questionnaire, they performed an IAT (Greenwald et
al., 1998) to measure their implicit attitude. The IAT is described in greater detail in § 3.5.2.
The sample consisted of 103 male students. The sample only included men in order to
prevent sex differences from influencing the experiment and only students in order to limit
age-related influences. It is likely that there are distinct differences in the ways men and
women experience computer games (Hartmann & Klimmt, 2006) and in the pace at which
men of differing ages can process information (Deary & Der, 2005). Hence it is possible that
the effects of in-game advertising differ for people of different sexes or ages.
§ 3.3 - Materials
The game Need for Speed: Underground 2 (Electronic Arts, 2004) was used for the
experiment. Similar to games used in many other research articles (Lee & Faber, 2007;
Nelson, 2002; Nelson et al. 2006; Schneider & Cornwell, 2005; Sharma et al., 2007; Winkler
& Buckner, 2006; Yang et al., 2006), this is a racing game. The racing game genre offers
certain advantages for use in experiments, because it often includes in-game advertising and is
easy to pick up for the participants (Bosch, 2013, p. 15). Moreover, this specific racing game
has been used in previous studies (Bosch, 2013; Ho, 2007; Janssen & Helmich, 2011), which
makes the results more easily comparable.
Need for Speed: Underground 2 is the eighth title of the popular Need for Speed racing
games. The goal in the game is to win races that are situated in a city. In-game advertising can
be found on many billboards along the tracks throughout the city, as decoration on the racing
cars and in the brands of the cars themselves. All of the playable cars in the game resemble
real cars and brands outside of the game and the billboards in the city feature both fake and
15. 13
real brands such as Burger King, Autozone and Old Spice. For the purpose of this research, the
branded decoration on the cars was not used in the experiment and the brand of the car was
kept constant for all conditions. The brands of the fast cars in the high difficulty condition
were randomly selected and the slow moving cars in the medium and high difficulty
conditions were not branded.
Billboards were manipulated in this study, resulting in two modified versions of the game,
each with a different brand represented on certain billboards (Appendix I). Because
differences in size and position of the billboards can have an impact on the effect of the
advertisement (Janssen & Helmich, 2011) the exact same billboards were manipulated in both
versions and the same track was used for all participants. In order to equalize the amount of
exposure to the manipulated billboards as much as possible for all participants, special care
was taken when selecting the billboards that were used in the experiment. The billboards that
were selected are in positions along the track where it is unlikely for a participant to make a
mistake which could cause the participant to slow down and be exposed to a manipulated
billboard for a longer amount of time. The track called 'Jackpot' was used for the experiment,
because it features very few alternate routes which could limit exposure to the manipulated
billboards.
The brands that were added to the game for the experiment were the beer brands
Carlsberg and Budweiser. The effects of the advertisements for the two different brands were
compared between subjects, because a standard IAT requires a comparison between two
independent variables. Each experimental condition featured one of these brands on a selected
numbers of billboards along the track. These brands are very well suited for researching the
development of the attitudes of Dutch students (Bosch, 2013, p. 16), because in order to
measurably affect an attitude with a manipulation as subtle as an in-game advertisement, the
participants need to be interested in the subject of the advertisements, without already having
a very strong attitude for the specific brand (Campbell & Keller, 2003). Carlsberg and
Budweiser are both well advertised internationally, so that Dutch students were likely to know
that they are beer brands. But because Carlsberg and Budweiser are hardly available in the
Netherlands, it was unlikely that the participants would have a very strong preference for
either of these brands. On top of that, these brands have clear brand logos that are both easily
recognizable and distinguishable and featured a white background. These logos could both be
implemented in the game and in the IAT.
16. 14
§ 3.4 - Experimental conditions
For each brand, there were three different conditions featuring different difficulty levels.
Each of these difficulty levels was meant to put a different amount of strain on the cognitive
capacity of the participant (Bosch, 2013, p. 16). In each of these difficulty levels the goal for
the participants was to race three laps as quickly as possible. In the low difficulty level, the
participant only had to race three laps as quickly as possible on an empty track. In the medium
difficulty level, the participant also had to race three laps as quickly as possible, but now had
to avoid a low amount of slow moving traffic along the way. In the high difficulty level, the
participant had to race three laps as quickly as possible while avoiding a high amount of slow
moving traffic and a number of fast moving opponents guided by artificial intelligence. The
easy difficulty level demanded the lowest amount of cognitive capacity to be directed at
playing the game, because it provided the lowest amount of important information to be
processed. The medium difficulty level demanded a moderate amount of cognitive capacity to
be directed at playing the game, because the participant had to pay extra attention to avoid
hitting other traffic on the track. Finally, the high difficulty level demanded the highest
amount of cognitive capacity to be directed at playing the game, because there was much
more traffic to avoid on the track and opponents to beat. Each of these difficulty levels
confronted the participant with a different amount of important information to process in
order to quickly reach the finish line.
§ 3.5 - Measures
All variables except for the implicit attitude were measured with a paper questionnaire.
The implicit attitude was measured with an IAT. After the participants had played their
gaming session, they first completed the paper questionnaire. Once they had completed the
questionnaire, they returned to the computer to do the IAT. How the implicit and the explicit
attitudes were measured and which control variables were taken into account is discussed in
the following paragraphs.
§ 3.5.1 - Explicit attitude
In previous in-game advertising research, the explicit attitude has often been measured
with a Likert-scale (Grigorovici & Constantin, 2004; Herrewijn & Poels, 2011; Lemon,
2006). However, Bosch (2013, p. 31) suggested that a single 7-point scale might not have a
level of sensitivity that is comparable to that of an IAT, making it possible that a similarly
subtle effect of exposure to in-game advertising is picked up by the IAT, but not by the
17. 15
explicit attitude self-report measure. In previous studies that compared explicit attitudes to
IAT outcomes (e.g., Greenwald et al. 1998; Karpinski & Hilton, 2001) respondents were often
asked to rate their level of liking on a thermometer scale ranging from 0 (cold) to 50 (neutral)
to 100 (hot). Hence, the explicit attitude was measured using a thermometer scale instead of
Likert-scales so that subtler effects might still be measurable. To prevent this measure from
influencing the participants for the recognition task at the end of the questionnaire,
participants were asked to rate their level of liking for 24 different beer and soda brands, only
one of which was the advertised beer brand.
§ 3.5.2 - Implicit attitude
The implicit attitude was measured by an IAT (Greenwald et al., 1998). There are many
different versions of the IAT available, but in this study a version was used which was
translated to Dutch and used pictures to represent the brands that are advertised in-game
(Bosch, 2013, pp. 49-58). This IAT was designed to measure which of two categories of
stimuli was associated most with either positive or negative evaluative terms. People associate
their favourite brand more with positive terms and can more easily categorize their favourite
brand in the same category as positive terms than negative terms, causing them to react
quicker than when they are tasked to categorize the same brand in the same category as
negative terms. The IAT can measure the reaction times for each task and compare them to
calculate their implicit preference.
The categories used in the IAT in this experiment represent Carlsberg and Budweiser - the
brands that were advertised in the game. Nosek, Greenwald & Banaji (2005) noted that two
pictures per category can be sufficient to find measurable effects. They also found that a
larger amount of pictures increased the accuracy of the IAT by only a minimal amount; it is
better to use a few pictures that are strong representations of the category than to use a
multitude of pictures that are weak representations. Thus only three different pictures were
chosen to form each category. The pictures were adopted from Bosch (2013, p. 48; Appendix
II). They all clearly represented their category and were clearly distinguishable due to the
dominance of the colour green in the pictures which represented Carlsberg and the colour red
in the pictures which represented Budweiser.
The IAT consisted of seven blocks. The participant received a different task for each
block. Before the start of each block, they were provided with an explanation of their specific
task for that block. During the IAT, the participants only had to use three keys: the 'e'-key and
the 'i'-key during the block and the spacebar to continue after the explanation (the
18. 16
international default US keyboard settings were used). In the top left and the top right side of
the screen, the categories which the keys represented were displayed. The 'e'-key, which is
situated on the left side of the keyboard, always represented the categories on the top left side
of the screen. The 'i'-key, which is situated on the right side of the 'e'-key, represented the
categories on the top right side of the screen. The participant was to place one finger of each
hand on one of these keys and every time an image or a word popped up in the middle of the
screen, they had to press the key that corresponded to the right category for that image or
word. For example: when the category 'good' was noted in the top left side of the screen and
the word 'pleasant' appeared in the middle of the screen, the participant was to hit the 'e'-key.
In the first two blocks, the participant was explained how they were supposed to use the
IAT. In the first block, they were introduced to the pictures for the beer brands, which they
were to categorize under Carlsberg or Budweiser. In the second block, they were introduced
to the positive and negative evaluative terms, which they were to categorize under 'good' or
'bad'. Every time they hit the right key, they instantly went on to the next picture or word until
they had completed the entire block. If the participant hit the wrong key, he was notified that
he made a mistake and had to click the other key before he moved on to the next picture or
word. All participants were explicitly asked to complete the tasks as fast as they could.
Once the introduction blocks had been completed, both tasks of categorizing the beer
brand pictures and categorizing the evaluative words were merged into one block. In each top
corner of the screen there were both the category 'good' or 'bad' and the name of one of the
beer brands. The participants were now randomly assigned either an evaluative term or a beer
brand picture to categorize. Once they had completed two blocks with this assignment, the
placement of the brands changed. The brand that had been in the top left was now in the top
right of the screen, causing the task to change: the beer brand the participants had been tasked
to categorize on the same side as the positive evaluative terms was now to be paired with the
negative evaluative terms and vice versa. For this task, the participants first completed one
block in which they were introduced to the new task. After this, they completed two more
blocks with this task. The recorded reaction times from the introduction blocks at the start and
the one in the middle were not used in the calculation of the implicit preference. Because the
task change in the middle of the IAT might have thrown off participants, causing them to
react significantly slower and make more mistakes in the second part of the IAT, the outcome
of the IAT might have been slightly biased towards the association in the first half of the IAT.
To prevent this effect from interfering with the results, half of the participants began with the
task to categorize Carlsberg on the same side as the positive evaluative terms in the first half
19. 17
of the IAT and the other half of the participants began with the task to categorize Budweiser
on the same side as the positive evaluative terms in the first half of the IAT.
§ 3.5.3 - Control variables
In addition to the explicit measure, several extra questions were added to the paper
questionnaire (Appendix III) about variables that might interfere with the results, but could
not be excluded from the experiment entirely.
Before the explicit self-report measure, a few questions about age, interest in beer, gaming
experience and enjoyment were asked. Even though only students were invited to participate
in the experiment, the influence of age differences could not be entirely excluded. To be able
to account for these age differences, the year of birth was asked. Students are generally known
for their interest in beer, but it was still possible that some students included in the experiment
did not drink beer and were thus less likely to form an attitude after seeing a beer
advertisement. It is also possible that extreme beer enthusiasts are less likely to change their
attitude after seeing a beer advertisement, because they already have strong attitudes about
beer brands. In order to be able to account for some of the varying levels of beer interest,
participants were asked how many glasses of beer they drank in an average week. Several
researchers have suggested that experienced gamers might process in-game advertising
differently from inexperienced gamers (Chaney et al., 2004; Lee & Faber, 2007; Lemon,
2006; Schneider & Cornwell, 2005). How experienced gamers' processing differs from
inexperienced gamers is yet unclear, because the results from previous studies seem to be
contradicting each other. Schneider and Cornwell (2005) found that experienced gamers were
more capable of recognizing and remembering in-game banners than inexperienced gamers.
Lemon (2006), however, could only partially confirm this. Lee and Faber (2007) found that
experienced gamers were slightly better at recognizing brands that were placed in focal
positions than brands placed in peripheral vision when they were only moderately involved
with the brand. They did not find this effect for inexperienced players, suggesting that
inexperienced players may distribute their attention more equally over all parts of the screen,
whereas experienced players pay extra attention to the most important parts. Chaney et al.
(2004) did not find any difference in the recall of advertised brands between experienced and
inexperienced gamers. The apparent contradictions in these findings may be caused by
different ways of measuring experience. This is why participants were asked whether they had
played the specific version of the game used in the experiment (Need for Speed: Underground
2, Electronic Arts, 2004) before and to make a judgement call about their own experience
20. 18
with racing games on a 7-point Likert-scale. The degree to which participants in the
experiment were frustrated with the gaming session might have a moderating effect on the
effects of in-game advertising, because gamers might form less favourable associations with
the brand if they were frustrated while playing the game (Herrewijn & Poels, 2011, p. 5). To
control for this effect, participants were asked to indicate their frustration during the gaming
session on a 7-point Likert-scale.
At the end of the questionnaire, after the participants had indicated their level of
preference for 24 different beer and soda brands for the explicit attitude measure, they were
asked which brands they believed to have seen during the gaming session. The same 24 beer
and soda brands were displayed on a new page and the participants were asked to indicate
which brands they had seen by marking them. For this task, the participants were asked to
mark a brand even if they had only a vague suspicion that they might have seen it. They were
able to mark any number of brands they liked and they were not told that there was only one
correct answer. If they truly had no idea, they were allowed to leave all brands unmarked.
§ 3.6 - Analysis
Before the data were analyzed, outliers on the dependent variables for the explicit and
implicit attitude and the outliers on the variable measuring beer interest were considered for
removal from the dataset in order to prevent them from significantly skewing the results. Due
to procedural mistakes, four participants were removed from the data entirely and for one
participant the IAT results were removed.
Greenwald, Banaji and Nosek (2003) have done extensive research on extrapolating a
meaningful measure from the IAT. Several different methods were analyzed using a very
large data pool in order to find the measure that has the highest correlation with the explicit
attitude. This might not be the best way to find a good measure for the implicit attitude,
because there is plenty of research indicating that the explicit and implicit attitudes are
independent from each other (Friese et al., 2008; Karpinski & Hilton, 2001; Olson & Fazio,
2001; Rydell & McConnell, 2006; Wilson et al., 2000). However, because the aim of this
study was to find differences between the explicit and the implicit attitude, it was a good idea
to follow the advice given by Greenwald et al. (2003) for data processing; if, after following
this advice, a clear distinction between the explicit and implicit attitude could still be found,
this could not be simply due to the manner in which the data were processed.
Following the recommendations of Greenwald et al. (2003), an extremely slow reaction
time of more than 10 seconds was removed from the data. Even though the first two reactions
21. 19
in each block were significantly slower than the rest of the reactions, Greenwald et al. (2003,
p. 202) found that including these two reactions would cause the implicit attitude measure to
correlate better with the explicit measure and correlate less with the extreme values in the
IAT. Hence, they were included in the data. Extremely fast reactions and mistakes may have
been caused by participants who did not take the IAT seriously and spammed random keys in
order to complete the IAT as fast as possible. This can lead to uninterpretable data and while
it is desirable to compensate for this, it is not desirable to delete too large an amount of data,
which would increase the risk that important data would be lost. Greenwald et al. (2003, pp.
204-205) recommended excluding participants with more than 10 % fast (< 300 milliseconds)
or slow (> 3000 milliseconds) reactions. None of the participants fitted this description, so no
data were removed on the basis of this criterion.
No special treatment was given to false answers, because they were recorded as slow
reactions (Greenwald et al., 2003, p. 202). Every time a participant made a mistake he was
prompted to correct his answer. The IAT then recorded the time between the first appearance
of the image or word and the moment the right answer was given. This way a false answer
was composed of both the time it took to give a false answer and the time it took to correct it
and thus the false answer was recorded as a slow reaction time. Because both slow and false
answers indicate that the participant had trouble associating the matched evaluative terms and
brand, there was no need to treat them differently.
The best measure to calculate the implicit attitude, according to Greenwald et al. (2003, p.
212), is the D-measure (p. 201; Bosch, 2013, p. 23), which was also used in this thesis. The
D-measure is the standardised difference between the compatible (where brand A is to be
categorized with positive evaluative terms) and the incompatible part (where brand A is to be
categorized with negative evaluative terms) of the IAT. Each of these parts is divided up into
introductory blocks which are not used in the calculation of the D-measure and two main
blocks: one block with 20 reaction tasks, followed by a block with 40 reaction tasks. For the
smaller blocks with the 20 reaction tasks and the larger blocks with the 40 reaction tasks, the
difference was calculated and standardised separately. In practise, this means that the D-
measure subtracts the mean reaction time in block 6 from the mean reaction time in block 3,
then divides this by the standard deviation of blocks 3 and 6 combined. Similarly, the mean
reaction time in block 7 is subtracted from the mean reaction time in block 4, then divided by
the standard deviation of blocks 4 and 7 combined. These two values are then averaged to
create the D-measure. Because the order of the IAT was varied to prevent order effects from
affecting the results, the D-measure had to be adjusted to be in accordance with the order in
22. 20
which the brands appeared in the IAT. This was done so that a negative value would indicate
an implicit preference for Budweiser and a positive value would indicate an implicit
preference for Carlsberg. The value 0 is an indication of a participant who has no preference
of one brand over the other. Values closer to 0 also indicate a smaller preference than values
further away from 0.
To strengthen the comparability to the implicit measure, the explicit measure was
calculated in a similar fashion. The grade for Budweiser was subtracted from the grade for
Carlsberg. In this way, a measure was created with negative values indicating an explicit
preference for Budweiser compared to Carlsberg and positive values indicating an explicit
preference for Carlsberg compared to Budweiser. The value 0 is an indication of a participant
who graded both brands equally. Values closer to 0 also indicate a smaller preference of one
brand over the other than values further away from 0.
The control variables did not lead to the exclusion of any participants. Participants had an
average age of M = 20 (SD = 2.46), the youngest being 17 and the oldest being 31. An
exploration of the interest in beer, measured by the amount of glasses consumed weekly, did
not produce more outliers than was to be expected. Other control variables were included in
the regression analyses discussed in § 5.2.
§ 3.7 - Analysis model
The hypotheses can be presented in a model (Figure 1) with in-game advertising as an
independent variable with direct effects on both the explicit and the implicit attitude, both
modified by the available cognitive capacity. In-game advertising is a nominal factor
consisting of two levels, determining the presence of one brand and the absence of the other.
In-game advertising is hypothesized to have a direct influence on the explicit and implicit
attitudes. This influence can be viewed as a positive relation, whereas the presence of a brand
in-game will increase the explicit and implicit preference for that brand. The cognitive load
acts as a moderating factor in this model. This ordinal factor consists of three levels: low,
medium and high cognitive load. It was hypothesized that a higher cognitive load would lead
to a lesser influence of the advertised brand on the explicit and implicit attitude. The implicit
D-measure is an interval variable with both positive (preference for Carlsberg) and negative
(preference for Budweiser) values, centred around 0, which means there was no preference for
either brand. The explicit measure is the difference between two ratio variables, ranging from
0 (cold) to 50 (neutral) to 100 (hot), to indicate the level of preference.
23. 21
H3 (-)
H4 (-) H1 (+)
H2 (+)
This model will be tested in § 5.1 using an analysis of variance, looking at the main effects
of the in-game brands and the difficulty level as indications of the effects of in-game
advertising and cognitive load. Furthermore, the moderating effect of cognitive load will be
tested by looking at the interaction effect of the in-game brands and the difficulty level. This
interaction will be more closely examined using pairwise comparisons and separate univariate
tests. This analysis will be done separately for the explicit attitude and the implicit attitude.
Chapter 4 - Qualitative results
§ 4.1 - In-game advertising in the present
Even though they had all worked on multiple in-game advertising projects in the past, the
interviewed were pessimistic about the usefulness of in-game advertising in the current
gaming business. All of the interviewed could explain from their own perspective that in-
game advertising was not that important.
To advertisers, in-game advertising is not important because it is either unknown to them
or they do not see the potential (Hufen, 2013). They will often contact advertising agencies to
do their advertising for them and those advertising agencies are hardly ever experienced with
in-game advertising (Hufen, 2013; Yildiz, 2013). According to Yildiz, advertising agencies
hardly use in-game advertising because of numerous reasons that hardly have anything to do
with in-game advertising itself. He was often confronted with people in advertising agencies
who had a very negative or niche depiction of games, who do not view themselves as gamers,
despite of them often playing games on their mobile phone or tablet. They would rather
advertise on television, because that is easier and they can better understand it. In-game
Cognitive load
In-game
advertising
Implicit attitude
Explicit attitude
Independent variables Dependent variables
Figure 1: Hypothesized model of analysis
24. 22
advertising seems risky and obscure to them since there is no model of effectiveness for in-
game advertising yet. This lack of certainty and familiarity coupled with the high investment
costs to do in-game advertising makes advertising agencies shy away from in-game
advertising (Yildiz, 2013).
Other issues for advertisers are the nature of the game and the nature of game
development. A lot of popular games are quite violent and most brands are hesitant to
associate themselves with a game like that (Yildiz, 2013). Only technological companies for
which gamers are specifically an important target demographic dare to advertise in and
around these games, because they know the gamers will associate the brand with a fun game,
not the violence that occurs in the game.
High budget games are complex and can take a long time to develop. Developing a game
is a creative process and creating processes have difficulty dealing with restrictions, hence
extended release dates for games are quite common. This is hard for advertisers to deal with,
because they often have to report short term results (Yildiz, 2013).
To big game developers, in-game advertising is not important because it delivers very
little revenue compared to the often enormous investments they make in order to create a new
game (Yildiz, 2013). For them, it can be more interesting as a part of an integrated
partnership. For example, Yildiz launched a partnership with Nivea, a large global brand
which would advertise in one of Ubisofts games and include the game in their global
marketing campaign. Another reason for game developers to use in-game advertising is to
strengthen the credibility of their game.
For investors, in-game advertising is not all that important either. When asked about the
importance of in-game advertising, Te Brake (2013) mentioned that it can be much more
interesting to invest in a game that is driven by in-game purchases than a game that is driven
by in-game advertising. Instead of relying on the income generated by advertisements, these
games allow the players to purchase in-game items with actual money. While there is more
risk involved with investing in a game that relies on in-game purchases to flourish because
this source of income is less certain, there is a chance that the game might be the next big
thing and profits go through the roof. Whereas if a game that relies on in-game advertising
goes big, the profits will most likely have to be shared with an advertising agency that's
shaving off a good percentage.
25. 23
§ 4.2 - In-game advertising in the future
Even though they were pessimistic about the current state of in-game advertising, the
interviewed were enthusiastic for the possibilities that games offer to advertising. Especially
with the far-stretching integration of social networking and the possibilities of mobile games
they saw potential in in-game advertising.
All three thought that in-game advertising should not be used as a onetime thing; instead,
it should be incorporated in a larger media campaign (Hufen, 2013; Te Brake, 2013) and
preferably as part of a longer running project. This is currently hindered by a generation gap;
the people who run advertising agencies are not familiar with games and do not see their
advertising potential. Only certain progressive advertising agents actually dare to go with an
in-game advertising project, but since these advertising agents are often only temporarily
working for the agency, these are short term projects which do not utilize the fact that games
can keep being played for years on end (Yildiz, 2013). Over time, these agents will be
replaced with people from newer generations who are more familiar with games and this
might give in-game advertising a new boost. Yildiz already mentioned that he had noticed the
start of a trend towards this, saying that for example car brands are no longer hesitant to let
game developers use their cars in games, where they used to be hesitant about this out of fear
that gamers would damage their car in the game and that this would make their brand look
bad. More recently the dynamic nature of games is more and more accepted and brands no
longer see this as a serious issue.
While game developers are in a good position to study the effectiveness of in-game
advertising, none of them are interested because they would rather not do any research than
risk finding out that in-game advertising is not effective. Mobile gaming might be a different
story, since game developers are using customer information tools as a unique selling point,
using the fact that most mobile phones are always online and have tools like GPS to gather
data. Yet this is still in its infancy (Yildiz, 2013). Using this technology to create databases
with customer profiles may help to advertise more efficiently (Hufen, 2013; Te Brake, 2013)
and makes it easier to sell in-game advertising. Other information about the effectiveness of
in-game advertising may also help convince advertising agencies of the value of in-game
advertising. The results of the quantitative part of this study may contribute to that.
26. 24
Chapter 5 - Quantitative results
§ 5.1 - Testing the main model
Univariate analyses of variance were performed with IBM's SPSS Statistics (SPSS) to test
the hypotheses with the explicit and implicit attitudes as the dependent variables. All analyses
included the in-game brand (either Carlsberg or Budweiser) and the difficulty level (low,
medium and high) as independent variables, including the interaction between the two. The
interaction effects were more closely examined using pairwise comparisons and separate
univariate tests. The results are discussed in this paragraph.
§ 5.1.1 - Explicit attitude
As can be seen in Table 1, the explicit attitude cannot be significantly predicted with brand
(F < 1) or difficulty (F(2,90) = 1.183, p = .311, η2
= .026) alone. This does not support the
first hypothesis, which stated that brands advertised in the game would affect the explicit
attitude.
There is, however, a significant interaction between the two (F(2,90) = 3.674, p = .029, η2
= .075). How effective the in-game branded billboards are at affecting the explicit attitude
differed between difficulty levels. In the low difficulty condition, the in-game brand did not
significantly affect the explicit attitude (F < 1), nor in the high difficulty condition (F(1,90) =
2.544, p = .114, η2
= .027). Only in the medium difficulty condition did the in-game
billboards significantly affect the explicit attitude (F(1,90) = 4.973, p = .028, η2
= .052). In the
medium difficulty condition, participants who had played the game with Carlsberg billboards
gave Carlsberg higher grades than Budweiser (M = 16.467, SE = 6.010) and participants who
had played the game with Budweiser billboards gave Budweiser higher grades than Carlsberg
(M = -2.188, SE = 5.819). This supports the first hypothesis that in-game brands influence the
explicit attitude of the gamer and partially supports the third hypothesis, which stated that the
Table 1: Analysis of Variance for the Effects on the Explicit Attitude
Source df F η2
p
Brand (B) 1 0.415 0.005 0.521
Difficulty (D) 2 1.183 0.026 0.311
Interaction (B x D) 2 3.674* 0.075 0.029
Error 90 (541.8)
Note. Value in parenthesis represents mean square error. *p <.05
27. 25
effect that brands have on the explicit attitude is modified by difficulty. However, the
effectiveness of brands does not simply decrease with increased difficulty; there seems to be a
certain optimum at the medium difficulty level where brands can have a significant effect on
the explicit attitude, whereas brands do not significantly affect the explicit attitude at a low or
high difficulty level (Figure 2).
Figure 2: Explicit preference for Carlsberg after playing a game with Carlsberg or
Budweiser billboards in low, medium and high difficulty conditions.
Note: Positive values correspond with a preference for Carlsberg; negative values
correspond with a preference for Budweiser.
§ 5.1.2 - Implicit attitude
As can be seen in Table 2, the results were rather different for the implicit attitude. There
was no significant interaction between brand and difficulty, but both brand (F(1,90) = 10.422,
p = .002, η2
= .102) and difficulty (F(2,90) = 3.203, p = .045, η2
= .065) could significantly
explain some of the variation in the implicit attitude. Participants who had played the game
with billboards advertising Carlsberg preferred Carlsberg over Budweiser (M = .206, SE =
.082) and participants who had played the game with billboards advertising Budweiser
preferred Budweiser over Carlsberg (M = -.168, SE = .082). The three difficulty levels
-10
-5
0
5
10
15
20
Low Medium* High
Budweiser
Carlsberg
Difficulty level *p < .05
28. 26
together showed a significant trend whereby participants in the easiest difficulty condition
preferred Carlsberg over Budweiser (M = .147, SE = .101), participants in the medium
difficulty condition preferred Carlsberg over Budweiser, but less strongly (M = .097, SE =
.098) and participants in the high difficulty condition preferred Budweiser over Carlsberg (M
= -.188, SE = .101), although pairwise comparisons did not yield any significant differences
between the average implicit attitudes of participants in conditions with different difficulty
levels. These results support the second hypothesis, stating that in-game brands can affect the
implicit attitude, but they do not support the fourth hypothesis. Instead of the hypothesized
moderating effect of the difficulty level, a main effect of difficulty level was found.
Table 2: Analysis of Variance for the Effects on the Implicit Attitude
Source df F η2
p
Brand (B) 1 10.422* 0.102 0.002
Difficulty (D) 2 3.203* 0.065 0.045
Interaction (B x D) 2 0.555 0.012 0.576
Error 92 (.329)
Note. Value in parenthesis represents mean square error. *p <.05
A closer inspection of the interaction of brand and difficulty level reveals that while the
interaction variable did not reach significance in the analysis of variance, the means do follow
a pattern that fits an interaction model. Participants who had played the game in which
Budweiser was advertised showed greater preference for Budweiser than participants who had
played the version of the game in which Carlsberg was advertised at all difficulty levels, but
the differences of the implicit attitude between the participants of both versions was largest in
the lowest difficulty condition (M = -.126, SE = .143 for Budweiser, M = .421, SE = .143 for
Carlsberg), smaller in the medium difficulty condition (M = -.056 SE = .139 for Budweiser, M
= .251, SE = .139 for Carlsberg) and smallest in the high difficulty condition (M = -.323, SE =
.143 for Budweiser, M = -.054, SE = .143 for Carlsberg). The difference between the two
versions in the low difficulty condition was significant (F(1,92) = 7.269, p = .008, η2
= .073),
whereas the differences in the medium difficulty condition (F(1,92) = 2.442, p = .122, η2
=
.026) and the high difficulty condition (F(1,92) = 1.754, p =.189, η2
= .019) were not. This
leads to a more easily interpretable model, detailed in Figure 3.
29. 27
Figure 3: Implicit preference for Carlsberg after playing a game with Carlsberg or
Budweiser billboards in low, medium and high difficulty conditions.
Note: Positive values correspond with a preference for Carlsberg; negative values
correspond with a preference for Budweiser.
§ 5.2 - Testing the associative explanation and other control variables
In the theory chapter, the explanation of the moderating effect of difficulty level by
Herrewijn and Poels (2011) was considered and several control variables were added to the
questionnaire. Using regression analysis, the predictive strength of their explanation and the
control variables can be compared to that of the variables used in the main analysis. The
results of these analyses are discussed in this paragraph.
§ 5.2.1 - Explicit attitude
In this analysis, the associative explanation of the moderating effect of difficulty level is
tested. It was theorized that difficulty level would interact with brand because a higher
difficulty level would introduce more information to process, leaving less cognitive capacity
to process the advertisements. However difficulty level is also likely to correlate with
frustration, meaning participants were more likely to get frustrated in higher difficulty levels.
This frustration could lead to less positive associations with the brand, reducing the
advertising effectiveness in higher difficulty levels. While it is not possible to completely
-0,4
-0,3
-0,2
-0,1
0
0,1
0,2
0,3
0,4
0,5
Low* Medium High
Budweiser
Carlsberg
Difficulty level *p < .05
30. 28
disentangle difficulty level in this study, it is possible to compare the predictors of difficulty
level and frustration in a regression analysis, using the standardized Beta coefficients. If the
associative explanation is superior to the cognitive explanation, an interaction between the
effects of frustration and in-game brand should be able to explain more variance than an
interaction between difficulty level and in-game brand. Additionally, recognition was added
to the analysis in order to check if it might be a prerequisite for the influence of the in-game
brands or if it is possible to be influenced by the in-game billboards without recognizing them
after the gaming session.
Several adaptations had to be made to properly execute the regression analysis. Even
though there were not more outliers than was to be expected, two outliers were extreme
outliers of more than three times the standard deviation, which is why they were filtered out
for the regression analyses regarding the explicit attitude. Because difficulty level is an
ordinal variable containing three categories, dummy variables were used. The variables
regarding difficulty level, frustration and recognition were then centred to prevent collinearity
from hindering the regression and new interaction variables were calculated using these
centred variables. After that, sheaf coefficients were calculated from the dummy variables to
allow them to be compared to the other variables.
The seven included predictors explained 20 % of the variance (R2
= .200, F(7,85) = 3.042,
p = .007, see Table 3). Of these seven predictors, only the interaction between brand and
difficulty (β = .258, t = 2.598, p = .011) and the interaction between brand and recognition (β
= .232, t = 2.243, p = .027) significantly predicted the explicit attitude. The interaction
between brand and frustration (β = .014, t < 1) could not significantly predict the explicit
Table 3: Regression Analysis for the Effects on the Explicit Attitude
Source β t p
Brand (B) 0.088 0.895 0.374
Difficulty (D) 0.111 1.115 0.268
Frustration (F) 0.102 1.109 0.311
Recognition (R) -0.147 -1.411 0.162
Interaction (B x D) 0.258 2.598* 0.011
Interaction (B x F) 0.014 0.014 0.891
Interaction (B x R) 0.232 2.243* 0.027
R2
= .200, *p <.05
31. 29
attitude and no significant main effects were found. These results do not suggest that the
associative explanation is superior to the cognitive capacity explanation regarding the
influence of in-game advertising on the explicit attitude. Additionally, it shows that
recognition of the brand may help the in-game brand to influence the explicit attitude.
An analysis of variance was used to examine the interaction between brand and
recognition more closely, using pairwise comparisons and separate univariate analyses. The
difference in the explicit attitude of participants that were able to indicate which brand was
advertised between the condition in which Budweiser was advertised (M = -9.429, SE =
5.223) and the condition in which Carlsberg was advertised (M = 12.604, SE = 6.691) was
significant (F = 6.410, p = .013, η2
= .069). This difference was also greater than the
difference in explicit attitude of participants that did not recognize the brand between the
condition in which Budweiser was advertised (M = 8.282, SE = 3.005) and the condition in
which Carlsberg was advertised (M = 6.148, SE = 2.897), which was not significant (F < 1).
This suggests that participants were more susceptible to the in-game advertising when they
were able to correctly guess the brand that was advertised in the game (Figure 4).
Figure 4: Explicit preference for Budweiser or Carlsberg when participants were able or
unable to recognize the advertised brand.
Note: Positive values correspond with a preference for Carlsberg; negative values
correspond with a preference for Budweiser.
-15
-10
-5
0
5
10
15
Not recognized Recognized
Budweiser
Carlsberg
Recognition *p < .05
32. 30
To test for any possible interference of gaming experience, another regression analysis
was performed. Gaming experience was measured in two ways: the participants indicated
whether they had previously played this specific version of the game and how much
experience they deemed themselves to have playing racing games. Both of these variables and
their interactions with the in-game brands and the main effect of in-game brands were
included. This model could not significantly predict the explicit attitude (R2
= .045, F(5,88) <
1). Both having previously played the game (β = -0.030, t < 1) and self-reported race game
experience (β = .053, t < 1) had no significant main effect on the explicit attitude, nor did they
significantly interact with the in-game brand conditions (respectively: β = .164, t = 1.471, p =
.145; β = .045, t < 1).
§ 5.2.2 - Implicit attitude
The same explanation and control variables were tested for the implicit attitude. The same
adaptations that were made for the regression analysis involving the explicit attitude were
made for this regression analysis, except no extreme outliers were found.
The seven included predictors explained 19.1 % of the variance (R2
= .191, F(7,88) =
2.968, p = .008, see Table 4). Of these seven predictors, only the main effects of brand (β =
.305, t = 3.158, p = .002) and difficulty (β = .232, t = 2.366, p = .020) significantly predicted
the implicit attitude. Interactions between brand and frustration (β = .121, t = 1.191, p = .237),
brand and difficulty (β = .127, t = 1.298, p = .198) and brand and recognition (β = .087, t < 1)
could not significantly predict the explicit attitude and no significant main effects were found
for frustration (β = -.061, t < 1) and recognition (β = .100, t < 1). These results do not suggest
that the associative explanation is superior to the cognitive capacity explanation regarding the
influence of in-game advertising on the implicit attitude. Additionally, it suggests that
recognition of the brand is not required for the in-game brand to influence the implicit
attitude.
To test for any possible interference of gaming experience, another regression analysis
was performed. Gaming experience was measured in two ways: the participants indicated
whether they had previously played this specific version of the game and how much
experience they deemed themselves to have playing racing games. Both of these variables and
their interactions with the in-game brands and the main effect of in-game brands were
included. This model could significantly predict the implicit attitude (R2
= .119, F(5,91) =
2.449, p = .040). However only the main effect of brand could significantly predict the
33. 31
implicit attitude (β = .301, t = 3.050, p = .003). Having previously played the game (β =
0.024, t < 1) and self-reported race game experience (β = -0.036, t < 1) had no significant
main effect on the implicit attitude, nor did they significantly interact with the in-game brand
conditions (β = -0.075, t < 1; β = -0.112, t = -1.064, p = .290, respectively).
Chapter 6 - Conclusion and discussion
§ 6.1 Conclusion
This thesis consists of two parts, a qualitative part with interviews to place the study in the
right professional context and a quantitative part with an experiment testing the effectiveness
of in-game advertising in influencing both the explicit and the implicit attitudes. In this
chapter, the conclusions to each of these will be given are discussed, followed by a short
paragraph touching on some of the limitations of the current study and finally a general
discussion.
§ 6.1.1 Interviews
The qualitative part of this study largely confirms the introduction to this thesis: in-game
advertising shows potential as a great advertising platform, yet it is currently underutilized.
Some of the reasons behind why in-game advertising is not more common became clear.
There are a lot of factors around games that complicate investing in in-game advertising. High
costs, long development and thus a late return on investment, a lack of knowledge about in-
game advertising and unfamiliarity with in-game advertising and games in general are some
of the major hindering factors for advertisers and the often involved advertising agencies. For
game developers in-game advertising adds only a relatively small amount to their revenue and
Table 4: Regression Analysis for the Effects on the Implicit Attitude
Source β t p
Brand (B) 0.305 3.158* 0.002
Difficulty (D) 0.232 2.366* 0.020
Frustration (F) -0.061 -0.598 0.522
Recognition (R) 0.100 0.981 0.329
Interaction (B x D) 0.127 1.298 0.198
Interaction (B x F) 0.121 1.191 0.237
Interaction (B x R) 0.087 0.865 0.389
R2
= .191, *p <.05
34. 32
investors can seek greater rewards from games that rely on in-game purchases instead of
advertising. Yet the interviewed still all saw potential for in-game advertising with some
trends in integrating social networking and mobile gaming. In addition, it was found that
game developers do not research the effectiveness of their in-game advertisements out of fear
that the results may be unbeneficial to them. The efforts of the current study can help create a
clearer image of the value of in-game advertising that may reassure the advertising agencies.
§ 6.1.2 Explicit attitude
The analysis of variance regarding the explicit attitude showed that the explicit attitude
was not affected by only the in-game advertisements. Rather, the influence of in-game
advertisements on the explicit attitude was moderated by difficulty level. Contrary to the
expectations, this moderating effect was not linear. Instead the effect of in-game brands was
absent in the low and high difficulty levels and only present in the medium difficulty level.
This suggests that there may be a optimum difficulty level where gamers are intrigued to
allocate enough cognitive resources to the game, yet are not overly burdened with information
processing that they can no longer process the advertisements.
Furthermore the regression analysis regarding the explicit attitude showed clear results. It
showed that frustration wasn't able to more accurately predict when in-game advertising was
effective than simply difficulty level, which means that both the associative and the cognitive
explanation of the moderating effect of difficulty still stand. Further research is needed to
settle which explanation is superior or whether both partially explain the moderating effect of
difficulty level. The interaction between brand and recognition in the regression analysis also
showed that gamers who manage to recognize the in-game brand after the playing session are
more influenced by the in-game advertisement than those who did not recognize the brand
after the play session. Finally, it did not matter whether the participants knew the game or
were experience with racing games.
§ 6.1.3 Implicit attitude
The results of the analysis of variance regarding the implicit attitude showed at first glance
that was no interaction effect between the in-game brands and the difficulty level. It was clear
that the in-game brand significantly predicted the implicit attitude. Furthermore, there seemed
to be a main effect of difficulty level, which would mean that participants preferred Carlsberg
in the low difficulty level and Budweiser in the high difficulty level. Such an effect proves
difficult to explain. However, pairwise comparisons showed that the participants in the
35. 33
different difficulty conditions did not have significantly different implicit attitudes and the
data showed a pattern that could be expected in case of an interaction between the in-game
brands and the difficulty level. This more easily comprehensible explanation showed that the
implicit attitudes of participants who had played the low difficulty version of the game with
in-game Carlsberg billboards were significantly more in favour of Carlsberg than the
participants who had played the version of the game with in-game Budweiser billboards,
while there were no significant differences between the versions in the medium and high
difficulty conditions. This explanation also fits the results found by Bosch (2013). While it is
clear that brands do affect the implicit attitude and the data do seem to indicate that difficulty
plays a role in this, it is not clear cut what that role is. Most of the data seems to point at an
interaction effect, but the analysis of variance did not confirm that.
The regression analysis suggests that frustration and recognition do not play any role of
significance. It is worth noting that while recognition did explain some of the variance in the
influence of the in-game advertisements on the explicit attitude, it does not significantly
predict any of the variance of the effect of the in-game advertisements on the implicit attitude.
This suggests that the effect of in-game advertising on the implicit attitude is indeed different
from the effect on the explicit attitude. Finally, it did not matter whether the participants knew
the game or were experienced with racing games.
§ 6.2 Limitations
Special care should be taken when generalizing the results of this thesis. The study was
conducted with Dutch male university students only and specifically involved a racing game.
It can be expected that these results do not directly translate to how in-game advertising
affects people of another nationality, gender, age or education and they may not apply to other
games or ways of advertising within a game other than billboards. Moreover, while it is
designed specifically to limit the hindrance of external factors, the laboratory where the
experiments were conducted was very different from a natural gaming environment. Next, the
statistical evidence pointing at an interaction effect of brand and difficulty on the implicit
attitude is weak at best. Finally, while the regression analysis did indicate that the associative
explanation of the moderating effect of difficulty was not superior to just difficulty level,
neither the associative nor the cognitive explanation can be ruled out at this point.
36. 34
§ 6.3 Discussion
This thesis bears theoretical and practical relevance. There is strong evidence that brands
on in-game billboards can affect both gamers' explicit and the implicit attitude towards those
brands. This information can provide advertisers and advertising agencies with some
reassurance that even a very short exposure to a game with in-game advertising can have a
significant positive effect on gamers' attitudes towards their brands. Moreover, this thesis
found that difficulty level likely moderates these effects, which provides some insight in
which conditions the in-game billboards may have an effect. More specifically, in-game
billboards are likely to have an effect on the implicit attitude on a low difficulty level,
whereas they are more likely to have an effect on the explicit attitude on a medium difficulty
level.
On a theoretical level this thesis provides more insight in the processes that are behind the
effects of in-game advertising. The results suggest that in-game billboards can both affect the
explicit and the implicit attitude. Moreover, an interaction between the in-game brand and
difficulty level seems to influence the explicit attitude, whereas Bosch (2013) did not find an
effect of in-game advertising on the explicit attitude at all. This fits better with other studies
that have found effects of in-game advertising on the explicit attitude (Grigorovici &
Constantin, 2004; Herrewijn & Poels, 2011; Sharma et al., 2007). The difference with the
study by Bosch (2013) can be explained by looking at the number of difficulty levels. Bosch
only used a low and a high difficulty level and found no effects of brand on the explicit
attitude. In the current study, no significant effects of the in-game brand could be found at
these difficulty levels either, despite improving on the sensitivity of the measurement. Only at
the newly added medium difficulty level could a significant effect of the in-game brand on the
explicit attitude be found. It seems that there may be an optimum difficulty level to stimulate
the gamer to assign a large amount of cognitive resources to the game without providing too
much information so that an optimum amount of unallocated resources is available. Further
research may add different difficulty levels in order to find out what the extend is of this
optimum level.
The results surrounding the effects of the in-game brands on the implicit attitude were less
conclusive. The main effects of both the in-game brand and difficulty level were significant,
but their interaction was not. A main effect of difficulty level is unexpected and has no
meaningful explanation. Pairwise comparisons may however have indicated that there was a
interaction effect that did not predict enough of the variance to reach significance, which
provides a more meaningful explanation and would fit the results of Bosch (2013). This does
37. 35
mean that an interaction between brand and difficulty level on the implicit attitude could not
be confirmed, perhaps because of noise caused by an unknown variable. Further research with
a different sample may be needed to confirm or reject the interaction hypothesis.
The associative explanation of the interaction effect between in-game brand and difficulty
level by Herrewijn and Poels (2011) involved varying levels of frustration. However,
frustration could not provide a better prediction than just difficulty level, so for now, the
cognitive explanation explains the moderating effect of difficulty just as well. A new study in
which an extremely easy condition is added which is more frustrating than the easy difficulty
level, but does not provide more information to process, may be able to settle this.
Other than that it was discovered that recognition may play an important role in the effect
of in-game brands on the explicit attitude. The in-game brands seem to predict the explicit
attitude better when gamers can recognize the in-game brand after the play session.
Interestingly, this was not the case for the effect of in-game brands on the implicit attitude,
which suggests that the processes involved with the effect on the explicit attitude may differ
from the processes involved with the effect on the implicit attitude. It would be interesting to
see what other differences there could be between the effects of in-game brands on these
attitudes.
38. 36
References
Batchelor, J. (September 15th, 2011). In-game ads to top $7.2bn by 2016. Retrieved April 7th,
2013, from: http://www.mcvuk.com/news/read/in-game-ads-to-top-7-2bn-by-
2016/084793
Baudoin, T. (2010). Het half-gestructureerde interview. Planning, uitvoering en
dataverwerking. Nijmegen: Radboud University, pp. 1-12.
Bosch, J. (2013). De effecten van in-game advertising op de merkattitude. Unpublished
master's thesis, Radboud University, Nijmegen, The Netherlands.
Campbell, M. C. & Keller, K. L. (2003). Brand familiarity and advertising repetition effects.
Journal of Consumer Research, 30, 292-304.
Chaney, I. M., Lin, K. H. & Chaney, J. (2004). The effect of billboards within the gaming
environment. Journal of Interactive Advertising, 5, 37-45.
Cho, C. H. & Cheon, H. J. (2004). Why do people avoid advertising on the internet? Journal
of Advertising, 33(4), 89-97.
Deary, I. J. & Der, G. (2005). Reaction time, age, and cognitive ability: Longitudinal findings
from 16 to 63 years in representative population samples. Aging, Neuropsychology,and
Cognition, 12, 187-215.
Donohew, L., Lorch, E. P,. & Palmgreen, P. (1998) Applications of a theoretic model of
information exposure to health interventions. Human Communication Research, 24(3),
454-468.
Entertainment Software Association (2012). The 2012 essential facts about the computer and
video game industry. ESA. Retrieved February 18th, 2013,
from:http://www.theesa.com/facts/pdfs/ESA_EF_2012.pdf
Fazio, R. H. & Olson, M. A. (2003). Implicit measures in social cognition research: Their
meaning and use. Annual Review of Psychology, 54, 297-327.
Friese, M., Hofmann, W. & Wänke, M. (2008). When impulses take over: Moderated
predictive validity of explicit and implicit attitude measures in predicting food choice
and consumption behavior. British Journal of Social Psychology, 47, 397-419.
Glass, Z. (2007). The effectiveness of product placement in video games. Journal of
Interactive Advertising, 8, 23-32.
39. 37
Graf, P. & Schacter, D. L. (1985). Implicit and explicit memory for new associations in
normal and amnesic subjects. Journal of Experimental Psychology: Learning, Memory,
and Cognition, 11(3), 501-518.
Greenwald, A. G. & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem,
and stereotypes. Psychological Review, 102(1), 4-27.
Greenwald, A. G., Banaji, M. R. & Nosek, B. A. (2003). Understanding and using the Implicit
Association Test: I An improved scoring algorithm. Journal of Personality and Social
Psychology, 85(2), 197-216.
Greenwald, A. G., McGhee, D. E. & Schwartz, J. L. K. (1998). Measuring individual
differences in individual cognition: The Implicit Association Test. Journal of
Personality and Social Psychology, 74(6), 1464-1480.
Grigorovici, D. M. & Constantin, C. D. (2004). Experiencing interactive advertising beyond
rich media: Impacts of ad type and presence on brand effectiveness in 3d gaming
immersive virtual environments. Journal of Interactive Advertising, 5(1), 22-36.
Hartmann, T. & Klimmt, C. (2006). Gender and computer games: Exploring females' dislikes.
Journal of Computer-Mediated Communication, 11, 910-931.
Herrewijn, L. & Poels, K. (2011). Putting brands into play: How player experiences influence
the effectiveness of in-game advertising. Retrieved on Januari 9th, 2013, from:
http://www.digra.org/dl/db/11305.46095.pdf
Hijmans, E. & Wester, F. (2006). De kwalitatieve interview-studie. In F. Wester, K.
Renckstorf, P. Scheepers (Eds.), Onderzoekstypen in de communicatiewetenschap, 2,
(507-532). Alphen aan de Rijn, Netherlands: Kluwer.
Ho, T-H. (2007). The power of videogame product placement. Unpublished master's thesis,
Radboud University, Nijmegen, The Netherlands.
Janssen, S. & Helmich, T. (2011). Effectieve product placement of prijzige pixels?
Unpublished master's thesis, Radboud University, Nijmegen, The Netherlands.
Karpinski, A. & Hilton, J. L. (2001). Attitudes and the Implicit Association Test. Journal of
Personality and Social Psychology, 81(5), 774-788.
Kim, J. K. & Pasadeos, Y. (2006). Ad avoidance by audiences across media: Modeling
beliefs, attitudes and behavior towards advertising. Proceedings of the American
Academy of Advertising, USA.
Lang, A. (2000). The Limited Capacity Model of Mediated Message Processing. International
Communication Association, 50(1), 46-79.
40. 38
Lee, M. & Faber, R. J. (2007). Effects of product placement in on-line games on brand
memory: A perspective of the limited capacity model of attention. Journal of
Advertising, 36(4), 75-90.
Lemon, M. (2006). Grinding the Axe body spray: Linking gamer experience and brand recall
in Guitar Hero 3. Unpublished master's thesis, Lousiana State University, Baton Rouge,
United States. Retrieved Januari 9th, 2013, from: http://etd.lsu.edu/docs/available/etd-
05282008-235527/unrestricted/Lemon_Thesis.pdf
Leng, H. K., Quah, S. L. & Zainuddin, F. (2010). The Obama effect: An exploratory study on
factors affecting brand recall in online games. International Journal of Trade, 1, 1- 5.
Lewis, B. & Porter, L. (2010). In-game advertising effects: Examining player perceptions of
advertising schema congruity in a massively multiplayer online role-playing game.
Journal of Interactive Advertising, 10(2), 46-60.
MacIsaac, T. (July 23rd, 2013). 'Candy Crush Saga' top grossing game: Profits in free gaming.
The Epoch Times. Retrieved August 8th, 2013, from:
http://www.theepochtimes.com/n3/197813-candy-crush-saga-top-grossing-game-
profits-in-free-gaming/
MarketingCharts (April 20th, 2007). In-game advertising revenue to reach $971MM by 2011.
Retrieved April 7th, 2013, from: http://www.marketingcharts.com/interactive/in-game-
advertising-revenue-to-reach-971mm-by-2011-1043/
Merrill, M. (October 11th, 2012). League of Legends infographic. League of Legends.
Retrieved August 8th, 2013, from:
http://forums.na.leagueoflegends.com/board/showthread.php?p=30135438
Milner, B., Corkin, S. & Teuber, H.-L. (1968). Further analysis of the hippocampal amnesic
syndrome: 14-year follow-up study of H.M. Neuropsychologia, 6, 215-234.
Nelson, M. R. (2002). Recall of brand placements in computer/video games. Journal of
Advertising Research, 42(2), 80-92.
Nelson, M. R., Keum, H. & Yaros, R. A. (2004). Advertainment or adcreep: Gameplayers'
attitudes toward advertising and product placements in computer games. Journal of
Interactive Advertising, 5, 3-21.
Nelson, M. R., Yaros, R. A. & Keum, H. (2006). Examining the influence of telepresence on
spectator and player processing of real and fictitious brands in a computer game.
Journal of Advertising, 35(4), 87-99.
41. 39
Newzoo (2011). Nationaal Gaming Onderzoek 2011. In NOS Nederland gameland, 2011,
April 28th. Retrieved August 8th, 2013, from: http://nos.nl/op3/artikel/236239-
nederland-gameland.html
Nosek, B. A., Greenwald, A. G. & Banaji, M. R. (2005). Understanding and using the Implicit
Association Test: II. Method variables and construct validity. Personality and Social
Psychology Bulletin, 31(2), 166-180.
Nunneley, S. (May 10th, 2013). Activision Q1: World of Warcraft subs down 1.3 million.
VG24 7. Retrieved August 8th, 2013, from: http://www.vg247.com/2013/05/08/acti-
blizz-q1-fy13-net-revenues-were-1-32-billion/
Olson, M. A. & Fazio, R. H. (2001). Implicit attitude formation through classical
conditioning. American Psychological Society, 12(5), 413-417.
Packard, V. (1960). De verborgen verleiders (5th ed., B. Suermondt, Trans.). Amsterdam:
Paris. (Original work published 1957)
Rovio (April 3rd, 2013). Rovio Entertainment reports 2012 financial reports. Rovio. Retrieved
August 8th, 2013, from: http://www.rovio.com/en/news/press-releases/284/rovio-
entertainment-reports-2012-financial-results
Rydell, R. J. & McConnell, A. R. (2006). Understanding implicit and explicit attitude change:
A systems of reasoning analysis. Journal of Personality and Social Psychology, 91(6),
995-1008.
Schacter, D. L. (1987). Implicit memory: History and current status. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 13(3), 501-518.
Schneider, L. P. & Cornwell, T. B. (2005). Cashing on crashes via brand placement in
computer games: The effects of experience and flow on memory. International Journal
of Advertising, 24(3), 321-343.
Shapiro, S. & Krishnan, H. S. (2001). Memory-based measures for assessing advertising
effects: A comparison of explicit and implicit memory effects. Journal of Advertising,
30(3), 1-13.
Shapiro, S., MacInnis, D. J. & Heckler, S. E. (1997). The effects of incidental ad exposure on
the formation of consideration sets. Journal of Consumer Research, 24(1), 94- 104.
Sharma, D., Mizerski, R. & Lee, A. (2007). The effect of product placement in computer
games on brand attitude and recall. University of Otago: Paper presented at the
ANZMAC 2007 Conference Proceedings.
Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional
blindness for dynamic events. Perception, 28, 1059-1260.
42. 40
Stühmeister, T. & Wenzel, T. (2010). Getting beer during commercials: Adverse effects of ad-
avoidance. Düsseldorf: Heinrich-Heine-Universität, Düsseldorf Institute for
Competition Economics.
Tassi, P. (May 9th, 2013). World of Warcraft lost 1.3M subscribers in the last three months.
Forbes. Retrieved August 8th, 2013, from:
http://www.forbes.com/sites/insertcoin/2013/05/09/world-of-warcraft-lost-1-3m-
subscribers-in-the-last-three-months/
Thier, D. (December 5th, 2013). Call of Duty: Black Ops 2 makes $1 billion in sales. Forbes.
Retrieved August 8th, 2013, from:
http://www.forbes.com/sites/davidthier/2012/12/05/call-of-duty-black-ops-2-makes-1-
billion-in-sales/
Tulving, E. & Schacter, D. L. (1990). Priming and human memory systems. Science, 247,
302-306.
Tulving, E., Schacter, D. L. & Stark, H. A. (1982). Priming effects in word-fragment
completion are independent of recognition memory. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 8(4), 336-342.
Wilbur, K. C. (2008). How the digital video recorder (DVR) changes traditional television
advertising. Journal of Advertising, 37, 143-149.
Wilson, T. D., Lindsey, S. & Schooler, T. Y. (2000). A model of dual attitudes. Psychological
Review, 107(1), 101-126.
Winkler, T. & Buckner, K. (2003). Receptiveness of gamers to embedded brand messages in
advergames: Attitudes towards product placement. Journal of Interactive Advertising,
7(1), 24-32.
Yang, M., Roskos-Ewoldsen, D. R., Dinu, L. & Arpan, L. M. (2006). The effectiveness of
“in-game” advertising: Comparing students’ explicit and implicit memory for brand
names. Journal of Advertising, 35(4), 143-152.
Yildiz, C. (April 5th, 2007). In-game advertising goed voor $1.9 miljard omzet in 2011.
Retrieved April 7th, 2013, from: http://www.marketingfacts.nl/berichten/
20070405_in_game_advertising_goed_voor_19_miljard_omzet_in_2011
Zajonc, R. B. (2001). Mere exposure: A gateway to the subliminal. Current Directions in
Psychological Science, 10, 224-228.
Zynga (July 25th, 2013). Q2 2013 earnings. Zynga. Retrieved August 8th, 2013, from:
http://files.shareholder.com/downloads/AMDA-
50. 48
Appendix III: Paper questionnaire
Vragenlijst (deel 1/4)
1) Vul hieronder het nummer in dat je hebt meegekregen voor deze vragenlijst:
. .
2) Wat was je eindtijd in de speelsessie?
. . minuten, . . seconden, . . honderdste.
3) Wat is je geboortejaar?
Hier volgen enkele vragen over jouw ervaring van de speelsessie:
4) Hoe leuk vond je de speelsessie?
Helemaal niet leuk 1 2 3 4 5 6 7 Erg leuk**
5) Hoe moeilijk vond je de speelsessie?
Erg gemakkelijk 1 2 3 4 5 6 7 Erg moeilijk**
6) Hoe goed vond je dat de speelsessie ging?
Helemaal niet goed 1 2 3 4 5 6 7 Erg goed**
Hier volgen enkele vragen over jouw ervaring met computerspellen voor de speelsessie:
7) Had je dit spel (Need for Speed: Underground 2) ooit al eerder gespeeld?
Ja / Nee*
8) Hoeveel ervaring heb jij met het spelen van racespellen?
Geen ervaring 1 2 3 4 5 6 7 Expert**
* Haal door wat niet van toepassing is.
** Omcirkel het getal dat het beste bij je past. Als je een fout hebt gemaakt, zet dan een kruis door
het foute antwoord en omcirkel het juiste antwoord.
